Vol.12, No.3, August 2023.                                                                                                                                                                               ISSN: 2217-8309

                                                                                                                                                                                                                        eISSN: 2217-8333


TEM Journal



Association for Information Communication Technology Education and Science

Android-Based Application for Real-Time Indonesian Sign Language Recognition Using Convolutional Neural Network


Raymond Sutjiadi


© 2023 Raymond Sutjiadi, published by UIKTEN. This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. (CC BY-NC-ND 4.0)


Citation Information: TEM Journal. Volume 12, Issue 3, Pages 1541-1549, ISSN 2217-8309, DOI: 10.18421/TEM123-35, August 2023.


Received: 03 May 2023.

Revised:   02 August 2023.
Accepted: 07 August 2023.
Published: 28 August 2023.




Individuals with hearing or speech impairments face challenges in communicating with others, requiring special techniques to express their thoughts and feelings. Sign language is an alternative way to communicate using a specific pattern of hand gestures to deliver messages instead of verbal speaking (oral communication). Unfortunately, most people do not know how to use and read sign language. Because of its complexity and many types of sign language worldwide, only well-trained personnel could use it as a communication medium. This research provides a solution in the form of a machine-learning Android-based application designed to recognize sign language captured by a smartphone camera and translate it into Latin characters. The recognition accommodates Convolutional Neural Network (CNN), one of the popular deep learning algorithms. This application recognizes 26 characters of Indonesian Sign Language (Bahasa Isyarat Indonesia/BISINDO) alphabets using MobileNetV3 architecture. To build the data model, dataset images were collected from 5 different models demonstrating 26 BISINDO characters in various lighting, background, and hand gesture position. These dataset images were also generated using image augmentation process to achieve the randomness by adjusting the image rotation, noise, and brightness. Based on the testing result using 6,240 dataset images, the application has 75.38% accuracy in recognizing Indonesian Sign Language alphabets.


Keywords –Machine learning, convolutional neural network, MobileNetV3, sign language, BISINDO.



Full text PDF >  



Copyright © 2023 UIKTEN
Copyright licence: All articles are licenced via Creative Commons CC BY-NC-ND 4.0 licence