Abstract
The recent development of disability studies in academic bodies has expedited the promotion of investigation on disability. With computer-aided tools, communication between the impaired person and someone who does not understand sign language could be accessible. A large number of people across the world are using sign language (e.g., British Sign Language (BSL), Asian Sign Language (ASL), Indian Sign Language (ISL), etc.) with hand gestures for communication. In BSL recognition, the involvement of both hands overlapping each other becomes the main challenge. Moreover, BSL comprises ambiguous signs concerning viewpoint. However, existing traditional techniques seem in-stable, less accurate, and inefficient. In this work, the BSL fingerspelling alphabet recognition problem explores using a Deep learning framework to address the above-mentioned concerns. Convolutional Neural Network (CNN) is employed to detect and recognize for classification of 26 alphabets after being trained on the BSL corpus dataset. The proposed work outperforms the existing works with better precision (6%), recall (4%), and F-measure (5̃%). It reported better results on the BSL corpus dataset and webcam videos. The model achieved better accuracy (98.0%) for a large lexicon of words than previous models (Goh & Holden [6]: 69.5%, Rambhau [9]: 79.2%, and Liwicki et al. [8]: 92.5%). The 3D CNN-based proposal performs robust hand detection, much more accurate sign recognition, more scalability, and less ambiguity in BSL finger-spelling recognition.
- [1] . 2004. A linguistic feature vector for the visual interpretation of sign language. In European Conference on Computer Vision. Springer, 390–401.Google Scholar
Cross Ref
- [2] . 2008. Long term arm and hand tracking for continuous sign language TV broadcasts. In Proceedings of the 19th British Machine Vision Conference. BMVA Press, 1105–1114.Google Scholar
Cross Ref
- [3] . 2016. Depth-projection-map-based bag of contour fragments for robust hand gesture recognition. IEEE Transactions on Human-Machine Systems 47, 4 (2016), 511–523.Google Scholar
Cross Ref
- [4] . 2004. Exploiting depth discontinuities for vision-based fingerspelling recognition. In 2004 Conference on Computer Vision and Pattern Recognition Workshop. IEEE, 155–155.Google Scholar
Cross Ref
- [5] . 2005. Automatic recognition of Auslan finger-spelling using hidden Markov models. Undergraduate Thesis, University of Western Australia (2005).Google Scholar
- [6] . 2006. Dynamic fingerspelling recognition using geometric and motion features. In 2006 International Conference on Image Processing. IEEE, 2741–2744.Google Scholar
Cross Ref
- [7] . 2006. A fast learning algorithm for deep belief nets. Neural Computation 18, 7 (2006), 1527–1554.Google Scholar
Digital Library
- [8] . 2004. Hand pose estimation for American sign language recognition. In Thirty-Sixth Southeastern Symposium on System Theory, 2004. Proceedings of the. IEEE, 132–136.Google Scholar
Cross Ref
- [9] . 2016. Metaphoric hand gestures for orientation-aware VR object manipulation with an egocentric viewpoint. IEEE Transactions on Human-Machine Systems 47, 1 (2016), 113–127.Google Scholar
- [10] . 2017. F-DES: Fast and deep event summarization. IEEE Transactions on Multimedia 20, 2 (2017), 323–334.Google Scholar
Digital Library
- [11] . 1981. British sign language.Special Education: Forward Trends 8, 1 (1981), 19–23.Google Scholar
- [12] . 1989. GEMINI: Gradient estimation through matrix inversion after noise injection. In Advances in Neural Information Processing Systems. 141–148.Google Scholar
- [13] . 2012. Efficient backprop. In Neural Networks: Tricks of the Trade. Springer, 9–48.Google Scholar
- [14] . 2014. Parsing the hand in depth images. IEEE Transactions on Multimedia 16, 5 (2014), 1241–1253.Google Scholar
Cross Ref
- [15] . 2009. Automatic recognition of fingerspelled words in British sign language. In 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops. IEEE, 50–57.Google Scholar
Cross Ref
- [16] . 2014. A hand gesture recognition framework and wearable gesture-based interaction prototype for mobile devices. IEEE Transactions on Human-Machine Systems 44, 2 (2014), 293–299.Google Scholar
Cross Ref
- [17] . 2015. VoxNet: A 3D convolutional neural network for real-time object recognition. In 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 922–928.Google Scholar
Digital Library
- [18] . 2014. Image-based and sensor-based approaches to Arabic sign language recognition. IEEE Transactions on Human-Machine Systems 44, 4 (2014), 551–557.Google Scholar
Cross Ref
- [19] . 2013. Recognition of two hand gestures of word in British sign language (BSL). International Journal of Scientific and Research Publications 3, 10 (2013).Google Scholar
- [20] . 2013. Robust part-based hand gesture recognition using Kinect sensor. IEEE Transactions on Multimedia 15, 5 (2013), 1110–1120.Google Scholar
Digital Library
- [21] . 2007. Spatio-temporal feature-extraction techniques for isolated gesture recognition in Arabic sign language. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 37, 3 (2007), 641–650.Google Scholar
Digital Library
- [22] . 2017. Sequential deep trajectory descriptor for action recognition with three-stream CNN. IEEE Transactions on Multimedia 19, 7 (2017), 1510–1520.Google Scholar
Digital Library
- [23] . 2015. Glove-based continuous Arabic sign language recognition in user-dependent mode. IEEE Transactions on Human-Machine Systems 45, 4 (2015), 526–533.Google Scholar
Cross Ref
- [24] . 2014. Superpixel-based hand gesture recognition with Kinect depth camera. IEEE Transactions on Multimedia 17, 1 (2014), 29–39.Google Scholar
Cross Ref
- [25] . 2014. Similarity assessment model for Chinese sign language videos. IEEE Transactions on Multimedia 16, 3 (2014), 751–761.Google Scholar
Digital Library
- [26] . 1987. British sign language. Language, Communication and Education (1987), 11–23.Google Scholar
- [27] . 2016. A deep neural network-driven feature learning method for multi-view facial expression recognition. IEEE Transactions on Multimedia 18, 12 (2016), 2528–2536.Google Scholar
Digital Library
Index Terms
DEAF-BSL: Deep lEArning Framework for British Sign Language recognition
Recommendations
A Machine Learning Based Sign Language Interpretation System for Communication with Deaf-mute People
Interacción '21: Proceedings of the XXI International Conference on Human Computer InteractionAn utmost necessity of the deaf-mute people is to communicate with the non-mute people without the knowledge of sign language in different environments including office premises, shopping centers, educational institutions, and the like. Although several ...
ASL-3DCNN: American sign language recognition technique using 3-D convolutional neural networks
AbstractThe communication between a person from the impaired community with a person who does not understand sign language could be a tedious task. Sign language is the art of conveying messages using hand gestures. Recognition of dynamic hand gestures in ...
American Sign Language Interpreter
T4E '12: Proceedings of the 2012 IEEE Fourth International Conference on Technology for EducationThere is currently an abundance of software in the market that is used to help teach people sign language. This software is very effective; however it is very difficult to know if you are making correct hand gestures. It is quite tedious to learn from ...






Comments