Abstract
Sign language recognition (SLR) is a promising research field that aims to blur boundaries between Deaf and hearing people by creating a system that can transcribe signs into a written or vocal language. There is a growing body of literature that investigates the recognition of different sign languages, especially American sign language. So far, to the best of our knowledge, no study has considered the Algerian SLR. This is mainly due to the lack of datasets. To address this issue, we created the Alabib-65, the first Algerian Sign Language dataset. It consists of up to 6,238 Videos recorded from 41 native signers under realistic settings. This dataset is challenging due to a variety of reasons. First, there is a little inter-class variability. The 65 sign classes are similar in terms of hands’ configuration, placement, or movement and can share the same sub-parts. Second, there is a large intra-class variability. Furthermore, compared to other SL datasets that were collected from an indoor environment with a static and simple background, our videos were recorded from both indoor and outdoor environments with 22 backgrounds varying from simple to cluttered, and from static to dynamic. To underpin future research, we provided baseline results on this new dataset using state-of-the-art machine learning methods, namely: IDTFs with Fisher vector and SVM-classifier, VGG16-GRU, I3D, I3D-GRU, and I3D-GRU-Attention. The results show the validity and the challenges of our dataset.
- [1] . 2005. Arabic Sign Language: A perspective. J. Deaf Stud. Deaf Educ. 10, 2 (2005), 212–221.Google Scholar
Cross Ref
- [2] . 2020. A review on computer vision-based methods for human action recognition. J. Imag. 6, 6 (2020), 46–46.Google Scholar
Cross Ref
- [3] K. Al-Fityani. 2010. Deaf people, modernity, and a contentious effort to unify arab sign languages: a dissertation submitted in partial satisfaction of the requirements for the degree Doctor of Philosophy. https://www.proquest.com/docview/821548359.Google Scholar
- [4] . 2008. A lexical comparison of sign languages in the Arab world. Trabalho Apresentado ao TISLR 9 (2008).Google Scholar
- [5] Kinda Al-Fityani and Carol Padden. 2010. Sign language geography in the Arab world. In Sign Languages (Cambridge Language Surveys), Diane Brentari (Ed.). Cambridge University Press New York, Cambridge, 433–450. Google Scholar
Cross Ref
- [6] . 2015. Arabic Sign Language benchmark database for different heterogeneous sensors. In 5th International Conference on Information & Communication Technology and Accessibility (ICTA). 1–9.Google Scholar
- [7] . 2018. Deep learned vs. hand-crafted features for action classification. In IEEE 1st International Conference on Artificial Intelligence and Knowledge Engineering (AIKE). 170–171.Google Scholar
- [8] . 2020. Trajectory-based recognition of dynamic Persian sign language using hidden Markov model. Comput. Speech Lang. 61 (2020), 101053.Google Scholar
Digital Library
- [9] . 2021. An integrated CNN-LSTM model for Bangla lexical sign language recognition. In International Conference on Trends in Computational and Cognitive Engineering. 695–707.Google Scholar
Cross Ref
- [10] . 2013. Hyperopt: A Python library for optimizing the hyperparameters of machine learning algorithms. In 12th Python in Science Conference.Google Scholar
Cross Ref
- [11] . 2023. Towards zero-shot sign language recognition. IEEE Trans. Pattern Anal. Mach. Intell. 45, 1 (2023), 1217–1232. .Google Scholar
Cross Ref
- [12] . 2014. Finding friends online: Online activities by deaf students and their well-being. PloS One 9, 2 (2014), e88351.Google Scholar
Cross Ref
- [13] . 2019. Sign language detection “in the wild” with recurrent neural networks. IEEE International Conference on Acoustics, Speech and Signal Processing. 1637–1641.Google Scholar
- [14] . 2013. Hand shape recognition using Hu and Legendre moments. In 6th International Conference on Security of Information and Networks. 272–276.Google Scholar
Digital Library
- [15] . 2022. Isolated video-based Arabic Sign Language recognition using convolutional and recursive neural networks. Arab. J. Sci. Eng. 47, 2 (2022), 2187–2199.Google Scholar
Cross Ref
- [16] . 2022. Exploring collection of sign language videos through crowdsourcing. Proc. ACM Hum.-Comput. Interact. 6, CSCW2 (2022), 1–24.Google Scholar
Digital Library
- [17] . 2019. Sign language recognition, generation, and translation: An interdisciplinary perspective. In 21st International ACM SIGACCESS Conference on Computers and Accessibility. 16–31.Google Scholar
Digital Library
- [18] Josueé Espejel Cabrera, Jair Cervantes, Farid Garciía Lamont, Joseé Sergio Ruiz Castilla, and Laura D. Jalili. 2021. Mexican sign language segmentation using color based neuronal networks to detect the individual skin color. Expert Systems with Applications 183 (2021), 115295. Google Scholar
Digital Library
- [19] . 2016. BosphorusSign: A Turkish sign language recognition corpus in health and finance domains. In 10th International Conference on Language Resources and Evaluation (LREC’16). 1383–1388.Google Scholar
- [20] . 2017. Quo vadis, action recognition? A new model and the kinetics dataset. In IEEE Conference on Computer Vision and Pattern Recognition. 6299–6308.Google Scholar
Cross Ref
- [21] . 2020. Fully convolutional networks for continuous sign language recognition. In European Conference on Computer Vision. 697–714.Google Scholar
Digital Library
- [22] . 2006. Human detection using oriented histograms of flow and appearance. In European Conference on Computer Vision. 428–441.Google Scholar
Digital Library
- [23] . 2009. ImageNet: A large-scale hierarchical image database. In IEEE Conference on Computer Vision and Pattern Recognition. 248–255.Google Scholar
- [24] . 2015. Long-term recurrent convolutional networks for visual recognition and description. In IEEE Conference on Computer Vision and Pattern Recognition. 2625–2634.Google Scholar
- [25] . 2019. A review of face recognition methods using deep learning network. J. Inf. Optim. Sci. 40, 2 (2019), 547–558.Google Scholar
- [26] . 2007. GSLC: Creation and annotation of a Greek sign language corpus for HCI. In International Conference on Universal Access in Human-Computer Interaction. 657–666.Google Scholar
- [27] . 2018. Multi-modality- based Arabic Sign Language recognition. IET Comput. Vis. 12, 7 (2018), 1031–1039.Google Scholar
Cross Ref
- [28] . 2015. Signer independent isolated Italian sign recognition based on hidden Markov models. Pattern Anal. Applic. 18, 2 (2015), 385–402.Google Scholar
Digital Library
- [29] Uzma Farooq, Mohd Shafry Mohd Rahim, Nabeel Sabir, Amir Hussain, and Adnan Abid. 2021. Advances in machine translation for sign language: approaches, limitations, and challenges. Neural Computing and Applications 33, 21 (2021), 14357–14399. Google Scholar
Digital Library
- [30] . 2018. A 3D stroke based representation of sign language signs using key maximum curvature points and 3D chain codes. Multim. Tools Applic. 77, 6 (2018), 7097–7130.Google Scholar
Digital Library
- [31] VIPL group, ICT, and CAS. 2015. DEVISIGN Database. http://vipl.ict.ac.cn/homepage/ksl/data.html.Google Scholar
- [32] . 2019. A survey on deep learning based face recognition. Comput. Vis. Image Underst. 189 (2019), 102805–102805.Google Scholar
Digital Library
- [33] . 2012. Local binary pattern based features for sign language recognition. Pattern Recog. Image Anal. 22, 4 (2012), 519–526.Google Scholar
Digital Library
- [34] . 1962. Visual pattern recognition by moment invariants. IRE Trans. Inf. Theor. 8, 2 (1962), 179–187.Google Scholar
Cross Ref
- [35] . 2019. Videograph: Recognizing minutes-long human activities in videos. arXiv preprint arXiv:1905.05143 (2019).Google Scholar
- [36] . 2018. An automatic Arabic Sign Language Recognition System (ArSLRS). J. King Saud Univ.-Comput. Inf. Sci. 30, 4 (2018), 470–477.Google Scholar
Cross Ref
- [37] . 2017. The THUMOS challenge on action recognition for videos “in the wild.” Comput. Vis. Image Underst. 155 (2017), 1–23.Google Scholar
Cross Ref
- [38] . 2014. Sign language recognition using dynamic time warping and hand shape distance based on histogram of oriented gradient features. In 7th International Conference on PErvasive Technologies Related to Assistive Environments. 1–6.Google Scholar
Digital Library
- [39] . 2022. MC-TRISLAN: A large 3D motion capture sign language data-set. In LREC2022 10th Workshop on the Representation and Processing of Sign Languages: Multilingual Sign Language Resources. 88–93.Google Scholar
- [40] . 2018. MS-ASL: A large-scale data set and benchmark for understanding American Sign Language. arXiv preprint arXiv:1812.01053 (2018).Google Scholar
- [41] . 2019. Method for multimodal recognition of one-handed sign language gestures through 3D convolution and LSTM neural networks. In International Conference on Speech and Computer. 191–200.Google Scholar
Digital Library
- [42] . 2017. The kinetics human action video dataset. arXiv preprint arXiv:1705.06950 (2017).Google Scholar
- [43] . 2014. Teaching Deaf Learners: Psychological and Developmental Foundations. Oxford University Press.Google Scholar
Cross Ref
- [44] . 2020. Quantitative survey of the state of the art in sign language recognition. arXiv preprint arXiv:2008.09918 (2020).Google Scholar
- [45] . 2022. DeepSign: Sign language detection and recognition using deep learning. Electronics 11, 11 (2022), 1780.Google Scholar
Cross Ref
- [46] . 2020. The utilisation of social media among users with hearing loss: An analysis of Facebook communities. Univ. Access Inf. Societ. 19, 3 (2020), 541–555.Google Scholar
Digital Library
- [47] . 2016. Community building among deaf and hard of hearing people by using written language on social networking sites. Comput. Hum. Behav. 65 (2016), 295–307.Google Scholar
Digital Library
- [48] . 2015. Enhancing universal access: Deaf and hard of hearing people on social networking sites. Univ. Access Inf. Societ. 14, 4 (2015), 537–545.Google Scholar
Digital Library
- [49] . 2022. Contemporary assistive technologies for students with hearing loss in higher education. In International Conference on Computers Helping People with Special Needs. 512–521.Google Scholar
Digital Library
- [50] . 2020. Word-level deep sign language recognition from video: A new large-scale dataset and methods comparison. In IEEE Winter Conference on Applications of Computer Vision. 1459–1469.Google Scholar
- [51] . 2015. Effective approaches to attention-based neural machine translation. arXiv preprint arXiv:1508.04025 (2015).Google Scholar
- [52] . 2013. Isolated dynamic Persian sign language recognition based on camshift algorithm and radon transform. In 1st Iranian Conference on Pattern Recognition and Image Analysis (PRIA). 1–5.Google Scholar
Cross Ref
- [53] Sarfaraz Masood, Adhyan Srivastava, Harish Chandra Thuwal, and Musheer Ahmad. 2018. Real-time sign language gesture (word) recognition from video sequences using CNN and RNN. In Intelligent Engineering Informatics, Vikrant Bhateja, Carlos A. Coello Coello, Suresh Chandra Satapathy, and Prasant Kumar Pattnaik (Eds.). Vol. 695. Springer Singapore, 623–632. Google Scholar
Cross Ref
- [54] . 2013. Polish sign language words recognition with Kinect. In 6th International Conference on Human System Interactions (HSI). 219–226.Google Scholar
- [55] . 2016. Isolated sign language recognition using improved dense trajectories. In 24th Signal Processing and Communication Application Conference (SIU). 1961–1964.Google Scholar
Cross Ref
- [56] . 2020. Bosphorussign22k sign language recognition dataset. arXiv preprint arXiv:2004.01283 (2020).Google Scholar
- [57] . 2009. Inside Deaf Culture. Harvard University Press.Google Scholar
- [58] . 2010. Improving the Fisher kernel for large-scale image classification. European Conference on Computer Vision. 143–156.Google Scholar
- [59] . 2017. A study of convolutional architectures for handshape recognition applied to sign language. In XXIII Congreso Argentino de Ciencias de la Computación.Google Scholar
- [60] . 2016. Indian sign language recognition using SVM. Pattern Recog. Image Anal. 26, 2 (2016), 434–441.Google Scholar
Digital Library
- [61] Razieh Rastgoo, Kourosh Kiani, and Sergio Escalera. 2021. Sign language recognition: A deep survey. Expert Systems with Applications 164 (2021), 113794. Google Scholar
Cross Ref
- [62] . 2020. Video-based isolated hand sign language recognition using a deep cascaded model. Multim. Tools Applic. 79 (2020), 22965–22987.Google Scholar
Cross Ref
- [63] . 2021. Hand pose aware multimodal isolated sign language recognition. Multim. Tools Applic. 80, 1 (2021), 127–163.Google Scholar
Digital Library
- [64] . 2016. LSA64: An Argentinian sign language dataset. In XXII Congreso Argentino de Ciencias de la Computación (CACIC 2016).Google Scholar
- [65] . 2021. Recognition of user-dependent and independent static hand gestures: Application to sign language. J. Vis. Commun. Image Represent. 79 (2021), 103193.Google Scholar
Digital Library
- [66] . 2011. User-independent recognition of Arabic Sign Language for facilitating communication with the deaf community. Digit. Sig. Process. 21, 4 (2011), 535–542.Google Scholar
Digital Library
- [67] S. Shivashankara and S. Srinath. 2022. Signer independent real-time hand gestures recognition using multi-features extraction and various classifiers. International Journal of Information Technology 14, 3 (2022), 1229–1240. Google Scholar
Cross Ref
- [68] . 2015. SignsWorld Atlas; a benchmark Arabic Sign Language database. J. King Saud Univ.-Comput. Inf. Sci. 27, 1 (2015), 68–76.Google Scholar
Digital Library
- [69] . 2021. KArSL: Arabic Sign Language database. ACM Trans. Asian Low-resour. L. Inf. Process. 20, 1 (2021), 1–19.Google Scholar
Digital Library
- [70] . 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).Google Scholar
- [71] . 2021. ChaLearn LAP large scale signer independent isolated sign language recognition challenge: Design, results and future research. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3472–3481.Google Scholar
Cross Ref
- [72] . 2020. AUTSL: A large scale multi-modal Turkish Sign Language dataset and baseline methods. IEEE Access 8 (2020), 181340–181355.Google Scholar
Cross Ref
- [73] . 2021. 3D-CNN based dynamic gesture recognition for Indian Sign Language modeling. Procedia Comput. Sci. 189 (2021), 76–83.Google Scholar
Cross Ref
- [74] . 2019. Video benchmarks of human action datasets: A review. Artif. Intell. Rev. 52, 2 (2019), 1107–1154.Google Scholar
Digital Library
- [75] . 2020. INCLUDE: A large scale dataset for Indian Sign Language recognition. In 28th ACM International Conference on Multimedia. 1366–1375.Google Scholar
- [76] . 2015. Latent support vector machine modeling for sign language recognition with Kinect. ACM Trans. Intell. Syst. Technol. 6, 2 (2015), 1–20.Google Scholar
Digital Library
- [77] . 2015. Going deeper with convolutions. In IEEE Conference on Computer Vision and Pattern Recognition. 1–9.Google Scholar
Cross Ref
- [78] . 2021. SIBI sign language recognition using convolutional neural network combined with transfer learning and non-trainable parameters. Procedia Comput. Sci. 179 (2021), 72–80.Google Scholar
Cross Ref
- [79] . 2021. Evaluation of hidden Markov models using deep CNN features in isolated sign recognition. Multim. Tools Applic. 80, 13 (2021), 19137–19155.Google Scholar
Digital Library
- [80] . 2022. SL-Animals-DVS: Event-driven sign language animals dataset. Pattern Anal. Applic. 25, 3 (2022), 505–520.Google Scholar
Digital Library
- [81] Adithya Venugopalan and Rajesh Reghunadhan. 2021. Applying deep neural networks for the automatic recognition of sign language words: A communication aid to deaf agriculturists. Expert Systems with Applications 185 (2021), 115601. Google Scholar
Digital Library
- [82] . 2016. Isolated sign language recognition with Grassmann covariance matrices. ACM Trans. Access. Comput. 8, 4 (2016), 1–21.Google Scholar
Digital Library
- [83] . 2013. Action recognition with improved trajectories. In IEEE International Conference on Computer Vision. 3551–3558.Google Scholar
- [84] . 2020. A review of object detection based on deep learning. Multim. Tools Applic. 79, 33 (2020), 23729–23791.Google Scholar
Digital Library
- [85] . 2020. The Korean sign language dataset for action recognition. In International Conference on Multimedia Modeling. 532–542.Google Scholar
Digital Library
- [86] . 2020. MediaPipe hands: On-device real-time hand tracking. arXiv preprint arXiv:2006.10214 (2020).Google Scholar
- [87] . 2015. A new system for Chinese Sign Language recognition. In IEEE China Summit and International Conference on Signal and Information Processing (ChinaSIP). 534–538.Google Scholar
Cross Ref
- [88] . 2016. Chinese Sign Language recognition with adaptive HMM. In IEEE International Conference on Multimedia and Expo (ICME). 1–6.Google Scholar
Cross Ref
- [89] . 2021. Real-time sign language recognition based on video stream. Int. J. Syst. Contr. Commun. 12, 2 (2021), 158–174.Google Scholar
- [90] . 2017. Multimodal gesture recognition using 3D convolution and convolutional LSTM. IEEE Access 5 (2017), 4517–4524.Google Scholar
Cross Ref
Index Terms
Alabib-65: A Realistic Dataset for Algerian Sign Language Recognition
Recommendations
A multimodal LIBRAS-UFOP Brazilian sign language dataset of minimal pairs using a microsoft Kinect sensor
AbstractSign language recognition has made significant advances in recent years. Many researchers show interest in encouraging the development of different applications to simplify the daily life of deaf people and to integrate them into the ...
Highlights- There are limited publicly datasets that provide complete RGB-D and skeleton data.
LSA-T: The First Continuous Argentinian Sign Language Dataset for Sign Language Translation
Advances in Artificial Intelligence – IBERAMIA 2022AbstractSign language translation (SLT) is an active field of study that encompasses human-computer interaction, computer vision, natural language processing and machine learning. Progress on this field could lead to higher levels of integration of deaf ...
The Korean Sign Language Dataset for Action Recognition
MultiMedia ModelingAbstractRecently, the development of computer vision technologies has shown excellent performance in complex tasks such as behavioral recognition. Therefore, several studies propose datasets for behavior recognition, including sign language recognition. ...






Comments