Fu-Chia Yang
Christos Mousas
ABSTRACT
We describe the implementation of a prototype system of 3D holographic sign language interpreters. The signing avatars, observed through wearable Mixed Reality (MR) smartglasses (e.g., Microsoft HoloLens), translate speech to Signed Exact English (SEE) in real-time. Such a system can be used by deaf and hard of hearing students in the classroom or other contexts to remove current accessibility barriers.
Supplemental Material
- N. Adamo-Villani and S. Anasingaraju. 2017. Holographic signing avatars for deaf education. In E-learning, E-education, and online training. 54–61.Google Scholar
- V. Constantinou, A. Ioannou, I. Klironomos, M. Antona, and C. Stephanidis. 2020. Technology support for the inclusion of deaf students in mainstream schools: a summary of research from 2007 to 2017. Universal Access in the Information Society 19, 1 (2020), 195–200.Google ScholarCross Ref
- A. Dachis. 2016. HoloHear is like Google translate for deaf people.Google Scholar
- J. M. De Martino, I. R. Silva, C. Z. Bolognini, P. D. P. Costa, K. M. O. Kumada, L. C. Coradine, P. H. da S. Brito, W. M. do Amaral, Â. B. Benetti, E. T. Poeta, L. M. G. Angare, C. M. Ferreira, and D. F. De Conti. 2017. Signing avatars: making education more inclusive. Universal Access in the Information Society 16, 3 (2017), 793–808.Google ScholarDigital Library
- S. Eden. 2020. Technology makes things possible: Improving the abilities of deaf and hard-of-hearing children with advanced technologies. The Oxford Handbook of deaf studies in learning and cognition (2020), 407–425.Google Scholar
- A. Miller, J. Malasig, B. Castro, V. L. Hanson, H. Nicolau, and A. Brandão. 2017. The use of smart glasses for lecture comprehension by deaf and hard of hearing students. In ACM CHI Conference Extended Abstracts on Human Factors in Computing Systems. 1909–1915.Google Scholar
- B. S. Parton. 2017. Glass vision 3D: digital discovery for the deaf. TechTrends 61, 2 (2017), 141–146.Google ScholarCross Ref
- Lorna C. Quandt, Athena Willis, Melody Schwenk, Kaitlyn Weeks, and Ruthie Ferster. 2022. Attitudes Toward Signing Avatars Vary Depending on Hearing Status, Age of Signed Language Acquisition, and Avatar Type. Frontiers in Psychology 13 (2022).Google Scholar
Recommendations
Sign Language Recognition: Learning American Sign Language in a Virtual Environment
CHI EA '19: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing SystemsIn this paper, we propose an approach, for sign language recognition, that makes use of a virtual reality headset to create an immersive environment. We show how features from data acquired by the Leap Motion controller, using an egocentric view, can be ...
Visual-gestural Interface for Auslan Virtual Assistant
SA '23: SIGGRAPH Asia 2023 Emerging TechnologiesFor Deaf people to use natural language interfaces, technologies must understand sign language input and respond in the same language. We developed a prototype smart home assistant that utilises gesture-based controls, improving accessibility and ...
Mobile sign language translation system for deaf community
W4A '12: Proceedings of the International Cross-Disciplinary Conference on Web AccessibilityNowadays, web technologies are a very efficient way to ensure communication between a large and heterogeneous audience. Furthermore, web information is mainly based on textual and multimedia content and consequently, some people with special needs, such ...
Comments