Hybrid-Brailler: Combining Physical and Gestural Interaction for Mobile Braille Input and Editing

ABSTRACT

Braille input enables fast nonvisual entry speeds on mobile touchscreen devices. Yet, the lack of tactile cues commonly results in typing errors, which are hard to correct. We propose Hybrid-Brailler, an input solution that combines physical and gestural interaction to provide fast and accurate Braille input. We use the back of the device for physical chorded input while freeing the touchscreen for gestural interaction. Gestures are used in editing operations, such as caret movement, text selection, and clipboard control, enhancing the overall text entry experience. We conducted two user studies to assess both input and editing performance. Results show that Hybrid-Brailler supports fast entry rates as its virtual counterpart, while significantly increasing input accuracy. Regarding editing performance, when compared with the mainstream technique, Hybrid-Brailler shows performance benefits of 21% in speed and increased editing accuracy. We finish with lessons learned for designing future nonvisual input and editing techniques.

References

1. Shiri Azenkot and Nicole B. Lee. 2013. Exploring the use of speech input by blind people on mobile devices. In Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility - ASSETS '13, 1--8.
2. Shiri Azenkot, Jacob O. Wobbrock, Sanjana Prasain, and Richard E. Ladner. 2012. Input finger detection for nonvisual touch screen text entry in Perkinput. In Proceedings of Graphics Interface, 121--129. https://doi.org/2305276.2305297
3. Matthew N. Bonner, Jeremy T. Brudvik, Gregory D. Abowd, and W. Keith Edwards. 2010. No-look notes: Accessible eyes-free multi-touch text entry. Lecture Notes in Computer Science 6030 LNCS: 409--426.
4. Maria Claudia Buzzi, Marina Buzzi, Barbara Leporini, and Amaury Trujillo. 2014. Designing a text entry multimodal keypad for blind users of touchscreen mobile phones. In Proceedings of the 16th international ACM SIGACCESS conference on Computers & accessibility (ASSETS '14), 131--136.
5. Chen Chen, Simon T Perrault, Shengdong Zhao, and Wei Tsang Ooi. 2014. BezelCopy: an efficient crossapplication copy-paste technique for touchscreen smartphones. In Proceedings of the 2014 International Working Conference on Advanced Visual Interfaces, 185--192.
6. Mark D Dunlop and Andrew Crossan. 2000. Predictive text entry methods for mobile phones. Personal Technologies 4, 2: 134--143.
7. Vittorio Fuccella, Poika Isokoski, and Benoit Martin. 2013. Gestures and Widgets: Performance in Text Editing on Multi-touch Capable Mobile Devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13), 2785--2794.
8. William Grussenmeyer and Eelke Folmer. 2017. Accessible Touchscreen Technology for People with Visual Impairments: A Survey. ACM Transactions on Accessible Computing (TACCESS) 9, 2: 6:1--6:31.
9. T Guerreiro, P Lagoá, H Nicolau, D Gonçalves, and J A Jorge. 2008. From tapping to touching: Making touch screens accessible to blind users. IEEE MultiMedia: 48--50.
10. Shaun K Kane, Jacob O Wobbrock, and Richard E Ladner. 2011. Usable Gestures for Blind People: Understanding Preference and Performance. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11), 413--422.
11. R C Littell, P R Henry, and C B Ammerman. 1998. Statistical analysis of repeated measures data using SAS procedures. Journal of animal science 76, 4: 1216--1231.
12. Kent Lyons, Thad Starner, Daniel Plaisted, James Fusia, Amanda Lyons, Aaron Drew, and E W Looney. 2004. Twiddler Typing: One-handed Chording Text Entry for Mobile Phones. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 671--678.
13. I S MacKenzie and R W Soukoreff. 2002. Text entry for mobile computing: Models and methods, theory and practice. Human-Computer Interaction 17, 2: 147-- 198.
14. I S MacKenzie and R W Soukoreff. 2003. Phrase sets for evaluating text entry techniques. In Extended abstracts of the SIGCHI Conference on Human Factors in Computing Systems, 754--755.
15. Sergio Mascetti, Cristian Bernareggi, and Matteo Belotti. 2011. TypeInBraille: A Braille-based Typing Application for Touchscreen Devices. In Proceedings of the 13th International ACM SIGACCESS Conference on Computers and Accessibility, 295.
16. Kyle Montague, Hugo Nicolau, and Vicki Hanson. 2014. Motor-Impaired Touchscreen Interactions in the Wild. In 16th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS).
17. Hugo Nicolau, Kyle Montague, Tiago Guerreiro, João Guerreiro, and Vicki L Hanson. 2014. B#: Chord-based Correction for Multitouch Braille Input. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'14), 1705--1708.
18. Hugo Nicolau, Kyle Montague, Tiago Guerreiro, André Rodrigues, and Vicki L Hanson. 2015. Typing Performance of Blind Users: An Analysis of Touch Behaviors, Learning Effect, and In-Situ Usage. In ACM SIGACCESS Conference on Computers and Accessibility.
19. Hugo Nicolau, Kyle Montague, Tiago Guerreiro, André Rodrigues, and Vicki L Hanson. 2017. Investigating Laboratory and Everyday Typing Performance of Blind Users. ACM Transactions on Accessible Computing (TACCESS) 10, 1: 4:1--4:26.
20. João Oliveira, Tiago Guerreiro, Hugo Nicolau, Joaquim Jorge, and Daniel Gonçalves. 2011. Blind people and mobile touch-based text-entry: Acknowledging the Need for Different Flavors. In The proceedings of the 13th international ACM SIGACCESS conference on Computers and accessibility - ASSETS '11, 179.
21. João Oliveira, Tiago Guerreiro, Hugo Nicolau, Joaquim Jorge, and Daniel Gonçalves. 2011. BrailleType: unleashing braille over touch screen mobile phones. Lecture Notes in Computer Science 6946 LNCS, PART 1: 100--107.
22. Jean-baptiste Scheibel, Cyril Pierson, Benoît Martin, Nathan Godard, Vittorio Fuccella, and Poika Isokoski. 2013. Virtual Stick in Caret Positioning on Touch Screens. In Proceedings of the 25th Conference on L'Interaction Homme-Machine (IHM '13), 107:107--107:114.
23. Kristen Shinohara and Jacob O Wobbrock. 2016. Selfconscious or self-confident? A diary study conceptualizing the social accessibility of assistive technology. ACM Transactions on Accessible Computing (TACCESS) 8, 2: 5.
24. R William Soukoreff and I Scott MacKenzie. 2003. Metrics for text entry research: an evaluation of MSD and KSPC, and a new unified error metric. In Proceedings of the SIGCHI conference on Human factors in computing systems, 113--120.
25. Caleb Southern, James Clawson, Brian Frey, Gregory Abowd, and Mario Romero. 2012. An Evaluation of BrailleTouch: Mobile Touchscreen Text Entry for the Visually Impaired. In Proceedings of the 14th International Conference on Human-computer Interaction with Mobile Devices and Services (MobileHCI '12), 317--326.
26. Kenji Suzuki, Kazumasa Okabe, Ryuuki Sakamoto, and Daisuke Sakamoto. 2015. Fix and Slide: Caret Navigation with Movable Background. In Adjunct Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology, 79--80.
27. Jacob O Wobbrock, Leah Findlater, Darren Gergle, and James J Higgins. 2011. The aligned rank transform for nonparametric factorial analyses using only anova procedures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 143--146.
28. Jacob O Wobbrock and Brad A Myers. 2006. Analyzing the input stream for character- level errors in unconstrained text entry evaluations. ACM Trans. Comput.-Hum. Interact. 13, 4: 458--489.
29. Georgios Yfantidis and Grigori Evreinov. 2004. Adaptive blind interaction technique for touchscreens. Universal Access in the Information Society 4, 4: 344-- 353.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

• Social and professional topics

  • Computing / technology policy