Investigating Laboratory and Everyday Typing Performance of Blind Users

Abstract

Over the last decade there have been numerous studies on touchscreen typing by blind people. However, there are no reports about blind users’ everyday typing performance and how it relates to laboratory settings. We conducted a longitudinal study involving five participants to investigate how blind users truly type on their smartphones. For 12 weeks, we collected field data, coupled with eight weekly laboratory sessions. This article provides a thorough analysis of everyday typing data and its relationship with controlled laboratory assessments. We improve state-of-the-art techniques to obtain intent from field data, and provide insights on real-world performance. Our findings show that users improve over time, even though it is at a slow rate. Substitutions are the most common type of error and have a significant impact on entry rates in both field and laboratory settings. Results show that participants are 1.3--2 times faster when typing during everyday tasks. On the other hand, they are less accurate. We finished by deriving some implications that should inform the design of a future virtual keyboard for nonvisual input. Moreover, findings should be of interest to keyboard designers and researchers looking to conduct field studies to understand everyday input performance.

References

1. Lisa Anthony, YooJin Kim, and Leah Findlater. 2013. Analyzing usergenerated youtube videos to understand touchscreen use by people with motor impairments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’13). ACM, New York, 1223--1232. http://dx.doi.org/10.1145/2470654.2466158.
2. Shiri Azenkot, Jacob O. Wobbrock, Sanjana Prasain, and Richard E. Ladner. 2012. Input finger detection for nonvisual touch screen text entry in Perkinput. In Proceedings of Graphics Interface (GI’12). Canadian Information Processing Society, Toronto, Ont., Canada, 121--129.
3. Matthew N. Bonner, Jeremy T. Brudvik, Gregory D. Abowd, and W. Keith Edwards. 2010. No-look notes: Accessible eyes-free multi-touch text entry. In Proceedings of the 8th International Conference on Pervasive Computing, 409--426.
4. Abigail Evans and Jacob Wobbrock. 2012. Taming wild behavior: The input observer for obtaining text entry and mouse pointing measures from everyday computer use. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’12). ACM, New York, 1947--1956. http://doi.acm.org/10.1145/2207676.2208338.
5. Leah Findlater, Jacob Wobbrock, and Daniel Wigdor. 2011. Typing on flat glass: Examining ten-finger expert typing patterns on touch surfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’11). ACM, New York, 2453--2462. http://dx.doi.org/10.1145/1978942.1979301.
6. Leah Findlater and Jacob Wobbrock. 2012. Personalized input: Improving ten-finger touchscreen typing through automatic adaptation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’12). ACM, New York, 815--824. http://dx.doi.org/10.1145/2207676.2208520.
7. Jon Froehlich, Mike Chen, Sunny Consolvo, Beverly Harrison, and James Landay. 2007. MyExperience: A system for in situ tracing and capturing of user feedback on mobile phones. In Proceedings of the 5th International Conference on Mobile Systems, Applications and Services (MobiSys’07). 57--70. http://dx.doi.org/10.1145/1247660.1247670.
8. Krzysztof Gajos, Katharina Reinecke, and Charles Herrmann. 2012. Accurate measurements of pointing performance from in situ observations. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’12). ACM, New York, 3157--3166. http://doi.acm.org/10.1145/2207676.2208733.
9. João Guerreiro, André Rodrigues, Kyle Montague, Tiago Guerreiro, Hugo Nicolau, and Daniel Gonçalves. 2015. TabLETS Get Physical: Non-visual text entry on tablet devices. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’15), 39--42. http://dx.doi.org/10.1145/2702123.2702373.
10. Tiago Guerreiro, Paulo Lagoá, Hugo Nicolau, Daniel Gonçalves, and Joaquim A. Jorge. 2008. From tapping to touching: Making touch screens accessible to blind users. IEEE MultiMedia 15, 4 (Oct. 2008), 48--50.
11. Faustina Hwang, Simeon Keates, Patrick Langdon, and John Clarkson. 2004. Mouse movements of motion-impaired users: A submovement analysis. In Proceedings of the 6th International ACM SIGACCESS Conference on Computers and Accessibility, 102--109. http://dx.doi.org/10.1145/1028630.1028649.
12. Amy Hurst, Jennifer Mankoff, and Scott E. Hudson. 2008. Understanding pointing problems in real world computing environments. In Proceedings of the 10th International ACM SIGACCESS Conference on Computers and Accessibility (Assets’08). ACM, New York, 43--50. http://doi.acm.org/10.1145/1414471.1414481.
13. Simeon Keates and Shari Trewin. 2005. Effect of age and Parkinson's disease on cursor positioning using a mouse. In Proceedings of the 7th International ACM SIGACCESS Conference on Computers and Accessibility (Assets’05). 68--75. http://dx.doi.org/10.1145/1090785.1090800.
14. Heidi Koester, Edmund LoPresti, and Richard Simpson. 2005. Toward Goldilocks’ pointing device: Determining a “just right” gain setting for users with physical impairments. In Proceedings of the 7th International ACM SIGACCESS Conference on Computers and Accessibility (Assets’05). ACM, New York, 84--89. http://dx.doi.org/10.1145/1090785.1090802.
15. Per Ola Kristensson. 2009. Five challenges for intelligent text entry methods. AI Magazine 30, 4 (2009), 85. http://dx.doi.org/10.1609/aimag.v30i4.2269.
16. I. Scott MacKenzie, Tatu Kauppinen, and Miika Silfverberg. 2001. Accuracy measures for evaluating computer pointing devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’01). 9--16. http://dx.doi.org/10.1145/365024.365028.
17. I. Scott MacKenzie and William Soukoreff. 2002. A character-level error analysis technique for evaluating text entry methods. Proceedings of the 2nd Nordic Conference on Human-Computer Interaction (NordiCHI’02), 243--246. http://dx.doi.org/10.1145/572020.572056.
18. I. Scott MacKenzie and William Soukoreff. 2002. Text entry for mobile computing: Models and methods, theory and practice. Human-Computer Interaction 17, 2 (2002), 147--198.
19. I. Scott MacKenzie and William Soujoreff. 2003. Phrase sets for evaluating text entry techniques. CHI’03 Extended Abstracts on Human Factors in Computing. 754--755. http://dx.doi.org/10.1145/765891.765971.
20. Sergio Mascetti, Cristian Bernareggi, and Matteo Belotti. 2012. TypeInBraille: Quick eyes-free typing on smartphones. In Proceedings of the International Conference on Computers for Handicapped Persons. 615--622. http://dx.doi.org/10.1007/978-3-642-31534-3_90.
21. Charles E. McCulloch and John M. Neuhaus. 2001. Generalized Linear Mixed Models. Wiley Online Library.
22. Kyle Montague, Hugo Nicolau, and Vicki L. Hanson. 2014. Motor-impaired touchscreen interactions in the wild. In Proceedings of the 16th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS’14). ACM, New York, 123--130. http://doi.acm.org/10.1145/2661334.2661362.
23. Kyle Montague, André Rodrigues, Hugo Nicolau, and Tiago Guerreiro. 2015. TinyBlackBox: Supporting mobile in-the-wild studies. In Proceedings of the International ACM SIGACCESS Conference on Computers and Accessibility (Assets’15), 379--380. http://dx.doi.org/10.1145/2700648.2811379.
24. Maia Naftali and Leah Findlater. 2014. Accessibility in context: Understanding the truly mobile experience of smartphone users with motor impairments. In Proceedings of the International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS’14). 209--216. http://dx.doi.org/10.1145/2661334.2661372.
25. Hugo Nicolau and Joaquim Jorge. 2012a. Elderly text-entry performance on touchscreens. In Proceedings of the 14th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS’12). 127--134. http://dx.doi.org/10.1145/2384916.2384939.
26. Hugo Nicolau and Joaquim Jorge. 2012b. Touch typing using thumbs: Understanding the effect of mobility and hand posture. In Proceedings of the 2012 ACM Annual Conference on Human Factors in Computing Systems (CHI’12), 2683--2686. http://dx.doi.org/10.1145/2207676.2208661.
27. Hugo Nicolau, Kyle Montague, Tiago Guerreiro, João Guerreiro, and Vicki L. Hanson. 2014. B#: Chord-based correction for multitouch braille input. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’14). ACM, 1705--1708. http://doi.acm.org/10.1145/2556288.2557269.
28. Hugo Nicolau, Kyle Montague, Tiago Guerreiro, André Rodrigues, and Vicki L. Hanson. 2015. Typing performance of blind users: An analysis of touch behaviors, learning effect, and in-situ usage. In Proceedings of the International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS’15). ACM, New York, 273--280. http://dx.doi.org/10.1145/2700648.2809861.
29. João Oliveira, Tiago Guerreiro, Hugo Nicolau, Joaquim Jorge, and Daniel Gonçalves. 2011. Blind people and mobile touch-based text-entry: Acknowledging the need for different flavors. In Proceedings of the 13th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS’11). ACM, New York, 179--186. http://doi.acm.org/10.1145/2049536.2049569.
30. Phillip T. Pasqual and Jacob Wobbrock. 2014. Mouse pointing endpoint prediction using kinematic template matching. In Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems (CHI’14), 743--752. http://dx.doi.org/10.1145/2556288.2557406.
31. André Rodrigues, Kyle Montague, Hugo Nicolau, and Tiago Guerreiro. 2015. Getting smartphones to talkback: Understanding the smartphone adoption process of blind users. In Proceedings of the International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS’15). ACM, New York, 23--32. http://doi.acm.org/10.1145/2700648.2809842.
32. André Rodrigues, Hugo Nicolau, Kyle Montague, Luís Carriço, and Tiago Guerreiro. 2016. Effect of target size on non-visual text-entry. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI’16). 47--52.
33. William Soukoreff and I. Scott MacKenzie. 2003. Metrics for text entry research: An evaluation of MSD and KSPC, and a new unified error metric. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’03), 113--120. http://dx.doi.org/10.1145/642611.642632.
34. Caleb Southern, James Clawson, Brian Frey, Gregory Abowd, and Mario Romero. 2012. An evaluation of brailletouch: Mobile touchscreen text entry for the visually impaired. In Proceedings of the 14th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI’12). ACM, New York, 317--326. http://doi.acm.org/10.1145/2371574.2371623.
35. Hussain Tinwala and I. Scott MacKenzie. 2010. Eyes-free text entry with error correction on touchscreen mobile devices. In Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries (NordiCHI’10), 511--520. http://dx.doi.org/10.1145/1868914.1868972.
36. Shari Trewin. 2004. Automating accessibility: The dynamic keyboard. In Proceedings of the 6th International ACM SIGACCESS Conference on Computers and Accessibility (Assets’04). ACM, New York, 71--78. http://dx.doi.org/10.1145/1028630.1028644.
37. Jacob O. Wobbrock and Brad A. Myers. 2006. Analyzing the input stream for character-level errors in unconstrained text entry evaluations. ACM Transactions on Computer-Human Interactions 13, 4 (December 2006), 458--489. http://doi.acm.org/10.1145/1188816.1188819.
38. Jacob O. Wobbrock. 2007. Measures of text entry performance. In Text Entry Systems, MacKenzie and Tanaka-Ishii (Eds.). Morgan Kaufmann, San Francisco. 47--74.
39. Jacob Wobbrock, James Fogarty, Shih-Yen Liu, Shunichi Kimuro, and Susumo Harada. 2009. The angle mouse: Target-agnostic dynamic gain adjustment based on angular deviation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’09), 1401--1410. http://dx.doi.org/10.1145/1518701.1518912.
40. Jacob O. Wobbrock, Leah Findlater, Darren Gergle, and James J. Higgins. 2011. The aligned rank transform for nonparametric factorial analyses using only anova procedures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’11). ACM, New York, 143--146. http://doi.acm.org/10.1145/1978942.1978963.
41. Georgios Yfantidis and Grigori Evreinov. 2006. Adaptive blind interaction technique for touchscreens. Universal Access in the Information Society 4, 4 (2006), 328--337. http://dx.doi.org/10.1007/s10209-004-0109-7.

Index Terms

• Human-centered computing

  • Accessibility

    • Accessibility systems and tools

    • Empirical studies in accessibility

  • Human computer interaction (HCI)

    • Interaction techniques

      • Text input