VelociWatch: Designing and Evaluating a Virtual Keyboard for the Input of Challenging Text

ABSTRACT

Virtual keyboard typing is typically aided by an auto-correct method that decodes a user's noisy taps into their intended text. This decoding process can reduce error rates and possibly increase entry rates by allowing users to type faster but less precisely. However, virtual keyboard decoders sometimes make mistakes that change a user's desired word into another. This is particularly problematic for challenging text such as proper names. We investigate whether users can guess words that are likely to cause auto-correct problems and whether users can adjust their behavior to assist the decoder. We conduct computational experiments to decide what predictions to offer in a virtual keyboard and design a smartwatch keyboard named VelociWatch. Novice users were able to use the features of VelociWatch to enter challenging text at 17 words-per-minute with a corrected error rate of 3%. Interestingly, they wrote slightly faster and just as accurately on a simpler keyboard with limited correction options. Our finding suggest users may be able to type difficult words on a smartwatch simply by tapping precisely without the use of auto-correct.

References

1. Ahmed Sabbir Arif and Ali Mazalek. 2016. A Survey of Text Entry Techniques for Smartwatches. In Proceedings, Part II, of the 18th International Conference on Human-Computer Interaction. Interaction Platforms and Techniques - Volume 9732. Springer-Verlag New York, Inc., New York, NY, USA, 255--267.
2. Ahmed Sabbir Arif and Wolfgang Stuerzlinger. 2013. Pseudo-pressure Detection and Its Use in Predictive Text Entry on Touchscreens. In Proceedings of the 25th Australian Computer-Human Interaction Conference: Augmentation, Application, Innovation, Collaboration (OzCHI '13). ACM, New York, NY, USA, 383--392.
3. Xiang 'Anthony' Chen, Tovi Grossman, and George Fitzmaurice. 2014. Swipeboard: A Text Entry Technique for Ultra-small Interfaces That Supports Novice to Expert Transitions. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST '14). ACM, New York, NY, USA, 615--620.
4. James Clawson, Kent Lyons, Alex Rudnick, Robert A. Iannucci, Jr., and Thad Starner. 2008. Automatic Whiteout++: Correcting mini-QWERTY Typing Errors Using Keypress Timing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '08). ACM, New York, NY, USA, 573--582.
5. John J. Dudley, Keith Vertanen, and Per Ola Kristensson. 2018. Fast and Precise Touch-Based Text Entry for Head-Mounted Augmented Reality with Variable Occlusion. ACM Transactions on ComputerHuman Interaction (TOCHI) 25, 6, Article 30 (12 2018), 40 pages.
6. Jun Gong, Zheer Xu, Qifan Guo, Teddy Seyed, Xiang 'Anthony' Chen, Xiaojun Bi, and Xing-Dong Yang. 2018. WrisText: One-handed Text Entry on Smartwatch Using Wrist Gestures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '18). ACM, New York, NY, USA, 181:1--181:14.
7. Joshua Goodman, Gina Venolia, Keith Steury, and Chauncey Parker. 2002. Language Modeling for Soft Keyboards. In Eighteenth National Conference on Artificial Intelligence (AAAI '02). American Association for Artificial Intelligence, Menlo Park, CA, USA, 419--424. http://dl. acm.org/citation.cfm?id=777092.777159
8. Mitchell Gordon, Tom Ouyang, and Shumin Zhai. 2016. WatchWriter: Tap and Gesture Typing on a Smartwatch Miniature Keyboard with Statistical Decoding. In Proceedings of the SIGCHI Conference on Human factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 3817--3821.
9. Timo Götzelmann and Pere-Pau Vázquez. 2015. InclineType: An Accelerometer-based Typing Approach for Smartwatches. In Proceedings of the XVI International Conference on Human Computer Interaction (Interacción '15). ACM, New York, NY, USA, Article 59, 4 pages.
10. Aakar Gupta and Ravin Balakrishnan. 2016. DualKey: Miniature Screen Text Entry via Finger Identification. In Proceedings of the SIGCHI Conference on Human factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 59--70.
11. Minako Hashimoto and Masatomo Togasi. 1995. A Virtual Oval Keyboard and a Vector Input Method for Pen-based Character Input. In Conference Companion on Human Factors in Computing Systems (CHI '95). ACM, New York, NY, USA, 254--255.
12. Jonggi Hong, Seongkook Heo, Poika Isokoski, and Geehyuk Lee. 2015. SplitBoard: A Simple Split Soft Keyboard for Wristwatch-sized Touch Screens. In Proceedings of the SIGCHI Conference on Human factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 1233--1236.
13. Clare-Marie Karat, Christine Halverson, Daniel Horn, and John Karat. 1999. Patterns of Entry and Correction in Large Vocabulary Continuous Speech Recognition Systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '99). ACM, New York, NY, USA, 568--575.
14. Per Ola Kristensson and Keith Vertanen. 2012. Performance Comparisons of Phrase Sets and Presentation Styles for Text Entry Evaluations. In Proceedings of the 2012 ACM International Conference on Intelligent User Interfaces (IUI '12). ACM, New York, NY, USA, 29--32.
15. Per Ola Kristensson and Shumin Zhai. 2004. SHARK2: A Large Vocabulary Shorthand Writing System for Pen-based Computers. In Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology (UIST '04). ACM, New York, NY, USA, 43--52.
16. Per Ola Kristensson and Shumin Zhai. 2005. Relaxing Stylus Typing Precision by Geometric Pattern Matching. In Proceedings of the 10th International Conference on Intelligent User Interfaces (IUI '05). ACM, New York, NY, USA, 151--158.
17. Kazutaka Kurihara, Masataka Goto, Jun Ogata, and Takeo Igarashi. 2006. Speech Pen: Predictive Handwriting Based on Ambient Multimodal Recognition. In Proceedings of the SIGCHI Conference on Human factors in Computing Systems (CHI '06). ACM, New York, NY, USA, 851--860.
18. Luis A. Leiva and Germán Sanchis-Trilles. 2014. Representatively Memorable: Sampling the Right Phrase Set to Get the Text Entry Experiment Right. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 1709--1712.
19. I Scott MacKenzie and R William Soukoreff. 2002. Text entry for mobile computing: Models and methods, theory and practice. Human-- Computer Interaction 17, 2--3 (2002), 147--198.
20. I. Scott MacKenzie and R. William Soukoreff. 2003. Phrase Sets for Evaluating Text Entry Techniques. In Extended Abstracts on Human Factors in Computing Systems (CHI EA '03). ACM, New York, NY, USA, 754--755.
21. I. Scott MacKenzie and Kumiko Tanaka-Ishii. 2007. Text Entry Systems: Mobility, Accessibility, Universality. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.
22. J. Ogata and M. Goto. 2005. Speech Repair: Quick Error Correction Just by Using Selection Operation for Speech Input Interfaces. In Proceedings of the International Conference on Spoken Language Processing. 133--136.
23. Stephen Oney, Chris Harrison, Amy Ogan, and Jason Wiese. 2013. ZoomBoard: A Diminutive Qwerty Soft Keyboard Using Iterative Zooming for Ultra-small Devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, New York, NY, USA, 2799--2802.
24. Germán Sanchis-Trilles and Luis A Leiva. 2014. A systematic comparison of 3 phrase sampling methods for text entry experiments in 10 languages. In Proceedings of the 16th International Conference on Human-computer Interaction with Mobile Devices & Services (MobileHCI '14). ACM, New York, NY, USA, 537--542.
25. Colton J. Turner, Barbara S. Chaparro, and Jibo He. 2017. Text Input on a Smartwatch QWERTY Keyboard: Tap vs. Trace. International Journal of Human-Computer Interaction 33, 2 (2017), 143--150. arXiv:http://dx.doi.org/10.1080/10447318.2016.1223265
26. Keith Vertanen, Crystal Fletcher, Dylan Gaines, Jacob Gould, and Per Ola Kristensson. 2018. The Impact of Word, Multiple Word, and Sentence Input on Virtual Keyboard Decoding Performance. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '18). ACM, New York, NY, USA, Article 626, 12 pages.
27. Keith Vertanen and Per Ola Kristensson. 2009. Parakeet: A Continuous Speech Recognition System for Mobile Touch-screen Devices. In Proceedings of the 14th International Conference on Intelligent User Interfaces (IUI '09). ACM, New York, NY, USA, 237--246.
28. Keith Vertanen and Per Ola Kristensson. 2011. A Versatile Dataset for Text Entry Evaluations Based on Genuine Mobile Emails. In Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices & Services (MobileHCI '11). ACM, New York, NY, USA, 295--298.
29. Keith Vertanen and Per Ola Kristensson. 2014. Complementing Text Entry Evaluations with a Composition Task. ACM Transactions of Computer Human Interaction 21, 2, Article 8 (February 2014), 33 pages.
30. Keith Vertanen, Haythem Memmi, Justin Emge, Shyam Reyal, and Per Ola Kristensson. 2015. VelociTap: Investigating Fast Mobile Text Entry Using Sentence-Based Decoding of Touchscreen Keyboard Input. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 659--668.
31. Daryl Weir, Henning Pohl, Simon Rogers, Keith Vertanen, and Per Ola Kristensson. 2014. Uncertain Text Entry on Mobile Devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 2307--2316.
32. Xin Yi, Chun Yu, Weinan Shi, Xiaojun Bi, and Yuanchun Shi. 2017. Word Clarity As a Metric in Sampling Keyboard Test Sets. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 4216--4228.
33. Xin Yi, Chun Yu, Weinan Shi, and Yuanchun Shi. 2017. Is it too small?: Investigating the performances and preferences of users when typing on tiny QWERTY keyboards. International Journal of Human-Computer Studies 106, Supplement C (2017), 44 -- 62.
34. Shumin Zhai and Per-Ola Kristensson. 2003. Shorthand Writing on Stylus Keyboard. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '03). ACM, New York, NY, USA, 97--104.
35. Shumin Zhai and Per Ola Kristensson. 2012. The word-gesture keyboard: reimagining keyboard interaction. Commun. ACM 55, 9 (2012), 91--101.
36. Shumin Zhai and Per Ola Kristensson. 2012. The Word-gesture Keyboard: Reimagining Keyboard Interaction. Commun. ACM 55, 9 (September 2012), 91--101.
37. Shumin Zhai, Per-Ola Kristensson, and Barton A Smith. 2005. In search of effective text input interfaces for off the desktop computing. Interacting with computers 17, 3 (2005), 229--250.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction techniques

      • Text input