Vidgets: modular mechanical widgets for mobile devices

Abstract

We present Vidgets, a family of mechanical widgets, specifically push buttons and rotary knobs that augment mobile devices with tangible user interfaces. When these widgets are attached to a mobile device and a user interacts with them, the widgets' nonlinear mechanical response shifts the device slightly and quickly, and this subtle motion can be detected by the accelerometer commonly equipped on mobile devices. We propose a physics-based model to understand the nonlinear mechanical response of widgets. This understanding enables us to design tactile force profiles of these widgets so that the resulting accelerometer signals become easy to recognize. We then develop a lightweight signal processing algorithm that analyzes the accelerometer signals and recognizes how the user interacts with the widgets in real time. Vidgets widgets are low-cost, compact, reconfigurable, and power efficient. They can form a diverse set of physical interfaces that enrich users' interactions with mobile devices in various practical scenarios. We demonstrate their use in three applications: photo capture with single-handed zoom, control of mobile games, and making a playable mobile music instrument.

References

1. Patrick Baudisch and Gerry Chu. 2009. Back-of-device Interaction Allows Creating Very Small Touch Devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09). ACM, New York, NY, USA, 1923--1932.
2. Donald J Berndt and James Clifford. 1994. Using dynamic time warping to find patterns in time series.. In KDD workshop, Vol. 10. Seattle, WA, 359--370.
3. Andrea Bianchi and Ian Oakley. 2013. Designing tangible magnetic appcessories. In Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction. ACM, 255--258.
4. Stephen Brewster, Stephen Brewster, Faraz Chohan, and Lorna Brown. 2007. Tactile feedback for mobile interactions. In Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 159--162.
5. Alex Butler, Shahram Izadi, and Steve Hodges. 2008. SideSight: multi-"touch" interaction around small devices. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST '08) (proceedings of the acm symposium on user interface software and technology (uist '08) ed.). Association for Computing Machinery, Inc. https://www.microsoft.com/en-us/research/publication/sidesight-multi-touch-interaction-around-small-devices/
6. Stuart K. Card, George G. Robertson, and Jock D. Mackinlay. 1991. The Information Visualizer, an Information Workspace. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '91). ACM, New York, NY, USA, 181--186.
7. Cherry GmbH. 2018. Cherry MX Switches. https://www.cherrymx.de/en.
8. Kimberly Chu and Chui Yin Wong. 2011. Mobile input devices for gaming experience. In User Science and Engineering (i-USEr), 2011 International Conference on. IEEE, 83--88.
9. Pedro F Felzenszwalb, Ross B Girshick, David McAllester, and Deva Ramanan. 2010. Object detection with discriminatively trained part-based models. IEEE transactions on pattern analysis and machine intelligence 32, 9 (2010), 1627--1645.
10. Google Inc. 2018. Sensor Overview of Android system. https://developer.android.com/guide/topics/sensors/sensors_overview.
11. Richard Hunter Harris. 1978. Buckling spring torsional snap actuator. US Patent 4,118,611.
12. Chris Harrison, Robert Xiao, and Scott Hudson. 2012. Acoustic Barcodes: Passive, Durable and Inexpensive Notched Identification Tags. In Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology (UIST '12). ACM, New York, NY, USA, 563--568.
13. Shigeo Hiraoka, Isshin Miyamoto, and Kiyoshi Tomimatsu. 2003. Behind touch, a text input method for mobile phones by the back and tactile sense interface. Information Processing Society of Japan, Interaction 2003 (2003), 131--138.
14. Eve Hoggan, Stephen A Brewster, and Jody Johnston. 2008. Investigating the effectiveness of tactile feedback for mobile touchscreens. In Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 1573--1582.
15. Sungjae Hwang, Myungwook Ahn, and Kwang-yun Wohn. 2013. MagGetz: customizable passive tangible controllers on and around conventional mobile devices. In Proceedings of the 26th annual ACM symposium on User interface software and technology. ACM, 411--416.
16. Kevin S Killourhy and Roy A Maxion. 2009. Comparing anomaly-detection algorithms for keystroke dynamics. In Dependable Systems & Networks, 2009. DSN'09. IEEE/IFIP international conference on. IEEE, 125--134.
17. Sunjun Kim, Byungjoo Lee, and Antti Oulasvirta. 2018. Impact Activation Improves Rapid Button Pressing. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, New York, NY, USA, Article 571, 8 pages.
18. Sven Kratz and Michael Rohs. 2009. HoverFlow: Expanding the Design Space of Around-device Interaction. In Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '09). ACM, New York, NY, USA, Article 4, 8 pages.
19. Gierad Laput, Eric Brockmeyer, Scott E Hudson, and Chris Harrison. 2015. Acoustruments: Passive, acoustically-driven, interactive controls for handheld devices. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 2161--2170.
20. Gierad Laput, Robert Xiao, and Chris Harrison. 2016. ViBand: High-Fidelity Bio-Acoustic Sensing Using Commodity Smartwatch Accelerometers. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16). ACM, New York, NY, USA, 321--333.
21. Byungjoo Lee and Antti Oulasvirta. 2016. Modelling error rates in temporal pointing. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 1857--1868.
22. Dingzeyu Li, David I.W. Levin, Wojciech Matusik, and Changxi Zheng. 2016. Acoustic Voxels: Computational Optimization of Modular Acoustic Filters. ACM Trans. Graph. (SIGGRAPH 2016) 35, 4 (2016).
23. Kevin A Li, Patrick Baudisch, and Ken Hinckley. 2008. Blindsight: eyes-free access to mobile phones. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1389--1398.
24. Rong-Hao Liang, Han-Chih Kuo, Liwei Chan, De-Nian Yang, and Bing-Yu Chen. 2014. GaussStones: shielded magnetic tangibles for multi-token interactions on portable displays. In Proceedings of the 27th annual ACM symposium on User interface software and technology. ACM, 365--372.
25. Nobutaka Matsushima, Wataru Yamada, and Hiroyuki Manabe. 2017. Attaching Objects to Smartphones Back Side for a Modular Interface. In Adjunct Publication of the 30th Annual ACM Symposium on User Interface Software and Technology. ACM, 51--52.
26. Robert B Miller. 1968. Response time in man-computer conversational transactions. In Proceedings of the December 9--11, 1968, fall joint computer conference, part I. ACM, 267--277.
27. Roderick Murray-Smith, John Williamson, Stephen Hughes, and Torben Quaade. 2008. Stane: Synthesized Surfaces for Tactile Input. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '08). ACM, New York, NY, USA, 1299--1302.
28. Nasser M Nasrabadi. 2007. Pattern recognition and machine learning. Journal of electronic imaging 16, 4 (2007), 049901.
29. Annie WY Ng and Alan HS Chan. 2012. Finger response times to visual, auditory and tactile modality stimuli. In Proceedings of the international multiconference of engineers and computer scientists, Vol. 2. 1449--1454.
30. Don Norman. 2013. The design of everyday things: Revised and expanded edition. Constellation.
31. Michal Piovarči, David IW Levin, Danny M Kaufman, and Piotr Didyk. 2018. Perception-aware modeling and fabrication of digital drawing tools. ACM Transactions on Graphics (SIGGRAPH 2018) 37, 4 (2018), 123.
32. David Rempel, Jack Dennerlein, CD Mote Jr, and Thomas Armstrong. 1994. A method of measuring fingertip loading during keyboard use. Journal of Biomechanics 27, 8 (1994), 1101--1104.
33. Valkyrie Savage, Andrew Head, Björn Hartmann, Dan B. Goldman, Gautham Mysore, and Wilmot Li. 2015. Lamello: Passive Acoustic Sensing for Tangible Input Components. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 1277--1280.
34. Katie A Siek, Yvonne Rogers, and Kay H Connelly. 2005. Fat finger worries: how older and younger users physically interact with PDAs. In IFIP Conference on Human-Computer Interaction. Springer, 267--280.
35. Evan Strasnick, Jackie Yang, Kesler Tanner, Alex Olwal, and Sean Follmer. 2017. shiftIO: Reconfigurable Tactile Elements for Dynamic Affordances and Mobile Interaction. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 5075--5086.
36. Masanori Sugimoto and Keiichi Hiroki. 2006. HybridTouch: an intuitive manipulation technique for PDAs using their front and rear surfaces. In Proceedings of the 8th conference on Human-computer interaction with mobile devices and services. ACM, 137--140.
37. Ge Wang. 2009. Designing Smule's Ocarina: The iPhone's Magic Flute. In NIME. 303--307.
38. Daniel Wigdor, Clifton Forlines, Patrick Baudisch, John Barnwell, and Chia Shen. 2007. Lucid touch: a see-through mobile device. In Proceedings of the 20th annual ACM symposium on User interface software and technology. ACM, 269--278.
39. Robert Xiao, Greg Lew, James Marsanico, Divya Hariharan, Scott Hudson, and Chris Harrison. 2014. Toffee: enabling ad hoc, around-device interaction with acoustic time-of-arrival correlation. In Proceedings of the 16th international conference on Human-computer interaction with mobile devices & services. ACM, 67--76.
40. Loutfouz Zaman, Daniel Natapov, and Robert J Teather. 2010. Touchscreens vs. traditional controllers in handheld gaming. In Proceedings of the international academic conference on the future of game design and technology. ACM, 183--190.
41. Cheng Zhang, Anhong Guo, Dingtian Zhang, Yang Li, Caleb Southern, Rosa I. Arriaga, and Gregory D. Abowd. 2016. Beyond the Touchscreen: An Exploration of Extending Interactions on Commodity Smartphones. ACM Trans. Interact. Intell. Syst. 6, 2, Article 16 (Aug. 2016), 23 pages.
42. Cheng Zhang, Anhong Guo, Dingtian Zhang, Caleb Southern, Rosa Arriaga, and Gregory Abowd. 2015. BeyondTouch: Extending the Input Language with Built-in Sensors on Commodity Smartphones. In Proceedings of the 20th International Conference on Intelligent User Interfaces (IUI '15). ACM, New York, NY, USA, 67--77.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction devices

      • Haptic devices