skip to main content
research-article

Designing Mid-Air Haptic Gesture Controlled User Interfaces for Cars

Published:18 June 2020Publication History
Skip Abstract Section

Abstract

We present advancements in the design and development of in-vehicle infotainment systems that utilize gesture input and ultrasonic mid-air haptic feedback. Such systems employ state-of-the-art hand tracking technology and novel haptic feedback technology and promise to reduce driver distraction while performing a secondary task therefore cutting the risk of road accidents. In this paper, we document design process considerations during the development of a mid-air haptic gesture-enabled user interface for human-vehicle-interactions. This includes an online survey, business development insights, background research, and an agile framework component with three prototype iterations and user-testing on a simplified driving simulator. We report on the reasoning that led to the convergence of the chosen gesture-input and haptic-feedback sets used in the final prototype, discuss the lessons learned, and give hints and tips that act as design guidelines for future research and development of this technology in cars.

References

  1. European Transport Safety Council https://etsc.eu/euroadsafetydata/Google ScholarGoogle Scholar
  2. Pope, C.N., Bell, T.R. and Stavrinos, D., 2017. Mechanisms behind distracted driving behaviour: The role of age and executive function in the engagement of distracted driving. Accident Analysis & Prevention, 98, pp.123--129.Google ScholarGoogle ScholarCross RefCross Ref
  3. Schroeder, P., Wilbur, M., Pena, R. and Abt, S.R.B.I., 2018. National Survey on Distracted Driving Attitudes and Behaviors-2015 (No. DOT HS 812 461). United States. National Highway Traffic Safety Administration.Google ScholarGoogle Scholar
  4. Kalra, N. and Paddock, S.M., 2016. Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability? Transportation Research Part A: Policy and Practice, 94, pp.182--193.Google ScholarGoogle ScholarCross RefCross Ref
  5. Car functions now controlled by waving a hand. https://eu.usatoday.com/story/money/cars/2013/01/10/cartech-gestures-ces/1820453/I.Google ScholarGoogle Scholar
  6. E. Ohn-bar. 2012. Hand Gesture based Visual User Interface for Infotainment. AutomotiveUI'12, 111--115.Google ScholarGoogle Scholar
  7. Keenan R May, Thomas M Gable, and Bruce N Walker. 2014. A Multimodal Air Gesture Interface for In Vehicle Menu Navigation. In AutoUI. 1--6.Google ScholarGoogle Scholar
  8. A. L. Eren. et. al. 2015. Can in-vehicle touchscreens be operated with zero visual demand? An exploratory driving simulator study. In the 4th International Conference on Driver Distraction and Inattention, 15345Google ScholarGoogle Scholar
  9. May, K.R., Gable, T.M. and Walker, B.N., 2017, September. Designing an in-vehicle air gesture set using elicitation methods. In Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (pp. 74--83). ACM.Google ScholarGoogle Scholar
  10. J. Sterkenburg et. al. 2016. Development Tool for Rapid Evaluation of Eyes-free In-vehicle Gesture Controls. AutomotiveUI '16, 15--19.Google ScholarGoogle Scholar
  11. P. I. M. Cornelio et. al. 2017. Agency in mid-air interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '17) 2426--2439.Google ScholarGoogle Scholar
  12. B. Long. et. al. 2014. Rendering volumetric haptic shapes in mid-air using ultrasound. Transactions on Graphics (TOG). 33(6):181.Google ScholarGoogle Scholar
  13. Hoshi, T., Takahashi, M., Iwamoto, T. and Shinoda, H., 2010. Noncontact tactile display based on radiation pressure of airborne ultrasound. IEEE Transactions on Haptics, 3(3), pp.155--165.Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Carter, T., Seah, S.A., Long, B., Drinkwater, B. and Subramanian, S., 2013, October. UltraHaptics: multi-point mid-air haptic feedback for touch surfaces. In Proceedings of the 26th annual ACM symposium on User interface software and technology (pp. 505--514). ACM.Google ScholarGoogle Scholar
  15. Frier, W., Ablart, D., Chilles, J., Long, B., Giordano, M., Obrist, M. and Subramanian, S., 2018, June. Using spatiotemporal modulation to draw tactile patterns in mid-air. In International Conference on Human Haptic Sensing and Touch Enabled Computer Applications (pp. 270--281). Springer, Cham.Google ScholarGoogle Scholar
  16. R. Sodhi, I. Poupyrev, M. Glisson, and A. Israr, ?Aireal: Interactive Tactile Experiences in Free Air," ACM Transactions on Graphics, Vol. 32, No. 4, pp. 134:1--134:10, July 2013.Google ScholarGoogle Scholar
  17. Corenthy, L., Giordano, M., Hayden, R., Griffiths, D., Jeffrey, C., Limerick, H., ... & Subramanian, S. (2018, April). Touchless tactile displays for digital signage: mid-air haptics meets large screens. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (p. D103). ACM.Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Shakeri, G., Williamson, J.H. and Brewster, S., 2018. May the Force Be with You: Ultrasound Haptic Feedback for Mid-Air Gesture Interaction in Cars.Google ScholarGoogle Scholar
  19. Harrington, K., Large, D.R., Burnett, G. and Georgiou, O., 2018, September. Exploring the Use of Mid-Air Ultrasonic Feedback to Enhance Automotive User Interfaces. In Proceedings of the 10th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (pp. 11--20). ACM.Google ScholarGoogle Scholar
  20. Georgiou, O., Biscione, V., Harwood, A., Griffiths, D., Giordano, M., Long, B. and Carter, T., 2017, September. Haptic In-Vehicle Gesture Controls. In Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications Adjunct (pp. 233--238). ACM.Google ScholarGoogle Scholar
  21. Schmidt, A., Dey, A.K., Kun, A.L. and Spiessl, W., 2010, April. Automotive user interfaces: human computer interaction in the car. In CHI'10 Extended Abstracts on Human Factors in Computing Systems (pp. 3177--3180). ACM.Google ScholarGoogle Scholar
  22. Weichert, F., Bachmann, D., Rudak, B. and Fisseler, D., 2013. Analysis of the accuracy and robustness of the Leap Motion Controller. Sensors, 13(5), pp.6380--6393.Google ScholarGoogle ScholarCross RefCross Ref
  23. Riener, A., Ferscha, A., Bachmair, F., Hagmüller, P., Lemme, A., Muttenthaler, D., Pühringer, D., Rogner, H., Tappe, A. and Weger, F., 2013, October. Standardization of the in-car gesture interaction space. In Proceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (pp. 14--21). ACM.Google ScholarGoogle Scholar
  24. Stickdorn, M., Schneider, J., Andrews, K. and Lawrence, A., 2011. This is service design thinking: Basics, tools, cases (Vol. 1). Hoboken, NJ: Wiley.Google ScholarGoogle Scholar
  25. Norman, D., 2009. The design of future things. Basic books.Google ScholarGoogle Scholar
  26. Michailidou, I., von Saucken, C. and Lindemann, U., 2013, July. How to create a user experience story. In International Conference of Design, User Experience, and Usability (pp. 554--563). Springer, Berlin, Heidelberg.Google ScholarGoogle Scholar
  27. Hokkanen, L., Kuusinen, K. and Väänänen, K., 2016, May. Minimum viable user experience: A framework for supporting product design in startups. In International Conference on Agile Software Development (pp. 66--78). Springer, Cham.Google ScholarGoogle Scholar
  28. Nelson Cowan. 2001. The magical number 4 in short term memory. A reconsideration of storage capacity. In Behavioral and Brain Sciences, Vol. 24. 87--186.Google ScholarGoogle ScholarCross RefCross Ref
  29. Gözel Shakeri, John H Williamson, and Stephen Brewster. 2017. Novel Multimodal Feedback Techniques for In-Car Mid-Air Gesture Interaction. In AutoUI.Google ScholarGoogle Scholar
  30. Jason Sterkenburg, Steven Landry, Myounghoon Jeon, and Joshua Johnson. 2016b. Towards an in-vehicle sonically-enhanced gesture control interface: a pilot study. In ICAD.Google ScholarGoogle Scholar
  31. Large, D.R. and Burnett, G.E., 2013. Drivers' preferences and emotional responses to satellite navigation voices. International Journal of Vehicle Noise and Vibration, 9(1--2), pp.28--46.Google ScholarGoogle Scholar
  32. Hore, J., Watts, S. and Tweed, D., 1999. Prediction and compensation by an internal model for back forces during finger opening in an overarm throw. Journal of Neurophysiology, 82(3), pp.1187--1197.Google ScholarGoogle ScholarCross RefCross Ref
  33. Burnett, G., Crundall, E., Large, D., Lawson, G. and Skrypchuk, L., 2013, October. A study of unidirectional swipe gestures on in-vehicle touch screens. In Proceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (pp. 22--29). ACM.Google ScholarGoogle Scholar
  34. Gable, T.M., May, K.R. and Walker, B.N., 2014, September. Applying popular usability heuristics to gesture interaction in the vehicle. In Adjunct Proceedings of the 6th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (pp. 1--7). ACM.Google ScholarGoogle Scholar
  35. Heimgärtner, R., Solanki, A. and Windl, H., 2017. Cultural User Experience in the Car-Toward a Standardized Systematic Intercultural Agile Automotive UI/UX Design Process. In Automotive User Interfaces (pp. 143--184). Springer, Cham.Google ScholarGoogle Scholar
  36. Brhel, M., Meth, H., Maedche, A. and Werder, K., 2015. Exploring principles of user-centered agile software development: A literature review. Information and software technology, 61, pp.163--181.Google ScholarGoogle Scholar
  37. Leap Motion Developer websites https://developer.leapmotion.com/ https://github.com/leapmotion http://blog.leapmotion.com/understanding-latency-part-2/Google ScholarGoogle Scholar
  38. Yeo, H. S., Lee, B. G., & Lim, H. (2015). Hand tracking and gesture recognition system for human-computer interaction using low-cost hardware. Multimedia Tools and Applications, 74(8), 2687--2715.Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Weidner, Florian, and Wolfgang Broll. "Interact with your car: a user-elicited gesture set to inform future in-car user interfaces." Proceedings of the 18th International Conference on Mobile and Ubiquitous Multimedia. 2019.Google ScholarGoogle Scholar
  40. Ultrahaptics Developer website http://developer.ultrahaptics.com/Google ScholarGoogle Scholar
  41. C. A. Pickering. 2007. A Research Study of Hand Gesture Recognition Technologies and Applications for Human Vehicle Interaction. 3rd Conf. on Automotive Electronics.Google ScholarGoogle Scholar
  42. Wobbrock, J. O., Morris, M. R., & Wilson, A. D. (2009, April). User-defined gestures for surface computing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 1083--1092). ACM.Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Rocchesso, Davide, et al. "Accessing and selecting menu items by in-air touch." Proceedings of the 13th Biannual Conference of the Italian SIGCHI Chapter: Designing the next interaction. 2019.Google ScholarGoogle Scholar
  44. Hambling, Brian, and Pauline Van Goethem. User acceptance testing: a step-by-step guide. BCS Learning & Development, 2013.Google ScholarGoogle Scholar
  45. Valentini, Pier Paolo, and Eugenio Pezzuti. "Accuracy in fingertip tracking using Leap Motion Controller for interactive virtual applications." International Journal on Interactive Design and Manufacturing (IJIDeM) 11.3 (2017): 641--650.Google ScholarGoogle Scholar
  46. Parada-Loira, Francisco, Elisardo González-Agulla, and José L. Alba-Castro. "Hand gestures to control infotainment equipment in cars." 2014 IEEE Intelligent Vehicles Symposium Proceedings. IEEE, 2014.Google ScholarGoogle Scholar

Index Terms

  1. Designing Mid-Air Haptic Gesture Controlled User Interfaces for Cars

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!