Abstract
We present advancements in the design and development of in-vehicle infotainment systems that utilize gesture input and ultrasonic mid-air haptic feedback. Such systems employ state-of-the-art hand tracking technology and novel haptic feedback technology and promise to reduce driver distraction while performing a secondary task therefore cutting the risk of road accidents. In this paper, we document design process considerations during the development of a mid-air haptic gesture-enabled user interface for human-vehicle-interactions. This includes an online survey, business development insights, background research, and an agile framework component with three prototype iterations and user-testing on a simplified driving simulator. We report on the reasoning that led to the convergence of the chosen gesture-input and haptic-feedback sets used in the final prototype, discuss the lessons learned, and give hints and tips that act as design guidelines for future research and development of this technology in cars.
- European Transport Safety Council https://etsc.eu/euroadsafetydata/Google Scholar
- Pope, C.N., Bell, T.R. and Stavrinos, D., 2017. Mechanisms behind distracted driving behaviour: The role of age and executive function in the engagement of distracted driving. Accident Analysis & Prevention, 98, pp.123--129.Google Scholar
Cross Ref
- Schroeder, P., Wilbur, M., Pena, R. and Abt, S.R.B.I., 2018. National Survey on Distracted Driving Attitudes and Behaviors-2015 (No. DOT HS 812 461). United States. National Highway Traffic Safety Administration.Google Scholar
- Kalra, N. and Paddock, S.M., 2016. Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability? Transportation Research Part A: Policy and Practice, 94, pp.182--193.Google Scholar
Cross Ref
- Car functions now controlled by waving a hand. https://eu.usatoday.com/story/money/cars/2013/01/10/cartech-gestures-ces/1820453/I.Google Scholar
- E. Ohn-bar. 2012. Hand Gesture based Visual User Interface for Infotainment. AutomotiveUI'12, 111--115.Google Scholar
- Keenan R May, Thomas M Gable, and Bruce N Walker. 2014. A Multimodal Air Gesture Interface for In Vehicle Menu Navigation. In AutoUI. 1--6.Google Scholar
- A. L. Eren. et. al. 2015. Can in-vehicle touchscreens be operated with zero visual demand? An exploratory driving simulator study. In the 4th International Conference on Driver Distraction and Inattention, 15345Google Scholar
- May, K.R., Gable, T.M. and Walker, B.N., 2017, September. Designing an in-vehicle air gesture set using elicitation methods. In Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (pp. 74--83). ACM.Google Scholar
- J. Sterkenburg et. al. 2016. Development Tool for Rapid Evaluation of Eyes-free In-vehicle Gesture Controls. AutomotiveUI '16, 15--19.Google Scholar
- P. I. M. Cornelio et. al. 2017. Agency in mid-air interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '17) 2426--2439.Google Scholar
- B. Long. et. al. 2014. Rendering volumetric haptic shapes in mid-air using ultrasound. Transactions on Graphics (TOG). 33(6):181.Google Scholar
- Hoshi, T., Takahashi, M., Iwamoto, T. and Shinoda, H., 2010. Noncontact tactile display based on radiation pressure of airborne ultrasound. IEEE Transactions on Haptics, 3(3), pp.155--165.Google Scholar
Digital Library
- Carter, T., Seah, S.A., Long, B., Drinkwater, B. and Subramanian, S., 2013, October. UltraHaptics: multi-point mid-air haptic feedback for touch surfaces. In Proceedings of the 26th annual ACM symposium on User interface software and technology (pp. 505--514). ACM.Google Scholar
- Frier, W., Ablart, D., Chilles, J., Long, B., Giordano, M., Obrist, M. and Subramanian, S., 2018, June. Using spatiotemporal modulation to draw tactile patterns in mid-air. In International Conference on Human Haptic Sensing and Touch Enabled Computer Applications (pp. 270--281). Springer, Cham.Google Scholar
- R. Sodhi, I. Poupyrev, M. Glisson, and A. Israr, ?Aireal: Interactive Tactile Experiences in Free Air," ACM Transactions on Graphics, Vol. 32, No. 4, pp. 134:1--134:10, July 2013.Google Scholar
- Corenthy, L., Giordano, M., Hayden, R., Griffiths, D., Jeffrey, C., Limerick, H., ... & Subramanian, S. (2018, April). Touchless tactile displays for digital signage: mid-air haptics meets large screens. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (p. D103). ACM.Google Scholar
Digital Library
- Shakeri, G., Williamson, J.H. and Brewster, S., 2018. May the Force Be with You: Ultrasound Haptic Feedback for Mid-Air Gesture Interaction in Cars.Google Scholar
- Harrington, K., Large, D.R., Burnett, G. and Georgiou, O., 2018, September. Exploring the Use of Mid-Air Ultrasonic Feedback to Enhance Automotive User Interfaces. In Proceedings of the 10th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (pp. 11--20). ACM.Google Scholar
- Georgiou, O., Biscione, V., Harwood, A., Griffiths, D., Giordano, M., Long, B. and Carter, T., 2017, September. Haptic In-Vehicle Gesture Controls. In Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications Adjunct (pp. 233--238). ACM.Google Scholar
- Schmidt, A., Dey, A.K., Kun, A.L. and Spiessl, W., 2010, April. Automotive user interfaces: human computer interaction in the car. In CHI'10 Extended Abstracts on Human Factors in Computing Systems (pp. 3177--3180). ACM.Google Scholar
- Weichert, F., Bachmann, D., Rudak, B. and Fisseler, D., 2013. Analysis of the accuracy and robustness of the Leap Motion Controller. Sensors, 13(5), pp.6380--6393.Google Scholar
Cross Ref
- Riener, A., Ferscha, A., Bachmair, F., Hagmüller, P., Lemme, A., Muttenthaler, D., Pühringer, D., Rogner, H., Tappe, A. and Weger, F., 2013, October. Standardization of the in-car gesture interaction space. In Proceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (pp. 14--21). ACM.Google Scholar
- Stickdorn, M., Schneider, J., Andrews, K. and Lawrence, A., 2011. This is service design thinking: Basics, tools, cases (Vol. 1). Hoboken, NJ: Wiley.Google Scholar
- Norman, D., 2009. The design of future things. Basic books.Google Scholar
- Michailidou, I., von Saucken, C. and Lindemann, U., 2013, July. How to create a user experience story. In International Conference of Design, User Experience, and Usability (pp. 554--563). Springer, Berlin, Heidelberg.Google Scholar
- Hokkanen, L., Kuusinen, K. and Väänänen, K., 2016, May. Minimum viable user experience: A framework for supporting product design in startups. In International Conference on Agile Software Development (pp. 66--78). Springer, Cham.Google Scholar
- Nelson Cowan. 2001. The magical number 4 in short term memory. A reconsideration of storage capacity. In Behavioral and Brain Sciences, Vol. 24. 87--186.Google Scholar
Cross Ref
- Gözel Shakeri, John H Williamson, and Stephen Brewster. 2017. Novel Multimodal Feedback Techniques for In-Car Mid-Air Gesture Interaction. In AutoUI.Google Scholar
- Jason Sterkenburg, Steven Landry, Myounghoon Jeon, and Joshua Johnson. 2016b. Towards an in-vehicle sonically-enhanced gesture control interface: a pilot study. In ICAD.Google Scholar
- Large, D.R. and Burnett, G.E., 2013. Drivers' preferences and emotional responses to satellite navigation voices. International Journal of Vehicle Noise and Vibration, 9(1--2), pp.28--46.Google Scholar
- Hore, J., Watts, S. and Tweed, D., 1999. Prediction and compensation by an internal model for back forces during finger opening in an overarm throw. Journal of Neurophysiology, 82(3), pp.1187--1197.Google Scholar
Cross Ref
- Burnett, G., Crundall, E., Large, D., Lawson, G. and Skrypchuk, L., 2013, October. A study of unidirectional swipe gestures on in-vehicle touch screens. In Proceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (pp. 22--29). ACM.Google Scholar
- Gable, T.M., May, K.R. and Walker, B.N., 2014, September. Applying popular usability heuristics to gesture interaction in the vehicle. In Adjunct Proceedings of the 6th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (pp. 1--7). ACM.Google Scholar
- Heimgärtner, R., Solanki, A. and Windl, H., 2017. Cultural User Experience in the Car-Toward a Standardized Systematic Intercultural Agile Automotive UI/UX Design Process. In Automotive User Interfaces (pp. 143--184). Springer, Cham.Google Scholar
- Brhel, M., Meth, H., Maedche, A. and Werder, K., 2015. Exploring principles of user-centered agile software development: A literature review. Information and software technology, 61, pp.163--181.Google Scholar
- Leap Motion Developer websites https://developer.leapmotion.com/ https://github.com/leapmotion http://blog.leapmotion.com/understanding-latency-part-2/Google Scholar
- Yeo, H. S., Lee, B. G., & Lim, H. (2015). Hand tracking and gesture recognition system for human-computer interaction using low-cost hardware. Multimedia Tools and Applications, 74(8), 2687--2715.Google Scholar
Digital Library
- Weidner, Florian, and Wolfgang Broll. "Interact with your car: a user-elicited gesture set to inform future in-car user interfaces." Proceedings of the 18th International Conference on Mobile and Ubiquitous Multimedia. 2019.Google Scholar
- Ultrahaptics Developer website http://developer.ultrahaptics.com/Google Scholar
- C. A. Pickering. 2007. A Research Study of Hand Gesture Recognition Technologies and Applications for Human Vehicle Interaction. 3rd Conf. on Automotive Electronics.Google Scholar
- Wobbrock, J. O., Morris, M. R., & Wilson, A. D. (2009, April). User-defined gestures for surface computing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 1083--1092). ACM.Google Scholar
Digital Library
- Rocchesso, Davide, et al. "Accessing and selecting menu items by in-air touch." Proceedings of the 13th Biannual Conference of the Italian SIGCHI Chapter: Designing the next interaction. 2019.Google Scholar
- Hambling, Brian, and Pauline Van Goethem. User acceptance testing: a step-by-step guide. BCS Learning & Development, 2013.Google Scholar
- Valentini, Pier Paolo, and Eugenio Pezzuti. "Accuracy in fingertip tracking using Leap Motion Controller for interactive virtual applications." International Journal on Interactive Design and Manufacturing (IJIDeM) 11.3 (2017): 641--650.Google Scholar
- Parada-Loira, Francisco, Elisardo González-Agulla, and José L. Alba-Castro. "Hand gestures to control infotainment equipment in cars." 2014 IEEE Intelligent Vehicles Symposium Proceedings. IEEE, 2014.Google Scholar
Index Terms
Designing Mid-Air Haptic Gesture Controlled User Interfaces for Cars
Recommendations
Haptic In-Vehicle Gesture Controls
AutomotiveUI '17: Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications AdjunctRecent efforts have proposed the use of hand gestures to control in-vehicle infotainment systems (IVISs) in an attempt to reduce visual demand and therefore reduce crash risk. These efforts however lack tactile feedback resulting in a decreased sense of ...
Novel Multimodal Feedback Techniques for In-Car Mid-Air Gesture Interaction
AutomotiveUI '17: Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular ApplicationsThis paper presents an investigation into the effects of different feedback modalities on mid-air gesture interaction for infotainment systems in cars. Car crashes and near-crash events are most commonly caused by driver distraction. Mid-air interaction ...
Ultrahapticons: “Haptifying” Drivers’ Mental Models to Transform Automotive Mid-Air Haptic Gesture Infotainment Interfaces
AutomotiveUI '20: 12th International Conference on Automotive User Interfaces and Interactive Vehicular ApplicationsIn-vehicle gesture interfaces show potential to reduce visual demand and improve task performance when supported with mid-air, ultrasound-haptic feedback. However, comparative studies have tended to select gestures and haptic sensations based either on ...






Comments