EarDynamic: An Ear Canal Deformation Based Continuous User Authentication Using In-Ear Wearables

Abstract

Biometric-based authentication is gaining increasing attention for wearables and mobile applications. Meanwhile, the growing adoption of sensors in wearables also provides opportunities to capture novel wearable biometrics. In this work, we propose EarDynamic, an ear canal deformation based user authentication using in-ear wearables. EarDynamic provides continuous and passive user authentication and is transparent to users. It leverages ear canal deformation that combines the unique static geometry and dynamic motions of the ear canal when the user is speaking for authentication. It utilizes an acoustic sensing approach to capture the ear canal deformation with the built-in microphone and speaker of the in-ear wearable. Specifically, it first emits well-designed inaudible beep signals and records the reflected signals from the ear canal. It then analyzes the reflected signals and extracts fine-grained acoustic features that correspond to the ear canal deformation for user authentication. Our extensive experimental evaluation shows that EarDynamic can achieve a recall of 97.38% and an F1 score of 96.84%. Results also show that our system works well under different noisy environments with various daily activities.

References

1. 2017. HUAWEI FreeBuds 3. https://consumer.huawei.com/en/audio/freebuds3/
2. 2020. Wireless Bluetooth Sports Headphones. https://www.jabra.com/sports-headphones
3. Aditya Abhyankar and Stephanie Schuckers. 2009. Integrating a wavelet based perspiration liveness check with fingerprint recognition. Pattern Recognition 42, 3 (2009), 452--464.
4. Amazon. 2020. Amazon Echo Buds. https://www.amazon.com/Echo-Buds/dp/B07F6VM1S3/
5. Takashi Amesaka, Hiroki Watanabe, and Masanori Sugimoto. 2019. Facial expression recognition using ear canal transfer function. In Proceedings of the 23rd International Symposium on Wearable Computers. 1--9.
6. Apple. 2020. Apple AirPods Pro. https://www.apple.com/airpods-pro/
7. Takayuki Arakawa, Takafumi Koshinaka, Shohei Yano, Hideki Irisawa, Ryoji Miyahara, and Hitoshi Imaoka. 2016. Fast and accurate personal authentication using ear acoustics. In 2016 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA). IEEE, 1--4.
8. Hind Baqeel and Saqib Saeed. 2019. Face Detection Authentication on Smartphones: End Users Usability Assessment Experiences. In 2019 International Conference on Computer and Information Sciences (ICCIS). IEEE, 1--6.
9. Abdelkareem Bedri, David Byrd, Peter Presti, Himanshu Sahni, Zehua Gue, and Thad Starner. 2015. Stick it in your ear: Building an in-ear jaw movement sensor. In Adjunct Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable Computers. 1333--1338.
10. Johan Carioli, Aidin Delnavaz, Ricardo J Zednik, and Jérémie Voix. 2017. Piezoelectric earcanal bending sensor. IEEE Sensors Journal 18, 5 (2017), 2060--2067.
11. Phillip L De Leon, Michael Pucher, Junichi Yamagishi, Inma Hernaez, and Ibon Saratxaga. 2012. Evaluation of speaker verification security and detection of HMM-based synthetic speech. IEEE Transactions on Audio, Speech, and Language Processing 20, 8 (2012), 2280--2290.
12. Aidin Delnavaz and Jérémie Voix. 2013. Energy harvesting for in-ear devices using ear canal dynamic motion. IEEE Transactions on Industrial Electronics 61, 1 (2013), 583--590.
13. Jianjiang Feng, Anil K Jain, and Arun Ross. 2009. Fingerprint alteration. submitted to IEEE TIFS (2009).
14. Matteo Ferrara, Annalisa Franco, and Davide Maltoni. 2016. On the effects of image alterations on face recognition accuracy. In Face recognition across the imaging spectrum. Springer, 195--222.
15. Yang Gao, Wei Wang, Vir V Phoha, Wei Sun, and Zhanpeng Jin. 2019. EarEcho: Using Ear Canal Echo for Wearable Authentication. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 3 (2019), 1--24.
16. Valentin Goverdovsky, David Looney, Preben Kidmose, and Danilo P Mandic. 2015. In-ear EEG from viscoelastic generic earpieces: Robust and unobtrusive 24/7 monitoring. IEEE Sensors Journal 16, 1 (2015), 271--277.
17. Malcolm J Grenness, Jon Osborn, and W Lee Weller. 2002. Mapping ear canal movement using area-based surface matching. The Journal of the Acoustical Society of America 111, 2 (2002), 960--971.
18. Rosa González Hautamäki, Tomi Kinnunen, Ville Hautamäki, Timo Leino, and Anne-Maria Laukkanen. 2013. I-vectors meet imitators: on vulnerability of speaker verification systems against voice mimicry.. In Interspeech. 930--934.
19. Mahdieh Izadpanahkakhk, Seyyed Mohammad Razavi, Mehran Taghipour-Gorjikolaie, Seyyed Hamid Zahiri, and Aurelio Uncini. 2018. Deep region of interest and feature extraction models for palmprint verification using convolutional neural networks transfer learning. Applied Sciences 8, 7 (2018), 1210.
20. Amin Jalali, Rommohan Mallipeddi, and Minho Lee. 2015. Deformation invariant and contactless palmprint recognition using convolutional neural network. In Proceedings of the 3rd International Conference on Human-Agent Interaction. 209--212.
21. Artur Janicki, Federico Alegre, and Nicholas Evans. 2016. An assessment of automatic speaker verification vulnerabilities to replay spoofing attacks. Security and Communication Networks 9, 15 (2016), 3030--3044.
22. Tomi Kinnunen, Md Sahidullah, Héctor Delgado, Massimiliano Todisco, Nicholas Evans, Junichi Yamagishi, and Kong Aik Lee. 2017. The ASVspoof 2017 challenge: Assessing the limits of replay spoofing attack detection. (2017).
23. Andreas Kipp, M-B Wesenick, and Florian Schiel. 1996. Automatic detection and segmentation of pronunciation variants in German speech corpora. In Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP'96, Vol. 1. IEEE, 106--109.
24. Thomas Kisler, Florian Schiel, and Han Sloetjes. 2012. Signal processing via web services: the use case WebMAUS. In Digital Humanities Conference 2012.
25. Rajamani Santhosh Kumar, KR Jothi Kumar, D Saravana Bhavan, and A Anandaraj. [n.d.]. Variations in the External Auditory Canal of 185 Adult Individuals: A Clinico-Morphological Study. International Journal of Scientific and Research Publications ([n. d.]), 305.
26. Nianfeng Liu, Man Zhang, Haiqing Li, Zhenan Sun, and Tieniu Tan. 2016. DeepIris: Learning pairwise filter bank for heterogeneous iris verification. Pattern Recognition Letters 82 (2016), 154--161.
27. Davide Maltoni, Dario Maio, Anil K Jain, and Salil Prabhakar. 2009. Handbook of fingerprint recognition. Springer Science & Business Media.
28. Emanuela Marasco and Arun Ross. 2014. A survey on antispoofing schemes for fingerprint recognition systems. ACM Computing Surveys (CSUR) 47, 2 (2014), 1--36.
29. Gian Luca Marcialis, Fabio Roli, and Alessandra Tidu. 2010. Analysis of fingerprint pores for vitality detection. In 2010 20th international conference on pattern recognition. IEEE, 1289--1292.
30. Amir Mohammadi, Sushil Bhattacharjee, and Sébastien Marcel. 2017. Deeply vulnerable: a study of the robustness of face recognition to presentation attacks. IET Biometrics 7, 1 (2017), 15--26.
31. Jang-Ho Park, Dae-Geun Jang, Jung Wook Park, and Se-Kyoung Youm. 2015. Wearable sensing of in-ear pressure for heart rate monitoring with a piezoelectric sensor. Sensors 15, 9 (2015), 23402--23417.
32. Chester Pirzanski and Brenda Berge. 2005. Ear canal dynamics: Facts versus perception. The Hearing Journal 58, 10 (2005), 50--52.
33. Kiran B Raja, Ramachandra Raghavendra, Vinay Krishna Vemuri, and Christoph Busch. 2015. Smartphone based visible iris recognition using deep sparse filtering. Pattern Recognition Letters 57 (2015), 33--42.
34. Daniel M Rasetshwane and Stephen T Neely. 2011. Inverse solution of ear-canal area function from reflectance. The Journal of the Acoustical Society of America 130, 6 (2011), 3873--3881.
35. Mosur K Ravishankar. 1996. Efficient Algorithms for Speech Recognition. Technical Report. Carnegie-Mellon Univ Pittsburgh pa Dept of Computer Science.
36. U Rosenhall. 1996. The Human Ear Canal. Theoretical Considerations and Clinical Applications including Cerumen Management by BB Ballachandra. JOURNAL OF AUDIOLOGICAL MEDICINE 5 (1996), 176--177.
37. Robert E Schapire and Yoav Freund. 2013. Boosting: Foundations and algorithms. Kybernetes (2013).
38. Ulrich Scherhag, Christian Rathgeb, Johannes Merkle, Ralph Breithaupt, and Christoph Busch. 2019. Face recognition systems under morphing attacks: A survey. IEEE Access 7 (2019), 23012--23026.
39. Florian Schiel, Christoph Draxler, and Jonathan Harrington. 2011. Phonemic segmentation and labelling using the MAUS technique. (2011).
40. Stephanie AC Schuckers. 2002. Spoofing and anti-spoofing measures. Information Security technical report 7, 4 (2002), 56--62.
41. Chao Shen, Yuanxun Li, Yufei Chen, Xiaohong Guan, and Roy A Maxion. 2017. Performance analysis of multi-motion sensor behavior for active smartphone authentication. IEEE Transactions on Information Forensics and Security 13, 1 (2017), 48--62.
42. Kaavya Sriskandaraja, Vidhyasaharan Sethu, Eliathamby Ambikairajah, and Haizhou Li. 2016. Front-end for antispoofing counter-measures in speaker verification: Scattering spectral decomposition. IEEE Journal of Selected Topics in Signal Processing 11,4 (2016), 632--643.
43. Michael R Stinson and BW Lawton. 1989. Specification of the geometry of the human ear canal for the prediction of sound-pressure level distribution. The Journal of the Acoustical Society of America 85, 6 (1989), 2492--2503.
44. Boudewijn Venema, Johannes Schiefer, Vladimir Blazek, Nikolai Blanik, and Steffen Leonhardt. 2013. Evaluating innovative in-ear pulse oximetry for unobtrusive cardiovascular and pulmonary monitoring during sleep. IEEE journal of translational engineering in health and medicine 1 (2013), 2700208--2700208.
45. Rudolf M Verdaasdonk and Niels Liberton. 2019. The Iphone X as 3D scanner for quantitative photography of faces for diagnosis and treatment follow-up (Conference Presentation). In Optics and Biophotonics in Low-Resource Settings V, Vol. 10869. International Society for Optics and Photonics, 1086902.
46. Jérémie Voix. 2017. The ear beyond hearing: From smart earplug to in-ear brain computer interfaces. In Proc. 24th Int. Congr. Sound Vibrat.(ICSV). 1--11.
47. Chen Wang, Junping Zhang, Jian Pu, Xiaoru Yuan, and Liang Wang. 2010. Chrono-gait image: A novel temporal template for gait recognition. In European Conference on Computer Vision. Springer, 257--270.
48. Mao-Che Wang, Chia-Yu Liu, An-Suey Shiao, and Tyrone Wang. 2005. Ear problems in swimmers. Journal of the chinese medical association 68, 8 (2005), 347--352.
49. Zhizheng Wu, Tomi Kinnunen, Nicholas Evans, Junichi Yamagishi, Cemal Hanilçi, Md Sahidullah, and Aleksandr Sizov. 2015. ASVspoof 2015: the first automatic speaker verification spoofing and countermeasures challenge. In Sixteenth Annual Conference of the International Speech Communication Association.

Index Terms

• Human-centered computing

  • Ubiquitous and mobile computing

    • Ubiquitous and mobile computing systems and tools

• Security and privacy

  • Security services

    • Authentication

      • Biometrics