Periscope: A Keystroke Inference Attack Using Human Coupled Electromagnetic Emanations

ABSTRACT

This study presents Periscope, a novel side-channel attack that exploits human-coupled electromagnetic (EM) emanations from touchscreens to infer sensitive inputs on a mobile device. Periscope is motivated by the observation that finger movement over the touchscreen leads to time-varying coupling between these two. Consequently, it impacts the screen's EM emanations that can be picked up by a remote sensory device. We intend to map between EM measurements and finger movements to recover the inputs. As the significant technical contribution of this work, we build an analytic model that outputs finger movement trajectories based on given EM readings. Our approach does not need a large amount of labeled dataset for offline model training, but instead a couple of samples to parameterize the user-specific analytic model. We implement Periscope with simple electronic components and conduct a suite of experiments to validate this attack's impact. Experimental results show that Periscope achieves a recovery rate over 6-digit PINs of 56.2% from a distance of 90 cm. Periscope is robust against environment dynamics and can well adapt to different device models and setting contexts.

References

1. Kamran Ali, Alex X Liu, Wei Wang, and Muhammad Shahzad. 2015. Keystroke recognition using wifi signals. In Proceedings of the Annual International Conference on Mobile computing and networking.
2. Suleyman AlShowarah. 2017. The Effectiveness of Dynamic Features of Finger Based Gestures on Smartphones' Touchscreens for User Identification. International Journal of Interactive Mobile Technologies (iJIM), Vol. 11, 1 (2017), 133--142.
3. Arduino. 2020. Arduino nano. (2020). https://www.arduino.cc
4. Dmitri Asonov and Rakesh Agrawal. 2004. Keyboard acoustic emanations. In Proceedings of the IEEE Symposium on Security and Privacy.
5. Michael Backes, Tongbo Chen, Markus Duermuth, Hendrik PA Lensch, and Martin Welk. Tempest in a teapot: Compromising reflections revisited. In Proceedings of the IEEE Symposium on Security and Privacy.
6. Michael Backes, Markus Dürmuth, and Dominique Unruh. 2008. Compromising reflections-or-how to read LCD monitors around the corner. In Proceedings of the IEEE Symposium on Security and Privacy.
7. Yigael Berger, Avishai Wool, and Arie Yeredor. 2006. Dictionary attacks using keyboard acoustic emanations. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security.
8. Eli Billauer. 2020. Peak detection algorithm. (2020). http://billauer.co.il/peakdet.html
9. Liang Cai and Hao Chen. 2011. TouchLogger: Inferring Keystrokes on Touch Screen from Smartphone Motion. In Proceedings of the USENIX Summit on Hot Topics in Security.
10. Jen-Shih Chang, Arnold J Kelly, and Joseph M Crowley. 1995. Handbook of electrostatic processes .CRC Press.
11. Yimin Chen, Tao Li, Rui Zhang, Yanchao Zhang, and Terri Hedgpeth. 2018. Eyetell: Video-assisted touchscreen keystroke inference from eye movements. In Proceedings of the IEEE Symposium on Security and Privacy.
12. Moumita Dey, Alireza Nazari, Alenka Zajic, and Milos Prvulovic. Emprof: Memory profiling via em-emanation in iot and hand-held devices. In Proceedings of the IEEE/ACM International Symposium on Microarchitecture.
13. Dnschecker.org. 2020. MAC Address Lookup. (2020). https://dnschecker.org/mac-lookup.php
14. YuLei Du, YingHua Lu, and JinLing Zhang. 2013. Novel method to detect and recover the keystrokes of ps/2 keyboard. Progress In Electromagnetics Research, Vol. 41 (2013), 151--161.
15. Wilfried Elmenreich. 2002. An introduction to sensor fusion. Vienna University of Technology, Austria, Vol. 502 (2002), 1--28.
16. Song Fang, Ian Markwood, Yao Liu, Shangqing Zhao, Zhuo Lu, and Haojin Zhu. 2018. No training hurdles: Fast training-agnostic attacks to infer your typing. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security.
17. Tao Feng, Ziyi Liu, Kyeong-An Kwon, Weidong Shi, Bogdan Carbunar, Yifei Jiang, and Nhung Nguyen. 2012. Continuous mobile authentication using touchscreen gestures. In Proceedings of the IEEE Conference on Technologies for Homeland Security.
18. Denis Foo Kune and Yongdae Kim. 2010. Timing attacks on pin input devices. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security.
19. Fujitsu. 2021. Capacitive Touch Sensors: application fields, technology overview and implementation example. (2021). https://www.fujitsu.com/downloads/MICRO/fme/articles/fujitsu-whitepaper-capacitive-touch-sensors.pdf
20. Daniel Genkin, Adi Shamir, and Eran Tromer. 2014. RSA key extraction via low-bandwidth acoustic cryptanalysis. In Proceedings of the Cryptology Conference.
21. Daniel Genkin, Adi Shamir, and Eran Tromer. 2017. Acoustic cryptanalysis. Journal of Cryptology, Vol. 30, 2 (2017), 392--443.
22. Chris Hoffman. 2020. How to See Who's Connected to Your Wi-Fi Network. (2020). hhttps://www.howtogeek.com/204057/how-to-see-who%E2%80%99s-connected-to-your-wi-fi-network/
23. Leander Kahney. 2019. Your iPhone could be ?unbreakable,' if it were just 1 mm thicker. (2019). https://www.cultofmac.com/624356/your-iphone-could-be-unbreakable-if-it-were-just-1mm-thicker/
24. Markus Guenther Kuhn. 2002. Compromising emanations: eavesdropping risks of computer displays. Ph.D. Dissertation. Citeseer.
25. Kyuwon Kyoung and Reiji Hattori. 2014. Electromagnetic field analysis of capacitive touch panels. Journal of Information Display, Vol. 15, 3 (2014), 145--155.
26. Chang-Ju Lee, Jong Kang Park, Canxing Piao, Han-Eol Seo, Jaehyuk Choi, and Jung-Hoon Chun. 2018. Mutual capacitive sensing touch screen controller for ultrathin display with extended signal passband using negative capacitance. Sensors, Vol. 18, 11 (2018), 3637.
27. Mengyuan Li, Yan Meng, Junyi Liu, Haojin Zhu, Xiaohui Liang, Yao Liu, and Na Ruan. 2016. When CSI meets public WiFi: Inferring your mobile phone password via WiFi signals. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security.
28. Kang Ling, Yuntang Liu, Ke Sun, Wei Wang, Lei Xie, and Qing Gu. 2020. SpiderMon: Towards Using Cell Towers as Illuminating Sources for Keystroke Monitoring. In Proceedings of the IEEE Conference on Computer Communications.
29. Jian Liu, Yan Wang, Gorkem Kar, Yingying Chen, Jie Yang, and Marco Gruteser. 2015a. Snooping keystrokes with mm-level audio ranging on a single phone. In Proceedings of the International Conference on Mobile Computing and Networking.
30. Xiangyu Liu, Zhe Zhou, Wenrui Diao, Zhou Li, and Kehuan Zhang. 2015b. When good becomes evil: Keystroke inference with smartwatch. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security.
31. Zhuoran Liu, Niels Samwel, Léo Weissbart, Zhengyu Zhao, Dirk Lauret, Lejla Batina, and Martha Larson. 2020. Screen Gleaning: A Screen Reading TEMPEST Attack on Mobile Devices Exploiting an Electromagnetic Side Channel. In Proceedings of the Network and Distributed System Security Symposium.
32. Li Lu, Jiadi Yu, Yingying Chen, Yanmin Zhu, Xiangyu Xu, Guangtao Xue, and Minglu Li. 2019. KeyListener: Inferring keystrokes on QWERTY keyboard of touch screen through acoustic signals. In Proceedings of the IEEE Conference on Computer Communications.
33. Anindya Maiti, Oscar Armbruster, Murtuza Jadliwala, and Jibo He. 2016. Smartwatch-based keystroke inference attacks and context-aware protection mechanisms. In Proceedings of the ACM on Asia Conference on Computer and Communications Security.
34. Seita Maruyama, Satohiro Wakabayashi, and Tatsuya Mori. 2019. Tap'n Ghost: A Compilation of Novel Attack Techniques against Smartphone Touchscreens. In Proceedings of the IEEE Symposium on Security and Privacy.
35. Mathworks. 2021. Envelope Extraction. (2021). https://www.mathworks.com/help/signal/ug/envelope-extraction-using-the-analytic-signal.html
36. Emiliano Miluzzo, Alexander Varshavsky, Suhrid Balakrishnan, and Romit Roy Choudhury. 2012. Tapprints: your finger taps have fingerprints. In Proceedings of the International Conference on Mobile Systems, Applications, and Services.
37. Fan Mo, Ying-Hua Lu, Jin-Ling Zhang, Qiang Cui, and Sihai Qiu. 2013. A support vector machine for identification of monitors based on their unintended electromagnetic emanation. Progress In Electromagnetics Research, Vol. 30 (2013), 211--224.
38. Nirsoft. 2020. Who is connected sniffer. (2020). https://www.nirsoft.net/utils/who_is_connected_sniffer.html
39. Emmanuel Owusu, Jun Han, Sauvik Das, Adrian Perrig, and Joy Zhang. 2012. Accessory: password inference using accelerometers on smartphones. In Proceedings of the Workshop on Mobile Computing Systems & Applications.
40. Mickael Pruvost, Wilbert J Smit, Cécile Monteux, Philippe Poulin, and Annie Colin. 2019. Polymeric foams for flexible and highly sensitive low-pressure capacitive sensors. npj Flexible Electronics, Vol. 3, 1 (2019), 1--6.
41. Lawrence R Rabiner. 1989. A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE, Vol. 77, 2 (1989), 257--286.
42. Rahul Raguram, Andrew M White, Dibyendusekhar Goswami, Fabian Monrose, and Jan-Michael Frahm. 2011. iSpy: automatic reconstruction of typed input from compromising reflections. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security.
43. J Patrick Reilly. 2012. Applied bioelectricity: from electrical stimulation to electropathology .Springer Science & Business Media.
44. Daniel Schneider, Alexander Otte, Travis Gesslein, Philipp Gagel, Bastian Kuth, Mohamad Shahm Damlakhi, Oliver Dietz, Eyal Ofek, Michel Pahud, Per Ola Kristensson, et al. 2019. Reconviguration: Reconfiguring physical keyboards in virtual reality. IEEE transactions on visualization and computer graphics, Vol. 25, 11 (2019), 3190--3201.
45. Diksha Shukla, Rajesh Kumar, Abdul Serwadda, and Vir V Phoha. 2014. Beware, your hands reveal your secrets!. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security.
46. Ilia Shumailov, Laurent Simon, Jeff Yan, and Ross Anderson. 2019. Hearing your touch: A new acoustic side channel on smartphones. arXiv preprint arXiv:1903.11137 (2019).
47. Jingchao Sun, Xiaocong Jin, Yimin Chen, Jinxue Zhang, Yanchao Zhang, and Rui Zhang. 2016. VISIBLE: Video-Assisted Keystroke Inference from Tablet Backside Motion.. In Proceedings of the Network and Distributed System Security Symposium.
48. Unite4buy. 2021. iPhone SE specification. (2021). https://unite4buy.com/Apple-iPhone-SE-2020--3--128Gb-buy/
49. Wim Van Eck. 1985. Electromagnetic radiation from video display units: An eavesdropping risk? Computers & Security, Vol. 4, 4 (1985), 269--286.
50. Tam Vu, Akash Baid, Simon Gao, Marco Gruteser, Richard Howard, Janne Lindqvist, Predrag Spasojevic, and Jeffrey Walling. 2013. Capacitive touch communication: A technique to input data through devices' touch screen. IEEE Transactions on Mobile Computing, Vol. 13, 1 (2013), 4--19.
51. Martin Vuagnoux and Sylvain Pasini. 2009. Compromising Electromagnetic Emanations of Wired and Wireless Keyboards.. In Proceedings of the USENIX Security Symposium.
52. Junjue Wang, Kaichen Zhao, Xinyu Zhang, and Chunyi Peng. 2014. Ubiquitous keyboard for small mobile devices: harnessing multipath fading for fine-grained keystroke localization. In Proceedings of the International Conference on Mobile Systems, Applications, and Services.
53. Litao Wang and Bin Yu. 2011. Analysis and measurement on the electromagnetic compromising emanations of computer keyboards. In Proceedings of the International Conference on Computational Intelligence and Security.
54. Yao Wang, Wandong Cai, Tao Gu, and Wei Shao. 2020. Your Eyes Reveal Your Secrets: An Eye Movement Based Password Inference on Smartphone. IEEE Transactions on Mobile Computing, Vol. 19, 11 (2020), 2714--2730.
55. Yao Wang, Wandong Cai, Tao Gu, Wei Shao, Ibrahim Khalil, and Xianghua Xu. 2018. GazeRevealer: Inferring password using smartphone front camera. In Proceedings of the International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services.
56. Yi Xu, Jared Heinly, Andrew M White, Fabian Monrose, and Jan-Michael Frahm. 2013. Seeing double: Reconstructing obscured typed input from repeated compromising reflections. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security.
57. Zhi Xu, Kun Bai, and Sencun Zhu. 2012. Taplogger: Inferring user inputs on smartphone touchscreens using on-board motion sensors. In Proceedings of the ACM conference on Security and Privacy in Wireless and Mobile Networks.
58. Zhenyu Yan, Qun Song, Rui Tan, Yang Li, and Adams Wai Kin Kong. 2019. Towards touch-to-access device authentication using induced body electric potentials. In Proceedings of the International Conference on Mobile Computing and Networking.
59. Yanli Yang. 2017. A signal theoretic approach for envelope analysis of real-valued signals. IEEE Access, Vol. 5 (2017), 5623--5630.
60. Qinggang Yue, Zhen Ling, Xinwen Fu, Benyuan Liu, Kui Ren, and Wei Zhao. 2014. Blind recognition of touched keys on mobile devices. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security.
61. Jie Zhang, Xiaolong Zheng, Zhanyong Tang, Tianzhang Xing, Xiaojiang Chen, Dingyi Fang, Rong Li, Xiaoqing Gong, and Feng Chen. 2016. Privacy leakage in mobile sensing: Your unlock passwords can be leaked through wireless hotspot functionality. Mobile Information Systems, Vol. 2016 (2016).
62. Tong Zhu, Qiang Ma, Shanfeng Zhang, and Yunhao Liu. 2014. Context-free attacks using keyboard acoustic emanations. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security.
63. Li Zhuang, Feng Zhou, and J Doug Tygar. 2009. Keyboard acoustic emanations revisited. Transactions on Information and System Security, Vol. 13, 1 (2009), 1--26.
64. Anya Zhukova. 2020. How To See Who Is Connected To My WiFi. (2020). https://helpdeskgeek.com/how-to/determine-computers-connected-to-wireless-network/

Index Terms

• Security and privacy

  • Network security

    • Mobile and wireless security