skip to main content
research-article

ThermoSecure: Investigating the Effectiveness of AI-Driven Thermal Attacks on Commonly Used Computer Keyboards

Published:13 March 2023Publication History
Skip Abstract Section

Abstract

Thermal cameras can reveal heat traces on user interfaces, such as keyboards. This can be exploited maliciously to infer sensitive input, such as passwords. While previous work considered thermal attacks that rely on visual inspection of simple image processing techniques, we show that attackers can perform more effective artificial intelligence (AI)–driven attacks. We demonstrate this by presenting the development of ThermoSecure and its evaluation in two user studies (N = 21, N = 16), which reveal novel insights about thermal attacks. We detail the implementation of ThermoSecure and make a dataset of 1,500 thermal images of keyboards with heat traces resulting from input publicly available. Our first study shows that ThermoSecure successfully attacks 6-symbol, 8-symbol, 12-symbol, and 16-symbol passwords with an average accuracy of 92%, 80%, 71%, and 55% respectively, and even higher accuracy when thermal images are taken within 30 seconds. We found that typing behavior significantly impacts vulnerability to thermal attacks: hunt-and-peck typists are more vulnerable than fast typists (92% vs. 83% thermal attack success. respectively, if performed within 30 seconds). The second study showed that keycap material has a statistically significant effect on the effectiveness of thermal attacks: ABS keycaps retain the thermal trace of user presses for a longer period of time, making them more vulnerable to thermal attacks, with a 52% average attack accuracy compared with 14% for keyboards with PBT keycaps. Finally, we discuss how systems can leverage our results to protect from thermal attacks and present 7 mitigation approaches that are based on our results and previous work.

REFERENCES

  1. [1] Abdelrahman Yomna, Khamis Mohamed, Schneegass Stefan, and Alt Florian. 2017. Stay cool! Understanding thermal attacks on mobile-based user authentication. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, Colorado, (CHI’17). ACM, New York, NY, 37513763. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. [2] Abdelrahman Yomna, Shirazi Alireza Sahami, Henze Niels, and Schmidt Albrecht. 2015. Investigation of material properties for thermal imaging-based interaction. In CHI Conference on Human Factors in Computing Systems, Seoul, Republic of Korea, April 18–23, 2015. ACM, New York, NY, 1518. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. [3] Abdrabou Yasmeen, Abdelrahman Yomna, Ayman Ahmed, Elmougy Amr, and Khamis Mohamed. 2020. Are thermal attacks ubiquitous? When non-expert attackers use off the shelf thermal cameras. In Proceedings of the International Conference on Advanced Visual Interfaces, Salerno, Italy (AVI’20). ACM, New York, NY, Article 47, 5 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. [4] Abdrabou Yasmeen, Hatem Reem, Abdelrahman Yomna, Elmougy Amr, and Khamis Mohamed. 2021. Passphrases beat thermal attacks: Evaluating text input characteristics against thermal attacks on laptops and smartphones. In Human-Computer Interaction – INTERACT 2021, Ardito Carmelo, Lanzilotti Rosa, Malizia Alessio, Petrie Helen, Piccinno Antonio, Desolda Giuseppe, and Inkpen Kori (Eds.). Springer International Publishing, Cham, 712721. Google ScholarGoogle Scholar
  5. [5] Adams Anne and Sasse Martina Angela. 1999. Users are not the enemy. Communications of the ACM 42, 12 (Dec.1999), 4046. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. [6] Alotaibi Norah, Islam Md Shafiqul, Marky Karola, and Khamis Mohamed. 2022. Advanced techniques for preventing thermal imaging attacks. In 27th International Conference on Intelligent User Interfaces, Helsinki, Finland (IUI’22 Companion). ACM, New York, NY, 1821. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. [7] Aviv Adam J., Gibson Katherine, Mossop Evan, Blaze Matt, and Smith Jonathan M.. 2010. Smudge attacks on smartphone touch screens. In Proceedings of the 4th USENIX Conference on Offensive Technologies, Washington, DC (WOOT’10). USENIX Association, 17.Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. [8] Bekaert Paul, Alotaibi Norah, Mathis Florian, Gerber Nina, Rafferty Aiden, Khamis Mohamed, and Marky Karola. 2022. Are thermal attacks a realistic threat? Investigating the preconditions of thermal attacks in users’ daily lives. In Proceedings of the 12th Nordic Conference on Human-Computer Interaction (NordiCHI’22). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. [9] Joseph Bonneau, Cormac Herley, Paul C. van Oorschot, and Frank Stajano. 2012. The Quest to Replace Passwords: A Framework for Comparative Evaluation of Web Authentication Schemes. In 2012 IEEE Symposium on Security and Privacy (San Francisco Bay Area, California). IEEE, 553–567. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. [10] Luca Alexander De, Hang Alina, Brudy Frederik, Lindner Christian, and Hussmann Heinrich. 2012. Touch Me Once and I Know It’s You! Implicit Authentication Based on Touch Screen Patterns. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 987996. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. [11] Funk Markus, Schneegass Stefan, Behringer Michael, Henze Niels, and Schmidt Albrecht. 2015. An interactive curtain for media usage in the shower. In Proceedings of the 4th International Symposium on Pervasive Displays, Saarbruecken, Germany (PerDis’15). ACM, New York, NY, 225231. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. [12] He Kaiming, Gkioxari Georgia, Dollár Piotr, and Girshick Ross. 2017. Mask R-CNN. In Proceedings of the IEEE International Conference on Computer Vision. 29612969.Google ScholarGoogle ScholarCross RefCross Ref
  13. [13] Kaczmarek Tyler, Ozturk Ercan, and Tsudik Gene. 2019. Thermanator: Thermal residue-based post factum attacks on keyboard data entry. In Proceedings of the 2019 ACM Asia Conference on Computer and Communications Security, Auckland, New Zealand (Asia CCS’19). ACM, New York, NY, 586593. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. [14] Katsini Christina, Abdrabou Yasmeen, Raptis George E., Khamis Mohamed, and Alt Florian. 2020. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. The Role of Eye Gaze in Security and Privacy Applications: Survey and Future HCI Research Directions. ACM, New York, NY, 121. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. [15] Keith Mark, Shao Benjamin, and Steinbart Paul John. 2007. The usability of passphrases for authentication: An empirical field study. International Journal of Human-Computer Studies 65, 1 (2007), 1728. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. [16] Kim Han, Richardson Clark, Roberts Jeanette, Gren Lisa, and Lyon Joseph L.. 1998. Cold hands, warm heart. The Lancet 351, 9114 (1998), 1492.Google ScholarGoogle ScholarCross RefCross Ref
  17. [17] Kurz Daniel. 2020. Method and device for detecting a touch between a first object and a second object. US Patent 10,877,605.Google ScholarGoogle Scholar
  18. [18] Vladimir I. Levenshtein et al. 1966. Binary codes capable of correcting deletions, insertions, and reversals. In Soviet Physics Doklady, Vol. 10. Soviet Union, 707–710.Google ScholarGoogle Scholar
  19. [19] Li Duo, Zhang Xiao-Ping, Hu Menghan, Zhai Guangtao, and Yang Xiaokang. 2018. Physical password breaking via thermal sequence analysis. IEEE Transactions on Information Forensics and Security 14, 5 (2018), 11421154.Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. [20] Mathis Florian, Vaniea Kami, and Khamis Mohamed. 2021. RepliCueAuth: Validating the Use of a Lab-Based Virtual Reality Setup for Evaluating Authentication Systems. ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. [21] Mathis Florian, Williamson John H., Vaniea Kami, and Khamis Mohamed. 2021. Fast and secure authentication in virtual reality using coordinated 3D manipulation and pointing. ACM Transactions on Computer-Human Interaction 28, 1, Article 6 (Jan.2021), 44 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. [22] Monrose Fabian and Rubin Aviel. 1997. Authentication via keystroke dynamics. In Proceedings of the 4th ACM Conference on Computer and Communications Security, Zurich, Switzerland (CCS’97). ACM, New York, NY, 4856. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. [23] Mowery Keaton, Meiklejohn Sarah, and Savage Stefan. 2011. Heat of the moment: Characterizing the efficacy of thermal camera-based attacks. In Proceedings of the 5th USENIX Conference on Offensive Technologies. 66.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. [24] Shirazi Alireza Sahami, Abdelrahman Yomna, Henze Niels, Schneegass Stefan, Khalilbeigi Mohammadreza, and Schmidt Albrecht. 2014. Exploiting thermal reflection for interactive systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Toronto, Ontario, Canada (CHI’14). ACM, New York, NY, 34833492. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. [25] Schaub Florian, Walch Marcel, Könings Bastian, and Weber Michael. 2013. Exploring the design space of graphical passwords on smartphones. In Proceedings of the 9th Symposium on Usable Privacy and Security, Newcastle, United Kingdom (SOUPS’13). ACM, New York, NY, Article 11, 14 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. [26] Schneegass Stefan, Steimle Frank, Bulling Andreas, Alt Florian, and Schmidt Albrecht. 2014. SmudgeSafe: Geometric image transformations for smudge-resistant user authentication. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing, Seattle, Washington (UbiComp’14). ACM, New York, NY, 775786. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. [27] Shay Richard, Komanduri Saranga, Durity Adam L., Huh Phillip (Seyoung), Mazurek Michelle L., Segreti Sean M., Ur Blase, Bauer Lujo, Christin Nicolas, and Cranor Lorrie Faith. 2016. Designing password policies for strength and usability. ACM Transactions on Information and System Security 18, 4, Article 13 (May2016), 34 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. [28] Zezschwitz Emanuel von, Luca Alexander De, Brunkow Bruno, and Hussmann Heinrich. 2015. SwiPIN: Fast and Secure PIN-Entry on Smartphones. ACM, New York, NY, 14031406. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. [29] Zezschwitz Emanuel von, Koslow Anton, Luca Alexander De, and Hussmann Heinrich. 2013. Making graphic-based authentication secure against smudge attacks. In Proceedings of the 2013 International Conference on Intelligent User Interfaces, Santa Monica, CA, (IUI’13). ACM, New York, NY, 277286. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. [30] Waalen Jill and Buxbaum Joel N.. 2011. Is older colder or colder older? The association of age with body temperature in 18,630 individuals. The Journals of Gerontology: Series A 66A, 5 (022011), 487492. arXiv:https://academic.oup.com/biomedgerontology/article-pdf/66A/5/487/1 529621/glr001.pdf.Google ScholarGoogle ScholarCross RefCross Ref
  31. [31] Wodo Wojciech and Hanzlik Lucjan. 2016. Thermal imaging attacks on keypad security systems. In SECRYPT. 458464.Google ScholarGoogle Scholar
  32. [32] Yadav Garima, Maheshwari Saurabh, and Agarwal Anjali. 2014. Contrast limited adaptive histogram equalization based enhancement for real time video system. In 2014 International Conference on Advances in Computing, Communications and Informatics (ICACCI’14). 23922397. Google ScholarGoogle ScholarCross RefCross Ref
  33. [33] Yang Yong. 2007. Thermal Conductivity. Springer, New York, NY, 155163. Google ScholarGoogle ScholarCross RefCross Ref
  34. [34] Ye Guixin, Tang Zhanyong, Fang Dingyi, Chen Xiaojiang, Wolff Willy, Aviv Adam J., and Wang Zheng. 2018. A video-based attack for Android pattern lock. ACM Transactions on Privacy and Security 21, 4, Article 19 (July2018), 31 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. ThermoSecure: Investigating the Effectiveness of AI-Driven Thermal Attacks on Commonly Used Computer Keyboards

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM Transactions on Privacy and Security
        ACM Transactions on Privacy and Security  Volume 26, Issue 2
        May 2023
        335 pages
        ISSN:2471-2566
        EISSN:2471-2574
        DOI:10.1145/3572849
        Issue’s Table of Contents

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 13 March 2023
        • Online AM: 15 September 2022
        • Accepted: 1 September 2022
        • Revised: 29 August 2022
        • Received: 3 September 2021
        Published in tops Volume 26, Issue 2

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Full Text

      View this article in Full Text.

      View Full Text

      HTML Format

      View this article in HTML Format .

      View HTML Format
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!