skip to main content
research-article
Public Access

Alexa, Who Am I Speaking To?: Understanding Users’ Ability to Identify Third-Party Apps on Amazon Alexa

Published:14 September 2021Publication History
Skip Abstract Section

Abstract

Many Internet of Things devices have voice user interfaces. One of the most popular voice user interfaces is Amazon’s Alexa, which supports more than 50,000 third-party applications (“skills”). We study how Alexa’s integration of these skills may confuse users. Our survey of 237 participants found that users do not understand that skills are often operated by third parties, that they often confuse third-party skills with native Alexa functions, and that they are unaware of the functions that the native Alexa system supports. Surprisingly, users who interact with Alexa more frequently are more likely to conclude that a third-party skill is a native Alexa function. The potential for misunderstanding creates new security and privacy risks: attackers can develop third-party skills that operate without users’ knowledge or masquerade as native Alexa functions. To mitigate this threat, we make design recommendations to help users better distinguish native functionality and third-party skills, including audio and visual indicators of native and third-party contexts, as well as a consistent design standard to help users learn what functions are and are not possible on Alexa.

References

  1. Sarah Perez. 2019. Over a quarter of US adults now own a smart speaker, typically an Amazon Echo. Tech Crunch. Retrieved February 2, 2020 from https://techcrunch.com/2019/03/08/over-a-quarter-of-u-s-adults-now-own- a-smart-speaker-typically-an-amazon-echo/.Google ScholarGoogle Scholar
  2. Amazon. n.d. Number of English Skills on Amazon Alexa (Internet Archive). Retrieved July 29, 2021 from https://bit.ly/366Z70G.Google ScholarGoogle Scholar
  3. Amazon. 2019. Alexa Skills Store. Retrieved September 11, 2019 from https://www.amazon.com/alexa-skills/b?ie=UTF8&node=13727921011.Google ScholarGoogle Scholar
  4. Nathaniel Fruchter and Ilaria Liccardi. 2018. Consumer attitudes towards privacy and security in home assistants. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. 1–6. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Josephine Lau, Benjamin Zimmerman, and Florian Schaub. 2018. Alexa, are you listening? Privacy perceptions, concerns and privacy-seeking behaviors with smart speakers. Proceedings of the ACM on Human-Computer Interaction 2, CSCW (Nov. 2018), Article 102, 31 pages. DOI:http://dx.doi.org/10.1145/3274371 Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Nathan Malkin, Joe Deatrick, Allen Tong, Primal Wijesekera, Serge Egelman, and David Wagner. 2019. Privacy attitudes of smart speaker users. Proceedings on Privacy Enhancing Technologies 2019, 4 (2019), 250–271.Google ScholarGoogle Scholar
  7. Noura Abdi, Kopo M. Ramokapane, and Jose M. Such. 2019. More than smart speakers: security and privacy perceptions of smart home personal assistants. In Proceedings of the 15th Symposium on Usable Privacy and Security. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Y. Gao, Z. Pan, H. Wang, and G. Chen. 2018. Alexa, my love: Analyzing reviews of Amazon Echo. In Proceedings of the 2018 IEEE SmartWorld, Ubiquitous Intelligence Computing, Advanced Trusted Computing, Scalable Computing Communications, Cloud Big Data Computing, Internet of People, and Smart City Innovation (SmartWorld/ SCALCOM/UIC/ATC/CBDCom/IOP/SCI’18). 372–380. DOI:http://dx.doi.org/10.1109/SmartWorld.2018.00094Google ScholarGoogle Scholar
  9. Irene Lopatovska and Harriet Williams. 2018. Personification of the Amazon Alexa: BFF or a mindless companion. In Proceedings of the 2018 Conference on Human Information Interaction and Retrieval (CHIIR’18). ACM, New York, NY, 265–268. DOI:http://dx.doi.org/10.1145/3176349.3176868 Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Amanda Purington, Jessie G. Taft, Shruti Sannon, Natalya N. Bazarova, and Samuel Hardman Taylor. 2017. “Alexa Is My New BFF”: Social roles, user satisfaction, and personification of the Amazon Echo. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA’17). ACM, New York, NY, 2853–2859. DOI:http://dx.doi.org/10.1145/3027063.3053246 Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Martin Porcheron, Joel E. Fischer, Stuart Reeves, and Sarah Sharples. 2018. Voice interfaces in everyday life. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI’18). ACM, New York, NY, Article 640, 12 pages. DOI:http://dx.doi.org/10.1145/3173574.3174214 Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Alex Sciuto, Arnita Saini, Jodi Forlizzi, and Jason I. Hong. 2018. “Hey Alexa, What’s Up?”: A mixed-methods studies of in-home conversational agent usage. In Proceedings of the 2018 Designing Interactive Systems Conference (DIS’18). ACM, New York, NY, 857–868. DOI:http://dx.doi.org/10.1145/3196709.3196772 Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Aarthi Easwara Moorthy and Kim-Phuong L. Vu. 2015. Privacy concerns for use of voice activated personal assistant in the public space. International Journal of Human–Computer Interaction 31, 4 (2015), 307–335. DOI:http://dx.doi.org/10.1080/10447318.2014.986642Google ScholarGoogle Scholar
  14. Deepak Kumar, Riccardo Paccagnella, Paul Murley, Eric Hennenfent, Joshua Mason, Adam Bates, and Michael Bailey. 2018. Skill squatting attacks on Amazon Alexa. In Proceedings of the 27th USENIX Conference on Security Symposium (SEC’18). 33–47. http://dl.acm.org/citation.cfm?id=3277203.3277207. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Nan Zhang, Xianghang Mi, Xuan Feng, XiaoFeng Wang, Yuan Tian, and Feng Qian. 2018. Understanding and mitigating the security risks of voice-controlled third-party skills on Amazon Alexa and Google Home. arxiv:1805.01525Google ScholarGoogle Scholar
  16. Don Norman. 2013. The psychology of everyday actions. In The Design of Everyday Things (revised, expanded ed.). Basic Books, 37–122. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. XXX. 2018. Amazon Echo Has 23% Share of Smart Speakers in Use: Strategy Analytics. Retrieved May 3, 2019 from https://news.strategyanalytics.com/press-release/intelligent-home/amazo n-echo-has-23-share-smart-speakers-use-strategy-analytics.Google ScholarGoogle Scholar
  18. Alexa. 2019. Alexa Voice Service. Retrieved May 3, 2019 from https://developer.amazon.com/alexa-voice-service.Google ScholarGoogle Scholar
  19. Alexa. 2019. Host a Custom Skill as a Web Service. Retrieved May 3, 2019 from https://developer.amazon.com/docs/custom-skills/host-a-custom-skill-as- a-web-service.html.Google ScholarGoogle Scholar
  20. Alexa. 2019. Understanding How Users Invoke Custom Skills. Retrieved May 6, 2019 from https://developer.amazon.com/docs/custom-skills/understanding-how-users -invoke-custom-skills.html.Google ScholarGoogle Scholar
  21. Amazon.com help: What do the lights on your echo device mean? [Online]. Retrieved from https://www.amazon.com/gp/help/customer/display.html?nodeId=GKLDRFT7FP4FZE56.Google ScholarGoogle Scholar
  22. Choose the invocation name for a custom skill | alexa skills kit. [Online]. Retrieved from https://developer.amazon.com/en-US/docs/alexa/customskills/choose-the-invocation-name-for-a-custom-skill.html.Google ScholarGoogle Scholar
  23. Amazon.com: Home wifi: Alexa skills. [Online]. Retrieved from https://voiceapp.store/listing/home-wifi/.Google ScholarGoogle Scholar
  24. Guoming Zhang, Chen Yan, Xiaoyu Ji, Tianchen Zhang, Taimin Zhang, and Wenyuan Xu. 2017. DolphinAttack: Inaudible voice commands. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. ACM, New York, NY, 103–117. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Noah Apthorpe, Danny Yuxing Huang, Dillon Reisman, Arvind Narayanan, and Nick Feamster. 2019. Keeping the smart home private with smart(er) IoT traffic shaping. Proceedings on Privacy Enhancing Technologies 2019, 3 (2019), 128–148.Google ScholarGoogle ScholarCross RefCross Ref
  26. Robert Nyman. 2012. Using the Fullscreen API in web browsers. Mozilla Hacks. Retrieved July 29, 2021 from https://hacks.mozilla.org/2012/01/using-the-fullscreen-api-in-web-browser s.Google ScholarGoogle Scholar
  27. Apple Insider Staff. 2017. Proof of concept phishing attack mimics iOS popups to steal user passwords. AI. Retrieved July 29, 2021 from https://appleinsider.com/articles/17/10/10/proof-of-concept-phishing-atta ck-mimics-ios-popups-to-steal-user-passwords.Google ScholarGoogle Scholar
  28. Joseph Weizenbaum. 1966. ELIZA—A computer program for the study of natural language communication between man and machine. Communications of the ACM 9, 1 (Jan. 1966), 36–45. DOI:http://dx.doi.org/10.1145/365153.365168 Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Brenda Laurel and S. Joy Mountford (Eds.). 1990. The Art of Human-Computer Interface Design. Addison-Wesley-Longman, Boston, MA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Cathy Pearl. 2016. Designing Voice User Interfaces. O’Reilly Media. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Gustavo López, Luis Quesada, and Luis A. Guerrero. 2018. Alexa vs. Siri vs. Cortana vs. Google Assistant: A comparison of speech-based natural user interfaces. In Advances in Human Factors and Systems Interaction, Isabel L. Nunes (Ed.). Springer International Publishing, Cham, Switzerland, 241–250. Google ScholarGoogle Scholar
  32. H. Chung, M. Iorga, J. Voas, and S. Lee. 2017. “Alexa, Can I Trust You?”Computer 50, 9 (2017), 100–104. DOI:http://dx.doi.org/10.1109/MC.2017.3571053Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Deepak Kumar, Riccardo Paccagnella, Paul Murley, Eric Hennenfent, Joshua Mason, Adam Bates, and Michael Bailey. 2018. Skill squatting attacks on Amazon Alexa. In Proceedings of the 27th USENIX Security Symposium (USENIX Security’18). 33–47. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Nan Zhang, Xianghang Mi, Xuan Feng, XiaoFeng Wang, Yuan Tian, and Feng Qian. 2019. Dangerous skills: Understanding and mitigating security risks of voice-controlled third-party functions on virtual personal assistant systems. In Proceedings of the 2019 IEEE Symposium on Security and Privacy (SP’19). IEEE, Los Alamitos, CA.Google ScholarGoogle ScholarCross RefCross Ref
  35. Madiha Tabassum, Tomasz Kosiński, Alisa Frik, Nathan Malkin, Primal Wijesekera, Serge Egelman, and Heather Richter Lipford. 2019. Investigating users’ preferences and expectations for always-listening voice assistants. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 4 (2019), 1–23. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Jide S. Edu, Jose M. Such, and Guillermo Suarez-Tangil. 2020. Smart home personal assistants: A security and privacy review. ACM Computing Surveys 53, 6 (2020), 116. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Elissa M. Redmiles, Sean Kross, and Michelle L. Mazurek. 2019. How well do my results generalize? Comparing security and privacy survey results from MTurk, web, and telephone samples. In Proceedings of the 2019 IEEE Symposium on Security and Privacy (SP’19). IEEE, Los Alamitos, CA, 227–244.Google ScholarGoogle Scholar
  38. Johnny Saldañna. 2013. The Coding Manual for Qualitative Researchers (2nd ed.). SAGE, Los Angeles, CA. Google ScholarGoogle Scholar
  39. Amazon Alexa. 2019. Save Data Between Sessions. Retrieved July 29, 2021 from https://developer.amazon.com/docs/custom-skills/manage-skill-session-and- session-attributes.html#save-data-between-sessions.Google ScholarGoogle Scholar
  40. Alexa. 2019. Alexa Conversations: Creating Natural Voice Experiences Faster. Retrieved September 14, 2019 from https://developer.amazon.com/en-US/alexa/alexa-skills-kit/alexa-convers ations.Google ScholarGoogle Scholar
  41. Taylor Martin. 2019. The Complete List of Alexa Commands So Far. Retrieved September 14, 2019 from https://www.cnet.com/how-to/amazon-echo-the-complete-list-of-alexa-comm ands/.Google ScholarGoogle Scholar
  42. Zhixiu Guo, Zijin Lin, Pan Li, and Kai Chen. 2020. SkillExplorer: Understanding the behavior of skills in large scale. In Proceedings of the 29th USENIX Security Symposium (USENIX Security’20). 2649–2666.Google ScholarGoogle Scholar
  43. Security Research Labs. n.d. Smart Spies: Alexa and Google Home Expose Users to Vishing and Eavesdropping. Retrieved July 29, 2021 from https://srlabs.de/bites/smart-spies/.Google ScholarGoogle Scholar

Index Terms

  1. Alexa, Who Am I Speaking To?: Understanding Users’ Ability to Identify Third-Party Apps on Amazon Alexa

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM Transactions on Internet Technology
        ACM Transactions on Internet Technology  Volume 22, Issue 1
        February 2022
        717 pages
        ISSN:1533-5399
        EISSN:1557-6051
        DOI:10.1145/3483347
        • Editor:
        • Ling Liu
        Issue’s Table of Contents

        Copyright © 2021 Association for Computing Machinery.

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 14 September 2021
        • Accepted: 1 December 2020
        • Revised: 1 November 2020
        • Received: 1 June 2020
        Published in toit Volume 22, Issue 1

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
        • Refereed

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format .

      View HTML Format
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!