skip to main content
research-article
Open Access

A Study on the Use of Checksums for Integrity Verification of Web Downloads

Published:28 September 2020Publication History
Skip Abstract Section

Abstract

App stores provide access to millions of different programs that users can download on their computers. Developers can also make their programs available for download on their websites and host the program files either directly on their website or on third-party platforms, such as mirrors. In the latter case, as users download the software without any vetting from the developers, they should take the necessary precautions to ensure that it is authentic. One way to accomplish this is to check that the published file’s integrity verification code—the checksum—matches that (if provided) of the downloaded file. To date, however, there is little evidence to suggest that such a process is effective. Even worse, very few usability studies about it exist.

In this article, we provide the first comprehensive study that assesses the usability and effectiveness of the manual checksum verification process. First, by means of an in-situ experiment with 40 participants and eye-tracking technology, we show that the process is cumbersome and error-prone. Second, after a 4-month-long in-the-wild experiment with 134 participants, we demonstrate how our proposed solution—a Chrome extension that verifies checksums automatically—significantly reduces human errors, improves coverage, and has only limited impact on usability. It also confirms that, sadly, only a tiny minority of websites that link to executable files in our sample provide checksums (0.01%), which is a strong call to action for web standards bodies, service providers, and content creators to increase the use of file integrity verification on their properties.

References

  1. Mauro Cherubini, Alexandre Meylan, Bertil Chapuis, Mathias Humbert, Igor Bilogrevic, and Kévin Huguenin. 2018. Towards usable checksums: Automating the integrity verification of web downloads for the masses. In Proceedings of the Conference on Computer and Communications Security (CCS). ACM, 1256--1271. DOI:http://dx.doi.org/10.1145/3243734.3243746Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Karen Turner. 2016-07-15. Developers consider Apple’s app store restrictive and anticompetitive, report shows. Washington Post (2016-07-15).Google ScholarGoogle Scholar
  3. Swati Khandelwal. 2018. Flaw in Popular Transmission BitTorrent Client Lets Hackers Control Your PC Remotely. https://thehackernews.com/2018/01/bittorent-transmission-hacking.html. (2018).Google ScholarGoogle Scholar
  4. Linux Mint Website Hacked; ISO Downloads Replaced with a Backdoor. Security News - Trend Micro USA. https://www.trendmicro.com/vinfo/us/security/news/cybercrime-and-digital-threats/linux-mint-website-hacked-iso-downloads-replaced-with-a-backdoor. ([n.d.]).Google ScholarGoogle Scholar
  5. Hsu-Chun Hsiao, Yue-Hsun Lin, Ahren Studer, Cassandra Studer, King-Hang Wang, Hiroaki Kikuchi, Adrian Perrig, Hung-Min Sun, and Bo-Yin Yang. 2009. A study of user-friendly hash comparison schemes. In Proceedings of the Computer Security Applications Conference (ACSAC). IEEE, 105--114. DOI:http://dx.doi.org/10.1109/ACSAC.2009.20Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Sergej Dechand, Dominik Schürmann, Karoline Busse, Yasemin Acar, Sascha Fahl, and Matthew Smith. 2016. An empirical study of textual key-fingerprint representations. In Proceedings of the USENIX Security Symposium (USENIX Security). USENIX.Google ScholarGoogle Scholar
  7. W3C. 2016. Subresource Integrity. https://www.w3.org/TR/SRI/. (2016).Google ScholarGoogle Scholar
  8. S. M. Furnell, P. Bryant, and A. D. Phippen. 2007. Assessing the security perceptions of personal Internet users. Computers 8 Security 26, 5 (Aug. 2007), 410--417. DOI:http://dx.doi.org/10.1016/j.cose.2007.03.001Google ScholarGoogle Scholar
  9. Catherine L. Anderson and Ritu Agarwal. 2010. Practicing safe computing: A multimedia empirical examination of home computer user security behavioral intentions. MIS Q. 34, 3 (Sept. 2010), 613--643.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Vaidya Rishi. 2018. Cyber Security Breaches Survey 2018. Survey. United Kingdom.Google ScholarGoogle Scholar
  11. Elissa M. Redmiles, Sean Kross, and Michelle L. Mazurek. 2017. Where is the digital divide?: A survey of security, privacy, and socioeconomics. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI). ACM, 931--936. DOI:http://dx.doi.org/10.1145/3025453.3025673Google ScholarGoogle Scholar
  12. Elissa M. Redmiles, Sean Kross, and Michelle L. Mazurek. 2016. How I learned to be secure: A census-representative survey of security advice sources and behavior. In Proceedings of the ACM Conference on Computer and Communications Security (CCS). ACM, 666--677. DOI:http://dx.doi.org/10.1145/2976749.2978307Google ScholarGoogle Scholar
  13. Elisa M. Redmiles, Amelia R. Malone, and Michelle L. Mazurek. 2016. I think they’re trying to tell me something: Advice sources and selection for digital security. In Proceedings of the IEEE Symposium on Security and Privacy (S8P). 272--288. DOI:http://dx.doi.org/10.1109/SP.2016.24Google ScholarGoogle Scholar
  14. Serge Egelman, Lorrie Faith Cranor, and Jason Hong. 2008. You’ve been warned: An empirical study of the effectiveness of web browser phishing warnings. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI). ACM, 1065--1074. DOI:http://dx.doi.org/10.1145/1357054.1357219Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Joshua Sunshine, Serge Egelman, Hazim Almuhimedi, Neha Atri, and Lorrie Faith Cranor. 2009. Crying wolf: An empirical study of SSL warning effectiveness. In Proceedings of the USENIX Security Symposium (USENIX Security). USENIX, 399--416.Google ScholarGoogle Scholar
  16. Devdatta Akhawe and Adrienne Porter Felt. 2013. Alice in warningland: A large-scale field study of browser security warning effectiveness. In Proceedings of the USENIX Security Symposium (USENIX Security). USENIX.Google ScholarGoogle Scholar
  17. Serge Egelman and Stuart Schechter. 2013. The importance of being earnest [in security warnings]. In Proceedings of the International Conference on Financial Cryptography and Data Security (FC). Springer, 52--59. DOI:http://dx.doi.org/10.1007/978-3-642-39884-1_5Google ScholarGoogle ScholarCross RefCross Ref
  18. David Modic and Ross Anderson. 2014. Reading this may harm your computer: The psychology of malware warnings. Computers in Human Behavior 41 (2014), 71--79.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Antonio Bianchi, Jacopo Corbetta, Luca Invernizzi, Yanick Fratantonio, Christopher Kruegel, and Giovanni Vigna. 2015. What the app is that? Deception and countermeasures in the Android user interface. In Proceedings of the IEEE Symposium on Security and Privacy (S8P). IEEE, 931--948. DOI:http://dx.doi.org/10.1109/SP.2015.62Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Jeffrey L. Jenkins, Bonnie Brinton Anderson, Anthony Vance, C. Brock Kirwan, and David Eargle. 2016. More harm than good? How messages that interrupt can make us vulnerable. Information Systems Research 27, 4 (2016), 880--896.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Mario Silic and Andrea Back. 2017. Deterrent effects of warnings on user’s behavior in preventing malicious software use. In Proceedings of the Hawaii International Conference on System Sciences (HICSS).Google ScholarGoogle ScholarCross RefCross Ref
  22. Robert W. Reeder, Adrienne Porter Felt, Sunny Consolvo, Nathan Malkin, Christopher Thompson, and Serge Egelman. 2018. An experience sampling study of user reactions to browser warnings in the field. In Proceedings of the Conference on Human Factors in Computing Systems (CHI). ACM, 512:1–512:13. DOI:http://dx.doi.org/10.1145/3173574.3174086Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Cristian Bravo-Lillo, Saranga Komanduri, Lorrie Faith Cranor, Robert W. Reeder, Manya Sleeper, Julie Downs, and Stuart Schechter. 2013. Your attention please: Designing security-decision UIs to make genuine risks harder to ignore. In Proceedings of the Symposium on Usable Privacy and Security (SOUPS). ACM, 6:1–6:12. DOI:http://dx.doi.org/10.1145/2501604.2501610Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Catherine S. Weir, Gary Douglas, Martin Carruthers, and Mervyn Jack. 2009. User perceptions of security, convenience and usability for ebanking authentication tokens. Computers 8 Security 28, 1-2 (2009), 47--62.Google ScholarGoogle Scholar
  25. Leona Tam, Myron Glassman, and Mark Vandenwauver. 2010. The psychology of password management: A tradeoff between security and convenience. Behaviour 8 Information Technology 29, 3 (2010), 233--244.Google ScholarGoogle Scholar
  26. Michael Fagan and Mohammad Maifi Hasan Khan. 2016. Why do they do what they do?: A study of what motivates users to (not) follow computer security advice. In Proceedings of the Symposium on Usable Privacy and Security (SOUPS). ACM, 59--75.Google ScholarGoogle Scholar
  27. K. Krombholz, K. Busse, K. Pfeffer, M. Smith, and E. von Zezschwitz. 2019. “If HTTPS were secure, I wouldn’t need 2FA” - End user and administrator mental models of HTTPS. In Proceedings of the IEEE Symposium on Security and Privacy (S8P). IEEE, 1138--1155. DOI:http://dx.doi.org/10.1109/SP.2019.00060Google ScholarGoogle ScholarCross RefCross Ref
  28. Joshua Tan, Lujo Bauer, Joseph Bonneau, Lorrie Faith Cranor, Jeremy Thomas, and Blase Ur. 2017. Can unicorns help users compare crypto key fingerprints? In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI). ACM, 3787--3798. DOI:http://dx.doi.org/10.1145/3025453.3025733Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. N. Unger, S. Dechand, J. Bonneau, S. Fahl, H. Perl, I. Goldberg, and M. Smith. 2015. SoK: Secure messaging. In Proceedings of the IEEE Symposium on Security and Privacy (S8P). IEEE, 232--249. DOI:http://dx.doi.org/10.1109/SP.2015.22Google ScholarGoogle Scholar
  30. Ruba Abu-Salma, M. Angela Sasse, Joseph Bonneau, Anastasia Danilova, Alena Naiakshina, and Matthew Smith. 2017. Obstacles to the adoption of secure communication tools. In Proceedings of the IEEE Symposium on Security and Privacy (S8P). IEEE, 137--153. DOI:http://dx.doi.org/10.1109/SP.2017.65Google ScholarGoogle ScholarCross RefCross Ref
  31. Elham Vaziripour, Justin Wu, Mark O’Neill, Ray Clinton, Jordan Whitehead, Scott Heidbrink, Kent Seamons, and Daniel Zappala. 2017. Is that you, Alice? A usability study of the authentication ceremony of secure messaging applications. In Proceedings of the Symposium on Usable Privacy and Security (SOUPS). ACM.Google ScholarGoogle Scholar
  32. Checksum On the Go. Chrome Webstore. https://chrome.google.com/webstore/detail/checksum-on-the-go/fholnooplijidhdagedffljaphholpea. ([n.d.]).Google ScholarGoogle Scholar
  33. Files MD5 SHA1 Calculate 8 Compare. Add-Ons for Firefox. https://addons.mozilla.org/en-US/firefox/addon/calculate-md5-sha1-hash-che-1/?src=search. ([n.d.]).Google ScholarGoogle Scholar
  34. Certificates and Digitally Signed Applications: A Double Edged Sword. https://eventtracker.com/tech-articles/certificates-and-digitally-signed-applications-a-double-edged-sword/. (Feb. 2016).Google ScholarGoogle Scholar
  35. Nevena Vratonjic, Julien Freudiger, Vincent Bindschaedler, and Jean-Pierre Hubaux. 2013. The inconvenient truth about web certificates. In Proceedings of the Workshop on Economics of Information Security and Privacy (WEIS). Springer, 79--117. DOI:http://dx.doi.org/10.1007/978-1-4614-1981-5_5Google ScholarGoogle ScholarCross RefCross Ref
  36. Justin Cappos, Justin Samuel, Scott Baker, and John H. Hartman. 2008. A look in the mirror: Attacks on package managers. In Proceedings of the ACM Conference on Computer and Communications Security (CCS). ACM, 565--574. DOI:http://dx.doi.org/10.1145/1455770.1455841Google ScholarGoogle Scholar
  37. Bart Preneel. 1994. Cryptographic hash functions. Transactions on Emerging Telecommunications Technologies 5, 4 (1994), 431--448.Google ScholarGoogle ScholarCross RefCross Ref
  38. Computer Security Division, Information Technology Laboratory. NIST Policy on Hash Functions - Hash Functions | CSRC. https://csrc.nist.gov/projects/hash-functions/nist-policy-on-hash-functions. ([n.d.]).Google ScholarGoogle Scholar
  39. Bertil Chapuis, Olamide Omolola, Mauro Cherubini, Mathias Humbert, and Kévin Huguenin. 2020. An empirical study of the use of integrity verification mechanisms for web subresources. In Proceedings of the Web Conference. ACM, 34--45. DOI:http://dx.doi.org/10.1145/3366423.3380092Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Bonnie Brinton Anderson, C. Brock Kirwan, Jeffrey L. Jenkins, David Eargle, Seth Howard, and Anthony Vance. 2015. How polymorphic warnings reduce habituation in the brain: Insights from an fMRI study. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI). ACM, 2883--2892. DOI:http://dx.doi.org/10.1145/2702123.2702322Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Adrienne Porter Felt, Alex Ainslie, Robert W. Reeder, Sunny Consolvo, Somas Thyagaraja, Alan Bettes, Helen Harris, and Jeff Grimes. 2015. Improving SSL warnings: Comprehension and adherence. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI). ACM, 2893--2902. DOI:http://dx.doi.org/10.1145/2702123.2702442Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Mario Silic, Jordan Barlow, and Dustin Ormond. 2015. Warning! A comprehensive model of the effects of digital information security warning messages. In Proceedings of the IFIP Workshop on Information Systems Security Research. IFIP.Google ScholarGoogle Scholar
  43. Alex Poole and Linden J. Ball. 2006. Eye tracking in human-computer interaction and usability research: Current status and future prospects. In Encyclopedia of Human Computer Interaction. 13.Google ScholarGoogle Scholar
  44. Joseph H. Goldberg, Mark J. Stimson, Marion Lewenstein, Neil Scott, and Anna M. Wichansky. 2002. Eye tracking in web search tasks: Design implications. In Proceedings of the Symposium on Eye Tracking Research 8 Applications (ETRA). ACM, 51. DOI:http://dx.doi.org/10.1145/507072.507082Google ScholarGoogle Scholar
  45. Marcel Adam Just and Patricia A. Carpenter. 1976. Eye fixations and cognitive processes. Cognitive Psychology 8, 4 (Oct. 1976), 441--480. DOI:http://dx.doi.org/10.1016/0010-0285(76)90015-3Google ScholarGoogle Scholar
  46. Mauro Cherubini and Nuria Oliver. 2009. A refined experience sampling method to capture mobile user experience. arXiv:0906.4125 [cs] (June 2009). arxiv:cs/0906.4125Google ScholarGoogle Scholar
  47. S. Consolvo and M. Walker. 2003. Using the experience sampling method to evaluate ubicomp applications. IEEE Pervasive Computing 2, 2 (Jun 2003), 24--31. DOI:http://dx.doi.org/10.1109/MPRV.2003.1203750Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Giovanni Iachello, Khai N. Truong, Gregory D. Abowd, Gillian R. Hayes, and Molly Stevens. 2006. Prototyping and sampling experience to evaluate ubiquitous computing privacy in the real world. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI). ACM, 1009--1018. DOI:http://dx.doi.org/10.1145/1124772.1124923Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Clara Mancini, Keerthi Thomas, Yvonne Rogers, Blaine A. Price, Lukazs Jedrzejczyk, Arosha K. Bandara, Adam N. Joinson, and Bashar Nuseibeh. 2009. From spaces to places: Emerging contexts in mobile privacy. In Proceedings of the International Conference on Ubiquitous Computing (UbiComp). ACM, 1--10. DOI:http://dx.doi.org/10.1145/1620545.1620547Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Stephen S. Intille, John Rondoni, Charles Kukla, Isabel Ancona, and Ling Bao. 2003. A context-aware experience sampling tool. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI) - Extended Abstracts. ACM, 972--973. DOI:http://dx.doi.org/10.1145/765891.766101Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Florian Schmitt, Jan Gassen, and Elmar Gerhards-Padilla. 2012. PDF scrutinizer: Detecting JavaScript-based attacks in PDF documents. In Proceedings of the International Conference on Privacy, Security and Trust (PST). IEEE, 104--111. DOI:http://dx.doi.org/10.1109/PST.2012.6297926Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Michael Crabb and Vicki L. Hanson. 2014. Age, technology usage, and cognitive characteristics in relation to perceived disorientation and reported website ease of use. In Proceedings of the International ACM SIGACCESS Conference on Computers 8 Accessibility (ASSETS). ACM, 193--200. DOI:http://dx.doi.org/10.1145/2661334.2661356Google ScholarGoogle Scholar

Index Terms

  1. A Study on the Use of Checksums for Integrity Verification of Web Downloads

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM Transactions on Privacy and Security
          ACM Transactions on Privacy and Security  Volume 24, Issue 1
          February 2021
          191 pages
          ISSN:2471-2566
          EISSN:2471-2574
          DOI:10.1145/3426975
          Issue’s Table of Contents

          Copyright © 2020 Owner/Author

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 28 September 2020
          • Revised: 1 July 2020
          • Accepted: 1 July 2020
          • Received: 1 December 2019
          Published in tops Volume 24, Issue 1

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article
          • Research
          • Refereed

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        HTML Format

        View this article in HTML Format .

        View HTML Format
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!