Abstract
Phishing attacks, in which criminals lure Internet users to Web sites that spoof legitimate Web sites, are occurring with increasing frequency and are causing considerable harm to victims. While a great deal of effort has been devoted to solving the phishing problem by prevention and detection of phishing emails and phishing Web sites, little research has been done in the area of training users to recognize those attacks. Our research focuses on educating users about phishing and helping them make better trust decisions. We identified a number of challenges for end-user security education in general and anti-phishing education in particular: users are not motivated to learn about security; for most users, security is a secondary task; it is difficult to teach people to identify security threats without also increasing their tendency to misjudge nonthreats as threats. Keeping these challenges in mind, we developed an email-based anti-phishing education system called “PhishGuru” and an online game called “Anti-Phishing Phil” that teaches users how to use cues in URLs to avoid falling for phishing attacks. We applied learning science instructional principles in the design of PhishGuru and Anti-Phishing Phil. In this article we present the results of PhishGuru and Anti-Phishing Phil user studies that demonstrate the effectiveness of these tools. Our results suggest that, while automated detection systems should be used as the first line of defense against phishing attacks, user education offers a complementary approach to help people better recognize fraudulent emails and websites.
- Abu-Nimeh, S., Nappa, D., Wang, X., and Nair, S. 2007. A comparison of machine learning techniques for phishing detection. e-Crime Researchers Summit, Anti-Phishing Working Group. Google Scholar
Digital Library
- Account Guard. 2006. Account Guard. http://pages.ebay.com/ebay_toolbar/.Google Scholar
- Adams, A. and Sasse, M. A. 1999. Users are not the enemy. Comm. ACM 42, 12, 40--46. DOI=http://doi.acm.org/10.1145/322796.322806. Google Scholar
Digital Library
- Aleven, V. and Koedinger, K. R. 2002. An effective metacognitive strategy: learning by doing and explaining with a computer-based cognitive tutor. Cogn. Sci. 26, 2, 147--179.Google Scholar
Cross Ref
- Anandpara, V., Dingman, A., Jakobsson, M., Liu, D., and Roinestad, H. 2007. Phishing IQ tests measure fear, not ability. Usable Security Workshop (USEC'07). http://usablesecurity.org/papers/anandpara.pdf. Google Scholar
Digital Library
- Anderson, J. R. 1993. Rules of the Mind. Lawrence Erlbaum Associates, Inc.Google Scholar
- Anderson, J. R., Corbett, A. T., Koedinger, K. R., and Pelletier, R. 1995. Cognitive tutors: Lessons learned. J. Learn. Sci. 4, 2, 167--207.Google Scholar
Cross Ref
- Anderson, J. R. and Simon, H. A. 1996. Situated learning and education. Educ. Resear. 25, 5--11.Google Scholar
Cross Ref
- Anti-Phishing Working Group. 2007. Anti-Phishing Working Group. http://www.antiphishing.org/.Google Scholar
- Bahrick, H. P. 1979. Maintenance of knowledge: Questions about memory we forgot to ask. J. Exper. Psych. 108, 3, 296--308.Google Scholar
Cross Ref
- Baker, R., Habgood, J., and Ainsworth, S. E. 2007. Modeling the acquistion of fluent skill in educational action games. Proceedings of the Conference on User Modeling, 17--26. Google Scholar
Digital Library
- Barnett, S. M. and Ceci, S. J. 2002. When and where do we apply what we learn? a taxonomy for far transfer. Psych. Bull. 128, 612--637.Google Scholar
Cross Ref
- Bransford, J. D. and Schwartz, D. L. 2001. Rethinking transfer: A simple proposal with multiple implications. In Review of Research in Education, A. Iran-Nejad and P. D. Pearson., Eds. Vol. 24, American Educational Research Association (AERA), Washington, DC, 61--100.Google Scholar
- Burmester, G. M., Stottler, D., and Hart, J. L. 2005. Embedded training intelligent tutoring systems (ITS) for the future combat systems (FCS) command and control (C2) vehicle. Tech. rep., Defense Technical Information Center. http://www.stottlerhenke.com/papers/IITSEC-02-ITSFCS.pdf.Google Scholar
- Chandrasekaran, M., Narayanan, K., and Upadhyaya, S. 2006. Phishing email detection based on structural properties. Proceedings of the NYS Cyber Security Conference.Google Scholar
- Clark, R. C. 1989. Developing Technical Training: A Structured Approach for the Development of Classroom and Computer-Based Instructional Materials. Addison Wesley Publishing Company.Google Scholar
- Clark, R. C. and Mayer, R. E. 2002. E-Learning and the Science of Instruction: Proven Guidelines for Consumers and Designers of Multimedia Learning. John Wiley & Sons, Inc. Google Scholar
Digital Library
- Committee on Developments in the Science of Learning and National Research Council. 2000. How People Learn: Bridging Research and Practice. National Academies Press.Google Scholar
- Corbett, A. T. and Anderson, J. R. 2001. Locus of feedback control in computer-based tutoring: impact on learning rate, achievement and attitudes. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'01). ACM Press, New York, NY, 245--252. Google Scholar
Digital Library
- Cordova, D. I. and Lepper, M. R. 1996. Intrinsic motivation and the process of learning: Beneficial effects of contextualization, personalization, and choice. J. Educ. Psych. 88, 4, 715--730.Google Scholar
Cross Ref
- Cranor, L. F. 2008. A framework for reasoning about the human in the loop. In Proceedings of the Conference on Usability, Psychology and Security. Google Scholar
Digital Library
- Cranor, L. F. and Garfinkel, S. Aug, 2005. Security and Usability: Designing Secure Systems that People Can Use. O'Reilly, Sebastopol, CA. Google Scholar
Digital Library
- Dhamija, R. and Tygar, J. 2005. The battle against phishing: Dynamic security skins. In Proceedings of the Symposium on Usable Privacy and Security (SOUPS'05). ACM Press, New York, NY, 77--88. Google Scholar
Digital Library
- eBay. 2006. Spoof email tutorial. http://pages.ebay.com/education/spooftutorial.Google Scholar
- Eberts, R. E. 1997. Handbook of Human-Computer Interaction. Elsevier Science, 825--847.Google Scholar
- Egelman, S., Cranor, L. F., and Hong, J. 2007. You've been warned: An empirical study of the effectiveness of Web browser phishing warnings. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'08). Google Scholar
Digital Library
- Emigh, A. 2005. Online identity theft: Phishing technology, chokepoints and countermeasures. Tech. rep., Radix Labs. October. http://www.antiphishing.org/Phishing-dhs-report.pdf.Google Scholar
- Evers, J. 2006. User education is pointless. http://news.com.com/2100-7350_3-6125213.html.Google Scholar
- Federal Trade Commission. 2006a. An e-card for you game. http://www.ftc.gov/bcp/conline/ecards/phishing/index.html.Google Scholar
- Federal Trade Commission. 2006b. How not to get hooked by a phishing scam. Consumer alert news. http://www.ftc.gov/bcp/edu/pubs/consumer/alerts/alt127.htm.Google Scholar
- Ferguson, A. J. 2005. Fostering e-mail security awareness: The west point carronade. EDUCASE Quart. 1. http://www.educause.edu/ir/library/pdf/eqm0517.pdf.Google Scholar
- Fette, I., Sadeh, N., and Tomasic, A. 2006. Learning to detect phishing emails. In Proceedings of the 16th International Conference on World Wide Web. Google Scholar
Digital Library
- Florencio, D. and Herley, C. 2005. Stopping a phishing attack, even when the victims ignore warnings. Tech. rep., Microsoft.Google Scholar
- Fong, G. T. and Nisbett, R. E. 1991. Immediate and delayed transfer of training effects in statistical reasoning. J. Exper. Psych. 120, 34--45.Google Scholar
Cross Ref
- Gagne, R. M., Foster, H., and Crowley, M. E. 1948. The measurement of transfer of training. Psych. Bull. 45, 2, 97--130.Google Scholar
Cross Ref
- Gordon, L. A., Loeb, M. P., Lucyshyn, W., and Richardson, R. 2006. CSI/FBI computer crime and security survey. Report, Computer Security Institute.Google Scholar
- Gorling, S. 2006. The myth of user education. In Proceedings of the 16th Virus Bulletin International Conference.Google Scholar
- Hight, S. D. 2005. The importance of a security, education, training and awareness program. http://www.infosecwriters.com/text_resources/pdf/SETA_SHight.pdf.Google Scholar
- Jackson, C., Simon, D., Tan, D., and Barth, A. 2007. An evaluation of extended validation and picture-in-picture phishing attacks. In Proceedings of the Usable Security Workshop (USEC'07). http://usablesecurity.org/papers/jackson.pdf. Google Scholar
Digital Library
- Jagatic, T., Johnson, N., Jakobsson, M., and Menczer, F. 2007. Social phishing. Comm. ACM 50, 10, 94--100. Google Scholar
Digital Library
- Jakobsson, M. 2007. The human factor in phishing. In Privacy & Security of Consumer Information. http://www.informatics.indiana.edu/markus/papers/aci.pdf.Google Scholar
- Jakobsson, M. and Myers, S., Eds. 2006. Phishing and Countermeasures: Understanding the Increasing Problem of Electronic Identity Theft. Wiley-Interscience. Google Scholar
Digital Library
- James, L. 2005. Phishing Exposed. Syngress Publishing, Canada. Google Scholar
Digital Library
- Johnson, B. R. and Koedinger, K. R. 2002. Comparing instructional strategies for integrating conceptual and procedural knowledge. In Proceedings of the Annual Meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education. Vol. 1--4. 969--978.Google Scholar
- Kirkley, J. R. and et al. 2003. Problem-based embedded training: An instructional methodology for embedded training using mixed and virtual reality technologies. In Proceedings of the Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC). http://www.iforces.org/downloads/problem-based.pdf.Google Scholar
- Koedinger, K. R. 2002. Toward evidence for instruction design principles: Examples from cognitive tutor math 6. Proocedings of the Annual Meeting of the Norh American Chapter of the International Group for the Psychology of Mathematics Education 1--4.Google Scholar
- Kumaraguru, P., Rhee, Y., Acquisti, A., Cranor, L. F., Hong, J., and Nunge, E. 2007a. Protecting people from phishing: the design and evaluation of an embedded training email system. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'07). ACM Press, New York, NY, 905--914. Google Scholar
Digital Library
- Kumaraguru, P., Rhee, Y., Sheng, S., Hasan, S., Acquisti, A., Cranor, L. F., and Hong, J. 2007b. Getting users to pay attention to anti-phishing education: Evaluation of retention and transfer. e-Crime Researchers Summit, Anti-Phishing Working Group. Google Scholar
Digital Library
- Lininger, R. and Vines, R. D. 2005. Phishing: Cutting the Identity Theft Line. IN. John Wiley and Sons. Google Scholar
Digital Library
- Macmillan, N. A. and Creelman, C. D. 2004. Detection Theory: A User's Guide. Lawrence Erlbaum.Google Scholar
Cross Ref
- Mail Frontier. 2006. Mailfrontier phishing IQ test. http://survey.mailfrontier.com/survey/quiztest.html.Google Scholar
- Mandl, H. and Levin, J. R. 1989. Knowledge Acquisition from Text and Pictures. North-Holland.Google Scholar
- Mathan, S. A. and Koedinger, K. R. 2003. Artificial Intelligence in Education: Shaping the Future of Learning Through Intelligent Technolgis. IOS Press, 13--20.Google Scholar
- Mathan, S. A. and Koedinger, K. R. 2005. Fostering the intelligent novice: Learning from errors with metacognitive tutoring. Educ. Psych. 40, 4, 257--265.Google Scholar
Cross Ref
- Mayer, R. E. 2001. Multimedia Learning. Cambridge University Press, Cambidge, UK. Google Scholar
Digital Library
- Mayer, R. E. and Anderson, R. B. 1992. The instructive animation: Helping students build connections between words and pictures in multimedia learning. J. Educ. Psych. 84, 4, 444--452.Google Scholar
Cross Ref
- McBride, C. M., Emmons, K. M., and Lipkus, I. M. 2003. Understanding the potential of teachable moments: the case of smoking cessation. Health Educ. Resear. 18, 2, 156--170.Google Scholar
Cross Ref
- Merrienboer, J. V., de Croock, M., and Jelsma, O. 1997. The transfer paradox: Effects of contextual interference on retention and transfer performance of a complex cognitive skill. Percept. Motor Skills 84, 784--786.Google Scholar
Cross Ref
- Microsoft Corporation. 2006. Consumer awareness page on phishing. http://www.microsoft.com/athome/security/email/phishing.mspx.Google Scholar
- Miller, R. C. and Wu, M. 2005. Fighting phishing at the user interface. In L. Cranor and S. Garfinkel Eds. Security and Usability: Designing Secure Systems that People Can Use. O'Reilly.Google Scholar
- Moreno, R. and Mayer, R. E. 1999. Cognitive principles of multimedia learning: The role of modality and contiguity. J. Educ. Psych. 91, 358--368.Google Scholar
Cross Ref
- MySecureCyberspace. 2007. Uniform resource locator (URL). http://www.mysecurecyberspace.com/encyclopedia/index/uniform-resource-locator-url-.html.Google Scholar
- Netcraft. 2006. Netcraf. http://toolbar.netcraft.com/.Google Scholar
- New York State Office of Cyber Security & Critical Infrastructure Coordination. 2005. Gone phishing&ldots;&ldots; a briefing on the anti-phishing exercise initiative for New York State government. Aggregate Exercise Results for public release.Google Scholar
- Nielsen, J. 2004. User education is not the answer to security problems. http://www.useit.com/alertbox/20041025.html.Google Scholar
- Robila, S. A. and Ragucci, J. W. 2006. Don't be a phish: steps in user education. In Proceedings of the 11th Annual SIGCSE Conference on Innovation and Technology in Computer Science Education (ITICSE'06). ACM Press, New York, NY, 237--241. DOI=http://doi.acm.org/10.1145/1140124.1140187. Google Scholar
Digital Library
- Rubin, D. C. and Wenzel, A. E. 1996. One hundred years of forgetting: A quantitative description of retention. Psych. Rev. 103, 4, 734--760.Google Scholar
Cross Ref
- Salkind, N. J. 2006. Encyclopedia of Measurement and Statistics. Sage Publications.Google Scholar
- Schmidt, R. A. and Bjork, R. A. 1992. New conceptualizations of practice: Common principles in three paradigms suggest new concepts for training. Psych. Sci. 3, 4, 207--217.Google Scholar
Cross Ref
- Schneier, B. 2000. Semantic attacks: The third wave of network attacks. Crypto-Gram Newsletter. http://www.schneier.com/crypto-gram-0010.html#1.Google Scholar
- Schwartz, D. L. and Bransford, J. D. 1998. A time for telling. Cogn. Instruc., 475--522.Google Scholar
- Sender Policy Framework. 2006. Sender Policy Framework. http://www.openspf.org/.Google Scholar
- Sheng, S., Magnien, B., Kumaraguru, P., Acquisti, A., Cranor, L. F., Hong, J., and Nunge, E. 2007. Anti-phishing phil: The design and evaluation of a game that teaches people not to fall for phish. Proceedings of the Symposium on Usable Privacy and Security. Google Scholar
Digital Library
- Sheng, S., Wardman, B., Warner, G., Cranor, L., Hong, J., and Zhang, C. 2009. An empirical analysis of phishing blacklists. Proceedings of the 6th Conference on Email and Anti-Spam.Google Scholar
- Singley, M. and Anderson, J. R. 1989. The Transfer of Cognitive Skill. Harvard University Press. Google Scholar
Digital Library
- SpoofGuard. 2006. Spoofguard. http://crypto.stanford.edu/SpoofGuard/.Google Scholar
- SpoofStick. 2006. Spoofstick. http://www.spoofstick.com/.Google Scholar
- Whitten, A. 2004. Making security usable. Ph.D. thesis, Carnegie Mellon University.Google Scholar
- Wu, M., Miller, R. C., and Garfinkel, S. L. 2006. Do security toolbars actually prevent phishing attacks? In Proceedings of the Conference on Human Factors in Computing Systems (CHI), 601--610. Google Scholar
Digital Library
- Yahoo. 2007. DomainKeys: Proving and Protecting Email Sender Identity. http://antispam.yahoo.com/domainkeys.Google Scholar
- Ye, Z. E. and Smith, S. 2002. Trusted paths for browsers. In Proceedings of the 11th USENIX Security Symposium. USENIX Association, Berkeley, CA, 263--279. Google Scholar
Digital Library
- Zhang, Y., Egelman, S., Cranor, L., and Hong, J. 2007. Phinding phish: Evaluating anti-phishing tools. In Proceedings of the 14th Annual Network and Distributed System Security Symposium. http://lorrie.cranor.org/pubs/ndss-phish-tools-final.pdf.Google Scholar
Index Terms
Teaching Johnny not to fall for phish
Recommendations
Protecting people from phishing: the design and evaluation of an embedded training email system
CHI '07: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsPhishing attacks, in which criminals lure Internet users to websites that impersonate legitimate sites, are occurring with increasing frequency and are causing considerable harm to victims. In this paper we describe the design and evaluation of an ...
Getting users to pay attention to anti-phishing education: evaluation of retention and transfer
eCrime '07: Proceedings of the anti-phishing working groups 2nd annual eCrime researchers summitEducational materials designed to teach users not to fall for phishing attacks are widely available but are often ignored by users. In this paper, we extend an embedded training methodology using learning science principles in which phishing education ...
School of phish: a real-world evaluation of anti-phishing training
SOUPS '09: Proceedings of the 5th Symposium on Usable Privacy and SecurityPhishGuru is an embedded training system that teaches users to avoid falling for phishing attacks by delivering a training message when the user clicks on the URL in a simulated phishing email. In previous lab and real-world experiments, we validated ...






Comments