skip to main content
research-article
Open Access

A General Framework for Adversarial Examples with Objectives

Published:10 June 2019Publication History
Skip Abstract Section

Abstract

Images perturbed subtly to be misclassified by neural networks, called adversarial examples, have emerged as a technically deep challenge and an important concern for several application domains. Most research on adversarial examples takes as its only constraint that the perturbed images are similar to the originals. However, real-world application of these ideas often requires the examples to satisfy additional objectives, which are typically enforced through custom modifications of the perturbation process. In this article, we propose adversarial generative nets (AGNs), a general methodology to train a generator neural network to emit adversarial examples satisfying desired objectives. We demonstrate the ability of AGNs to accommodate a wide range of objectives, including imprecise ones difficult to model, in two application domains. In particular, we demonstrate physical adversarial examples—eyeglass frames designed to fool face recognition—with better robustness, inconspicuousness, and scalability than previous approaches, as well as a new attack to fool a handwritten-digit classifier.

References

  1. Brandon Amos, Bartosz Ludwiczuk, and Mahadev Satyanarayanan. 2016. OpenFace: A General-purpose Face Recognition Library with Mobile Applications. Technical Report. CMU-CS-16-118, CMU School of Computer Science.Google ScholarGoogle Scholar
  2. Martin Arjovsky, Soumith Chintala, and Léon Bottou. 2017. Wasserstein GAN. In Proceedings of the International Conference on Machine Learning (ICML’17).Google ScholarGoogle Scholar
  3. David Arthur and Sergei Vassilvitskii. 2007. k-means++: The advantages of careful seeding. In Proceedings of the ACM-SIAM Symposium on Discrete Algorithms (SODA’07).Google ScholarGoogle Scholar
  4. Anish Athalye and Nicholas Carlini. 2018. On the robustness of the CVPR 2018 white-box adversarial example defenses. arXiv:1804.03286 (2018).Google ScholarGoogle Scholar
  5. Anish Athalye, Nicholas Carlini, and David Wagner. 2018. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In Proceedings of the International Conference on Machine Learning (ICML’18).Google ScholarGoogle Scholar
  6. Autodesk. {n.d.}. Measuring light levels. Retrieved from https://goo.gl/hkBWbZ.Google ScholarGoogle Scholar
  7. Tadas Baltrušaitis, Peter Robinson, and Louis-Philippe Morency. 2016. Openface: An open source facial behavior analysis toolkit. In Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV’16).Google ScholarGoogle ScholarCross RefCross Ref
  8. Shumeet Baluja and Ian Fischer. 2018. Learning to attack: Adversarial transformation networks. In Proceedings of the AAAI Conference on Artificial Intelligence.Google ScholarGoogle Scholar
  9. Battista Biggio and Fabio Roli. 2018. Wild patterns: Ten years after the rise of adversarial machine learning. Pattern Recogn. 84 (2018), 317--331.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Šrndić, Pavel Laskov, Giorgio Giacinto, and Fabio Roli. 2013. Evasion attacks against machine learning at test time. In Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases (ECML/PKDD’13).Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. CaretDashCaret. {n.d.}. 3D printable frames from eyeglasses SVGs. Retrieved from https://github.com/caretdashcaret/pince-nez.Google ScholarGoogle Scholar
  12. Nicholas Carlini and David Wagner. 2017. Adversarial examples are not easily detected: Bypassing ten detection methods. In Proceedings of the ACM Workshop on Artificial Intelligence and Security (AISec’17).Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Nicholas Carlini and David Wagner. 2017. Towards evaluating the robustness of neural networks. In Proceedings of the IEEE Symposium on Security and Privacy (S8P’17).Google ScholarGoogle ScholarCross RefCross Ref
  14. Nicholas Carlini, Pratyush Mishra, Tavish Vaidya, Yuankai Zhang, Micah Sherr, Clay Shields, David Wagner, and Wenchao Zhou. 2016. Hidden voice commands. In Proceedings of the USENIX Security Symposium. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Yizheng Chen, Yacin Nadji, Athanasios Kountouras, Fabian Monrose, Roberto Perdisci, Manos Antonakakis, and Nikolaos Vasiloglou. 2017. Practical attacks against graph-based clustering. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS’17). Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Moustapha Cisse, Yossi Adi, Natalia Neverova, and Joseph Keshet. 2017. Houdini: Fooling deep structured prediction models. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS’17).Google ScholarGoogle Scholar
  17. Simon Eberz, Nicola Paoletti, Marc Roeschlin, Marta Kwiatkowska, I. Martinovic, and A. Patané. 2017. Broken hearted: How to attack ECG biometrics. In Proceedings of the ISOC Annual Network and Distributed System Security Symposium (NDSS’17).Google ScholarGoogle Scholar
  18. Alexei A. Efros and Thomas K. Leung. 1999. Texture synthesis by non-parametric sampling. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’99). Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Logan Engstrom, Dimitris Tsipras, Ludwig Schmidt, and Aleksander Madry. 2017. A rotation and a translation suffice: Fooling CNNs with simple transformations. In Proceedings of the NeurIPS Machine Learning and Computer Security Workshop.Google ScholarGoogle Scholar
  20. Ivan Evtimov, Kevin Eykholt, Earlence Fernandes, Tadayoshi Kohno, Bo Li, Atul Prakash, Amir Rahmati, and Dawn Song. 2018. Robust physical-world attacks on machine learning models. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’18).Google ScholarGoogle Scholar
  21. Alhussein Fawzi, Seyed-Mohsen Moosavi-Dezfooli, and Pascal Frossard. 2016. Robustness of classifiers: From adversarial to random noise. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS’16).Google ScholarGoogle Scholar
  22. Reuben Feinman, Ryan R. Curtin, Saurabh Shintre, and Andrew B. Gardner. 2017. Detecting adversarial samples from artifacts. arXiv:1703.00410 (2017).Google ScholarGoogle Scholar
  23. Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. 2015. Model inversion attacks that exploit confidence information and basic countermeasures. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS’15).Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Justin Gilmer, Ryan P. Adams, Ian Goodfellow, David Andersen, and George E. Dahl. 2018. Motivating the rules of the game for adversarial example research. arXiv:1807.06732 (2018).Google ScholarGoogle Scholar
  25. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS’14).Google ScholarGoogle Scholar
  26. Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. MIT Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and harnessing adversarial examples. In Proceedings of the International Conference on Learning Representations (ICLR’15).Google ScholarGoogle Scholar
  28. Kathrin Grosse, Praveen Manoharan, Nicolas Papernot, Michael Backes, and Patrick McDaniel. 2017. On the (statistical) detection of adversarial examples. arXiv:1702.06280 (2017).Google ScholarGoogle Scholar
  29. Kathrin Grosse, Nicolas Papernot, Praveen Manoharan, Michael Backes, and Patrick McDaniel. 2017. Adversarial perturbations against deep neural networks for malware classification. In Proceedings of the European Symposium on Research in Computer Security (ESORICS’17).Google ScholarGoogle Scholar
  30. Chuan Guo, Mayank Rana, Moustapha Cisse, and Laurens van der Maaten. 2018. Countering adversarial images using input transformations. In Proceedings of the International Conference on Learning Representations (ICLR’18).Google ScholarGoogle Scholar
  31. Adam Harvey. 2010. CV Dazzle: Camouflage from face detection. Master’s thesis. New York University. Retreieved from http://cvdazzle.com.Google ScholarGoogle Scholar
  32. Gary B. Huang, Manu Ramesh, Tamara Berg, and Erik Learned-Miller. 2007. Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments. Technical Report 07-49. University of Massachusetts, Amherst.Google ScholarGoogle Scholar
  33. Ruitong Huang, Bing Xu, Dale Schuurmans, and Csaba Szepesvári. 2015. Learning with a strong adversary. arXiv:1511.03034 (2015).Google ScholarGoogle Scholar
  34. Lucas Introna and Helen Nissenbaum. 2009. Facial Recognition Technology: A Survey of Policy and Implementation Issues. Technical Report. Center for Catastrophe Preparedness and Response, New York University.Google ScholarGoogle Scholar
  35. Can Kanbak, Seyed-Mohsen Moosavi-Dezfooli, and Pascal Frossard. 2018. Geometric robustness of deep networks: Analysis and improvement. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’18).Google ScholarGoogle ScholarCross RefCross Ref
  36. Harini Kannan, Alexey Kurakin, and Ian Goodfellow. 2018. Adversarial logit pairing. arXiv:1803.06373 (2018).Google ScholarGoogle Scholar
  37. Alex Kantchelian, J. D. Tygar, and Anthony D. Joseph. 2016. Evasion and hardening of tree ensemble classifiers. In Proceedings of the International Conference on Machine Learning (ICML’16).Google ScholarGoogle Scholar
  38. Diederik Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of the International Conference on Learning Representations (ICLR’15).Google ScholarGoogle Scholar
  39. J. Zico Kolter and Eric Wong. 2018. Provable defenses against adversarial examples via the convex outer adversarial polytope. In Proceedings of the International Conference on Machine Learning (ICML’18).Google ScholarGoogle Scholar
  40. Neeraj Kumar, Alexander C. Berg, Peter N. Belhumeur, and Shree K. Nayar. 2009. Attribute and simile classifiers for face verification. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’09).Google ScholarGoogle Scholar
  41. Alexey Kurakin, Ian Goodfellow, and Samy Bengio. 2017. Adversarial machine learning at scale. In Proceedings of the International Conference on Learning Representations (ICLR’17).Google ScholarGoogle Scholar
  42. Alexey Kurakin, Ian Goodfellow, and Samy Bengio. 2017. Adversarial examples in the physical world. In Proceedings of the International Conference on Learning Representations Workshop (ICLRW’17).Google ScholarGoogle Scholar
  43. Yann LeCun, Corinna Cortes, and Christopher J. C. Burges. {n.d.}. The MNIST database of handwritten digits. Retrieved from http://yann.lecun.com/exdb/mnist/.Google ScholarGoogle Scholar
  44. Fangzhou Liao, Ming Liang, Yinpeng Dong, Tianyu Pang, Jun Zhu, and Xiaolin Hu. 2018. Defense against adversarial attacks using high-level representation guided denoiser. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’18).Google ScholarGoogle ScholarCross RefCross Ref
  45. Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2018. Towards deep learning models resistant to adversarial attacks. In Proceedings of the International Conference on Learning Representations (ICLR’18).Google ScholarGoogle Scholar
  46. Tom Malzbender, Dan Gelb, and Hans Wolters. 2001. Polynomial texture maps. In Proceedings of the ACM Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH’01).Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Mary L. McHugh. 2013. The Chi-square test of independence. Biochem. Med. 23, 2 (2013), 143--149.Google ScholarGoogle ScholarCross RefCross Ref
  48. Dongyu Meng and Hao Chen. 2017. MagNet: A two-pronged defense against adversarial examples. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS’17).Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Jan Hendrik Metzen, Tim Genewein, Volker Fischer, and Bastian Bischoff. 2017. On detecting adversarial perturbations. In Proceedings of the International Conference on Learning Representations (ICLR’17).Google ScholarGoogle Scholar
  50. Matthew Mirman, Timon Gehr, and Martin Vechev. 2018. Differentiable abstract interpretation for provably robust neural networks. In Proceedings of the International Conference on Machine Learning (ICML’18).Google ScholarGoogle Scholar
  51. Takeru Miyato, Shin-ichi Maeda, Masanori Koyama, Ken Nakae, and Shin Ishii. 2016. Distributional smoothing with virtual adversarial training. In Proceedings of the International Conference on Learning Representations (ICLR’16).Google ScholarGoogle Scholar
  52. Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. 2016. DeepFool: A simple and accurate method to fool deep neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16).Google ScholarGoogle ScholarCross RefCross Ref
  53. Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. 2017. Universal adversarial perturbations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17).Google ScholarGoogle ScholarCross RefCross Ref
  54. Nina Narodytska and Shiva Kasiviswanathan. 2017. Simple black-box adversarial attacks on deep neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW’17).Google ScholarGoogle ScholarCross RefCross Ref
  55. George L. Nemhauser, Laurence A. Wolsey, and Marshall L. Fisher. 1978. An analysis of approximations for maximizing submodular set functions. Math. Program. 14, 1 (1978), 265--294.Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Jorge Nocedal. 1980. Updating quasi-Newton matrices with limited storage. Math. Comp. 35, 151 (1980), 773--782.Google ScholarGoogle ScholarCross RefCross Ref
  57. Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z. Berkay Celik, and Ananthram Swami. 2016. The limitations of deep learning in adversarial settings. In Proceedings of the IEEE European Symposium on Security and Privacy (EuroS8P’16).Google ScholarGoogle ScholarCross RefCross Ref
  58. Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z. Berkay Celik, and Ananthram Swami. 2017. Practical black-box attacks against machine learning. In Proceedings of the ACM Symposium on Information, Computer and Communications Security (AsiaCCS’17).Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. Omkar M. Parkhi, Andrea Vedaldi, and Andrew Zisserman. 2015. Deep face recognition. In Proceedings of the British Machine Vision Conference (BMVC’15).Google ScholarGoogle ScholarCross RefCross Ref
  60. Jose Pinheiro, Douglas Bates, Saikat DebRoy, Deepayan Sarkar, and R Core Team. 2015. nlme: Linear and Nonlinear Mixed Effects Models. R package version 3.1--122.Google ScholarGoogle Scholar
  61. Omid Poursaeed, Isay Katsman, Bicheng Gao, and Serge Belongie. 2018. Generative adversarial perturbations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’18).Google ScholarGoogle ScholarCross RefCross Ref
  62. Alec Radford, Luke Metz, and Soumith Chintala. 2016. Unsupervised representation learning with deep convolutional generative adversarial networks. In Proceedings of the International Conference on Learning Representations (ICLR’16).Google ScholarGoogle Scholar
  63. Andras Rozsa, Ethan M. Rudd, and Terrance E. Boult. 2016. Adversarial diversity and hard positive generation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW’16).Google ScholarGoogle Scholar
  64. Sara Sabour, Yanshuai Cao, Fartash Faghri, and David J. Fleet. 2016. Adversarial manipulation of deep representations. In Proceedings of the International Conference on Learning Representations (ICLR’16).Google ScholarGoogle Scholar
  65. Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. 2016. Improved techniques for training gans. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS’16).Google ScholarGoogle Scholar
  66. Pouya Samangouei, Maya Kabkab, and Rama Chellappa. 2018. Defense-GAN: Protecting classifiers against adversarial attacks using generative models. In Proceedings of the International Conference on Learning Representations (ICLR’18).Google ScholarGoogle Scholar
  67. Adi Shamir, Itay Safran, Eyal Ronen, and Orr Dunkelman. 2019. A simple explanation for the existence of adversarial examples with small hamming distance. arXiv:1901.10861 (2019).Google ScholarGoogle Scholar
  68. Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, and Michael K. Reiter. 2016. Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS’16).Google ScholarGoogle Scholar
  69. Vignesh Srinivasan, Arturo Marban, Klaus-Robert Müller, Wojciech Samek, and Shinichi Nakajima. 2018. Counterstrike: Defending deep learning architectures against adversarial samples by Langevin dynamics with supervised denoising autoencoder. arXiv:1805.12017 (2018).Google ScholarGoogle Scholar
  70. Nedim Srndic and Pavel Laskov. 2014. Practical evasion of a learning-based classifier: A case study. In Proceedings of the IEEE Symposium on Security and Privacy (S8P’14). Google ScholarGoogle ScholarDigital LibraryDigital Library
  71. K. O. Stanley. 2007. Compositional pattern producing networks: A novel abstraction of development. Gen. Program. Evolv. Mach. 8, 2 (2007), 131--162. Google ScholarGoogle ScholarDigital LibraryDigital Library
  72. Yaron Steinbuch. 2017. JetBlue ditching boarding passes for facial recognition. New York Post (May 31 2017).Google ScholarGoogle Scholar
  73. Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. 2014. Intriguing properties of neural networks. In Proceedings of the International Conference on Learning Representations (ICLR’14).Google ScholarGoogle Scholar
  74. Yaniv Taigman, Ming Yang, Marc’Aurelio Ranzato, and Lior Wolf. 2014. DeepFace: Closing the gap to human-level performance in face verification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’14). Google ScholarGoogle ScholarDigital LibraryDigital Library
  75. A. Vedaldi and K. Lenc. 2015. MatConvNet—Convolutional neural networks for MATLAB. In Proceedings of the Annual ACM Conference on Multimedia (MM’15). Google ScholarGoogle ScholarDigital LibraryDigital Library
  76. David Warde-Farley and Ian Goodfellow. 2016. Adversarial perturbations of deep neural networks. Perturbations, Optimization, and Statistics, Tamir Hazan, George Papandreou, and Daniel Tarlow (Eds.). MIT Press, 1--32.Google ScholarGoogle Scholar
  77. Chaowei Xiao, Bo Li, Jun-Yan Zhu, Warren He, Mingyan Liu, and Dawn Song. 2018. Generating adversarial examples with adversarial networks. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI’18).Google ScholarGoogle ScholarCross RefCross Ref
  78. Weilin Xu, Yanjun Qi, and David Evans. 2016. Automatically evading classifiers. In Proceedings of the ISOC Annual Network and Distributed System Security Symposium (NDSS’16).Google ScholarGoogle Scholar
  79. Takayuki Yamada, Seiichi Gohshi, and Isao Echizen. 2013. Privacy Visor: Method based on light absorbing and reflecting properties for preventing face image detection. In Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics (SMC’13). Google ScholarGoogle ScholarDigital LibraryDigital Library
  80. Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. 2014. How transferable are features in deep neural networks? In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS’14). Google ScholarGoogle ScholarDigital LibraryDigital Library
  81. Guoming Zhang, Chen Yan, Xiaoyu Ji, Taimin Zhang, Tianchen Zhang, and Wenyuan Xu. 2017. DolphinAttack: Inaudible voice commands. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS’17). Google ScholarGoogle ScholarDigital LibraryDigital Library
  82. Zhengli Zhao, Dheeru Dua, and Sameer Singh. 2018. Generating natural adversarial examples. In Proceedings of the International Conference on Learning Representations (ICLR’18).Google ScholarGoogle Scholar

Index Terms

  1. A General Framework for Adversarial Examples with Objectives

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM Transactions on Privacy and Security
          ACM Transactions on Privacy and Security  Volume 22, Issue 3
          August 2019
          143 pages
          ISSN:2471-2566
          EISSN:2471-2574
          DOI:10.1145/3328797
          Issue’s Table of Contents

          Copyright © 2019 Owner/Author

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 10 June 2019
          • Accepted: 1 March 2019
          • Revised: 1 October 2018
          • Received: 1 April 2018
          Published in tops Volume 22, Issue 3

          Check for updates

          Qualifiers

          • research-article
          • Research
          • Refereed

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        HTML Format

        View this article in HTML Format .

        View HTML Format
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!