skip to main content
research-article

SPGAN: Face Forgery Using Spoofing Generative Adversarial Networks

Published:31 March 2021Publication History
Skip Abstract Section

Abstract

Current face spoof detection schemes mainly rely on physiological cues such as eye blinking, mouth movements, and micro-expression changes, or textural attributes of the face images [9]. But none of these methods represent a viable mechanism for makeup-induced spoofing, especially since makeup has been widely used. Compared with face alteration techniques such as plastic surgery, makeup is non-permanent and cost efficient, which makes makeup-induced spoofing become a realistic threat to the integrity of a face recognition system. To solve this problem, we propose a generative model to construct spoofing face images (confusing face images) for improving the accuracy and robustness of automatic face recognition. Our network structure is composed of two separate parts, with one using inter-attention mechanism to obtain interested face region, and another using intra-attention to translate imitation style with preserving imitation style-excluding details. These two attention mechanisms can precisely learn imitation style, where inter-attention pays more attention to imitation regions of image and intra-attention learns face attributes with long distance in image. To effectively discriminate generated images, we introduce an imitation style discriminator. Our model (SPGAN) generates face images that transfer the imitation style from target to subject image and preserve the imitation-excluding features. Experimental results demonstrate the performance of our model in improving quality of imitated face images.

References

  1. Andre Anjos and Sebastien Marcel. 2011. Counter-measures to photo attacks in face recognition: A public database and a bas. In IJCB. 1–7. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Martin Arjovsky, Soumith Chintala, and Léon Bottou. 2017. Wasserstein GAN. In ICML. DOI:https://doi.org/arXiv:1701.07875Google ScholarGoogle Scholar
  3. Konstantinos Bousmalis, Nathan Silberman, David Dohan, Dumitru Erhan, and Dilip Krishnan. 2017. Unsupervised pixel-level domain adaptation with generative adversarial networks. In CVPR. DOI:https://doi.org/10.1109/CVPR.2017.18Google ScholarGoogle Scholar
  4. Huiwen Chang, Jingwan Lu, Fisher Yu, and Adam Finkelstein. 2018. PairedCycleGAN asymmetric style transfer for applying and removing makeup. In CVPR. 40–48. DOI:https://doi.org/10.1109/CVPR.2018.00012Google ScholarGoogle Scholar
  5. Tong Che, Yanran Li, Athul Paul Jacob, Yoshua Bengio, and Wenjie Li. 2017. Mode regularized generative adversarial networks. In ICLR. DOI:https://doi.org/arXiv:1612.02136Google ScholarGoogle Scholar
  6. Cunjian Chen, Antitza Dantcheva, and Arun Ross. 2016. An ensemble of patch-based subspaces for makeup-robust face recognition. Inf. Fusion 32, 32 (2016), 80–92. DOI:https://doi.org/10.1016/j.inffus.2015.09.005 Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Li Chen, Kun Zhou, and Stephen Lin. 2015. Simulating makeup through physics-based manipulation of intrinsic image layers. In CVPR.Google ScholarGoogle Scholar
  8. Liang Chieh Chen, George Papandreou, Iasonas Kokkinos, and Kevin Murphy. 2018. DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 40, 4 (2018), 834–848. DOI:https://doi.org/10.1109/TPAMI.2017.2699184Google ScholarGoogle ScholarCross RefCross Ref
  9. WenTing Chen, Xinpeng Xie, Xi Jia, and Linlin Shen. 2018. Texture deformation based generative adversarial networks for face editing. ArXiv Preprint ArXiv:1812.09832 (2018).Google ScholarGoogle Scholar
  10. Jianpeng Cheng, Dong Li, and Mirella Lapata. 2016. Long short-term memory-networks for machine reading. In EMNLP. 551–561. DOI:https://doi.org/10.18653/v1/D16-1053Google ScholarGoogle Scholar
  11. Songtao Ding, Shiru Qu, Yuling Xi, and Shaohua Wan. 2019. Stimulus-driven and concept-driven analysis for image caption generation. Neurocomputing (2019), 520–530. DOI:https://doi.org/10.1016/j.neucom.2019.04.095Google ScholarGoogle Scholar
  12. Harshala Gammulle, Tharindu Fernando, Simon Denman, Sridha Sridharan, and Clinton Fookes. 2019. Coupled generative adversarial network for continuous fine-grained action segmentation. In WACV. 200–209. DOI:https://doi.org/10.1109/wacv.2019.00027Google ScholarGoogle Scholar
  13. Zan Gao, Yinming Li, and Shaohua Wan. 2020. Exploring deep learning for view-based 3D model retrieval. ACM Trans. Multimedia Comput. Commun. Applic. 16, 1 (2020), 1–21. DOI:https://doi.org/10.1145/3377876 Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron Courville. 2017. Improved training of Wasserstein GANs. In NIPS. 5769–5779. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Dong Hao, Neekhara Paarth, and Wu Chao. 2017. Unsupervised image-to-image translation with generative adversarial networks. arXiv:CVPR (2017). DOI:https://doi.org/arXiv:1701.02676Google ScholarGoogle Scholar
  16. Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In ICML. 448–456. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. T. Darrell, J. Long, and E. Shelhamer. 2015. Fully convolutional networks for semantic segmentation. In CVPR. 3431–3440.Google ScholarGoogle Scholar
  18. Justin Johnson, Alexandre Alahi, and Li Fei Fei. 2016. Perceptual losses for real-time style transfer and super-resolution. In ECCV. 694–711. DOI:https://doi.org/10.1007/978-3-319-46475-6_43Google ScholarGoogle Scholar
  19. Zhu Jun Yan, Park Taesung, and Isola Phillip. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In ICCV. 2242–2251. DOI:https://doi.org/10.1109/iccv.2017.244Google ScholarGoogle Scholar
  20. Mohammad Ahangar Kiasari, Dennis Singh Moirangthem, and Minho Lee. 2018. Coupled generative adversarial stacked auto-encoder: CoGASA. Neural Netw. 100 (2018), 1–9. DOI:https://doi.org/10.1016/j.neunet.2018.01.002Google ScholarGoogle ScholarCross RefCross Ref
  21. Olga Komarova. 2016. A neural algorithm of artistic style. J. Vision 16, 12 (2016), 326–326. DOI:https://doi.org/10.1167/16.12.326Google ScholarGoogle ScholarCross RefCross Ref
  22. Guillaume Lample, Neil Zeghidour, Nicolas Usunier, Antoine Bordes, Ludovic Denoyer, and Marc’Aurelio Ranzato. 2017. Fader networks manipulating images by sliding attributes. In NIPS. 5967–5976. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Anders Boesen Lindbo Larsen, Søren Kaae Sønderby, Hugo Larochelle, and Ole Winther. 2016. Autoencoding beyond pixels using a learned similarity metric. In ICML. 1558–1566. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Christian Ledig, Lucas Theis, and Huszar. 2017. Photo-realistic single image super-resolution using a generative adversarial network. In CVPR. 105–114. DOI:https://doi.org/10.1109/CVPR.2017.19Google ScholarGoogle Scholar
  25. Christian Ledig, Lucas Theis, Ferenc Huszar, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, and Zehan Wang. 2017. Photo-realistic single image super-resolution using a generative adversarial network. In CVPR. 105–114. DOI:https://doi.org/10.1109/CVPR.2017.19Google ScholarGoogle Scholar
  26. Lingzhi Li, Jianmin Bao, Yang Hao, Dong Chen, and Fang Wen. 2019. FaceShifter: Towards high fidelity and occlusion aware face swapping. In CVPR.Google ScholarGoogle Scholar
  27. Minjun Li, Haozhi Huang, Lin Ma, Wei Liu, Tong Zhang, and Yu-Gang Jiang. 2018. Unsupervised image-to-image translation with stacked cycle-consistent adversarial networks. In ECCV. 184–199. DOI:https://doi.org/10.1007/978-3-030-01240-3_12Google ScholarGoogle Scholar
  28. Mu Li, Wangmeng Zuo, and David Zhang. 2016. Convolutional network for attribute-driven and identity-preserving human face generation. Preprint arXiv 1608.06434 (2016).Google ScholarGoogle Scholar
  29. Mu Li, Wangmeng Zuo, and David Zhang. 2016. Deep identity aware transfer of facial attributes. Preprint arXiv 1610.05586 (2016).Google ScholarGoogle Scholar
  30. Zhouhan Lin, Minwei Feng, Cicero Nogueira Dos Santos, Yu Mo, and Yoshua Bengio. 2017. A structured self-attentive sentence embedding. Preprint arXiv:1703.03130 (2017).Google ScholarGoogle Scholar
  31. Xiaowei Liu, Kenli Li, and Keqin Li. 2019. Attentive semantic and perceptual faces completion using self-attention generative adversarial networks. Neural Proc. Lett. (2019). DOI:https://doi.org/10.1007/s11063-019-10080-2Google ScholarGoogle Scholar
  32. Chen Long, Hanwang Zhang, Jun Xiao, Liqiang Nie, and Tat Seng Chua. 2017. SCA-CNN: Spatial and channel-wise attention in convolutional networks for image captioning. In CVPR. DOI:https://doi.org/10.1109/CVPR.2017.667Google ScholarGoogle Scholar
  33. Xiaofeng Mao, Shuhui Wang, Liying Zheng, and Qingming Huang. 2018. Semantic invariant cross-domain image generation with generative adversarial networks [J]. Neurocomputing 293, 7 (2018), 55--63.Google ScholarGoogle ScholarCross RefCross Ref
  34. Luke Metz, Ben Poole, David Pfau, and Jascha Sohl-Dickstein. 2017. Unrolled generative adversarial networks. In ICLR. DOI:https://doi.org/arXiv:1611.02163Google ScholarGoogle Scholar
  35. Mehdi Mirza and Simon Osindero. 2014. Conditional generative adversarial nets. Preprint arXiv 1411.1784 (2014).Google ScholarGoogle Scholar
  36. Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. 2018. Spectral normalization for generative adversarial networks. Preprint ArXiv (2018). DOI:https://doi.org/arXiv:1802.05957Google ScholarGoogle Scholar
  37. Thomas Brox, Olaf Ronneberger, and Philipp Fischer. 2015. U-Net: Convolutional networks for biomedical image segmentation. In MICCAI.Google ScholarGoogle Scholar
  38. Andrew Zisserman, Omkar M. Parkhi, and Andrea Vedaldi. 2015. Deep face recognition. In BMVC.Google ScholarGoogle Scholar
  39. Ankur P. Parikh, Oscar Tckstrm, Dipanjan Das, and Jakob Uszkoreit. 2016. A decomposable attention model for natural language inference. In EMNLP. 2249–2255. DOI:https://doi.org/10.18653/v1/D16-1244Google ScholarGoogle Scholar
  40. Keyurkumar Patel, Hu Han, and Anil K. Jain. 2016. Secure face unlock: Spoof detection on smartphones. IEEE Trans. Inf. Forens. Secur. 11, 10 (2016), 2268–2283. DOI:https://doi.org/10.1109/TIFS.2016.2578288 Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Romain Paulus, Caiming Xiong, and Richard Socher. 2017. A deep reinforced model for abstractive summarization. ArXiv Preprint ArXiv:1705.04304 (2017).Google ScholarGoogle Scholar
  42. Guim Perarnau, Joost van de Weijer, Bogdan Raducanu, and Jose M. Álvarez 2016. Invertible conditional GANs for image editing. arXiv preprint arXiv 1611.06355 (2016).Google ScholarGoogle Scholar
  43. Albert Pumarola, Antonio Agudo, Aleix M. Martinez, Alberto Sanfeliu, and Francesc Moreno-Noguer. 2018. GANimation: Anatomically-aware facial animation from a single image. In ECCV. 835–851. DOI:https://doi.org/10.1007/978-3-030-01249-6_50Google ScholarGoogle Scholar
  44. Weichao Qiu and Alan Yuille. 2016. UnrealCV: Connecting computer vision to unreal engine. In ECCV. 909–916. DOI:https://doi.org/10.1007/978-3-319-49409-8_75Google ScholarGoogle Scholar
  45. Alec Radford, Luke Metz, and Soumith Chintala. 2016. Unsupervised representation learning with deep convolutional generative adversarial networks. Preprint ArXiv. DOI:https://doi.org/arXiv:1511.06434Google ScholarGoogle Scholar
  46. Stephan R. Richter, Vibhav Vineet, Stefan Roth, and Vladlen Koltun. 2016. Playing for data: Ground truth from computer games. In ECCV. 102–118. DOI:https://doi.org/10.1007/978-3-319-46475-6_7Google ScholarGoogle Scholar
  47. Shao Rui, Xiangyuan Lan, Jiawei Li, and Yuen Pong. 2019. Multi-adversarial discriminative deep domain generalization for face presentation attack detection. In CVPR. 10023–10031.Google ScholarGoogle Scholar
  48. Andrei A. Rusu, Matej Vecerik, Thomas Rothorl, Nicolas Heess, Razvan Pascanu, and Raia Hadsell. 2016. Sim-to-real robot learning from pixels with progressive nets. Preprint arXiv:1809.07480 (2016).Google ScholarGoogle Scholar
  49. Tim Salimans, Han Zhang, Alec Radford, and Dimitris Metaxas. 2018. Improving GANs using optimal transport. In ICLR. DOI:https://arxiv.org/abs/1803.05573.Google ScholarGoogle Scholar
  50. Mathew Salvaris, Danielle Dean, and Wee Hyong Tok. 2018. Generative adversarial networks. Preprint arXiv 1406.2661 (2018).Google ScholarGoogle Scholar
  51. Wei Shen and Rujie Liu. 2017. Learning residual images for face attribute manipulation. In CVPR. 1225–1233. DOI:https://doi.org/10.1109/cvpr.2017.135Google ScholarGoogle Scholar
  52. Casper Kaae Sonderby, Jose Caballero, Lucas Theis, and Wenzhe Shi. 2017. Amortised MAP inference for image super-resolution. In ICLR.Google ScholarGoogle Scholar
  53. C. Dong, T. Li, R. Qian. 2018. BeautyGAN: Instrance-level facial makeup transfer with deep generative adversarial network. In MM. 645–653. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Dickson Tong, Chi Keung Tang, Michael S. Brown, and Ying Qing Xu. 2008. Example-based cosmetic transfer. Cosmet. Toilet.211–218. DOI:https://doi.org/10.1109/PG.2007.31 Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Eric Tzeng, Coline Devin, Judy Hoffman, Chelsea Finn, and Xingchao Peng. 2016. Towards adapting deep visuomotor representations from simulated to real environments. arXiv preprint arXiv 1511.07111 (2016).Google ScholarGoogle Scholar
  56. Paul Upchurch, Jacob Gardner, Geoff Pleiss, Robert Pless, Noah Snavely, Kavita Bala, and Kilian Weinberger. 2017. Deep feature interpolation for image content changes. In CVPR. 6090–6099. DOI:https://doi.org/10.1109/CVPR.2017.645Google ScholarGoogle Scholar
  57. Shaohua Wan, Yu Xia, Lianyong Qi, Yee Hong Yang, and Mohammed Atiquzzaman. 2020. Automated colorization of a grayscale image with seed points propagation. IEEE Trans. Multimedia 99 (2020), 1–1. DOI:https://doi.org/10.1109/TMM.2020.2976573Google ScholarGoogle Scholar
  58. Shuyang Wang and Fu Yun. 2016. Face behind makeup. In AAAI. Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. E. Kolve, Y. Zhu, and R. Mottaghi. 2017. Target-driven visual navigation in indoor scenes using deep reinforcement learning. In ICRA. 3357–3364. DOI:https://doi.org/10.1109/ICRA.2017.7989381Google ScholarGoogle Scholar
  60. Junichi Yamagishi, Tomi H. Kinnunen, and Nicholas Evans. 2017. Introduction to the issue on spoofing and countermeasures for automatic speaker verification. IEEE J. Select. Topics Sig. Proc. 11, 4 (2017), 585–587. DOI:https://doi.org/10.1109/JSTSP.2017.2698143Google ScholarGoogle ScholarCross RefCross Ref
  61. Donggeun Yoo, Namil Kim, Sunggyun Park, Anthony S. Paek, and In So Kweon. 2016. Pixel-level domain transfer. In ECCV. 517–532. DOI:https://doi.org/10.1007/978-3-319-46484-8_31Google ScholarGoogle Scholar
  62. Munyoung Kim Yunjey Choi, Minje Choi, Jung-Woo Ha, Sunghun Kim, and Jaegul Choo. 2018. StarGAN:Unified generative adversarial networks for multidomain image-to-image translation. In CVPR. 8789–8797.Google ScholarGoogle Scholar
  63. H. Zhang, I. Goodfellow, D. Metaxas, and A. Odena. 2019. Self-attention generative adversarial networks. In ICML. 7354–7363.Google ScholarGoogle Scholar
  64. Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaogang Wang, Xiaolei Huang, and Dimitris Metaxas. 2017. StackGAN: Text to photo-realistic image synthesis with stacked generative adversarial networks. In ICCV. 5908–5916.Google ScholarGoogle Scholar
  65. Richard Zhang, Phillip Isola, and Alexei A. Efros. 2016. Colorful image colorization. In ECCV. 649–666. DOI:https://doi.org/10.1007/978-3-319-46487-9_40Google ScholarGoogle Scholar
  66. Junbo Zhao, Michael Mathieu, and Yann Lecun. 2016. Energy-based generative adversarial network. Preprint ArXiv (2016).Google ScholarGoogle Scholar
  67. Shuchang Zhou, Taihong Xiao, Yi Yang, Dieqiao Feng, Qinyao He, and Weiran He. 2017. GeneGAN: Learning object transfiguration and attribute subspace from unpaired data. Preprint arXiv 1705.04932 (2017).Google ScholarGoogle Scholar
  68. Jun Yan Zhu, Taesung Park, and Phillip Isola. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In ICCV. 2242–2251. DOI:https://doi.org/10.1109/iccv.2017.244Google ScholarGoogle Scholar
  69. Jun Yan Zhu, Philipp Krähenbühl, Eli Shechtman, and Alexei A. Efros. 2016. Generative visual manipulation on the natural image manifold. In ECCV. DOI:https://doi.org/10.1007/978-3-319-46454-1_36Google ScholarGoogle Scholar

Index Terms

  1. SPGAN: Face Forgery Using Spoofing Generative Adversarial Networks

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!