Abstract
Recently, AI-manipulated face techniques have developed rapidly and constantly, which has raised new security issues in society. Although existing detection methods consider different categories of fake faces, the performance on detecting the fake faces with “unseen” manipulation techniques is still poor due to the distribution bias among cross-manipulation techniques. To solve this problem, we propose a novel framework that focuses on mining intrinsic features and further eliminating the distribution bias to improve the generalization ability. First, we focus on mining the intrinsic clues in the channel difference image (CDI) and spectrum image (SI) view of two different aspects, including the camera imaging process and the indispensable step in AI manipulation process. Then, we introduce the Octave Convolution and an attention-based fusion module to effectively and adaptively mine intrinsic features from CDI and SI view of these two different but intrinsic aspects. Finally, we design an alignment module to eliminate the bias of manipulation techniques to obtain a more generalized detection framework. We evaluate the proposed framework on four categories of fake faces datasets with the most popular and state-of-the-art manipulation techniques and achieve very competitive performances. We further conduct experiments on cross-manipulation techniques, and the results of our method show the superior advantages on improving generalization ability.
- [1] . 2018. Mesonet: A compact facial video forgery detection network. In Proceedings of the IEEE International Workshop on Information Forensics and Security (WIFS’18). IEEE, 1–7.Google Scholar
Cross Ref
- [2] . 2019. Deepfake video detection through optical flow based cnn. In Proceedings of the IEEE International Conference on Computer Vision Workshops. 0–0.Google Scholar
Cross Ref
- [3] . 2020. What makes fake images detectable? Understanding properties that generalize. In Proceedings of the European Conference on Computer Vision. Springer, 103–120.Google Scholar
Digital Library
- [4] . 2019. Drop an octave: Reducing spatial redundancy in convolutional neural networks with octave convolution. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 3435–3444.Google Scholar
Cross Ref
- [5] . 2019. Homomorphic latent space interpolation for unpaired image-to-image translation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2408–2416.Google Scholar
Cross Ref
- [6] . 2018. Stargan: Unified generative adversarial networks for multi-domain image-to-image translation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 8789–8797.Google Scholar
Cross Ref
- [7] . 2020. Not made for each other-audio-visual dissonance-based deepfake detection and localization. In Proceedings of the 28th ACM International Conference on Multimedia. 439–447.Google Scholar
Digital Library
- [8] . 2020. Fakecatcher: Detection of synthetic portrait videos using biological signals. IEEE Trans. Pattern Anal. Mach. Intell. (2020).Google Scholar
Cross Ref
- [9] . 2018. Forensictransfer: Weakly-supervised domain adaptation for forgery detection. arXiv:1812.02510. Retrieved from https://arxiv.org/abs/1812.02510.Google Scholar
- [10] . 2020. On the detection of digital face manipulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5781–5790.Google Scholar
Cross Ref
- [11] . 2019. Face image manipulation detection based on a convolutional neural network. Expert Syst. Appl. 129 (2019), 156–168.Google Scholar
Digital Library
- [12] . 2005. A tutorial on the cross-entropy method. Ann. Operat. Res. 134, 1 (2005), 19–67.Google Scholar
Cross Ref
- [13] . 2018. Exprgan: Facial expression editing with controllable expression intensity. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence.Google Scholar
Cross Ref
- [14] . 2019. Predicting heart rate variations of deepfake videos using neural ODE. In Proceedings of the IEEE International Conference on Computer Vision Workshops. 0–0.Google Scholar
Cross Ref
- [15] . 2020. Leveraging frequency analysis for deep fake image recognition. In Proceedings of the International Conference on Machine Learning. PMLR, 3247–3258.Google Scholar
- [16] . 2012. A kernel two-sample test. J. Mach. Learn. Res. 13, 1 (2012), 723–773.Google Scholar
Digital Library
- [17] . 2018. Deepfake video detection using recurrent neural networks. In Proceedings of the 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS’18). IEEE, 1–6.Google Scholar
Cross Ref
- [18] . 2002. Color plane interpolation using alternating projections. IEEE Trans. Image Process. 11, 9 (2002), 997–1013.Google Scholar
Digital Library
- [19] . 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 770–778.Google Scholar
Cross Ref
- [20] . 2019. Detection of fake images via the ensemble of deep representations from multi color spaces. In Proceedings of the IEEE International Conference on Image Processing (ICIP’19). IEEE, 2299–2303.Google Scholar
Cross Ref
- [21] . 2019. Attgan: Facial attribute editing by only changing what you want. IEEE Trans. Image Process. 28, 11 (2019), 5464–5478.Google Scholar
Digital Library
- [22] . 2008. Labeled faces in the wild: A database forstudying face recognition in unconstrained environments. In Workshop on Faces in ‘Real-Life’ Images: Detection, Alignment, and Recognition.Google Scholar
- [23] . 2020. T-GD: Transferable GAN-generated images detection framework. In Proceedings of the International Conference on Machine Learning. PMLR, 4746–4761.Google Scholar
- [24] . 2018. Progressive growing of GANs for improved quality, stability, and variation. In Proceedings of the International Conference on Learning Representations.Google Scholar
- [25] . 2019. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4401–4410.Google Scholar
Cross Ref
- [26] . 2020. Analyzing and improving the image quality of stylegan. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8110–8119.Google Scholar
Cross Ref
- [27] . 2014. Adam: A method for stochastic optimization. arXiv:1412.6980. Retrieved from https://arxiv.org/abs/1412.6980.Google Scholar
- [28] . 2017. Fast face-swap using convolutional neural networks. In Proceedings of the IEEE International Conference on Computer Vision. 3677–3685.Google Scholar
Cross Ref
- [29] . 2020. Identification of deep network generated images using disparities in color components. Sign. Process. 174 (2020), 107616.Google Scholar
Cross Ref
- [30] . 2020. Face x-ray for more general face forgery detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5001–5010.Google Scholar
Cross Ref
- [31] . 2018. In ictu oculi: Exposing ai created fake videos by detecting eye blinking. In Proceedings of the IEEE International Workshop on Information Forensics and Security (WIFS’18). IEEE, 1–7.Google Scholar
Cross Ref
- [32] . 2019. Exposing DeepFake videos by detecting face warping artifacts. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 46–52.Google Scholar
- [33] . 2019. Stgan: A unified selective transfer network for arbitrary image attribute editing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3673–3682.Google Scholar
Cross Ref
- [34] . 2019. Efficient image hashing with geometric invariant vector distance for copy detection. ACM Trans. Multimedia Comput. Commun. Appl. 15, 4 (2019), 1–22.Google Scholar
Digital Library
- [35] . 2015. Deep learning face attributes in the wild. In Proceedings of the IEEE International Conference on Computer Vision. 3730–3738.Google Scholar
Digital Library
- [36] . 2020. Global texture enhancement for fake face detection in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8060–8069.Google Scholar
Cross Ref
- [37] . 2008. Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 11(Nov.2008), 2579–2605.Google Scholar
- [38] . 2019. Incremental learning for the detection and classification of GAN-generated images. In Proceedings of the IEEE International Workshop on Information Forensics and Security (WIFS’19). IEEE, 1–6.Google Scholar
Cross Ref
- [39] . 2019. Exploiting visual artifacts to expose deepfakes and face manipulations. In Proceedings of the IEEE Winter Applications of Computer Vision Workshops (WACVW’19). IEEE, 83–92.Google Scholar
Cross Ref
- [40] . 2019. Detecting GAN-generated imagery using saturation cues. In Proceedings of the IEEE International Conference on Image Processing (ICIP’19). IEEE, 4584–4588.Google Scholar
Cross Ref
- [41] . 2020. GAN-generated image detection with self-attention mechanism against GAN generator defect. IEEE J. Select. Top. Sign. Process. 14, 5 (2020), 969–981.Google Scholar
Cross Ref
- [42] . 2020. Emotions don’t lie: An audio-visual deepfake detection method using affective cues. In Proceedings of the 28th ACM International Conference on Multimedia. 2823–2832.Google Scholar
Digital Library
- [43] . 2013. Content-based copy detection through multimodal feature representation and temporal pyramid matching. ACM Trans. Multimedia Comput. Commun. Appl. 10, 1 (2013), 1–20.Google Scholar
Digital Library
- [44] . 2019. Capsule-forensics: Using capsule networks to detect forged images and videos. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP’19). IEEE, 2307–2311.Google Scholar
Cross Ref
- [45] . 2018. Ganimation: Anatomically-aware facial animation from a single image. In Proceedings of the European Conference on Computer Vision (ECCV’18). 818–833.Google Scholar
Cross Ref
- [46] . 2020. Thinking in frequency: Face forgery detection by mining frequency-aware clues. In Proceedings of the European Conference on Computer Vision. Springer, 86–103.Google Scholar
Digital Library
- [47] . 2019. Faceforensics++: Learning to detect manipulated facial images. In Proceedings of the IEEE International Conference on Computer Vision. 1–11.Google Scholar
Cross Ref
- [48] . 2019. Recurrent convolutional strategies for face manipulation detection in videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 80–87.Google Scholar
- [49] . 2017. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer Vision. 618–626.Google Scholar
Cross Ref
- [50] . 2019. Deferred neural rendering: Image synthesis using neural textures. ACM Trans. Graph. 38, 4 (2019), 1–12.Google Scholar
Digital Library
- [51] . 2016. Face2face: Real-time face capture and reenactment of rgb videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2387–2395.Google Scholar
Cross Ref
- [52] . 2020. Deepfakes and beyond: A survey of face manipulation and fake detection. Inf. Fus. 64 (2020), 131–148.Google Scholar
Cross Ref
- [53] . 2019. FakeSpotter: A simple yet robust baseline for spotting AI-synthesized fake faces. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20. 3444–3451.Google Scholar
- [54] . 2020. CNN-generated images are surprisingly easy to spot... for now. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vol. 8695–8704.Google Scholar
Cross Ref
- [55] . 2020. Controlling neural learning network with multiple scales for image splicing forgery detection. ACM Trans. Multimedia Comput. Commun. Appl. 16, 4 (2020), 1–22.Google Scholar
Digital Library
- [56] . 2019. Exposing deep fakes using inconsistent head poses. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP’19). IEEE, 8261–8265.Google Scholar
Cross Ref
- [57] . 2019. Exposing GAN-synthesized faces using landmark locations. In Proceedings of the ACM Workshop on Information Hiding and Multimedia Security. 113–118.Google Scholar
Digital Library
- [58] . 2014. How transferable are features in deep neural networks?Advances in Neural Information Processing Systems 27 (2014), 3320–3328.Google Scholar
- [59] . 2019. Attributing fake images to gans: Learning and analyzing gan fingerprints. In Proceedings of the IEEE International Conference on Computer Vision. 7556–7566.Google Scholar
Cross Ref
- [60] . 2016. Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Sign. Process. Lett. 23, 10 (2016), 1499–1503.Google Scholar
Cross Ref
- [61] . 2019. Detecting and simulating artifacts in gan fake images. In Proceedings of the IEEE International Workshop on Information Forensics and Security (WIFS’19). IEEE, 1–6.Google Scholar
Cross Ref
- [62] . 2011. Facial expression recognition from near-infrared videos. Image Vis. Comput. 29, 9 (2011), 607–619.Google Scholar
Digital Library
Index Terms
Detection of AI-Manipulated Fake Faces via Mining Generalized Features
Recommendations
Faces and Facial Features Detection in Color Images
GMAI '06: Proceedings of the conference on Geometric Modeling and Imaging: New TrendsHuman face detection is concerned with finding the location and size of every human face in a given image. Face detection plays a very important role in human computer interaction field. It represents the first step in a fully automatic face re ...
Enhanced discriminative global-local feature learning with priority for facial expression recognition
AbstractFacial expression recognition (FER) with disturbances is challenging because of contamination of global features and inappropriate patch cropping of local features. Although methods based on global and local features have been combined,...
Mining of frequent itemsets with JoinFI-mine algorithm
AIKED'11: Proceedings of the 10th WSEAS international conference on Artificial intelligence, knowledge engineering and data basesAssociation rule mining among frequent items has been widely studied in data mining field. Many researches have improved the algorithm for generation of all the frequent itemsets. In this paper, we proposed a new algorithm to mine all frequents itemsets ...






Comments