Abstract
With the widespread use of smartphones and the rise of intelligent software, we can manipulate captured photos anytime and anywhere, so the fake photos finally obtained look “Real.” If these intelligent operation methods are maliciously applied to our daily life, then fake news, fake photos, rumors, slander, fraud, threats, and other information security issues around us can happen all the time. Today’s intelligent retouching software can make various modifications to photos, some of which do not change the content that the photos themselves want to express, such as retouching, contrast improvement, and so on. In this article, we mainly study the three operation modes of changing the authenticity of photo contents, which are Copy-move, Splicing, and Removal. Few scholars have done relevant research due to the lack of a corresponding dataset. To address this issue, we elaborately collect a novel dataset, called the multi-realistic scene manipulation dataset (MSM30K), which consists of 30,000 images, including three types of tampering methods, and covering 32 different tampering scenes in life. In addition, we propose a unified detection network: the efficient search and recognition network (ESRNet) for three tampering methods. It mainly includes four main modules: Efficient feature pyramid network (EFPN), Residual receptive field block with attention (RFBA), Hierarchical decoding identification (HDI), and Cascaded group-reversal attention (GRA) blocks. On these three datasets, ESRNet can reach 0.81 on the S-measure, 0.72 on the F-measure, and 0.85 on the E-measure. The inference speed is ~53 fps on a single GPU without I/O time. ESRNet outperforms various state-of-the-art manipulation detection baselines on three image manipulation datasets.
- [1] , , , and (Eds.). 2010. In Proceedings of the 2nd ACM Workshop on Multimedia in Forensics, Security and Intelligence. ACM.Google Scholar
Digital Library
- [2] . 2021. HINet: Half instance normalization network for image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops. 182–192.Google Scholar
Cross Ref
- [3] . 2014. Forensic analysis of SIFT keypoint removal and injection. IEEE Trans. Inf. Forens. Secur. 9, 9 (2014), 1450–1464.Google Scholar
Digital Library
- [4] . 2015. Efficient dense-field copy-move forgery detection. IEEE Trans. Inf. Forens. Secur. 10, 11 (2015), 2284–2297.Google Scholar
Digital Library
- [5] . 2013. CASIA image tampering detection evaluation database. In Proceedings of the IEEE China Summit and International Conference on Signal and Information Processing. IEEE, 422–426.Google Scholar
Cross Ref
- [6] . 2021. Concealed object detection. CoRR abs/2102.10274 (2021).Google Scholar
- [7] . 2020. Camouflaged object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 2774–2784.Google Scholar
Cross Ref
- [8] . 2021. Rethinking RGB-D salient object detection: Models, data sets, and large-scale benchmarks. IEEE Trans. Neural Netw. Learn. Syst. 32, 5 (2021), 2075–2089.Google Scholar
Cross Ref
- [9] . 2020. PraNet: Parallel reverse attention network for polyp segmentation. In Medical Image Computing and Computer Assisted Intervention – MICCAI (MICCAI’20). Lecture Notes in Computer Science, vol. 12266. Springer, Cham. Google Scholar
Digital Library
- [10] . 2008. Image authentication by detecting traces of demosaicing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, 1–8.Google Scholar
Cross Ref
- [11] . 2017. Mask R-CNN. In Proceedings of the IEEE International Conference on Computer Vision. IEEE Computer Society, 2980–2988.Google Scholar
Cross Ref
- [12] . 2017. Robustness of copy-move forgery detection under high JPEG compression artifacts. Multimed. Tools Applic. 76, 1 (2017), 1509–1530.Google Scholar
Digital Library
- [13] . 2020. DOA-GAN: Dual-order attentive generative adversarial network for image copy-move forgery detection and localization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 4675–4684.Google Scholar
Cross Ref
- [14] . 2016. Evaluation of random field models in multi-modal unsupervised tampering localization. In Proceedings of the IEEE International Workshop on Information Forensics and Security. IEEE, 1–6.Google Scholar
Cross Ref
- [15] . 2019. Selective kernel networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Computer Vision Foundation/IEEE, 510–519.Google Scholar
Cross Ref
- [16] . 2017. Feature pyramid networks for object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, 936–944.Google Scholar
Cross Ref
- [17] . 2018. PiCANet: Learning pixel-wise contextual attention for saliency detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, 3089–3098.Google Scholar
Cross Ref
- [18] . 2019. Mask scoring R-CNN. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’19).Google Scholar
- [19] . 2018. Receptive field block net for accurate and fast object detection. In Computer Vision - ECCV 2018-15th European Conference, Munich, Germany, September 8–14, 2018, Proceedings, Part XI(
Lecture Notes in Computer Science , Vol. 11215), , , , and (Eds.). Springer, 404–419.Google ScholarCross Ref
- [20] . 2019. Adversarial learning for constrained image splicing detection and localization based on atrous convolution. IEEE Trans. Inf. Forens. Secur. 14, 10 (2019), 2551–2566.Google Scholar
Cross Ref
- [21] . 2007. Detection of copy-move forgery using a method based on blur moment invariants. Forens. Sci. Int. 171, 2–3 (2007), 180–189.Google Scholar
Cross Ref
- [22] . 2019. DEFACTO: Image and face manipulation dataset. In Proceedings of the 27th European Signal Processing Conference. IEEE, 1–5.Google Scholar
Cross Ref
- [23] . 2016. Copy-move forgery detection technique for forensic analysis in digital images. Math. Prob. Eng. 2016, pt.5 (2016), 1–13.Google Scholar
Cross Ref
- [24] . 2020. IMD2020: A large-scale annotated dataset tailored for detecting manipulated images. In Proceedings of the IEEE Winter Applications of Computer Vision Workshops. IEEE, 71–80.Google Scholar
Cross Ref
- [25] . 2015. U-Net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention - MICCAI 2015-18th International Conference Munich, Germany, October 5–9, 2015, Proceedings, Part III(
Lecture Notes in Computer Science , Vol. 9351), , , , and (Eds.). Springer, 234–241.Google ScholarCross Ref
- [26] . 2019. Fake images: The effects of source, intermediary, and digital media literacy on contextual assessment of image credibility online. New Media Soc. 21, 2 (2019).Google Scholar
Cross Ref
- [27] . 2019. Robust image hashing with tensor decomposition. IEEE Trans. Knowl. Data Eng. 31, 3 (2019), 549–560.Google Scholar
Digital Library
- [28] . 2016. Robust image hashing with ring partition and invariant vector distance. IEEE Trans. Inf. Forens. Secur. 11, 1 (2016), 200–214.Google Scholar
Digital Library
- [29] . 2014. Robust perceptual image hashing based on ring partition and NMF. IEEE Trans. Knowl. Data Eng. 26, 3 (2014), 711–724.Google Scholar
Digital Library
- [30] . 2015. Detection of copy-move forgery in images using segmentation and SURF. In Advances in Signal Processing and Intelligent Recognition Systems Proceedings of Second International Symposium on Signal Processing and Intelligent Recognition Systems (SIRS’15), December 16–19, 2015, Trivandrum, India, Sabu M. Thampi, Sanghamitra Bandyopadhyay, Sri Krishnan, Kuan-Ching Li, Sergey G. Mosin, and Maode Ma (Eds.), Vol. 425. Springer, 645–654.
DOI: Google ScholarCross Ref
- [31] . 2021. Toward offloading internet of vehicles applications in 5G networks. IEEE Trans. Intell. Transport. Syst. 22, 7 (2021), 4151–4159.Google Scholar
Digital Library
- [32] . 2019. Efficient computation offloading for Internet of Vehicles in edge computing-assisted 5G networks. J. Supercomput.4 (2019).Google Scholar
Digital Library
- [33] . 2020. Automated colorization of a grayscale image with seed points propagation. IEEE Trans. Multim. 22, 7 (2020), 1756–1768.Google Scholar
Cross Ref
- [34] . 2021. Controlling neural learning network with multiple scales for image splicing forgery detection. ACM Trans. Multim. Comput. Commun. Appl. 16, 4 (2021), 124:1–124:22.Google Scholar
- [35] . 2016. COVERAGE—A novel database for copy-move forgery detection. In Proceedings of the IEEE International Conference on Image Processing. IEEE, 161–165.Google Scholar
Cross Ref
- [36] . 2017. Deep matching and validation network: An end-to-end solution to constrained image splicing localization and detection. In Proceedings of the 2017 ACM on Multimedia Conference, (MM’17), Mountain View, CA, USA, October 23-27, 2017, Qiong Liu, Rainer Lienhart, Haohong Wang, Sheng–Wei “Kuan–Ta” Chen, Susanne Boll, Yi–Ping Phoebe Chen, Gerald Friedland, Jia Li, and Shuicheng Yan (Eds.). ACM, 1480–1502.
DOI: Google ScholarDigital Library
- [37] . 2018. BusterNet: Detecting copy-move image forgery with source/target localization. In Computer Vision - ECCV 2018-15th European Conference, Munich, Germany, September 8–14, 2018, Proceedings, Part VI(
Lecture Notes in Computer Science , Vol. 11210), , , , and (Eds.). Springer, 170–186.Google ScholarCross Ref
- [38] . 2019. ManTra-Net: Manipulation tracing network for detection and localization of image forgeries with anomalous features. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
Cross Ref
- [39] . 2019. Stacked cross refinement network for edge-aware salient object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision. IEEE, 7263–7272.Google Scholar
Cross Ref
- [40] . 2021. EDPN: Enhanced deep pyramid network for blurry image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops. 414–423.Google Scholar
Cross Ref
- [41] . [n.d.]. Nimble challenge 2017 evaluation data and tool. ([n. d.]).Google Scholar
- [42] . 2021. Multi-stage progressive image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 14821–14831.Google Scholar
Cross Ref
- [43] . 2017. Large-scale evaluation of splicing localization algorithms for web images. Multim. Tools & Applic. 76, 4 (2017), 1–34.Google Scholar
Digital Library
- [44] . 2016. Image region forgery detection: A deep learning approach. In Proceedings of the Singapore Cyber-Security Conference (SG-CRC) 2016 - Cyber-Security by Design, Singapore, January 14–15, 2016(
Cryptology and Information Security Series , Vol. 14), and (Eds.). IOS Press, 1–11.Google Scholar - [45] . 2017. Pyramid scene parsing network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, 6230–6239.Google Scholar
Cross Ref
- [46] . 2019. EGNet: Edge guidance network for salient object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision. IEEE, 8778–8787.Google Scholar
Cross Ref
- [47] . 2018. Learning rich features for image manipulation detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, 1053–1061.Google Scholar
Cross Ref
Index Terms
ESRNet: Efficient Search and Recognition Network for Image Manipulation Detection
Recommendations
ERINet: efficient and robust identification network for image copy-move forgery detection and localization: Efficient and robust identification network
AbstractImages can be maliciously manipulated to hide content or duplicate certain objects. Detecting an elaborate copy-move forgery is very challenging for both humans and machines, and current methods cannot detect copy-move images with the precision ...
Steganalysis aided by fragile detection of image manipulations
AbstractSteganalysis is usually considered as a two-class classification problem of differentiating between covers and stegos. However, in the real world, the cover image may have undergone various operations, which causes two problems that some processed ...
Multi-scale attention context-aware network for detection and localization of image splicing: Efficient and robust identification network
AbstractNowadays, advanced image editing tools and techniques produce more realistic tampered images, especially the addition of intelligent retouching technology makes the threshold of tampered operations lower and lower, which can easily evade image ...






Comments