skip to main content
research-article

ESRNet: Efficient Search and Recognition Network for Image Manipulation Detection

Authors Info & Claims
Published:04 March 2022Publication History
Skip Abstract Section

Abstract

With the widespread use of smartphones and the rise of intelligent software, we can manipulate captured photos anytime and anywhere, so the fake photos finally obtained look “Real.” If these intelligent operation methods are maliciously applied to our daily life, then fake news, fake photos, rumors, slander, fraud, threats, and other information security issues around us can happen all the time. Today’s intelligent retouching software can make various modifications to photos, some of which do not change the content that the photos themselves want to express, such as retouching, contrast improvement, and so on. In this article, we mainly study the three operation modes of changing the authenticity of photo contents, which are Copy-move, Splicing, and Removal. Few scholars have done relevant research due to the lack of a corresponding dataset. To address this issue, we elaborately collect a novel dataset, called the multi-realistic scene manipulation dataset (MSM30K), which consists of 30,000 images, including three types of tampering methods, and covering 32 different tampering scenes in life. In addition, we propose a unified detection network: the efficient search and recognition network (ESRNet) for three tampering methods. It mainly includes four main modules: Efficient feature pyramid network (EFPN), Residual receptive field block with attention (RFBA), Hierarchical decoding identification (HDI), and Cascaded group-reversal attention (GRA) blocks. On these three datasets, ESRNet can reach 0.81 on the S-measure, 0.72 on the F-measure, and 0.85 on the E-measure. The inference speed is ~53 fps on a single GPU without I/O time. ESRNet outperforms various state-of-the-art manipulation detection baselines on three image manipulation datasets.

REFERENCES

  1. [1] Battiato Sebastiano, Emmanuel Sabu, Ulges Adrian, and Worring Marcel (Eds.). 2010. In Proceedings of the 2nd ACM Workshop on Multimedia in Forensics, Security and Intelligence. ACM.Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. [2] Chen Liangyu, Lu Xin, Zhang Jie, Chu Xiaojie, and Chen Chengpeng. 2021. HINet: Half instance normalization network for image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops. 182192.Google ScholarGoogle ScholarCross RefCross Ref
  3. [3] Costanzo Andrea, Amerini Irene, Caldelli Roberto, and Barni Mauro. 2014. Forensic analysis of SIFT keypoint removal and injection. IEEE Trans. Inf. Forens. Secur. 9, 9 (2014), 14501464.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. [4] Cozzolino Davide, Poggi Giovanni, and Verdoliva Luisa. 2015. Efficient dense-field copy-move forgery detection. IEEE Trans. Inf. Forens. Secur. 10, 11 (2015), 22842297.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. [5] Dong Jing, Wang Wei, and Tan Tieniu. 2013. CASIA image tampering detection evaluation database. In Proceedings of the IEEE China Summit and International Conference on Signal and Information Processing. IEEE, 422426.Google ScholarGoogle ScholarCross RefCross Ref
  6. [6] Fan Deng-Ping, Ji Ge-Peng, Cheng Ming-Ming, and Shao Ling. 2021. Concealed object detection. CoRR abs/2102.10274 (2021).Google ScholarGoogle Scholar
  7. [7] Fan Deng-Ping, Ji Ge-Peng, Sun Guolei, Cheng Ming-Ming, Shen Jianbing, and Shao Ling. 2020. Camouflaged object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 27742784.Google ScholarGoogle ScholarCross RefCross Ref
  8. [8] Fan Deng-Ping, Lin Zheng, Zhang Zhao, Zhu Menglong, and Cheng Ming-Ming. 2021. Rethinking RGB-D salient object detection: Models, data sets, and large-scale benchmarks. IEEE Trans. Neural Netw. Learn. Syst. 32, 5 (2021), 20752089.Google ScholarGoogle ScholarCross RefCross Ref
  9. [9] Fan D. P., Ji G. P., Zhou T., Chen G., Fu H., Shen J., and Shao L.. 2020. PraNet: Parallel reverse attention network for polyp segmentation. In Medical Image Computing and Computer Assisted Intervention – MICCAI (MICCAI’20). Lecture Notes in Computer Science, vol. 12266. Springer, Cham. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. [10] Gallagher Andrew C. and Chen Tsuhan. 2008. Image authentication by detecting traces of demosaicing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, 18.Google ScholarGoogle ScholarCross RefCross Ref
  11. [11] He Kaiming, Gkioxari Georgia, Dollár Piotr, and Girshick Ross B.. 2017. Mask R-CNN. In Proceedings of the IEEE International Conference on Computer Vision. IEEE Computer Society, 29802988.Google ScholarGoogle ScholarCross RefCross Ref
  12. [12] Huang D. Y., Huang C. N., Hu W. C., and Chou C. H.. 2017. Robustness of copy-move forgery detection under high JPEG compression artifacts. Multimed. Tools Applic. 76, 1 (2017), 15091530.Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. [13] Islam Ashraful, Long Chengjiang, Basharat Arslan, and Hoogs Anthony. 2020. DOA-GAN: Dual-order attentive generative adversarial network for image copy-move forgery detection and localization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 46754684.Google ScholarGoogle ScholarCross RefCross Ref
  14. [14] Korus Pawel and Huang Jiwu. 2016. Evaluation of random field models in multi-modal unsupervised tampering localization. In Proceedings of the IEEE International Workshop on Information Forensics and Security. IEEE, 16.Google ScholarGoogle ScholarCross RefCross Ref
  15. [15] Li Xiang, Wang Wenhai, Hu Xiaolin, and Yang Jian. 2019. Selective kernel networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Computer Vision Foundation/IEEE, 510519.Google ScholarGoogle ScholarCross RefCross Ref
  16. [16] Lin Tsung-Yi, Dollár Piotr, Girshick Ross B., He Kaiming, Hariharan Bharath, and Belongie Serge J.. 2017. Feature pyramid networks for object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, 936944.Google ScholarGoogle ScholarCross RefCross Ref
  17. [17] Liu Nian, Han Junwei, and Yang Ming-Hsuan. 2018. PiCANet: Learning pixel-wise contextual attention for saliency detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, 30893098.Google ScholarGoogle ScholarCross RefCross Ref
  18. [18] Wang Zhaojin Huang, Lichao Huang, Yongchao Gong, Chang Huang, and Xinggang. 2019. Mask scoring R-CNN. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’19).Google ScholarGoogle Scholar
  19. [19] Liu Songtao, Huang Di, and Wang Yunhong. 2018. Receptive field block net for accurate and fast object detection. In Computer Vision - ECCV 2018-15th European Conference, Munich, Germany, September 8–14, 2018, Proceedings, Part XI(Lecture Notes in Computer Science, Vol. 11215), Ferrari Vittorio, Hebert Martial, Sminchisescu Cristian, and Weiss Yair (Eds.). Springer, 404419.Google ScholarGoogle ScholarCross RefCross Ref
  20. [20] Liu Yaqi, Zhu Xiaobin, Zhao Xianfeng, and Cao Yun. 2019. Adversarial learning for constrained image splicing detection and localization based on atrous convolution. IEEE Trans. Inf. Forens. Secur. 14, 10 (2019), 25512566.Google ScholarGoogle ScholarCross RefCross Ref
  21. [21] Mahdian B. and Saic S.. 2007. Detection of copy-move forgery using a method based on blur moment invariants. Forens. Sci. Int. 171, 2–3 (2007), 180189.Google ScholarGoogle ScholarCross RefCross Ref
  22. [22] Mahfoudi Gaël, Tajini Badr, Retraint Florent, Morain-Nicolier Frédéric, Dugelay Jean-Luc, and Pic Marc. 2019. DEFACTO: Image and face manipulation dataset. In Proceedings of the 27th European Signal Processing Conference. IEEE, 15.Google ScholarGoogle ScholarCross RefCross Ref
  23. [23] Mahmood T., Nawaz T., Irtaza A., Ashraf R., Shah M., and Mahmood M. T.. 2016. Copy-move forgery detection technique for forensic analysis in digital images. Math. Prob. Eng. 2016, pt.5 (2016), 113.Google ScholarGoogle ScholarCross RefCross Ref
  24. [24] Novozámský Adam, Mahdian Babak, and Saic Stanislav. 2020. IMD2020: A large-scale annotated dataset tailored for detecting manipulated images. In Proceedings of the IEEE Winter Applications of Computer Vision Workshops. IEEE, 7180.Google ScholarGoogle ScholarCross RefCross Ref
  25. [25] Ronneberger Olaf, Fischer Philipp, and Brox Thomas. 2015. U-Net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention - MICCAI 2015-18th International Conference Munich, Germany, October 5–9, 2015, Proceedings, Part III(Lecture Notes in Computer Science, Vol. 9351), Navab Nassir, Hornegger Joachim, III William M. Wells, and Frangi Alejandro F. (Eds.). Springer, 234241.Google ScholarGoogle ScholarCross RefCross Ref
  26. [26] Shen Cuihua, Kasra Mona, Pan Wenjing, Bassett Grace A., Malloch Yining, and O’Brien James F.. 2019. Fake images: The effects of source, intermediary, and digital media literacy on contextual assessment of image credibility online. New Media Soc. 21, 2 (2019).Google ScholarGoogle ScholarCross RefCross Ref
  27. [27] Tang Zhenjun, Chen Lv, Zhang Xianquan, and Zhang Shichao. 2019. Robust image hashing with tensor decomposition. IEEE Trans. Knowl. Data Eng. 31, 3 (2019), 549560.Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. [28] Tang Zhenjun, Zhang Xianquan, Li Xianxian, and Zhang Shichao. 2016. Robust image hashing with ring partition and invariant vector distance. IEEE Trans. Inf. Forens. Secur. 11, 1 (2016), 200214.Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. [29] Tang Zhenjun, Zhang Xianquan, and Zhang Shichao. 2014. Robust perceptual image hashing based on ring partition and NMF. IEEE Trans. Knowl. Data Eng. 26, 3 (2014), 711724.Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. [30] Mehtre V. T. Manu and Babu M.. 2015. Detection of copy-move forgery in images using segmentation and SURF. In Advances in Signal Processing and Intelligent Recognition Systems Proceedings of Second International Symposium on Signal Processing and Intelligent Recognition Systems (SIRS’15), December 16–19, 2015, Trivandrum, India, Sabu M. Thampi, Sanghamitra Bandyopadhyay, Sri Krishnan, Kuan-Ching Li, Sergey G. Mosin, and Maode Ma (Eds.), Vol. 425. Springer, 645–654. DOI:Google ScholarGoogle ScholarCross RefCross Ref
  31. [31] Wan Shaohua, Gu Renhao, Umer Tariq, Salah Khaled, and Xu Xiaolong. 2021. Toward offloading internet of vehicles applications in 5G networks. IEEE Trans. Intell. Transport. Syst. 22, 7 (2021), 41514159.Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. [32] Wan S., Li X., Xue Y., Lin W., and Xu X.. 2019. Efficient computation offloading for Internet of Vehicles in edge computing-assisted 5G networks. J. Supercomput.4 (2019).Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. [33] Wan Shaohua, Xia Yu, Qi Lianyong, Yang Yee-Hong, and Atiquzzaman Mohammed. 2020. Automated colorization of a grayscale image with seed points propagation. IEEE Trans. Multim. 22, 7 (2020), 17561768.Google ScholarGoogle ScholarCross RefCross Ref
  34. [34] Wei Yang, Wang Zhuzhu, Xiao Bin, Liu Ximeng, Yan Zheng, and Ma Jianfeng. 2021. Controlling neural learning network with multiple scales for image splicing forgery detection. ACM Trans. Multim. Comput. Commun. Appl. 16, 4 (2021), 124:1–124:22.Google ScholarGoogle Scholar
  35. [35] Wen Bihan, Zhu Ye, Subramanian Ramanathan, Ng Tian-Tsong, Shen Xuanjing, and Winkler Stefan. 2016. COVERAGE—A novel database for copy-move forgery detection. In Proceedings of the IEEE International Conference on Image Processing. IEEE, 161165.Google ScholarGoogle ScholarCross RefCross Ref
  36. [36] Wu Yue, Abd-Almageed Wael, and Natarajan Prem. 2017. Deep matching and validation network: An end-to-end solution to constrained image splicing localization and detection. In Proceedings of the 2017 ACM on Multimedia Conference, (MM’17), Mountain View, CA, USA, October 23-27, 2017, Qiong Liu, Rainer Lienhart, Haohong Wang, Sheng–Wei “Kuan–Ta” Chen, Susanne Boll, Yi–Ping Phoebe Chen, Gerald Friedland, Jia Li, and Shuicheng Yan (Eds.). ACM, 1480–1502. DOI:Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. [37] Wu Yue, Abd-Almageed Wael, and Natarajan Prem. 2018. BusterNet: Detecting copy-move image forgery with source/target localization. In Computer Vision - ECCV 2018-15th European Conference, Munich, Germany, September 8–14, 2018, Proceedings, Part VI(Lecture Notes in Computer Science, Vol. 11210), Ferrari Vittorio, Hebert Martial, Sminchisescu Cristian, and Weiss Yair (Eds.). Springer, 170186.Google ScholarGoogle ScholarCross RefCross Ref
  38. [38] Wu Y., Abdalmageed W., and Natarajan P.. 2019. ManTra-Net: Manipulation tracing network for detection and localization of image forgeries with anomalous features. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Google ScholarGoogle ScholarCross RefCross Ref
  39. [39] Wu Zhe, Su Li, and Huang Qingming. 2019. Stacked cross refinement network for edge-aware salient object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision. IEEE, 72637272.Google ScholarGoogle ScholarCross RefCross Ref
  40. [40] Xu Ruikang, Xiao Zeyu, Huang Jie, Zhang Yueyi, and Xiong Zhiwei. 2021. EDPN: Enhanced deep pyramid network for blurry image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops. 414423.Google ScholarGoogle ScholarCross RefCross Ref
  41. [41] Yates A. N., Guan H., Lee Y., Delgado A. P., Zhou D. F., and Fiscus J. G.. [n.d.]. Nimble challenge 2017 evaluation data and tool. ([n. d.]).Google ScholarGoogle Scholar
  42. [42] Zamir Syed Waqas, Arora Aditya, Khan Salman, Hayat Munawar, Khan Fahad Shahbaz, Yang Ming-Hsuan, and Shao Ling. 2021. Multi-stage progressive image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 1482114831.Google ScholarGoogle ScholarCross RefCross Ref
  43. [43] Zampoglou M., Papadopoulos S., and Kompatsiaris Y.. 2017. Large-scale evaluation of splicing localization algorithms for web images. Multim. Tools & Applic. 76, 4 (2017), 134.Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. [44] Zhang Ying, Goh Jonathan, Win Lei Lei, and Thing Vrizlynn L. L.. 2016. Image region forgery detection: A deep learning approach. In Proceedings of the Singapore Cyber-Security Conference (SG-CRC) 2016 - Cyber-Security by Design, Singapore, January 14–15, 2016(Cryptology and Information Security Series, Vol. 14), Mathur Aditya and Roychoudhury Abhik (Eds.). IOS Press, 111.Google ScholarGoogle Scholar
  45. [45] Zhao Hengshuang, Shi Jianping, Qi Xiaojuan, Wang Xiaogang, and Jia Jiaya. 2017. Pyramid scene parsing network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, 62306239.Google ScholarGoogle ScholarCross RefCross Ref
  46. [46] Zhao Jiaxing, Liu Jiangjiang, Fan Deng-Ping, Cao Yang, Yang Jufeng, and Cheng Ming-Ming. 2019. EGNet: Edge guidance network for salient object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision. IEEE, 87788787.Google ScholarGoogle ScholarCross RefCross Ref
  47. [47] Zhou Peng, Han Xintong, Morariu Vlad I., and Davis Larry S.. 2018. Learning rich features for image manipulation detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, 10531061.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. ESRNet: Efficient Search and Recognition Network for Image Manipulation Detection

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM Transactions on Multimedia Computing, Communications, and Applications
        ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 18, Issue 4
        November 2022
        497 pages
        ISSN:1551-6857
        EISSN:1551-6865
        DOI:10.1145/3514185
        • Editor:
        • Abdulmotaleb El Saddik
        Issue’s Table of Contents

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 4 March 2022
        • Accepted: 1 December 2021
        • Revised: 1 November 2021
        • Received: 1 August 2021
        Published in tomm Volume 18, Issue 4

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
        • Refereed

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Full Text

      View this article in Full Text.

      View Full Text

      HTML Format

      View this article in HTML Format .

      View HTML Format
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!