skip to main content
research-article

A Dual-Domain Perceptual Framework for Generating Visual Inconspicuous Counterparts

Published:26 April 2017Publication History
Skip Abstract Section

Abstract

For a given image, it is a challenging task to generate its corresponding counterpart with visual inconspicuous modification. The complexity of this problem reasons from the high correlativity between the editing operations and vision perception. Essentially, a significant requirement that should be emphasized is how to make the object modifications hard to be found visually in the generative counterparts. In this article, we propose a novel dual-domain perceptual framework to generate visual inconspicuous counterparts, which applies the perceptual bidirectional similarity metric (PBSM) and appearance similarity metric (ASM) to create the dual-domain perception error minimization model. The candidate targets are yielded by the well-known PatchMatch model with the strokes-based interactions and selective object library. By the dual-perceptual evaluation index, all candidate targets are sorted to select out the best result. For demonstration, a series of objective and subjective measurements are used to evaluate the performance of our framework.

References

  1. Connelly Barnes, Dan Goldman, Eli Shechtman, and Adam Finkelstein. 2011. The PatchMatch randomized matching algorithm for image manipulation. Communications of the ACM 54, 11, 103--110. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Connelly Barnes, Eli Shechtman, Adam Finkelstein, and Dan Goldman. 2009. PatchMatch: A randomized correspondence algorithm for structural image editing. In Proceedings of ACM SIGGRAPH 2009 Papers, Vol. 28. 1--11. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Subhabrata Bhattacharya, Rahul Sukthankar, and Mubarak Shah. 2011. A holistic approach to aesthetic enhancement of photographs. ACM Transactions on Multimedia Computing, Communications and Applications 7S, 1, 21:1--21:21. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Ali Borji, Dicky Sihite, and Laurent Itti. 2012. Salient object detection: A benchmark. In Proceedings of the 12th European Conference on Computer Vision. 414--429.Google ScholarGoogle ScholarCross RefCross Ref
  5. Tao Chen, Mingming Cheng, Ping Tan, Ariel Shamir, and Shimin Hu. 2009. Sketch2Photo: Internet image montage. In Proceedings of ACM SIGGRAPH Asia 2009, Vol. 28. 124:1--124:10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Mingming Cheng, Fanglue Zhang, Niloy Mitra, and Xiaolei and Huang. 2010. RepFinder: Finding approximately repeated scene elements for image editing. ACM Transactions on Graphics 29, 4, 1--8. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Mingming Cheng, Guoxin Zhang, Niloy J. Mitra, Xiaolei Huang, and Shimin Hu. 2011. Global contrast based salient region detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 409--416. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Mingming Cheng, Guoxin Zhang, Niloy J. Mitra, Xiaolei Huang, and Shimin Hu. 2015. Global contrast based salient region detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 37, 3, 569--582.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. T. Cho, M. Butman, S. Avidan, and W. Freeman. 2008. The Patch Transform and its applications to image editing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1--8.Google ScholarGoogle Scholar
  10. A. Criminisi, P. Perez, and K. Toyama. 2003. Object removal by exemplar-based inpainting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vol. 2. 721--728.Google ScholarGoogle Scholar
  11. Kostas Daniilidis, Petros Maragos, Nikos Paragios, Connelly Barnes, Eli Shechtman, Dan Goldman, and Adam Finkelstein. 2010. The generalized PatchMatch correspondence algorithm. In Proceedings of the European Conference on Computer Vision. 29--43. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Carl Doersch, Saurabh Singh, Abhinav Gupta, Josef Sivic, and Alexei Efros. 2012. What makes Paris look like Paris? ACM Transactions on Graphics 31, 4, 1--9. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Mathias Eitz, Kristian Hildebrand, Tamy Boubekeur, and Marc Alexa. 2009. PhotoSketch: A sketch based image query and compositing system. In Proceedings of SIGGRAPH 2009: Talks. 1--4. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Zeev Farbman, Gil Hoffer, Yaron Lipman, Daniel Cohen-Or, and Dani Lischinski. 2009. Coordinates for instant image cloning. In Proceedings of ACM SIGGRAPH 2009 Papers, Vol. 28. 1--9. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Chen Goldberg, Tao Chen, Fanglue Zhang, Ariel Shamir, and Shimin Hu. 2012. Data-driven object manipulation in images. Computer Graphics Forum 31, 2, 265--274. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Jonathan Harel, Christof Koch, and Pietro Perona. 2007. Graph-based visual saliency. In Proceedings of the 20th Annual Conference on Neural Information Processing Systems. 545--552. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Shimin Hu, Fanglue Zhang, Miao Wang, Ralph Martin, and Jue Wang. 2013. PatchNet: A patch-based image representation for interactive library-driven image editing. ACM Transactions on Graphics 32, 6, 196:1--196:12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Hui Huang, Kangxue Yin, Minglun Gong, Dani Lischinski, Daniel Cohen-Or, Uri Ascher, and Baoquan Chen. 2013. “Mind the gap”: Tele-registration for structure-driven image completion. ACM Transactions on Graphics 32, 6, 1--10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. L. Itti, C. Koch, and E. Niebur. 1998. A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 20, 11, 1254--1259. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Jiaya Jia, Jian Sun, Chikeung Tang, and Heungyeung Shum. 2006. Drag-and-drop pasting. In Proceedings of ACM SIGGRAPH 2006 Papers. 631--637. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Weisi Lin and C. Jay Kuo. 2011. Perceptual visual quality metrics: A survey. Journal of Visual Communication and Image Representation 22, 4, 297--312. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Tie Liu, Jian Sun, Nanning Zheng, Xiaoou Tang, and Heungyeung Shum. 2007. Learning to detect a salient object. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vol. 33. 1--8.Google ScholarGoogle ScholarCross RefCross Ref
  23. Cewu Lu, Li Xu, and Jiaya Jia. 2014. Contrast preserving decolorization with perception-based quality metrics. International Journal of Computer Vision 110, 2, 222--239. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Anush Moorthy and Alan Bovik. 2011. Blind image quality assessment: From natural scene statistics to perceptual quality. IEEE Transactions on Image Processing 20, 12, 3350--3364. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Patrick Perez, Michel Gangnet, and Andrew Blake. 2003. Poisson image editing. In Proceedings of ACM SIGGRAPH 2003 Papers. 313--318. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Yael Pritch, Eitam Kav-Venaki, and Shmuel Peleg. 2009. Shift-Map image editing. In Proceedings of the 12th International Conference on Computer Vision. 151--158.Google ScholarGoogle Scholar
  27. Carsten Rother, Vladimir Kolmogorov, and Andrew Blake. 2004. “GrabCut”—interactive foreground extraction using iterated graph cuts. In Proceedings of ACM SIGGRAPH 2004 Papers. 309--314. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. M. Rubinstein, D. Gutierrez, O. Sorkine, and A. Shamir. 2010. A comparative study of image retargeting. ACM Transactions on Graphics 29, 5, 160:1--160:10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Bryan Russell, Antonio Torralba, Kevin Murphy, and William Freeman. 2008. LabelMe: A database and Web-based tool for image annotation. International Journal of Computer Vision 77, 1--3, 157--173. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Ariel Shamir and Olga Sorkine. 2009. Visual media retargeting. In Proceedings of ACM SIGGRAPH Asia 2009 Courses. 1--13. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. X. Shen, C. Zhou, L. Xu, and J. Jia. 2015. Mutual-structure for joint filtering. In Proceedings of the 2015 IEEE International Conference on Computer Vision. 3406--3414. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. D. Simakov, Y. Caspi, E. Shechtman, and M. Irani. 2008. Summarizing visual data using bidirectional similarity. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1--8.Google ScholarGoogle Scholar
  33. Mingli Song, Dacheng Tao, Chun Chen, Xuelong Li, and Chang Chen. 2010. Color to gray: Visual cue preservation. IEEE Transactions on Pattern Analysis and Machine Intelligence 32, 9, 1537--1552. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Z. Su, K. Zeng, L. Liu, B. Li, and X. Luo. 2014. Corruptive artifacts suppression for example-based color transfer. IEEE Transactions on Multimedia 16, 4, 988--999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Jian Sun, Lu Yuan, Jiaya Jia, and Heungyeung Shum. 2005. Image completion with structure propagation. ACM Transactions on Graphics 24, 3, 861--868. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Shaoyan Sun, Wengang Zhou, Qi Tian, and Houqiang Li. 2015. Scalable object retrieval with compact image representation from generic object regions. ACM Transactions on Multimedia Computing, Communications and Applications 12, 2, 29:1--29:21. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Michael Tao, Micah Johnson, and Sylvain Paris. 2013. Error-tolerant image compositing. International Journal of Computer Vision 103, 2, 178--189. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Joseph Tighe and Svetlana Lazebnik. 2013. Superparsing: Scalable nonparametric image parsing with superpixels. International Journal of Computer Vision 101, 2, 329--349. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Zhou Wang, Alan Bovik, Hamid Sheikh, and Eero Simoncelli. 2004. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing 13, 4, 600--612. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Pohung Wu, Chienchi Chen, Jianjiun Ding, Chiyu Hsu, and Yingwun Huang. 2013. Salient region detection improved by principle component analysis and boundary information. IEEE Transactions on Image Processing 22, 9, 3614--3624. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Yulin Xie, Huchuan Lu, and Minghsuan Yang. 2013. Bayesian saliency via low and mid level cues. IEEE Transactions on Image Processing 22, 5, 1689--1698. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Li Xu, Qiong Yan, and Jiaya Jia. 2013. A sparse control model for image and video editing. ACM Transactions on Graphics 32, 6, 197:1--197:10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Yang Yang, Linjun Yang, Gangshan Wu, and Shipeng Li. 2012. A bag-of-objects retrieval model for Web image search. In Proceedings of the 20th ACM International Conference on Multimedia. 49--58. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Kun Zeng, Mingtian Zhao, Caiming Xiong, and Songchun Zhu. 2009. From image parsing to painterly rendering. ACM Transactions on Graphics 29, 1, 1--11. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Fanglue Zhang, Mingming Cheng, Jiaya Jia, and Shimin Hu. 2012. ImageAdmixture: Putting together dissimilar objects from groups. IEEE Transactions on Visualization and Computer Graphics 18, 11, 1849--1857. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Fanglue Zhang, Miao Wang, and Shimin Hu. 2013. Aesthetic image enhancement by dependence-aware object recomposition. IEEE Transactions on Multimedia 15, 7, 1480--1490. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Mingtian Zhao and Songchun Zhu. 2013. Abstract painting with interactive control of perceptual entropy. ACM Transactions on Applied Perception 10, 1, 1--21. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Wang Zhou and Li Qiang. 2011. Information content weighting for perceptual image quality assessment. IEEE Transactions on Image Processing 20, 5, 1185--1198. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. A Dual-Domain Perceptual Framework for Generating Visual Inconspicuous Counterparts

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Article Metrics

      • Downloads (Last 12 months)1
      • Downloads (Last 6 weeks)1

      Other Metrics

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!