skip to main content
research-article

Vertical Retargeting for Stereoscopic Images via Stereo Seam Carving

Authors Info & Claims
Published:17 December 2020Publication History
Skip Abstract Section

Abstract

Vertical retargeting for stereoscopic images using seam manipulation-based approaches has remained an open challenge over the years. Even though horizontal retargeting had attracted a huge amount of interest, its seam coupling strategies were not capable to construct valid seam pairs for vertical retargeting. In this article, we propose two seam coupling strategies for vertical retargeting, namely, real mapping and virtual mapping. Our proposed mapping strategies were implemented to address the problems of multiple assignments and missing assignments, which are able to occur in the straightforward generalization from horizontal retargeting to vertical retargeting. On the basis of our proposed method, stereo seams were allowed to lay across occluded regions and occluding regions in stereo images. We maintained the geometric consistency by removing occluded pixels and corresponding occluding pixels in both stereo images. As a result, our method guarantees valid and geometrically consistent stereo seam pairs to be found in the horizontal direction. We generate vertically retargeted stereo images by removing or adding horizontal seam pairs iteratively. We conducted experiments on a number of indoor and outdoor scenes. Experimental results demonstrated that our method overcomes the limitations of vertical retargeting and is effective in preserving the geometric consistency.

References

  1. Shai Avidan and Ariel Shamir. 2007. Seam carving for content-aware image resizing. ACM Trans. Graph. 26, 3 (July 2007), 10. DOI:https://doi.org/10.1145/1276377.1276390Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. B. Bare, K. Li, B. Yan, X. Qi, and H. Gharavi. 2015. Pixel fusion based stereo image retargeting. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME’15). IEEE, Turin, Italy, 1--6. DOI:https://doi.org/10.1109/ICME.2015.7177529Google ScholarGoogle Scholar
  3. Tali Dekel Basha, Yael Moses, and Shai Avidan. 2013. Stereo seam carving a geometrically consistent approach. IEEE Trans. Pattern Anal. Mach. Intell. 35, 10 (2013), 2513--2525.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Li Bing. 2014. Region-based depth-preserving stereoscopic image retargeting. In Proceedings of the IEEE International Conference on Image Processing (ICIP’14). IEEE, Paris, France, 2903--2907.Google ScholarGoogle Scholar
  5. Li Bing, Duan Ling-Yu, Lin Chia-Wen, Huang Tiejun, and Gao Wen. 2015. Depth-preserving warping for stereo image retargeting. IEEE Trans. Image Process. 24, 9 (2015), 2811--2826.Google ScholarGoogle ScholarCross RefCross Ref
  6. Umaima A. Bohari and S. M. Kamalapur. 2015. Efficient 3D image retargeting using stereo seam carving. Int. J. Electron. Commun. Soft Comput. Sci. Eng. (2015), 124.Google ScholarGoogle Scholar
  7. Xiongli Chai, Feng Shao, Qiuping Jiang, and Yo Sung Ho. 2019. Seam manipulator: Leveraging pixel fusion for depth-adjustable stereoscopic image retargeting. IEEE Access 7, 99 (2019), 25239--25252.Google ScholarGoogle ScholarCross RefCross Ref
  8. Che Han Chang, Chia Kai Liang, and Yung Yu Chuang. 2011. Content-aware display adaptation and interactive editing for stereoscopic images. IEEE Trans. Multimedia 13, 4 (2011), 589--601.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Ming Ming Cheng, Guo Xin Zhang, N. J. Mitra, Xiaolei Huang, and Shi Min Hu. 2015. Global contrast based salient region detection. IEEE Trans. Pattern Anal. Mach. Intell. 37, 3 (Mar. 2015), 569--582.Google ScholarGoogle ScholarCross RefCross Ref
  10. Sang-Hyun Cho and Hang-Bong Kang. 2013. A new visual comfort-based stereoscopic image retargeting method. In Advances in Visual Computing, George Bebis, Richard Boyle, Bahram Parvin, Darko Koracin, Baoxin Li, Fatih Porikli, Victor Zordan, James Klosowski, Sabine Coquillart, Xun Luo, Min Chen, and David Gotz (Eds.). Springer, Berlin, 290--300.Google ScholarGoogle Scholar
  11. Yuming Fang, Zhenzhong Chen, Weisi Lin, and Chia-Wen Lin. 2012. Saliency detection in the compressed domain for adaptive image retargeting. IEEE Trans. Image Process. 21, 9 (Sep 2012), 3888--3901.Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Yuming Fang, Junle Wang, Yuan Yuan, Jianjun Lei, Weisi Lin, and Patrick Le Callet. 2016. Saliency-based stereoscopic image retargeting. Info. Sci. 372 (2016), 347--358.Google ScholarGoogle Scholar
  13. Shao Feng, Zhutuan Li, Qiuping Jiang, Gangyi Jiang, Yu Mei, and Zongju Peng. 2015. Visual discomfort relaxation for stereoscopic 3D images by adjusting zero-disparity plane for projection. Displays 39 (2015), 125--132.Google ScholarGoogle ScholarCross RefCross Ref
  14. Andrea Fusiello, Emanuele Trucco, and Alessandro Verri. 2000. A compact algorithm for rectification of stereo pairs. Mach. Vision Appl. 12, 1 (2000), 16--22.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Yongyi Gong, Shangru Li, Kanoksak Wattanachote, and Xiaonan Luo. 2019. Advanced stereo seam carving by considering occlusions on both sides. ACM Trans. Multimedia Comput. Commun. Appl. 15, 3 (Aug. 2019), 1--21. DOI:https://doi.org/10.1145/3321513Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Benjamin Guthier, Johannes Kiess, Stephan Kopf, and Wolfgang Effelsberg. 2013. Seam carving for stereoscopic video. In Proceedings of the IEEE Image, Video, and Multidimensional Signal Processing Workshop (IVMSP’13). IEEE, Seoul, South Korea, 1--4.Google ScholarGoogle ScholarCross RefCross Ref
  17. Benjamin Guthier, Johannes Kiess, Stephan Kopf, and Wolfgang Effelsberg. 2013. Stereoscopic seam carving with temporal consistency. Tech. Rep. 13 (2013).Google ScholarGoogle Scholar
  18. Hirschmüller Heiko. 2007. Stereo processing by semiglobal matching and mutual information. IEEE Trans. Pattern Anal. Mach. Intell. 30, 2 (2007), 328--341.Google ScholarGoogle Scholar
  19. Heiko Hirschmüller and Daniel Scharstein. 2007. Evaluation of cost functions for stereo matching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 1--8.Google ScholarGoogle ScholarCross RefCross Ref
  20. Seung-Won Jung. 2013. A modified model of the just noticeable depth difference and its application to depth sensation enhancement. IEEE Trans. Image Process. 22, 10 (Oct. 2013), 3892--3903.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Ken Yi Lee, Cheng Da Chung, and Yung Yu Chuang. 2012. Scene warping: Layer-based stereoscopic image resizing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 49--56.Google ScholarGoogle Scholar
  22. Jianjun Lei, Shiqiu Li, Bingren Wang, Kefeng Fan, and Chunping Hou. 2014. Stereoscopic visual attention guided disparity controlfor multiview images. J. Display Technol. 10, 5 (2014), 373--379.Google ScholarGoogle ScholarCross RefCross Ref
  23. Jianjun Lei, Wu Min, Changqing Zhang, Wu Feng, Nam Ling, and Chunping Hou. 2017. Depth-preserving stereo image retargeting based on pixel fusion. IEEE Trans. Multimedia 19, 7 (2017), 1--1.Google ScholarGoogle ScholarCross RefCross Ref
  24. Jianjun Lei, Cuicui Zhang, Yuming Fang, Zhouye Gu, Nam Ling, and Chunping Hou. 2015. Depth sensation enhancement for multiple virtual view rendering. IEEE Trans. Multimedia 17, 4 (2015), 457--469.Google ScholarGoogle ScholarCross RefCross Ref
  25. Kuo Chin Lien and Matthew Turk. 2015. On preserving structure in stereo seam carving. In Proceedings of the International Conference on 3D Vision. IEEE, 571--579.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Shih Syun Lin, Chao Hung Lin, Shu Huai Chang, and Tong Yee Lee. 2014. Object-coherence warping for stereoscopic image retargeting. IEEE Trans. Circ. Syst. Video Technol. 24, 5 (2014), 759--768.Google ScholarGoogle ScholarCross RefCross Ref
  27. Yuqing Lin, Yuzhen Niu, Shuai Zhang, and Junhao Chen. 2017. Anisotropic energy accumulation for stereoscopic image seam carving. In Proceedings of the International Conference on 3D Immersion (IC3D). IEEE, 1--7.Google ScholarGoogle ScholarCross RefCross Ref
  28. Dawei Lu, Huadong Ma, and Liang Liu. 2016. Visually preserving stereoscopic image retargeting using depth carving. J. Electron. Imag. 25, 2 (2016), 023029.Google ScholarGoogle ScholarCross RefCross Ref
  29. Ken Nakayama and Shinsuke Shimojo. 1990. Da Vinci stereopsis: Depth and subjective occluding contours from unpaired image points. Vision Res. 30, 11 (Jan. 1990), 1811--1825.Google ScholarGoogle ScholarCross RefCross Ref
  30. Hai Thanh Nguyen and Chee Sun Won. 2013. Stereo video retargeting with representative seams in a group of stereoscopic frames. ETRI J. 35, 6 (2013), 980--989.Google ScholarGoogle ScholarCross RefCross Ref
  31. Changjae Oh, Bumsub Ham, Sunghwan Choi, and Kwanghoon Sohn. 2015. Visual fatigue relaxation for stereoscopic video via nonlinear disparity remapping. IEEE Trans. Broadcast. 61, 2 (2015), 142--153.Google ScholarGoogle ScholarCross RefCross Ref
  32. Hanje Park, Hoonjae Lee, and Sanghoon Sull. 2014. Efficient viewer-centric depth adjustment based on virtual fronto-parallel planar projection in stereo 3D images. IEEE Trans. Multimedia 16, 2 (2014), 326--336.Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Diptiben Patel and Shanmuganathan Raman. 2019. Object occlusion guided stereo image retargeting. Pattern Recogn. Lett. 125 (July 2019), 798--805. DOI:https://doi.org/10.1016/j.patrec.2019.07.018Google ScholarGoogle ScholarCross RefCross Ref
  34. Shaoyu Qi, Yu Tseh Jason Chi, Adrian M. Peter, and Jeffrey Ho. 2016. CASAIR: Content and shape-aware image retargeting and its applications. IEEE Trans. Image Process. 25, 5 (2016), 2222--2232.Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Michael Rubinstein, Diego Gutierrez, Olga Sorkine, and Ariel Shamir. 2010. A comparative study of image retargeting. ACM Trans. Graph. 29, 6 (Dec. 2010), 1. DOI:https://doi.org/10.1145/1882261.1866186Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Michael Rubinstein, Ariel Shamir, and Shai Avidan. 2008. Improved seam carving for video retargeting. ACM Trans. Graph. 27, 3 (2008), 1.Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Feng Shao, Weisi Lin, Zhutuan Li, Gangyi Jiang, and Qionghai Dai. 2017. Toward simultaneous visual comfort and depth sensation optimization for stereoscopic 3-D experience. IEEE Trans. Cybern. 47, 12 (2017), 4521--4533.Google ScholarGoogle ScholarCross RefCross Ref
  38. Feng Shao, Wenchong Lin, Weisi Lin, Gangyi Jiang, Mei Yu, and Randi Fu. 2015. Stereoscopic visual attention guided seam carving for stereoscopic image retargeting. J. Display Technol. 12, 1 (2015), 22--30.Google ScholarGoogle ScholarCross RefCross Ref
  39. F. Shao, W. Lin, W. Lin, Q. Jiang, and G. Jiang. 2017. QoE-guided warping for stereoscopic image retargeting. IEEE Trans. Image Process. 26, 10 (2017), 4790--4805.Google ScholarGoogle ScholarCross RefCross Ref
  40. Feng Shao, Libo Shen, Qiuping Jiang, Fucui Li, and Yo-Sung Ho. 2019. User controllable content retargeting and depth adaptation for stereoscopic display. IEEE Access 7 (2019), 22541--22553.Google ScholarGoogle ScholarCross RefCross Ref
  41. Kei Utsugi, Takuma Shibahara, Takafumi Koike, Keita Takahashi, and Takeshi Naemura. 2010. Seam carving for stereo images. In Proceedings of the 3DTV-Conference: The True Vision-Capture, Transmission and Display of 3D Video. IEEE, 1--4.Google ScholarGoogle ScholarCross RefCross Ref
  42. Junle Wang, Yuming Fang, Manish Narwaria, Weisi Lin, and Patrick Le Callet. 2014. Stereoscopic image retargeting based on 3D saliency detection. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP’14). IEEE, 669--673.Google ScholarGoogle ScholarCross RefCross Ref
  43. K. Wattanachote, T. K. Shih, W. Chang, and H. Chang. 2015. Tamper detection of JPEG image due to seam modifications. IEEE Trans. Info. Forensics Secur. 10, 12 (2015), 2477--2491.Google ScholarGoogle ScholarCross RefCross Ref
  44. Bin Yue, Chun Ping Hou, and Yuan Zhou. 2013. Improved seam carving for stereo image resizing. EURASIP J. Wireless Commun. Netw. 2013, 1 (2013), 116.Google ScholarGoogle ScholarCross RefCross Ref
  45. Dengyong Zhang, Ting Yin, Gaobo Yang, Ming Xia, Leida Li, and Xingming Sun. 2017. Detecting image seam carving with low scaling ratio using multi-scale spatial and spectral entropies. J. Vis. Commun. Image Represent. 48 (2017), 281--291. DOI:https://doi.org/10.1016/j.jvcir.2017.07.006Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Yin Zhao, Zhenzhong Chen, Ce Zhu, Yap Peng Tan, and Lu Yu. 2010. Binocular just-noticeable-difference model for stereoscopic images. IEEE Signal Process. Lett. 18, 1 (2010), 19--22.Google ScholarGoogle ScholarCross RefCross Ref
  47. Yinzuo Zhou, Luming Zhang, Zhang Chao, Li Ping, and Xuelong Li. 2018. Perceptually aware image retargeting for mobile devices. IEEE Trans. Image Process. 27, 5 (2018), 2301--2313.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Vertical Retargeting for Stereoscopic Images via Stereo Seam Carving

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Article Metrics

      • Downloads (Last 12 months)16
      • Downloads (Last 6 weeks)0

      Other Metrics

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!