skip to main content
research-article

Advanced Stereo Seam Carving by Considering Occlusions on Both Sides

Authors Info & Claims
Published:20 August 2019Publication History
Skip Abstract Section

Abstract

Stereo image retargeting plays a significant role in the field of image processing, which aims at making major objects as prominent as possible when the resolution of an image is changed, including maintaining disparity and depth information at the same time. Some seam carving methods are proposed to preserve the geometric consistency of the images. However, the regions of occlusion on both sides are not considered properly. In this article, we propose a solution to solve this problem. A new strategy of seams finding is designed by considering occluded and occluding regions on both of the input images, and leaving geometric consistency in both images intact. We also introduced the method of line segment detection and superpixel segmentation to further improve the quality of the images. Imaging effects are optimized in the process and visual comfort, which is also influenced by other factors, can be boosted as well.

References

  1. Shai Avidan and Ariel Shamir. 2007. Seam carving for content-aware image resizing. In ACM Transactions on Graphics (TOG), Vol. 26. ACM, 10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Bahetiyaer Bare, Ke Li, Bo Yan, Xiaoyu Qi, and Hamid Gharavi. 2015. Pixel fusion based stereo image retargeting. In IEEE International Conference on Multimedia and Expo (ICME’15). IEEE, 1--6.Google ScholarGoogle ScholarCross RefCross Ref
  3. Connelly Barnes, Eli Shechtman, Adam Finkelstein, and Dan B Goldman. 2009. PatchMatch: A randomized correspondence algorithm for structural image editing. ACM Trans. Graph. 28, 3 (2009), 24--1. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Tali Dekel Basha, Yael Moses, and Shai Avidan. 2013. Stereo seam carving a geometrically consistent approach. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 10 (2013), 2513--2525. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Yanxiang Chen, Yifei Pan, Minglong Song, and Meng Wang. 2015. Improved seam carving combining with 3D saliency for image retargeting. Neurocomputing 151 (2015), 645--653.Google ScholarGoogle ScholarCross RefCross Ref
  6. Zaiqing Chen, Junsheng Shi, Xiaoqiao Huang, Lijun Yun, and Yonghan Tai. 2014. Visual comfort modeling for disparity in 3D contents based on Weber--Fechner’s Law. Journal of Display Technology 10, 12 (2014), 1001--1009.Google ScholarGoogle ScholarCross RefCross Ref
  7. Sang-Hyun Cho and Hang-Bong Kang. 2013. A new visual comfort-based stereoscopic image retargeting method. In International Symposium on Visual Computing. Springer, 290--300. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Yuming Fang, Junle Wang, Yuan Yuan, Jianjun Lei, Weisi Lin, and Patrick Le Callet. 2016. Saliency-based stereoscopic image retargeting. Information Sciences 372 (2016), 347--358. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Andrea Fusiello, Emanuele Trucco, and Alessandro Verri. 2000. A compact algorithm for rectification of stereo pairs. Machine Vision and Applications 12, 1 (2000), 16--22. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Yongyi Gong, Guan Boliang, and et al. 2016. Stereo seam carving for vertical retargeting. Chinese Journal of Computers (2016).Google ScholarGoogle Scholar
  11. Heiko Hirschmuller. 2008. Stereo processing by semiglobal matching and mutual information. IEEE Transactions on Pattern Analysis and Machine Intelligence 30, 2 (2008), 328--341. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Heiko Hirschmuller and Daniel Scharstein. 2007. Evaluation of cost functions for stereo matching. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR’07). IEEE, 1--8.Google ScholarGoogle ScholarCross RefCross Ref
  13. Xiaodi Hou and Liqing Zhang. 2007. Saliency detection: A spectral residual approach. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR’07). IEEE, 1--8.Google ScholarGoogle ScholarCross RefCross Ref
  14. Frédéric Huguet and Frédéric Devernay. 2007. A variational method for scene flow estimation from stereo sequences. In IEEE 11th International Conference on Computer Vision (ICCV’07). IEEE, 1--7.Google ScholarGoogle ScholarCross RefCross Ref
  15. Seung-Won Jung. 2013. A modified model of the just noticeable depth difference and its application to depth sensation enhancement. IEEE Transactions on Image Processing 22, 10 (2013), 3892--3903. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Frank L. Kooi and Alexander Toet. 2004. Visual comfort of binocular and 3D displays. Displays 25, 2-3 (2004), 99--108.Google ScholarGoogle ScholarCross RefCross Ref
  17. Ken-Yi Lee, Cheng-Da Chung, and Yung-Yu Chuang. 2012. Scene warping: Layer-based stereoscopic image resizing. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR’12). IEEE, 49--56. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Jianjun Lei, Min Wu, Changqing Zhang, Feng Wu, Nam Ling, and Chunping Hou. 2017. Depth-preserving stereo image retargeting based on pixel fusion. IEEE Transactions on Multimedia (2017).Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Bing Li, Lingyu Duan, Chia-Wen Lin, and Wen Gao. 2014. Region-based depth-preserving stereoscopic image retargeting. In IEEE International Conference on Image Processing (ICIP’14). IEEE, 2903--2907.Google ScholarGoogle ScholarCross RefCross Ref
  20. Bing Li, Ling-Yu Duan, Chia-Wen Lin, Tiejun Huang, and Wen Gao. 2015. Depth-preserving warping for stereo image retargeting. IEEE Transactions on Image Processing 24, 9 (2015), 2811--2826.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Yuqing Lin, Yuzhen Niu, Shuai Zhang, and Junhao Chen. 2017. Anisotropic energy accumulation for stereoscopic image seam carving. In International Conference on 3D Immersion (IC3D’17). IEEE, 1--7.Google ScholarGoogle ScholarCross RefCross Ref
  22. Yi Liu, Lifeng Sun, and Shiqiang Yang. 2013. Disparity-constrained resolution adaptation for stereoscopic images. In Pacific-Rim Conference on Multimedia. Springer, 13--24. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Ken Nakayama and Shinsuke Shimojo. 1990. Da Vinci stereopsis: Depth and subjective occluding contours from unpaired image points. Vision Research 30, 11 (1990), 1811--1825.Google ScholarGoogle ScholarCross RefCross Ref
  24. Jianteng Peng, Jianbing Shen, and Xuelong Li. 2016. High-order energies for stereo segmentation. IEEE Transactions on Cybernetics 46, 7 (2016), 1616--1627.Google ScholarGoogle ScholarCross RefCross Ref
  25. Shaoyu Qi and Jeffrey Ho. 2012. Shift-map based stereo image retargeting with disparity adjustment. In Asian Conference on Computer Vision. Springer, 457--469. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Vidya Setlur, Saeko Takagi, Ramesh Raskar, Michael Gleicher, and Bruce Gooch. 2005. Automatic image retargeting. In Proceedings of the 4th International Conference on Mobile and Ubiquitous Multimedia. ACM, 59--68. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Feng Shao, Wenchong Lin, Weisi Lin, Gangyi Jiang, Mei Yu, and Randi Fu. 2016. Stereoscopic visual attention guided seam carving for stereoscopic image retargeting. Journal of Display Technology 12, 1 (2016), 22--30.Google ScholarGoogle ScholarCross RefCross Ref
  28. Feng Shao, Wenchong Lin, Weisi Lin, Qiuping Jiang, and Gangyi Jiang. 2017. QoE-guided warping for stereoscopic image retargeting. IEEE Transactions on Image Processing 26, 10 (2017), 4790--4805.Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Feng Shao, Libo Shen, Qiuping Jiang, Randi Fu, and Gangyi Jiang. 2017. StereoEditor: Controllable stereoscopic display by content retargeting. Optics Express 25, 26 (2017), 33202--33215.Google ScholarGoogle ScholarCross RefCross Ref
  30. Kei Utsugi, Takuma Shibahara, Takafumi Koike, Keita Takahashi, and Takeshi Naemura. 2010. Seam carving for stereo images. In 3DTV-Conference: The True Vision-Capture, Transmission and Display of 3D Video (3DTV-CON), 2010. IEEE, 1--4.Google ScholarGoogle Scholar
  31. Rafael Grompone Von Gioi, Jérémie Jakubowicz, Jean-Michel Morel, and Gregory Randall. 2012. LSD: A line segment detector. Image Processing On Line 2 (2012), 35--55.Google ScholarGoogle ScholarCross RefCross Ref
  32. Qi Wang and Yuan Yuan. 2014. Learning to resize image. Neurocomputing 131 (2014), 357--367. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Qi Wang, Yuan Yuan, and Pingkun Yan. 2013. Visual saliency by selective contrast. IEEE Transactions on Circuits and Systems for Video Technology 23, 7 (2013), 1150--1155. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Wenguan Wang, Jianbing Shen, Yizhou Yu, and Kwan-Liu Ma. 2017. Stereoscopic thumbnail creation via efficient stereo saliency detection. IEEE Transactions on Visualization and Computer Graphics 23, 8 (2017), 2014--2027.Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Kanoksak Wattanachote, Timothy K. Shih, Wen-Lung Chang, and Hon-Hang Chang. 2015. Tamper detection of JPEG image due to seam modifications. IEEE Transactions on Information Forensics and Security 10, 12 (2015), 2477--2491.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Weiqing Yan, Chunping Hou, Baoliang Wang, and Laihua Wang. 2017. Content-aware disparity adjustment for different stereo displays. Multimedia Tools and Applications 76, 8 (2017), 10465--10479. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Weiqing Yan, Chunping Hou, Yuan Zhou, and Wei Xiang. 2016. Content-and disparity-adaptive stereoscopic image retargeting. Journal of Modern Optics 63, 3 (2016), 207--218.Google ScholarGoogle ScholarCross RefCross Ref
  38. Ting Yin, Gaobo Yang, Leida Li, Dengyong Zhang, and Xingming Sun. 2015. Detecting seam carving based image resizing using local binary patterns. Computers 8 Security 55 (2015), 130--141. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Jin Woo Yoo, Sehoon Yea, and In Kyu Park. 2013. Content-driven retargeting of stereoscopic images. IEEE Signal Processing Letters 20, 5 (2013), 519--522.Google ScholarGoogle ScholarCross RefCross Ref
  40. Dengyong Zhang, Ting Yin, Gaobo Yang, Ming Xia, Leida Li, and Xingming Sun. 2017. Detecting image seam carving with low scaling ratio using multi-scale spatial and spectral entropies. Journal of Visual Communication and Image Representation 48 (2017), 281--291. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Advanced Stereo Seam Carving by Considering Occlusions on Both Sides

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format .

      View HTML Format
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!