skip to main content
research-article

Probability Model-Based Early Merge Mode Decision for Dependent Views Coding in 3D-HEVC

Published:01 October 2018Publication History
Skip Abstract Section

Abstract

As a 3D extension to the High Efficiency Video Coding (HEVC) standard, 3D-HEVC was developed to improve the coding efficiency of multiview videos. It inherits the prediction modes from HEVC, yet both Motion Estimation (ME) and Disparity Estimation (DE) are required for dependent views coding. This improves coding efficiency at the cost of huge computational costs. In this article, an early Merge mode decision approach is proposed for dependent texture views and dependent depth maps coding in 3D-HEVC based on priori and posterior probability models. First, the priori probability model is established by exploiting the hierarchical and interview correlations from those previously encoded blocks. Second, the posterior probability model is built by using the Coded Block Flag (CBF) of the current coding block. Finally, the joint priori and posterior probability model is adopted to early terminate the Merge mode decision for both dependent texture views and dependent depth maps coding. Experimental results show that the proposed approach saves 45.2% and 30.6% encoding time on average for dependent texture views and dependent depth maps coding while maintaining negligible loss of coding efficiency, respectively.

References

  1. G. Bjontegaard. 2001. Calculation of average PSNR differences between RD curves. no. ITU-T SC16/Q6, VCEG-M33, Austin, USA (April 2001).Google ScholarGoogle Scholar
  2. H. Chen, C. H. Fu, Y. Zhang, Y. L. Chan, and W. C. Siu. 2017. Early merge mode decision for depth maps in 3D-HEVC. In Proceedings of the 22nd International Conference on Digital Signal Processing (DSP) (Aug. 2017), 1--5.Google ScholarGoogle Scholar
  3. X. Ding, Y. Li, M. Xia, J. He, and G. Yang. 2018. Detection of motion compensated frame interpolation via motion-aligned temporal difference. Multimedia Tools and Applications (Aug. 2018).Google ScholarGoogle Scholar
  4. Q. Hu, X. Zhang, Z. Shi, and Z. Gao. 2016. Neyman-pearson-based early mode decision for HEVC encoding. IEEE Transactions on Multimedia 18, 3 (March 2016), 379--391. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. S. Jung and H. W. Park. 2016. A fast mode decision method in HEVC using adaptive ordering of modes. IEEE Transactions on Circuits and Systems for Video Technology 26, 10 (Oct. 2016), 1846--1858. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. L. Lei, J. Duan, F. Wu, N. Ling, and C. Hou. 2018. Fast mode decision based on grayscale similarity and inter-view correlation for depth map coding in 3D-HEVC. IEEE Transactions on Circuits and Systems for Video Technology 28, 3 (March 2018), 706--718.Google ScholarGoogle ScholarCross RefCross Ref
  7. Y. Li, G. Yang, N. Chen, Y. Zhu, and X. Ding. 2016. Early DIRECT mode decision for MVC using MB mode homogeneity and RD cost correlation. IEEE Transactions on Broadcasting 62, 3 (May 2016), 700--708.Google ScholarGoogle ScholarCross RefCross Ref
  8. Y. Li, G. Yang, Y. Zhu, X. Ding, and X. Sun. 2017. Adaptive inter CU depth decision for HEVC using optimal selection model and encoding parameters. IEEE Transactions on Broadcasting 63, 3 (Sept. 2017), 535--546.Google ScholarGoogle ScholarCross RefCross Ref
  9. Y. Li, G. Yang, Y. Zhu, X. Ding, and X. Sun. 2017. Unimodal stopping model-based early SKIP mode decision for high efficiency video coding. IEEE Transactions on Multimedia 19, 7 (July 2017), 1431--1441.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Y. Li, G. Yang, Y. Zhu, C. Liu, and K. Liu. 2016. Adaptive mode decision for multiview video coding based on macroblock position constraint model. Journal of Real-Time Image Processing 12, 3 (Oct. 2016), 575--582. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. K. Müller and A. Vetro. 2011. Common test conditions of 3DV core experiments. document JCT3V-G1100, San Jose, CA, USA (2011).Google ScholarGoogle Scholar
  12. Z. Pan, J. Lei, Y. Zhang, and F. Wang. 2018. Adaptive fractional-pixel motion estimation skipped algorithm for efficient HEVC motion estimation. ACM Transactionson Multimedia Computing and Communications Applications 14, 1 (Jan. 2018), Article 12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Z. Pan, S. Kwong, M.-T. Sun, and J. Lei. 2014. Early merge mode decision based on motion estimation and hierarchical depth correlation for HEVC. IEEE Transactions on Broadcasting 60, 2 (June 2014), 405--412.Google ScholarGoogle ScholarCross RefCross Ref
  14. Z. Pan, Y. Zhang, and S. Kwong. 2015. Efficient motion and disparity estimation optimization for low complexity multiview video coding. IEEE Transactions on Broadcasting 61, 2 (June 2015), 166--176.Google ScholarGoogle ScholarCross RefCross Ref
  15. Z. Pan, Y. Zhang, J. Lei, L. Xu, and X. Sun. 2016. Early DIRECT mode decision based on all-zero block and rate distortion cost for multiview video coding. IET Image Processing 10, 1 (Jan. 2016), 9--15.Google ScholarGoogle ScholarCross RefCross Ref
  16. L. Shen, P. An, Z. Zhang, Q. Hu, and Z. Chen. 2015. A 3D-HEVC fast mode decision algorithm for real-time applications. ACM Transactions on Multimedia Computing and Communications Applications 11, 3 (Jan. 2015), Article 34. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. L. Shen, Z. Liu, R. Ma, P. An, and Z. Zhang. 2011. Low-complexity mode decision for MVC. IEEE Transactions on Circuits and Systems for Video Technology 21, 6 (June 2011), 837--843. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. L. Shen, Z. Liu, Z. Zhang, S. Liu, and P. An. 2009. Selective disparity estimation and variable size motion estimation based on motion homogeneity for multi-view coding. IEEE Transactions on Broadcasting 55, 4 (Dec. 2009), 761--766.Google ScholarGoogle Scholar
  19. L. Shen, Z. Liu, T. Yan, Z. Zhang, and P. An. 2010. Early SKIP mode decision for MVC using inter-view correlation. Signal Processing: Image Communication 25, 2 (Feb. 2010), 88--93. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. L. Shen, Z. Zhang, and Z. Liu. 2014. Adaptive inter-mode decision for HEVC jointly utilizing inter-level and spatiotemporal correlations. IEEE Transactions on Circuits and Systems for Video Technology 24, 10 (Oct. 2014), 1709--1722.Google ScholarGoogle Scholar
  21. G. J. Sullivan, J. Ohm, W.-J. Han, and T. Wiegand. 2012. Overview of the high efficiency video coding (HEVC) standard. IEEE Transactions on Circuits and Systems for Video Technology 22, 12 (Dec. 2012), 1649--1668. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. M. Tanimoto, M. P. Tehrani, T. Fujii, and T. Yendo. 2011. Free-viewpoint TV. IEEE Signal Processing Magazine 28, 1 (Jan. 2011), 67--76.Google ScholarGoogle ScholarCross RefCross Ref
  23. J. Tariq, S. Kwong, and H. Yuan. 2017. Spatial/temporal motion consistency based MERGE mode early decision for HEVC. Journal of Visual Communication and Image Representation 44 (April 2017), 198--213. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. G. Tech, Y. Chen, K. Müller, J.-R. Ohm, A. Vetro, and Y. Wang. 2016. Overview of the multiview and 3D extensions of high efficiency video coding. IEEE Transactions on Circuits and Systems for Video Technology 26, 1 (Jan. 2016), 35--49.Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. A. Vetro, A. M. Tourapis, K. Muller, and T. Chen. 2011. 3D-TV content storage and transmission. IEEE Transactions on Broadcasting 52, 7 (June 2011), 384--394.Google ScholarGoogle Scholar
  26. A. Vetro, T. Wiegand, and G. J. Sullivan. 2011. Overview of the stereo and multi-view video coding extensions of the H. 264/MPEG-4 AVC standard. Proceedings of the IEEE 99, 4 (April 2011), 626--642.Google ScholarGoogle ScholarCross RefCross Ref
  27. F. Wang, H. Zeng, Q. Shen, and S. Du. 2013. Efficient early direct mode decision for multi-view video coding. Signal Processing: Image Communication 28, 7 (Aug. 2013), 736--744. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. T. Wiegand, G. J. Sullivan, G. Bjontegaard, and A. Luthra. 2003. Overview of the H.264/AVC video coding standard. IEEE Transactions on Circuits and Systems for Video Technology 13, 7 (Aug. 2003), 560--576. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Y. X. Song and K. B. Jia. 2015. Early merge mode decision for texture coding in 3D-HEVC. Journal of Visual Communication and Image Representation 33 (Nov. 2015), 60--68. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. M. Xia, G. Yang, L. Li, R. Li, and X. Sun. 2017. Detecting video frame rate up-conversion based on frame-level analysis of average texture variation. Multimedia Tools and Applications 76, 6 (March 2017), 8399--8421. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. J. Yang, J. Kim, K. Won, H. Lee, and B. Jeon. 2011. Early SKIP detection for HEVC. document JCTVC-G543, JCT-VC, Geneva, Switzerland (2011).Google ScholarGoogle Scholar
  32. H. Zeng, X. Wang, J. Chen, C. Cai, and Y. Zhang. 2014. Fast multiview video coding using adaptive prediction structure and hierarchical mode decision. IEEE Transactions on Circuits and Systems for Video Technology 24, 9 (March 2014), 1566--1578.Google ScholarGoogle Scholar
  33. D. Zhang, T. Yinand, G. Yang, M. Xia, L. Li, and X. Sun. 2017. Detecting image seam carving with low scaling ratio using multi-scale spatial and spectral entropies. Journal of Visual Communication and Image Representation 48 (Aug. 2017), 281--291. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. J. Zhang, B. Li, and H. Li. 2016. An efficient fast mode decision method for inter prediction in HEVC. IEEE Transactions on Circuits and Systems for Video Technology 28, 6 (Aug. 2016), 1502--1515.Google ScholarGoogle Scholar
  35. N. Zhang, D. Zhao, Y.-W. Chen, J.-L. Lin, and W. Gao. 2014. Fast encoder decision for texture coding in 3D-HEVC. Signal Processing: Image Communication 29, 9 (Oct. 2014), 951--961. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Q. Zhang, K. Huang, X. Wang, B. Jiang, and Y. Gan. 2017. Efficient multiview video plus depth coding for 3D-HEVC based on complexity classification of the treeblock. Journal of Real-Time Image Processing (May 2017), 1--18.Google ScholarGoogle ScholarCross RefCross Ref
  37. Q. Zhang, Q. Wu, X. Wang, and Y. Gan. 2014. Early SKIP mode decision for three-dimensional high efficiency video coding using spatial and interview correlations. Journal of Electronic Imaging 23, 5 (Oct. 2014), 053017--053024.Google ScholarGoogle ScholarCross RefCross Ref
  38. Q. Zhang, N. Zhang, T. Wei, X. Qian, K. Huang, and Y. Gan. 2017. Fast depth map mode decision based on depth-texture correlation and edge classification for 3D-HEVC. Journal of Visual Communication and Image Representation 45 (May 2017), 170--180. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Y. Zhang, S. Kwong, G. Jiang, X. Wang, and M. Yu. 2012. Statistical early termination model for fast mode decision and reference frame selection in multiview video coding. IEEE Transactions on Broadcasting 58, 1 (Dec. 2012), 10--23.Google ScholarGoogle ScholarCross RefCross Ref
  40. Y. Zhang, S. Kwong, L. Xu, and G. Jiang. 2013. DIRECT mode early decision optimization based on rate distortion cost property and inter-view correlation. IEEE Transactions on Broadcasting 59, 2 (April 2013), 390--398.Google ScholarGoogle ScholarCross RefCross Ref
  41. T. Zhao, S. Kwong, H. Wang, Z. Wang, Z. Pan, and C.-C. J. Kuo. 2013. Multiview coding mode decision with hybrid optimal stopping model. IEEE Transactions on Image Processing 22, 4 (Dec. 2013), 1598--1609. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. W. Zhao, T. Onoye, and T. Song. 2015. Hierarchical structure-based fast mode decision for H.265/HEVC. IEEE Transactions on Circuits and Systems for Video Technology 25, 10 (Oct. 2015), 1651--1664.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Probability Model-Based Early Merge Mode Decision for Dependent Views Coding in 3D-HEVC

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Multimedia Computing, Communications, and Applications
      ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 14, Issue 4
      Special Section on Deep Learning for Intelligent Multimedia Analytics
      November 2018
      221 pages
      ISSN:1551-6857
      EISSN:1551-6865
      DOI:10.1145/3282485
      Issue’s Table of Contents

      Copyright © 2018 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 1 October 2018
      • Accepted: 1 August 2018
      • Revised: 1 May 2018
      • Received: 1 March 2018
      Published in tomm Volume 14, Issue 4

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!