skip to main content
research-article

A 3D-HEVC Fast Mode Decision Algorithm for Real-Time Applications

Published:05 February 2015Publication History
Skip Abstract Section

Abstract

3D High Efficiency Video Coding (3D-HEVC) is an extension of the HEVC standard for coding of multiview videos and depth maps. It inherits the same quadtree coding structure as HEVC for both components, which allows recursively splitting into four equal-sized coding units (CU). One of 11 different prediction modes is chosen to code a CU in inter-frames. Similar to the joint model of H.264/AVC, the mode decision process in HM (reference software of HEVC) is performed using all the possible depth levels and prediction modes to find the one with the least rate distortion cost using a Lagrange multiplier. Furthermore, both motion estimation and disparity estimation need to be performed in the encoding process of 3D-HEVC. Those tools achieve high coding efficiency, but lead to a significant computational complexity. In this article, we propose a fast mode decision algorithm for 3D-HEVC. Since multiview videos and their associated depth maps represent the same scene, at the same time instant, their prediction modes are closely linked. Furthermore, the prediction information of a CU at the depth level X is strongly related to that of its parent CU at the depth level X-1 in the quadtree coding structure of HEVC since two corresponding CUs from two neighboring depth levels share similar video characteristics. The proposed algorithm jointly exploits the inter-view coding mode correlation, the inter-component (texture-depth) correlation and the inter-level correlation in the quadtree structure of 3D-HEVC. Experimental results show that our algorithm saves 66% encoder runtime on average with only a 0.2% BD-Rate increase on coded views and 1.3% BD-Rate increase on synthesized views.

References

  1. G. Bjontegaard. 2001. Calculation of average PSNR difference between RD-curves. In Proceedings of the 13th VCEG-M33 Meeting.Google ScholarGoogle Scholar
  2. G. Cernigliaro, F. Jaureguizar, J. Cabrera, and N. Garcia. 2013. Low complexity mode decision and motion estimation for H.264/AVC based depth maps encoding in free viewpoint video. IEEE Trans. Circuits Syst. Video Technol. 23, 5, 769--783. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. G. Correa, P. Assuncao, L. Agostini, and L. A. Da Silva Cruz. 2013. Complexity control of HEVC through quadtree depth estimation. In Proceedings of IEEE EUROCON. 81--86.Google ScholarGoogle Scholar
  4. I. Daribo, C. Tillier, and B. Pesquet-Popescu. 2009. Motion vector sharing and bitrate allocation for 3D video-plus-depth coding. EURASIP J. Applied Signal Process. 1--13. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. M. Domanski and O. Stankiewicz, and, et al. 2013. High efficiency 3D video coding using new tools based on view synthesis. IEEE Trans. Image Process. 22, 9, 3517--3527. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. S. Grewatsch and E. Muller. 2004. Sharing of motion vectors in 3D video coding. In Proceedings of the IEEE 11th International Conference on Image Processing. 3271--3274.Google ScholarGoogle Scholar
  7. Z. Gu, J. Zheng, N. Ling, and P. Zhang. 2013. Fast depth modeling mode selection for 3D HEVC depth intra coding. In Proceedings of the IEEE International Conference on Multimedia and Expo Workshops. 1--4.Google ScholarGoogle Scholar
  8. ISO-N11829. 2011. Applications and requirements on 3D video coding. ISO/IECJTC1/SC29/WG11.Google ScholarGoogle Scholar
  9. Y. V. Lvanov and C. J. Bleakley. 2010. Real-time H.264 video encoding in software with fast mode decision and dynamic complexity control. ACM Trans. Multimedia Comput. Commun. Appl. 6, 1, Article 5. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. JCTVC-F045. 2011. Early termination of CU encoding to reduce HEVC complexity. ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11.Google ScholarGoogle Scholar
  11. JCTVC-F092. 2011. Coding tree pruning based CU early termination. ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11.Google ScholarGoogle Scholar
  12. JCT3V-A0044. 2012. Depth quadtree prediction for HTM. ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11.Google ScholarGoogle Scholar
  13. JCT3V-A1100. 2012. Common test conditions of 3DV core experiments. ITU-T SG16 WP3 & ISO/IECJTC1/SC29/WG11.Google ScholarGoogle Scholar
  14. M. Karczewicz, P. Chen, and et al. 2010. A hybrid video coder based on extended macroblock sizes, improved interpolation, and flexible motion representation. IEEE Trans. Circ. Syst. Video Technol. 20, 12, 1698--1708. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. J. Kim, J. Yang, K. Won, and B. Jeon. 2012. Early determination of mode decision for HEVC. In Proceedings of the Picture Coding Symposium. 449--452.Google ScholarGoogle Scholar
  16. Y. T. Kim, J. Y. Kim, and K. H. Sohn. 2007. Fast disparity and motion estimation for multi-view video coding. IEEE Trans. Consum. Electron. 53, 2, 712--719. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. J. Lee, H. Wey, and D. Park. 2011. A fast and efficient multi-view depth image coding method based on temporal and inter-view correlations of texture images. IEEE Trans. Circ. Syst. Video Technol. 21, 12, 1859--1868.Google ScholarGoogle ScholarCross RefCross Ref
  18. J. Lei, S. Li, C. Zhu, M. Sun, and C. Hou. 2014. Depth coding based on depth-texture motion and structure similarities. IEEE Trans. Circuits System Video Technol. DOI: 10.1109/TCSVT.2014.2335471.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Y. Lin and J. Wu. 2011. A depth information based fast mode decision algorithm for color plus depth-map 3D videos. IEEE Trans. Broadcast. 57, 2, 542--550.Google ScholarGoogle ScholarCross RefCross Ref
  20. S. Ma, S. Wang, S. Wang, L. Zhao, Q. Yu, and W. Gao. 2013. Low complexity rate distortion optimization for HEVC. In Proceedings of the Data Compression Conference. 73--82. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. B. W. Micallef, C. J. Debono, and R. A. Farrugia. 2012. Fast inter-mode decision in multi-view video plus depth coding. In Proceedings of the Picture Coding Symposium. 113--116.Google ScholarGoogle Scholar
  22. E. Mora, J. Jung, M. Cagnazzo, and B. Pesquet-Popescu. 2014. Initialization, limitation and predictive coding of the depth and texture quadtree in 3D-HEVC. IEEE Trans. Circ. Syst. Video Technol. DOI: 10.1109/TCSVT.2013.2283110Google ScholarGoogle Scholar
  23. K. Muller, H. Schwarz, and et al. 2013. 3D High-Efficiency Video Coding for Multi-View Video and Depth Data. IEEE Trans. Image Process. 22, 9, 3366--3378. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. B. Oh and K. Oh. 2014. View synthesis distortion estimation for AVC- and HEVC-compatible 3D video coding. IEEE Trans. Circ. Syst. Video Technol. DOI: 10.1109/TCSVT.2013.2290577Google ScholarGoogle Scholar
  25. H. Oh and Y. S. Ho. 2006. H.264-based depth map sequence coding using motion information of corresponding texture video. In Advances in Image and Video Technology, Springer. 898--907. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. M. T. Pourazad, P. Nasiopoulos, and R. K. Ward. 2006. An H.264-based video encoding scheme for 3DTV. In Proceedings of the European Signal Processing Conference. 1--5.Google ScholarGoogle Scholar
  27. L. Shen, Z. Liu, P. An, R. Ma, and Z. Zhang. 2011. Low-complexity mode decision for MVC. IEEE Trans. Circ. Syst. Video Technol. 21, 6, 837--843. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. L. Shen, Z. Liu, X. Zhang, W. Zhao, and Z. Zhang. 2013. An effective CU size decision method for HEVC encoders. IEEE Trans. Multimedia. 15, 2, 465--470. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. L. Shen, Z. Zhang, and Z. Liu. 2014. Effective CU size decision for HEVC intracoding. IEEE Trans. Image Process. 23, 10, 4232--4241.Google ScholarGoogle ScholarCross RefCross Ref
  30. L. Shen, Z. Zhang, and Z. Liu. 2014. Adaptive inter-mode decision for HEVC jointly utilizing inter-level and spatiotemporal correlations. IEEE Trans. Circ, System Video Technol. 24, 10, 1709--1722.Google ScholarGoogle ScholarCross RefCross Ref
  31. L. Shen, Z. Zhang, and Z. Liu. 2012. Inter mode selection for depth map coding in 3D video. IEEE Trans. Consumer Electron. 58, 3, 926--931.Google ScholarGoogle Scholar
  32. A. Smolic, G. Tech, and H. Brust. 2010. Report on generation of stereo video data base. Mobile3DTV Tech. Rep. 2, 1, 1--15.Google ScholarGoogle Scholar
  33. G. J. Sullivan, J. Ohm, W. J. Han, and T. Wiegand. 2012. Overview of the high efficiency video coding (HEVC) standard. IEEE Trans. Circ. Syst. Video Technol. 22, 12, 1649--1668. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. G. J. Sullivan and T. Wiegand. 1998. Rate-distortion optimization for video compression. IEEE Signal Process. Mag. 15, 6, 74--90.Google ScholarGoogle ScholarCross RefCross Ref
  35. H. Sun, D. Zhou, and S. Goto. 2012. A Low-complexity HEVC intra prediction algorithm based on level and mode filtering. In Proceedings of the IEEE International Conference on Multimedia and Expo. 1085--1090. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. S. Tao, Y. Chen, M. M. Hannuksela, Y. K. Wang, M. Gabbouj, and H. Li. 2009. Joint texture and depth map video coding based on the scalable extension of H.264/AVC. In Proceedings of the IEEE International Symposium on Circuits and Systems. 2353--2356.Google ScholarGoogle Scholar
  37. K. Ugur, K. Andersson, A. Fuldseth, G. Bjontegaard, L. P. Endresen, and et al. 2010. High performance, low complexity video coding and the emerging HEVC standard. IEEE Trans. Circ. Syst. Video Technol. 20, 12, 1688--1697. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. J. Vanne, M. Viitanen, T. D. Hamalainen, and A. Hallapuro. 2012. Comparative rate-distortion-complexity analysis of HEVC and AVC video codecs. IEEE Trans. Circ. Syst. Video Technol. 22, 12, 1885--1898. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. J. Xiong, H. Li, Q. Wu, and F. Meng. 2014. A fast HEVC inter CU selection method based on pyramid motion divergence. IEEE Trans. Multimedia 16, 2, 559--564. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Y. Yan, H. Li, and M. M. Hannuksela. 2013. Multiview-video-plus-depth coding and inter-component prediction in high-level-syntax extension of H.265/HEVC. In Proceedings of the Picture Coding Symposium. 406--409.Google ScholarGoogle Scholar
  41. H. Zhang and Z. Ma. 2013. Early termination schemes for fast intra mode decision in high efficiency video coding. In Proceedings of the IEEE International Symposium on Circuits and Systems. 45--48.Google ScholarGoogle Scholar
  42. J. Zhang, M. M. Hannuksela, and H. Li. 2010. Joint multiview video plus depth coding. In Proceedings of the IEEE International Conference on Image Processing. 2865--2868.Google ScholarGoogle Scholar
  43. N. Zhang, Y. W. Chen, J. L. Lin, X. P. Fan, S. W. Ma, D. B. Zhao, and W. Gao. 2013. Improved disparity vector derivation in 3DHEVC. In Proceedings of the Conference on Visual Communications and Image Processing. 1--5.Google ScholarGoogle Scholar
  44. Q. Zhang, P. An, Y. Zhang, L. Shen, and Z. Zhang. 2011. Low complexity multiview video plus depth coding. IEEE Trans. Consum. Electron. 57, 4, 1857--1865.Google ScholarGoogle ScholarCross RefCross Ref
  45. W. Zhu, X. Tian, F. Zhou, and Y. Chen. 2010. Fast disparity estimation using spatio-temporal correlation of disparity field for multiview video coding. IEEE Trans. Consum. Electron. 56, 2, 957--964. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. W. Zhu, X. Tian, F. Zhou, and Y. Chen. 2010. Fast inter mode decision based on textural segmentation and correlations for multiview video coding. IEEE Trans. Consum. Electron. 56, 3, 1696--1704. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

(auto-classified)
  1. A 3D-HEVC Fast Mode Decision Algorithm for Real-Time Applications

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Multimedia Computing, Communications, and Applications
      ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 11, Issue 3
      January 2015
      173 pages
      ISSN:1551-6857
      EISSN:1551-6865
      DOI:10.1145/2733235
      Issue’s Table of Contents

      Copyright © 2015 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 5 February 2015
      • Accepted: 1 September 2014
      • Revised: 1 May 2014
      • Received: 1 February 2014
      Published in tomm Volume 11, Issue 3

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!