skip to main content
research-article

A novel 3D video transcoding scheme for adaptive 3D video transmission to heterogeneous terminals

Published:16 October 2012Publication History
Skip Abstract Section

Abstract

Three-dimensional video (3DV) is attracting many interests with its enhanced viewing experience and more user driven features. 3DV has several unique characteristics different from 2D video: (1) It has a much larger amount of data captured and compressed, and corresponding video compression techniques can be much more complicated in order to explore data redundancy. This will lead to more constraints on users' network access and computational capability, (2) Most users only need part of the 3DV data at any given time, while the users' requirements exhibit large diversity, (3) Only a limited number of views are captured and transmitted for 3DV. View rendering is thus necessary to generate virtual views based on the received 3DV data. However, many terminal devices do not have the functionality to generate virtual views. To enable 3DV experience for the majority of users with limited capabilities, adaptive 3DV transmission is necessary to extract/generate the required data content and represent it with supported formats and bitrates for heterogeneous terminal devices. 3DV transcoding is an emerging and effective technique to achieve desired adaptive 3DV transmission. In this article, we propose the first efficient 3DV transcoding scheme that can obtain any desired view, either an encoded one or a virtual one, and compress it with more universal H.264/AVC. The key idea of the proposed scheme is to appropriately utilize motion information contained in the bitstream to generate candidate motion information. Original information of both the desired view and reference views are used to obtain this candidate information and a proper motion refinement process is carried out for certain blocks. Simulation results show that, compared to the straightforward cascade algorithm, the proposed scheme is able to output compressed bitstream of the required view with significantly reduced complexity while incurring negligible performance loss. Such a 3DV transcoding can be applied to most gateways that usually have constraints on computational complexity and time delay.

References

  1. Ahmand, I., Wei, X., Sun, Y., and Zhang, Y.-Q. 2005. Video transcoding: An Overview of various techniques and research issues. IEEE Trans. Multimed. 7, 5. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Bai, B., Boulanger, P., and Harms, J. 2005. A multiview video transcoder. In Proceedings of the ACM Multimedia. 503--506. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Cha, J., Eid, M., and Saddik, A. 2009. Touchable 3D video system. ACM Trans. Multimed. Comput. Commun. Appl. 5, 4. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Cheung, G., Ortega, A., and Cheung, N.-M. 2011. Interactive streaming of stored multiview video using redundant frame structures. IEEE Trans. Image Process. 20, 3. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Fehn, C. 2004. Depth-image-based-rendering (DIBR), compression and transmission for a new approach on 3D-TV. In SPIE Stereoscopic Displays and Virtual Reality Systems.Google ScholarGoogle Scholar
  6. Florêncio, D. and Zhang, C. 2009. Multiview video compression and streaming based on predicted viewer position. In Proceedings of ICASSP '09. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. JVT-AA209. 2008. Joint Draft 7.0 on multiview video coding.Google ScholarGoogle Scholar
  8. JVT-T207. 2006. Common Test Conditions for Multiview Video Coding. Klagenfurt, Austria.Google ScholarGoogle Scholar
  9. Kurutepe, E., Civanlar, M. R., and Tekalp, A. M. 2007. Client-driven selective streaming of multiview video for interactive 3DTV. IEEE Trans. Circuite Syst. Video. Technol. 17. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Levoy, M. and Hanrahan, P. 1996. Light field rendering, In Proceedings of SIGGRAPH'96, ACM, pp. 31--42. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Liu, S. and Chen, C. W. 2009. Multiview video transcoding: From multiple views to single view. In Proceedings of the Picture Coding Symposium (PCS'09). Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Liu, S. and Chen, C. W. 2010. 3D video transcoding for virtual views. In Proceedings of ACM Multimedia. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Liu, Y., Huang, Q., Ma, S., Zhao, D., and Gao, W. 2009. Joint video/depth rate allocation for 3D video coding based on view synthesis distortion model. Signal Process. Image Commun. 24, 8. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Liu, S., Lai, P., Tian, D., and Chen, C. W. 2011. New depth coding techniques with utilization of corresponding video. IEEE Trans. Broadcas. 57, 2.Google ScholarGoogle ScholarCross RefCross Ref
  15. Lou, J.-G., Cai, H., and Li, J. 2007. Interactive multiview video delivery based on IP multicast. In Advances in Multimedia. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Maitre, M. and Do, M. N. 2008. Joint encoding of the depth image based representation using shape-adaptive wavelets. In Proceedings of ICIP. 1768--1771Google ScholarGoogle Scholar
  17. Microsoft 3D Video Test Sequences (available online: http://research.microsoft.com/en-us/um/people/sbkang/3dvideodownload/)Google ScholarGoogle Scholar
  18. MPEG Video and Requirements Subgroup. 2009. Applications and requirements on 3D video coding. Document w11061. MPEG.Google ScholarGoogle Scholar
  19. Müller, K., Smolic, A., Dix, K., Merkle, P., Kauff, P., and Wiegand, T. 2008. Reliability-based generation and view synthesis in layered depth video. In Proceedings of the IEEE International Workshop on Multimedia Signal Processing.Google ScholarGoogle Scholar
  20. Oh, H. and Ho, Y.-S. 2006. H.264-based depth map coding using motion information of correspondingtexture video. Adv. Image Video Tech. 4319. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Pan, Z., Ikuta, Y., Bandai, M., and Watanabe, T. 2011. User dependent scheme for multi-view video transmission. In Proceedings of ICC.Google ScholarGoogle Scholar
  22. Shum, H.-Y., Chan, S.-C., and Kang, S. B. 2006. Image-Based Rendering. Springer. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Smolic, A., Müller, K., Dix, K., Merkle, P., Kauff, P., and Wiegand, T. 2008. Intermediate View Interpolation Based on Multiview Video Plus Depth for Advanced 3D Video Systems. In Proceedings of the IEEE International Conference on Image Processing.Google ScholarGoogle Scholar
  24. Smolic, A., Müller, K., Merkele, P., Kauff, P., and Wiegand, T. 2009. An overview of available and emerging 3D video formats and depth enhanced stereo as efficient generic solution. In Proceedings of the Picture Coding Symposium. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Tang, X.-L., Dai, S.-K., and Cai, C.-H. 2010. An analysis of TZSearch algorithm. In Proceedings of ICGCS.Google ScholarGoogle Scholar
  26. Tanimoto, M., Fuji, T., and Suzuki, K. 2009. View synthesis algorithm in view synthesis reference software 3.0 (VSRS3.0), Tech. rep. Document M16090, ISO/IEC JTC1/SC29/WG11.Google ScholarGoogle Scholar
  27. Vetro, A., Matusik, W., Pfister, H., and Xin, J. 2004. Coding approaches for end-to-end 3D TV systems. Proceedings of the Picture Coding Symposium.Google ScholarGoogle Scholar
  28. Yang, Z., Wu, W., Nahrstedt, K., Kurillo G., and Bajscy, R. 2010. Enabling Multi-Party 3D Tele-Immersive Environments with ViewCast. ACM Trans. Multimedia Comput. Commun. Appl. 6, 2. Google ScholarGoogle ScholarCross RefCross Ref
  29. Yang, Y., Yu, M., Jiang, G., and Peng, Z. 2007. A transmission and interaction oriented free-viewpoint video system. Int. J. Circ. Syst. Signal Process. 4, 1.Google ScholarGoogle Scholar
  30. Zhang, C. and Florêncio, D. 2010. Joint tracking and multiview video compression. In Proceedings of VCIP.Google ScholarGoogle Scholar
  31. Zitnick, L., Kang, S. B., Uyttendaele, M., Winder, S., and Szeliski, R. 2004. High-quality video view interpolation using a layered representation. ACM Trans. Graph. 23, 3, 600--608. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. A novel 3D video transcoding scheme for adaptive 3D video transmission to heterogeneous terminals

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Multimedia Computing, Communications, and Applications
      ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 8, Issue 3s
      Special section of best papers of ACM multimedia 2011, and special section on 3D mobile multimedia
      September 2012
      173 pages
      ISSN:1551-6857
      EISSN:1551-6865
      DOI:10.1145/2348816
      Issue’s Table of Contents

      Copyright © 2012 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 16 October 2012
      • Accepted: 1 June 2012
      • Revised: 1 April 2012
      • Received: 1 January 2010
      Published in tomm Volume 8, Issue 3s

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!