skip to main content
research-article

Energy-efficient multicasting of multiview 3D videos to mobile devices

Published:16 October 2012Publication History
Skip Abstract Section

Abstract

Multicasting multiple video streams over wireless broadband access networks enables the delivery of multimedia content to large-scale user communities in a cost-efficient manner. Three dimensional (3D) videos are the next natural step in the evolution of digital media technologies. In order to provide 3D perception, 3D video streams contain one or more views that greatly increase their bandwidth requirements. Due to the limited channel capacity and variable bit rate of the videos, multicasting multiple 3D videos over wireless broadband networks is a challenging problem. In this article, we consider a 4G wireless access network in which a number of 3D videos represented in two-view plus depth format and encoded using scalable video coders are multicast. We formulate the optimal 3D video multicasting problem to maximize the quality of rendered virtual views on the receivers' displays. We show that this problem is NP-complete and present a polynomial time approximation algorithm to solve it. We then extend the proposed algorithm to efficiently schedule the transmission of the chosen substreams from each video in order to maximize the power saving on the mobile receivers. Our simulation-based experimental results show that our algorithm provides solutions that are within 0.3 dB of the optimal solutions while satisfying real-time requirements of multicast systems. In addition, our algorithm results in an average power consumption reduction of 86%.

References

  1. ABI Research. 2010. 3D Mobile devices. http://www.abiresearch.com/research/1006094.Google ScholarGoogle Scholar
  2. Akar, G., Tekalp, A., Fehn, C., and Civanlar, M. 2007. Transport methods in 3DTV—a survey. IEEE Trans. Circ. Syst. Video Tech. 17, 11, 1622--1630. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Arican, Z., Yea, S., Sullivan, A., and Vetro, A. 2009. Intermediate view generation for perceived depth adjustment of stereo video. Appl. Digital Image Proc. XXXII 7443, 1, 74430U.Google ScholarGoogle Scholar
  4. Bosc, E., Jantet, V., Pressigout, M., Morin, L., and Guillemot, C. 2011. Bit-rate allocation for multi-view video plus depth. In Proceedings of the 3DTV Conference: The True Vision—Capture, Transmission and Display of 3D Video (3DTV-CON). 1--4.Google ScholarGoogle Scholar
  5. Chen, Y., Wang, Y.-K., Ugur, K., Hannuksela, M., Lainema, J., and Gabbouj, M. 2009. The emerging MVC standard for 3D video services. EURASIP J. Adv. Sig. Proc. 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Cheung, G., Velisavljevic, V., and Ortega, A. 2011. On dependent bit allocation for multiview image coding with depth-image-based rendering. IEEE Trans. Image Proc. 20, 11, 3179--3194. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. CPLEX 2011. IBM ILOG CPLEX Optimizer. http://www.ibm.com/software/integration/optimization/cplex-optimizer/.Google ScholarGoogle Scholar
  8. de Diego Balaguer, E., Fitzek, F., Olsen, O., and Gade, M. 2005. Performance evaluation of power saving strategies for DVB-H services using adaptive MPE-FEC decoding. In Proceedings of the IEEE International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC'05). Vol. 4. 2221--2226.Google ScholarGoogle Scholar
  9. De Silva, D., Ekmekcioglu, E., Abdul-Hameed, O., Fernando, W., Worrall, S., and Kondoz, A. 2010. Performance evaluation of 3D-TV transmission over WiMAX broadband access networks. In Proceedings of the International Conference on Information and Automation for Sustainability (ICIAFs). 298--303.Google ScholarGoogle Scholar
  10. Dodgson, N. 2005. Autostereoscopic 3D displays. IEEE Computer 38, 8, 31--36. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Dyer, M. 1984. An O(n) algorithm for the multiple-choice knapsack linear program. Mathemat. Prog. 29, 57--63.Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Fehn, C. 2004. Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV. Stereoscop. Displ. Virt. Reality Syst. XI 5291, 1, 93--104.Google ScholarGoogle Scholar
  13. Gotfryd, M., Wegner, K., and Domański, M. 2008. View Synthesis Software and Assessment of Its Performance. ISO/IEC JTC1/SC29/WG11, MPEG 2008/M15672.Google ScholarGoogle Scholar
  14. Harrison, D. 2010. Hockey Night in Canada offers games in 3D. http://www.cbc.ca/sports/hockey/story/2010/09/29/sp-hockey-night-panasonic.html.Google ScholarGoogle Scholar
  15. Hefeeda, M. and Hsu, C.-H. 2008. Energy optimization in mobile TV broadcast networks. In Proceedings of the International Conference on Innovations in Information Technology. 430--434.Google ScholarGoogle Scholar
  16. Hsu, C.-H. and Hefeeda, M. 2010. Achieving viewing time scalability in mobile video streaming using scalable video coding. In Proceedings of the 1st ACM SIGMM Conference on Multimedia Systems (MMSys'10). 111--122. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Informa Telecoms and Media. 2010. 22.5 million homes will tune into 3D by 2015. http://shop.informatm.com/itmgcontent/icoms/s/press-releases/20017774108.html.Google ScholarGoogle Scholar
  18. Järvinen, K., Bouazizi, I., Laaksonen, L., Ojala, P., and Rämö, A. 2010. Media coding for the next generation mobile system LTE. Comput. Comm. 33, 16, 1916--1927. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. JSVM 2011. Joint Scalable Video Model (JSVM)—Reference Software. http://ip.hhi.de/imagecom_G1/savce/downloads/SVC-Reference-Software.htm.Google ScholarGoogle Scholar
  20. Kauff, P., Atzpadin, N., Fehn, C., Müller, M., Schreer, O., Smolic, A., and Tanger, R. 2007. Depth map creation and image-based rendering for advanced 3DTV services providing interoperability and scalability. Sig. Proc. Image Comm. 22, 2, 217--234. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Kellerer, H., Pferschy, U., and Pisinger, D. 2004. Knapsack Problems. Springer-Verlag.Google ScholarGoogle Scholar
  22. Kim, W., Ortega, A., Lai, P., Tian, D., and Gomila, C. 2010. Depth map coding with distortion estimation of rendered view. Proc. SPIE 7543.Google ScholarGoogle Scholar
  23. Kimata, H., Kitahara, M., Kamikura, K., Yashima, Y., Fujii, T., and Tanimoto, M. 2004. System design of free viewpoint video communication. In Proceedings of the IEEE International Conference on Computer and Information Technology. 52--59. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Kumar, A. 2008. Mobile Broadcasting with WiMAX: Principles, Technology, and Applications. Elsevier Inc. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Liu, Y., Huang, Q., Ma, S., Zhao, D., and Gao, W. 2009. Joint video/depth rate allocation for 3D video coding based on view synthesis distortion model. Signal Proc. Image Comm. 24, 8, 666--681. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. MasterImage. 2012. MasterImage 3D Autostereoscopic 3D LCD. http://masterimage3d.com/products/3d-lcd.Google ScholarGoogle Scholar
  27. Mathew, R. and Taubman, D. 2010. Quad-tree motion modeling with leaf merging. IEEE Trans. Circ. Syst. Video Tech. 20, 10, 1331--1345. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Merkle, P., Smolic, A., Müller, K., and Wiegand, T. 2007. Efficient prediction structures for multiview video coding. IEEE Trans. Circ. Syst. Video Tech. 17, 11, 1461--1473. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Müller, K., Merkle, P., and Wiegand, T. 2011. 3-D video representation using depth maps. Proc. IEEE 99, 4, 643--656.Google ScholarGoogle ScholarCross RefCross Ref
  30. Multimedia Scalable 3D for Europe (MUSCADE) Project. 2010. Mission scenarios and service requirements. http://www. muscade.eu/deliverables/MUS.RP.00001.EBU%20D1.1.1.pdf.Google ScholarGoogle Scholar
  31. Petrovic, G., Do, L., Zinger, S., and de With, P. H. N. 2010. Virtual view adaptation for 3d multiview video streaming. Stereoscopic Displays and Appl. XXI 7524, 1, 752410.Google ScholarGoogle Scholar
  32. Philips Electronics. 2012. Philips WOWvx Autostereoscopic Display. http://www.business-sites.philips.com/3dsolutions/home/index.page.Google ScholarGoogle Scholar
  33. Pilkington, E. 2010. ESPN viewers can watch World Cup matches in 3D—at a price. http://www.guardian.co.uk/technology/2010/jan/06/espn-3d-world-cup-ces.Google ScholarGoogle Scholar
  34. Schwarz, H., Marpe, D., and Wiegand, T. 2007. Overview of the scalable video coding extension of the H.264/AVC standard. IEEE Trans. Circ. Syst. Video Tech. 17, 9, 1103--1120. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Sequans Communications. 2007. Datasheet: SQN1130 System-on-Chip for WiMAX Moblie Stations. http://www.sequans.com/products-solutions/mobile-wimax/sqn1130/.Google ScholarGoogle Scholar
  36. Shao, F., Jiang, G., Yu, M., Chen, K., and Ho, Y.-S. 2012. Asymmetric coding of multi-view video plus depth based 3-D video for view rendering. IEEE Trans. Multimed. 14, 1, 157--167. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Su, G.-M., Lai, Y.-C., Kwasinski, A., and Wang, H. 2011. 3D video communications: Challenges and opportunities. Int. J. Comm. Syst. 24, 10, 1261--1281. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Tanimoto, M., Fujii, T., Suzuki, K., Fukushima, N., and Mori, Y. 2008. Reference Softwares for Depth Estimation and View Synthesis. ISO/IEC JTC1/SC29/WG11, MPEG2008/M15377.Google ScholarGoogle Scholar
  39. Uehara, S., Hiroya, T., Kusanagi, H., Shigemura, K., and Asada, H. 2008. 1-inch diagonal transflective 2D and 3D LCD with HDDP arrangement. In Proc. SPIE. 6803. 68030O.Google ScholarGoogle Scholar
  40. Urey, H., Chellappan, K. V., Erden, E., and Surman, P. 2011. State of the art in stereoscopic and autostereoscopic displays. Proc. IEEE 99, 4.Google ScholarGoogle ScholarCross RefCross Ref
  41. Vetro, A., Matusik, W., Pfister, H., and Xin, J. 2004. Coding approaches for end-to-end 3D TV systems. In Proceedings of the 23rd Picture Coding Symposium (PCS'04). 319--324.Google ScholarGoogle Scholar
  42. Vetro, A., Wiegand, T., and Sullivan, G. 2011. Overview of the stereo and multiview video coding extensions of the H.264/MPEG-4 AVC standard. Proc. IEEE 99, 4, 626--642.Google ScholarGoogle ScholarCross RefCross Ref
  43. Wang, Z., Bovik, A., Sheikh, H., and Simoncelli, E. 2004. Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Proc. 13, 4, 600--612. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Wearden, G. 2010. 3D TV dominates IFA electronics show. http://www.guardian.co.uk/technology/blog/2010/sep/02/3d-television-ifa-2010.Google ScholarGoogle Scholar
  45. Yang, X., Song, Y., Owens, T., Cosmas, J., and Itagaki, T. 2004. Performance analysis of time slicing in DVB-H. In Proceedings of the Joint Ist Workshop on Mobile Future, and the Symposium on Trends in Communications (SympoTIC'04). 183--186.Google ScholarGoogle Scholar
  46. Yuan, G., Zhang, X., Wang, W., and Yang, Y. 2010. Carrier aggregation for LTE-advanced mobile communication systems. IEEE Comm. Mag. 48, 2, 88--93. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Yuan, H., Chang, Y., Huo, J., Yang, F., and Lu, Z. 2011. Model-based joint bit allocation between texture videos and depth maps for 3-D video coding. IEEE Trans. Circ. Syst. Video Tech. 21, 4, 485--497. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Zemel, E. 1984. An O(n) algorithm for the linear multiple choice knapsack problem and related problems. Inf. Proc. Lett. 18, 123--128. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Energy-efficient multicasting of multiview 3D videos to mobile devices

          Recommendations

          Reviews

          Bernard Kuc

          The world of mobile content delivery moves fast-very fast. Not that long ago, we were discussing multimedia messaging; now, here we are discussing the delivery of 3D media that supports multiple viewpoints. Broadcasting and distributing 3D video that can be rendered from many different viewpoints is not a trivial undertaking, hence this project in which the authors build on top of the multiview video coding extensions of the H.264/MPEG-4 standard. The starting premise is a video encoding method that contains multiple streams of independent depth and texture data. Each additional stream enhances the quality of the depth or texture information available for the rendering node. Typically, streams for more than two viewpoints are provided so that the end display device can render custom viewpoints. The authors focus on two problems. The first is how to choose how many (and which) of the depth and texture streams to send to a mobile device for the best possible rendering quality of the unknown user viewpoint. The second is how to schedule the data into radio frames for transfer between the base station and the mobile device. Assuming a positive linear relationship between the bit rate of depth or texture stream and the synthesized viewpoint quality, the authors decompose the first problem into the nondeterministic polynomial time (NP)-complete multiple-choice knapsack problem. They then apply an existing linear time partitioning algorithm for solving the linear programming relaxed multiple-choice knapsack problem. To solve the radio frame allocation problem, they use double buffering at the mobile node and schedule burst transfers to the device that will run out of buffered media the soonest. This is not an ideal solution given that both problems should be optimized jointly rather than independently for the best user experience. I would also question the energy savings claimed by the authors, as even a naive round-robin implementation could be shown to have similar energy-saving gains. This paper will be of most interest to practitioners and researchers in the telecommunications industry working in the area of 3D content delivery via mobile networks. Online Computing Reviews Service

          Access critical reviews of Computing literature here

          Become a reviewer for Computing Reviews.

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          • Published in

            cover image ACM Transactions on Multimedia Computing, Communications, and Applications
            ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 8, Issue 3s
            Special section of best papers of ACM multimedia 2011, and special section on 3D mobile multimedia
            September 2012
            173 pages
            ISSN:1551-6857
            EISSN:1551-6865
            DOI:10.1145/2348816
            Issue’s Table of Contents

            Copyright © 2012 ACM

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 16 October 2012
            • Revised: 1 May 2012
            • Accepted: 1 May 2012
            • Received: 1 January 2012
            Published in tomm Volume 8, Issue 3s

            Permissions

            Request permissions about this article.

            Request Permissions

            Check for updates

            Qualifiers

            • research-article
            • Research
            • Refereed

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!