skip to main content
research-article

Depth Personalization and Streaming of Stereoscopic Sports Videos

Published:08 March 2016Publication History
Skip Abstract Section

Abstract

Current three-dimensional displays cannot fully reproduce all depth cues used by a human observer in the real world. Instead, they create only an illusion of looking at a three-dimensional scene. This leads to a number of challenges during the content creation process. To assure correct depth reproduction and visual comfort, either the acquisition setup has to be carefully controlled or additional postprocessing techniques have to be applied. Furthermore, these manipulations need to account for a particular setup that is used to present the content, for example, viewing distance or screen size. This creates additional challenges in the context of personal use when stereoscopic content is shown on TV sets, desktop monitors, or mobile devices. We address this problem by presenting a new system for streaming stereoscopic content. Its key feature is a computationally efficient depth adjustment technique which can automatically optimize viewing experience for videos of field sports such as soccer, football, and tennis. Additionally, the method enables depth personalization to allow users to adjust the amount of depth according to their preferences. Our stereoscopic video streaming system was implemented, deployed, and tested with real users.

References

  1. 3 DeeCentral. 2015. 3DeeCentral Web site. Retrieved from http://www.3deecentral.com/.Google ScholarGoogle Scholar
  2. 3 DVisionLive. 2015. 3DVisionLive Web site. Retrieved from https://www.3dvisionlive.com/.Google ScholarGoogle Scholar
  3. Thomas Brox, Andrés Bruhn, Nils Papenberg, and Joachim Weickert. 2004. High accuracy optical flow estimation based on a theory for warping. In Proc. of European Conference on Computer Vision (ECCV’04). 25--36.Google ScholarGoogle ScholarCross RefCross Ref
  4. Peter Burt and Bela Julesz. 1980. Modifications of the classical notion of Panum’s fusional area. Perception 9, 6 (1980), 671--682.Google ScholarGoogle ScholarCross RefCross Ref
  5. Kiana Calagari, Krzysztof Templin, Tarek Elgamal, Khaled Diab, Piotr Didyk, Wojciech Matusik, and Mohamed Hefeeda. 2014. Anahita: A system for 3d video streaming with depth customization. In Proc. of the ACM International Conference on Multimedia (ACM MM’14). 337--346. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Pablo Carballeira, Julián Cabrera, Antonio Ortega, Fernando Jaureguizar, and Narciso García. 2012. A framework for the analysis and optimization of encoding latency for multiview video. IEEE J. Select. Top. Signal Process. 6, 5 (2012), 583--596.Google ScholarGoogle ScholarCross RefCross Ref
  7. Ben E. Coutant and Gerald Westheimer. 1993. Population distribution of stereoscopic ability. Ophthal. Physiol. Opt. 13, 1 (1993), 3--7.Google ScholarGoogle ScholarCross RefCross Ref
  8. Khaled Diab, Tarek Elgamal, Kiana Calagari, and Mohamed Hefeeda. 2014. Storage optimization for 3d streaming systems. In Proc. of ACM Conference on Multimedia Systems (MMSys’14). 59--69. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Piotr Didyk, Tobias Ritschel, Elmar Eisemann, Karol Myszkowski, and Hans-Peter Seidel. 2011. A perceptual model for disparity. ACM Trans. Graphics 30, 4 (2011), 96:1--96:9. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Displaybank Co. 2010. 3D TV industry trend and market forecast. Special Report. Retrieved from http://www.displaybank.com/research_file/710527.pdf.Google ScholarGoogle Scholar
  11. Ahmet Ekin, A. Murat Tekalp, and Rajiv Mehrotra. 2003. Automatic soccer video analysis and summarization. IEEE Transactions on Image Processing 12, 7 (2003), 796--807. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Christoph Fehn. 2003. A 3d-tv system based on video plus depth information. In Asilomar Conference on Signals, Systems and Computers 2, 1529--1533.Google ScholarGoogle ScholarCross RefCross Ref
  13. Jonathan Freeman and Steve E. Avons. 2000. Focus group exploration of presence through advanced broadcast services. In Proc. of SPIE Human Vision and Electronic Imaging Conference. 530--539.Google ScholarGoogle Scholar
  14. C. Göktuğ Gurler, Burak Görkemli, Görkem Saygili, and others. 2011. Flexible transport of 3-d video over networks. Proc. IEEE 99, 4 (2011), 694--707.Google ScholarGoogle ScholarCross RefCross Ref
  15. Ahmed Hamza and Mohamed Hefeeda. 2014. A DASH-based free-viewpoint video streaming system. In Proc. of ACM NOSSDAV’14 Workshop, in Conjunction with ACM Multimedia Systems (MMSys’14) Conference. 55--60. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Nicolas S. Holliman. 2004. Mapping perceived depth to regions of interest in stereoscopic images. In Proc. of SPIE Stereoscopic Displays and Virtual Reality Systems Conference. 117--128.Google ScholarGoogle Scholar
  17. ISO/IEC 23009-1:2012. 2012. Information technology -- Dynamic adaptive streaming over HTTP (DASH) -- Part 1: Media presentation description and segment formats.Google ScholarGoogle Scholar
  18. ITU-R BT.2021. 2012. Subjective methods for the assessment of stereoscopic 3D TV systems. International Telecommunication Union, Geneva, Switzerland.Google ScholarGoogle Scholar
  19. Mathias Johanson. 2001. Stereoscopic video transmission over the internet. In Proc. of IEEE Workshop on Internet Applications (WIAPP’01). 12--19. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Hideaki Kimata, Katsuhiko Fukazawa, Akio Kameda, Yoshie Yamaguchi, and Norihiko Matsuura. 2011. Interactive 3D multi-angle live streaming system. In Proc. of IEEE International Symposium on Consumer Electronics (ISCE’01). 576--579.Google ScholarGoogle ScholarCross RefCross Ref
  21. Manuel Lang, Alexander Hornung, Oliver Wang, Steven Poulakos, Aljoscha Smolic, and Markus Gross. 2010. Nonlinear disparity mapping for stereoscopic 3D. ACM Trans. Graphics 29, 3 (2010), 75:1--75:10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Jian-Guang Lou, Hua Cai, and Jiang Li. 2005. A real-time interactive multi-view video system. In Proc. of ACM International Conference on Multimedia (ACMMM’05). 161--170. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Bernard Mendiburu. 2012. 3D Movie Making: Stereoscopic Digital Cinema from Script to Screen. CRC Press.Google ScholarGoogle Scholar
  24. Thomas Oskam, Alexander Hornung, Huw Bowles, Kenny Mitchell, and Markus Gross. 2011. OSCAM - optimized stereoscopic camera control for interactive 3D. ACM Trans. Graphics 30, 6 (2011), 189:1--189:8. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Dawid Pajak, Robert Herzog, Radosław Mantiuk, Piotr Didyk, Elmar Eisemann, Karol Myszkowski, and Kari Pulli. 2014. Perceptual depth compression for stereo applications. Comput. Graphics Forum 33, 2 (2014), 195--204. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Sylvain Paris and Frédo Durand. 2009. A fast approximation of the bilateral filter using a signal processing approach. Int. J. Comput. Vision 81, 1 (2009), 24--52. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Selen Pehlivan, Anil Aksay, Cagdas Bilen, Gozde Bozdagi Akar, and M. Reha Civanlar. 2006. End-to-end stereoscopic video streaming system. In Proc. of IEEE International Conference on Multimedia and Expo (ICME’06). 2169--2172.Google ScholarGoogle Scholar
  28. André Redert, Marc Op de Beeck, Christoph Fehn, Wijnand Jsselsteijn, Marc Pollefeys, Luc Van Gool, Eyal Ofek, Ian Sexton, and Philip Surman. 2002. ATTEST: Advanced three-dimensional television system technologies. In Proc. of International Symposium on 3D Data Processing Visualization and Transmission. 313--319.Google ScholarGoogle Scholar
  29. Takashi Shibata, Joohwan Kim, David M. Hoffman, and Martin S. Banks. 2011. The zone of comfort: Predicting visual discomfort with stereo displays. J. Vision 11, 8 (2011), 11:1--11:29.Google ScholarGoogle ScholarCross RefCross Ref
  30. Aljoscha Smolic, Peter Kauff, Sebastian Knorr, Alexander Hornung, Matthias Kunter, Marcus Mller, and Manuel Lang. 2011. Three-dimensional video postproduction and processing. Proc. IEEE 99, 4 (2011), 607--625.Google ScholarGoogle ScholarCross RefCross Ref
  31. Iraj Sodagar. 2011. The MPEG-DASH standard for multimedia streaming over the internet. IEEE Multimed. Mag. 18, 4 (2011), 62--67. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Thomas Stockhammer. 2011. Dynamic adaptive streaming over HTTP: Standards and design principles. In Proc. of ACM Conference on Multimedia Systems (MMSys’11). 133--144. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Wa James Tam, Filippo Speranza, Sumio Yano, Koichi Shimono, and Hiroshi Ono. 2011. Stereoscopic 3D-tv: Visual comfort. IEEE Trans. Broadcast. 57, 2 (2011), 335--346.Google ScholarGoogle ScholarCross RefCross Ref
  34. Jonathan R. Thorpe and Mark J. Russell. 2011. Perceptual effects when scaling screen size of stereo 3D presentations. In Proc. of Society of Motion Picture and Television Engineers Conferences (SMPTE’11). 1--10.Google ScholarGoogle Scholar
  35. Trivido. 2015. Trivido Web site. Retrieved from http://www.trivido.com/.Google ScholarGoogle Scholar
  36. Ugo Capeto. 2013. Depth Map Automatic Generator (DMAG). Retrieved from http://3dstereophoto.blogspot.com/2013/04/depth-map-automatic-generator-dmag.html.Google ScholarGoogle Scholar
  37. Anthony Vetro, Thomas Wiegand, and Gary J. Sullivan. 2011. Overview of the stereo and multiview video coding extensions of the H. 264/MPEG-4 AVC standard. Proc. IEEE 99, 4 (2011), 626--642.Google ScholarGoogle ScholarCross RefCross Ref
  38. Wanmin Wu, Ahsan Arefin, Gregorij Kurillo, Pooja Agarwal, Klara Nahrstedt, and Ruzena Bajcsy. 2011. Color-plus-depth level-of-detail in 3D tele-immersive video: A psychophysical approach. In Proc. of ACM International Conference on Multimedia (ACM MM’11). 13--22. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Baicheng Xin, Ronggang Wang, Zhenyu Wang, Wenmin Wang, Chenchen Gu, Quanzhan Zheng, and Wen Gao. 2012. AVS 3D video streaming system over internet. In Proc. of IEEE International Conference on Signal Processing, Communication and Computing (ICSPCC’12). 286--289.Google ScholarGoogle ScholarCross RefCross Ref
  40. Qingxiong Yang, Liang Wang, and Narendra Ahuja. 2010. A constant-space belief propagation algorithm for stereo matching. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR’10). 1458--1465.Google ScholarGoogle ScholarCross RefCross Ref
  41. YouTube. Retrieved from http://www.youtube.com/.Google ScholarGoogle Scholar

Index Terms

  1. Depth Personalization and Streaming of Stereoscopic Sports Videos

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!