Abstract
The increasing popularity of head-mounted devices and 360° video cameras allows content providers to provide virtual reality (VR) video streaming over the Internet, using a two-dimensional representation of the immersive content combined with traditional HTTP adaptive streaming (HAS) techniques. However, since only a limited part of the video (i.e., the viewport) is watched by the user, the available bandwidth is not optimally used. Recent studies have shown the benefits of adaptive tile-based video streaming; rather than sending the whole 360° video at once, the video is cut into temporal segments and spatial tiles, each of which can be requested at a different quality level. This allows prioritization of viewable video content and thus results in an increased bandwidth utilization. Given the early stages of research, there are still a number of open challenges to unlock the full potential of adaptive tile-based VR streaming. The aim of this work is to provide an answer to several of these open research questions. Among others, we propose two tile-based rate adaptation heuristics for equirectangular VR video, which use the great-circle distance between the viewport center and the center of each of the tiles to decide upon the most appropriate quality representation. We also introduce a feedback loop in the quality decision process, which allows the client to revise prior decisions based on more recent information on the viewport location. Furthermore, we investigate the benefits of parallel TCP connections and the use of HTTP/2 as an application layer optimization. Through an extensive evaluation, we show that the proposed optimizations result in a significant improvement in terms of video quality (more than twice the time spent on the highest quality layer), compared to non-tiled HAS solutions.
- M. Seufert, S. Egger, M. Slanina, T. Zinner, T. Hoßfeld, and P. Tran-Gia. 2015. A survey on quality of experience of HTTP adaptive streaming. IEEE Commun. Surveys Tutor. 17, 1 (2015), 469--492.Google Scholar
Digital Library
- R. Mok, E. Chan, and R. Chang. 2011. Measuring the quality of experience of HTTP video streaming. In Proceedings of the IFIP/IEEE International Symposium on Integrated Network Management. 485--492.Google Scholar
- ISO/IEC 23009-1 2014. Information Technology—Dynamic Adaptive Streaming over HTTP (DASH) - Part 1: Media Presentation Description and Segment Formats. Technical Report. International Organization for Standardization.Google Scholar
- OpenSignal. 2018. State of Mobile Networks: USA. Retrieved from https://opensignal.com/reports/2018/01/usa/state-of-the-mobile-network/.Google Scholar
- OpenSignal. 2018. State of Mobile Networks: Belgium. Retrieved from https://opensignal.com/reports/2018/03/belgium/state-of-the-mobile-network/.Google Scholar
- S. Petrangeli, V. Swaminathan, M. Hosseini, and F. De Turck. 2017. An HTTP/2-based adaptive streaming framework for 360-degree virtual reality videos. In Proceedings of the ACM Multimedia Conference. 306--314.Google Scholar
- J. van der Hooft, M. Torres Vega, S. Petrangeli, T. Wauters, and F. De Turck. 2019. Optimizing adaptive tile-based virtual reality video streaming. In Proceedings of the IFIP/IEEE International Symposium on Integrated Network Management.Google Scholar
- M. Yu, H. Lakshman, and B. Girod. 2015. A framework to evaluate omnidirectional video coding schemes. In Proceedings of the IEEE International Symposium on Mixed and Augmented Reality. 31--36.Google Scholar
- R. Skupin, Y. Sanchez, D. Podborski, C. Hellge, and T. Schierl. 2017. HEVC tile-based streaming to head-mounted displays. In Proceedings of the IEEE Annual Consumer Communications Networking Conference. 613--615.Google Scholar
- M. Budagavi, J. Furton, G. Jin, A. Saxena, J. Wilkinson, and A. Dickerson. 2015. 360 degrees video coding using region adaptive smoothing. In Proceedings of the IEEE International Conference on Image Processing. 750--754.Google Scholar
- M. Hosseini and V. Swaminathan. 2016. Adaptive 360 VR video streaming: Divide and conquer! In Proceedings of the IEEE International Symposium on Multimedia. 107--110.Google Scholar
- Team Pixvana. 2016. An Intro to FOVAS: Field of View Adaptive Streaming for Virtual Reality. Retrieved from https://pixvana.com/intro-to-field-of-view-adaptive-streaming-for-vr/.Google Scholar
- E. Kuzyakov and D. Pio. 2016. Next-Generation Video Encoding Techniques for 360 Video and VR. Retrieved from https://code.fb.com/virtual-reality/next-generation-video-encoding-techniques-for-360-video-and-vr/.Google Scholar
- A. Zare, A. Aminlou, and M. M. Hunnuksela. 2017. Virtual reality content streaming: Viewport-dependent projection and tile-based techniques. In Proceedings of the IEEE International Conference on Image Processing. 1432--1436.Google Scholar
- JCTVC-AC0038 2017. Temporal MCTS Coding Constraints Implementation. Technical Report. Joint Collaborative Team on Video Coding.Google Scholar
- J. Son, D. Jang, and E. Ryu. 2018. Implementing motion-constrained tile and viewport extraction for VR streaming. In Proceedings of the 28th ACM SIGMM Workshop on Network and Operating Systems Support for Digital Audio and Video. ACM, 61--66.Google Scholar
- A. Taghavi Nasrabadi, A. Mahzari, J. D. Beshay, and R. Prakash. 2017. Adaptive 360-degree video streaming using layered video coding. In Proceedings of the IEEE Virtual Reality Conference. 347--348.Google Scholar
- F. Qian, L. Ji, B. Han, and V. Gopalakrishnan. 2016. Optimizing 360 video delivery over cellular networks. In Proceedings of the Workshop on All Things Cellular: Operations, Applications and Challenges. 1--6.Google Scholar
- Z. Xu, X. Zhang, K. Zhang, and Z. Guo. 2018. Probabilistic viewport adaptive streaming for 360-degree videos. In Proceedings of the IEEE International Symposium on Circuits and Systems. 1--5.Google Scholar
- C. Fan, J. Lee, W. Lo, C. Huang, K. Chen, and C. Hsu. 2017. Fixation prediction for 360-degree; video streaming in head-mounted virtual reality. In Proceedings of the Workshop on Network and Operating Systems Support for Digital Audio and Video. 67--72.Google Scholar
- V. Sitzmann, A. Serrano, A. Pavel, M. Agrawala, D. Gutierrez, B. Masia, and G. Wetzstein. 2018. Saliency in VR: How do people explore virtual environments? IEEE Trans. Visual. Comput. Graph. 24, 4 (2018), 1633--1642.Google Scholar
Digital Library
- H. Hu, Y. Lin, M. Liu, H. Cheng, Y. Chang, and M. Sun. 2017. Deep 360 pilot: Learning a deep agent for piloting through 360-degree sports video. Comput. Res. Repos. abs/1705.01759 (2017).Google Scholar
- A. Ghosh, V. Aggarwal, and F. Qian. 2017. A rate adaptation algorithm for tile-vased 360-degree video streaming. Comput. Res. Repos. abs/1704.08215 (2017).Google Scholar
- A. Zare, A. Aminlou, M. M. Hannuksela, and M. Gabbouj. 2016. HEVC-compliant tile-based streaming of panoramic video for virtual reality applications. In Proceedings of the ACM International Conference on Multimedia. ACM, 601--605.Google Scholar
- J. Le Feuvre and C. Concolato. 2016. Tiled-based adaptive streaming using MPEG-DASH. In Proceedings of the International Conference on Multimedia Systems. 41:1--41:3.Google Scholar
- L. D’Acunto, J. van den Berg, E. Thomas, and O. Niamut. 2016. Using MPEG DASH SRD for zoomable and navigable video. In Proceedings of the International Conference on Multimedia Systems. 34:1--34:4.Google Scholar
- R. Fielding and J. Reschke. 2014. Hypertext Transfer Protocol (HTTP/1.1): Message Syntax and Routing. RFC 7230. RFC Editor. Retrieved from https://www.rfc-editor.org/rfc/rfc7230.txt.Google Scholar
- M. Belshe, G. Peon, and M. Thomson. 2015. Hypertext Transfer Protocol Version 2 (HTTP/2). RFC 7540. RFC Editor. Retrieved from https://www.rfc-editor.org/rfc/rfc7540.txt.Google Scholar
- W3Techs. 2019. Usage of HTTP/2 for Websites. Retrieved from https://w3techs.com/technologies/details/ce-http2/all/all.Google Scholar
- S. Wei and V. Swaminathan. 2014. Low latency live video streaming over HTTP 2.0. In Proceedings of the Network and Operating System Support on Digital Audio and Video Workshop. 37:37--37:42.Google Scholar
Digital Library
- L. Xie, Z. Xu, Y. Ban, X. Zhang, and Z. Guo. 2017. 360ProbDASH: Improving QoE of 360 video streaming using tile-based HTTP adaptive streaming. In Proceedings of the ACM Multimedia Systems Conference. 315--323.Google Scholar
- H. Ahmadi, O. Eltobgy, and M. Hefeeda. 2017. Adaptive multicast streaming of virtual reality content to mobile users. In Proceedings of the Thematic Workshops of ACM Multimedia. 170--178.Google Scholar
- R. I. T. da Costa Filho, M. C. Luizelli, M. Torres Vega, J. van der Hooft, S. Petrangeli, T. Wauters, F. De Turck, and L. P. Gaspary. 2018. Predicting the performance of virtual reality video streaming in mobile networks. In Proceedings of the ACM Multimedia Systems Conference. 270--283.Google Scholar
- Y. Rai, P. Le Callet, and P. Guillotel. 2017. Which saliency weighting for omni directional image quality assessment? In Proceedings of the International Conference on Quality of Multimedia Experience. 1--6.Google Scholar
- J. van der Hooft, M. Torres Vega, S. Petrangeli, T. Wauters, and F. De Turck. 2019. Quality assessment for adaptive virtual reality video streaming: A probabilistic approach on the user’s gaze. In Proceedings of the International Workshop on Quality of Experience Management.Google Scholar
- P. Juluri, V. Tamarapalli, and D. Medhi. 2015. SARA: Segment-aware rate adaptation algorithm for dynamic adaptive streaming over HTTP. In Proceedings of the IEEE International Conference on Communication Workshop. 1765--1770.Google Scholar
- C. Wu, Z. Tan, Z. Wang, and S. Yang. 2017. A dataset for exploring user behaviors in VR spherical video streaming. In Proceedings of the ACM Multimedia Systems Conference. 193--198.Google Scholar
- H. Kellerer, U. Pferschy, and D. Pisinger. 2004. The Multiple-Choice Knapsack Problem. Springer, Berlin, 317--347.Google Scholar
- J. van der Hooft, S. Petrangeli, T. Wauters, R. Huysegems, P. Rondao Alface, T. Bostoen, and F. De Turck. 2016. HTTP/2-based adaptive streaming of HEVC video over 4G/LTE networks. IEEE Commun. Lett. 20, 11 (2016), 2177--2180.Google Scholar
Cross Ref
Index Terms
Tile-based Adaptive Streaming for Virtual Reality Video
Recommendations
An HTTP/2-Based Adaptive Streaming Framework for 360° Virtual Reality Videos
MM '17: Proceedings of the 25th ACM international conference on MultimediaVirtual Reality (VR) devices are becoming accessible to a large public, which is going to increase the demand for 360° VR videos. VR videos are often characterized by a poor quality of experience, due to the high bandwidth required to stream the 360° ...
HEVC-compliant Tile-based Streaming of Panoramic Video for Virtual Reality Applications
MM '16: Proceedings of the 24th ACM international conference on MultimediaDelivering wide-angle and high-resolution spherical panoramic video content entails a high streaming bitrate. This imposes challenges when panorama clips are consumed in virtual reality (VR) head-mounted displays (HMD). The reason is that the HMDs ...
Improving Virtual Reality Streaming using HTTP/2
MMSys'17: Proceedings of the 8th ACM on Multimedia Systems ConferenceThe demand for 360° Virtual Reality (VR) videos is expected to grow in the near future, thanks to the diffusion of VR headsets. VR Streaming is however challenged by the high bandwidth requirements of 360° videos. To save bandwidth, we spatially tile ...






Comments