Abstract
To cope with the massive bandwidth demands of Virtual Reality (VR) video streaming, both the scientific community and the industry have been proposing optimization techniques such as viewport-aware streaming and tile-based adaptive bitrate heuristics. As most of the VR video traffic is expected to be delivered through mobile networks, a major problem arises: both the network performance and VR video optimization techniques have the potential to influence the video playout performance and the Quality of Experience (QoE). However, the interplay between them is neither trivial nor has it been properly investigated. To bridge this gap, in this article, we introduce VR-EXP, an open-source platform for carrying out VR video streaming performance evaluation. Furthermore, we consolidate a set of relevant VR video streaming techniques and evaluate them under variable network conditions, contributing to an in-depth understanding of what to expect when different combinations are employed. To the best of our knowledge, this is the first work to propose a systematic approach, accompanied by a software toolkit, which allows one to compare different optimization techniques under the same circumstances. Extensive evaluations carried out using realistic datasets demonstrate that VR-EXP is instrumental in providing valuable insights regarding the interplay between network performance and VR video streaming optimization techniques.
- Zahaib Akhtar, Yun Seong Nam, Ramesh Govindan, Sanjay Rao, Jessica Chen, Ethan Katz-Bassett, Bruno Ribeiro, Jibin Zhan, and Hui Zhang. 2018. Oboe: Auto-tuning video ABR algorithms to network conditions. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication (SIGCOMM’18). ACM, New York, NY, 44--58. DOI:https://doi.org/10.1145/3230543.3230558Google Scholar
Digital Library
- Mathias Almquist, Viktor Almquist, Vengatanathan Krishnamoorthi, Niklas Carlsson, and Derek Eager. 2018. The prefetch aggressiveness tradeoff in 360 video streaming. In Proceedings of the 9th ACM Multimedia Systems Conference (MMSys’18). ACM, New York, NY, 258--269. DOI:https://doi.org/10.1145/3204949.3204970Google Scholar
Digital Library
- Jill Boyce, Elena Alshina, Adeel Abbas, and Yan Ye. 2017. JVET common test conditions and evaluation procedures for 360 video. Joint Video Exploration Team of ITU-T SG 16 (2017). ITU - International Telecommunication Union. Retrieved on 21 June, 2010 from http://phenix.it sudparis.eu/jvet/doc_end_user/documents/16_Geneva/wg11/JVET-P0006-v1.zip.Google Scholar
- Zhenzhong Chen, Yiming Li, and Yingxue Zhang. 2018. Recent advances in omnidirectional video coding for virtual reality: Projection and evaluation. Sig. Proc. 146 (2018), 66--78. DOI:https://doi.org/10.1016/j.sigpro.2018.01.004Google Scholar
Cross Ref
- Cisco. 2019. Cisco Visual Networking Index: Forecast and Trends, 2017--2022. Technical Report. Cisco Systems.Google Scholar
- X. Corbillon, G. Simon, A. Devlic, and J. Chakareski. 2017. Viewport-adaptive navigable 360-degree video delivery. In Proceedings of the IEEE International Conference on Communications (ICC’17). 1--7. DOI:https://doi.org/10.1109/ICC.2017.7996611Google Scholar
- R. I. T. da Costa Filho, W. Lautenschlager, N. Kagami, V. Roesler, and L. P. Gaspary. 2016. Network fortune cookie: Using network measurements to predict video streaming performance and QoE. In Proceedings of the IEEE Global Communications Conference (GLOBECOM’16). 1--6. DOI:https://doi.org/10.1109/GLOCOM.2016.7842022Google Scholar
Digital Library
- Roberto Irajá Tavares da Costa Filho, Marcelo Caggiani Luizelli, Maria Torres Vega, Jeroen van der Hooft, Stefano Petrangeli, Tim Wauters, Filip De Turck, and Luciano Paschoal Gaspary. 2018. Predicting the performance of virtual reality video streaming in mobile networks. In Proceedings of the 9th ACM Multimedia Systems Conference (MMSys’18). ACM, New York, NY, 270--283. DOI:https://doi.org/10.1145/3204949.3204966Google Scholar
Digital Library
- Giorgos Dimopoulos, Ilias Leontiadis, Pere Barlet-Ros, and Konstantina Papagiannaki. 2016. Measuring video QoE from encrypted traffic. In Proceedings of the Internet Measurement Conference (IMC’16). ACM, New York, NY, 513--526. DOI:https://doi.org/10.1145/2987443.2987459Google Scholar
Digital Library
- Glederson Lessa dos Santos, Vinicius Tavares Guimaraes, Jorge Guedes Silveira, Alexandre T. Vieira, Jose Augusto de Oliveira Neto, R. I. T. da Costa, and Ricardo Balbinot. 2007. UAMA: A unified architecture for active measurements in IP networks; end-to-end objetive quality indicators. In Proceedings of the 10th IFIP/IEEE International Symposium on Integrated Network Management (IM’07). 246--253.Google Scholar
Cross Ref
- Ching-Ling Fan, Jean Lee, Wen-Chih Lo, Chun-Ying Huang, Kuan-Ta Chen, and Cheng-Hsin Hsu. 2017. Fixation prediction for 360 video streaming in head-mounted virtual reality. In Proceedings of the 27th Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV’17). ACM, New York, NY, 67--72. DOI:https://doi.org/10.1145/3083165.3083180Google Scholar
Digital Library
- Mario Graf, Christian Timmerer, and Christopher Mueller. 2017. Towards bandwidth-efficient adaptive streaming of omnidirectional video over HTTP: Design, implementation, and evaluation. In Proceedings of the 8th ACM Conference on Multimedia Systems Conference (MMSys’17). ACM, New York, NY, 261--271. DOI:https://doi.org/10.1145/3083187.3084016Google Scholar
Digital Library
- Jian He, Mubashir Adnan Qureshi, Lili Qiu, Jin Li, Feng Li, and Lei Han. 2018. Favor: Fine-grained video rate adaptation. In Proceedings of the 9th ACM Multimedia Systems Conference (MMSys’18). ACM, New York, NY, 64--75. DOI:https://doi.org/10.1145/3204949.3204957Google Scholar
Digital Library
- M. Hosseini and V. Swaminathan. 2016. Adaptive 360 VR video streaming: Divide and conquer. In Proceedings of the IEEE International Symposium on Multimedia (ISM’16). 107--110. DOI:https://doi.org/10.1109/ISM.2016.0028Google Scholar
- Xueshi Hou, Sujit Dey, Jianzhong Zhang, and Madhukar Budagavi. 2018. Predictive view generation to enable mobile 360-degree and VR experiences. In Proceedings of the Morning Workshop on Virtual Reality and Augmented Reality Network (VR/AR Network’18). ACM, New York, NY, 20--26. DOI:https://doi.org/10.1145/3229625.3229629Google Scholar
Digital Library
- H. Hristova, X. Corbillon, G. Simon, V. Swaminathan, and A. Devlic. 2018. Heterogeneous spatial quality for omnidirectional video. In Proceedings of the IEEE 20th International Workshop on Multimedia Signal Processing (MMSP’18). 1--6. DOI:https://doi.org/10.1109/MMSP.2018.8547114Google Scholar
- E. Jeong, D. You, C. Hyun, B. Seo, N. Kim, D. H. Kim, and Y. H. Lee. 2018. Viewport prediction method of 360 VR video using sound localization information. In Proceedings of the 10th International Conference on Ubiquitous and Future Networks (ICUFN’18). 679--681. DOI:https://doi.org/10.1109/ICUFN.2018.8436981Google Scholar
- Ron Kohavi et al. 1995. A study of cross-validation and bootstrap for accuracy estimation and model selection. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI’95), Vol. 14. 1137--1145.Google Scholar
- L. Ma, Y. Xu, J. Sun, W. Huang, S. Xie, Y. Li, and N. Liu. 2018. Buffer control in VR video transmission over MMT system. In Proceedings of the IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB’18). 1--5. DOI:https://doi.org/10.1109/BMSB.2018.8436817Google Scholar
- Hongzi Mao, Ravi Netravali, and Mohammad Alizadeh. 2017. Neural adaptive video streaming with Pensieve. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication (SIGCOMM’17). ACM, New York, NY, 197--210. DOI:https://doi.org/10.1145/3098822.3098843Google Scholar
Digital Library
- A. Morton. 2016. RFC 7799 - Active and Passive Metrics and Methods (with Hybrid Types In-Between). Technical Report. IETF.Google Scholar
- T. C. Nguyen and J. Yun. 2018. Predictive tile selection for 360-degree VR video streaming in bandwidth-limited networks. IEEE Commun. Lett. 22, 9 (Sept. 2018), 1858--1861. DOI:https://doi.org/10.1109/LCOMM.2018.2848915Google Scholar
Cross Ref
- Stefano Petrangeli, Jeroen Famaey, Maxim Claeys, Steven Latré, and Filip De Turck. 2015. QoE-driven rate adaptation heuristic for fair adaptive video streaming. ACM Trans. Multimedia Comput. Commun. Appl. 12, 2, Article 28 (Oct. 2015), 24 pages. DOI:https://doi.org/10.1145/2818361Google Scholar
Digital Library
- S. Petrangeli, G. Simon, and V. Swaminathan. 2018. Trajectory-based viewport prediction for 360-degree virtual reality videos. In Proceedings of the IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR’18). 157--160. DOI:https://doi.org/10.1109/AIVR.2018.00033Google Scholar
- Stefano Petrangeli, Viswanathan Swaminathan, Mohammad Hosseini, and Filip De Turck. 2017. An HTTP/2-Based adaptive streaming framework for 360 virtual reality videos. In Proceedings of the ACM on Multimedia Conference (MM’17). ACM, New York, NY, 306--314. DOI:https://doi.org/10.1145/3123266.3123453Google Scholar
Digital Library
- Feng Qian, Lusheng Ji, Bo Han, and Vijay Gopalakrishnan. 2016. Optimizing 360 video delivery over cellular networks. In Proceedings of the 5th Workshop on All Things Cellular: Operations, Applications and Challenges (ATC’16). ACM, New York, NY, 1--6. DOI:https://doi.org/10.1145/2980055.2980056Google Scholar
Digital Library
- Iraj Sodagar. 2011. The MPEG-DASH standard for multimedia streaming over the internet. IEEE Multimedia 18, 4 (2011), 62--67.Google Scholar
Digital Library
- Kevin Spiteri, Ramesh Sitaraman, and Daniel Sparacio. 2018. From theory to practice: Improving bitrate adaptation in the DASH reference player. In Proceedings of the 9th ACM Multimedia Systems Conference (MMSys’18). ACM, New York, NY, 123--137. DOI:https://doi.org/10.1145/3204949.3204953Google Scholar
Digital Library
- K. Spiteri, R. Urgaonkar, and R. K. Sitaraman. 2016. BOLA: Near-optimal bitrate adaptation for online videos. In Proceedings of the 35th IEEE International Conference on Computer Communications (INFOCOM’16). 1--9. DOI:https://doi.org/10.1109/INFOCOM.2016.7524428Google Scholar
Digital Library
- K. Stangherlin, R. C. Filho, W. Lautenschläger, V. Guadagnin, L. Balbinot, R. Balbinot, and V. Roesler. 2011. One-way delay measurement in wired and wireless mobile full-mesh networks. In Proceedings of the IEEE Wireless Communications and Networking Conference. 1044--1049. DOI:https://doi.org/10.1109/WCNC.2011.5779279Google Scholar
- TELCO. 2018. Mobile Operators Market Share in Brazil. Retrieved from: http://www.teleco.com.br/en/en_mshare.asp.Google Scholar
- J. van der Hooft, S. Petrangeli, T. Wauters, R. Huysegems, P. R. Alface, T. Bostoen, and F. De Turck. 2016. HTTP/2-Based adaptive streaming of HEVC video over 4G/LTE networks. IEEE Commun. Lett. 20, 11 (2016), 2177--2180.Google Scholar
Cross Ref
- M. Viitanen, A. Koivula, A. Lemmetti, J. Vanne, and T. D. Hämäläinen. 2015. Kvazaar HEVC encoder for efficient intra coding. In Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS’15). 1662--1665. DOI:https://doi.org/10.1109/ISCAS.2015.7168970Google Scholar
- Chenglei Wu, Zhihao Tan, Zhi Wang, and Shiqiang Yang. 2017. A dataset for exploring user behaviors in VR spherical video streaming. In Proceedings of the 8th ACM on Multimedia Systems Conference (MMSys’17). ACM, New York, NY, 193--198. DOI:https://doi.org/10.1145/3083187.3083210Google Scholar
Digital Library
- Xiaoqi Yin, Abhishek Jindal, Vyas Sekar, and Bruno Sinopoli. 2015. A control-theoretic approach for dynamic adaptive video streaming over HTTP. In Proceedings of the ACM Conference on Special Interest Group on Data Communication (SIGCOMM’15). ACM, New York, NY, 325--338. DOI:https://doi.org/10.1145/2785956.2787486Google Scholar
Digital Library
- Xiaoqi Yin, Abhishek Jindal, Vyas Sekar, and Bruno Sinopoli. 2015. A control-theoretic approach for dynamic adaptive video streaming over HTTP. SIGCOMM Comput. Commun. Rev. 45, 4 (Aug. 2015), 325--338. DOI:https://doi.org/10.1145/2829988.2787486Google Scholar
Digital Library
- Chao Zhou, Zhenhua Li, Joe Osgood, and Yao Liu. 2018. On the effectiveness of offset projections for 360-degree video streaming. ACM Trans. Multimedia Comput. Commun. Appl. 14, 3s, Article 62 (June 2018), 24 pages. DOI:https://doi.org/10.1145/3209660Google Scholar
Digital Library
Index Terms
Dissecting the Performance of VR Video Streaming through the VR-EXP Experimentation Platform
Recommendations
Predicting the performance of virtual reality video streaming in mobile networks
MMSys '18: Proceedings of the 9th ACM Multimedia Systems ConferenceThe demand of Virtual Reality (VR) video streaming to mobile devices is booming, as VR becomes accessible to the general public. However, the variability of conditions of mobile networks affects the perception of this type of high-bandwidth-demanding ...
VR Grabbers: Ungrounded Haptic Retargeting for Precision Grabbing Tools
UIST '18: Proceedings of the 31st Annual ACM Symposium on User Interface Software and TechnologyHaptic feedback in VR is important for realistic simulation in virtual reality. However, recreating the haptic experience for hand tools in VR traditionally requires hardware with precise actuators, adding complexity to the system. We propose Ungrounded ...
Power Evaluation of 360 VR Video Streaming on Head Mounted Display Devices
NOSSDAV'17: Proceedings of the 27th Workshop on Network and Operating Systems Support for Digital Audio and VideoVirtual reality (VR) video streaming with 360-degree views has become a trending video application recently. While providing the users with immersive video viewing experiences, the 360 video streaming introduces significantly higher overhead than the ...






Comments