Abstract
Dual-connectivity streaming is a key enabler of next-generation six Degrees Of Freedom (6DOF) Virtual Reality (VR) scene immersion. Indeed, using conventional sub-6 GHz WiFi only allows to reliably stream a low-quality baseline representation of the VR content, while emerging high-frequency communication technologies allow to stream in parallel a high-quality user viewport-specific enhancement representation that synergistically integrates with the baseline representation to deliver high-quality VR immersion. We investigate holistically as part of an entire future VR streaming system two such candidate emerging technologies, Free Space Optics (FSO) and millimeter-Wave (mmWave), that benefit from a large available spectrum to deliver unprecedented data rates. We analytically characterize the key components of the envisioned dual-connectivity 6DOF VR streaming system that integrates in addition edge computing and scalable 360° video tiling, and we formulate an optimization problem to maximize the immersion fidelity delivered by the system, given the WiFi and mmWave/FSO link rates, and the computing capabilities of the edge server and the users’ VR headsets. This optimization problem is mixed integer programming of high complexity and we formulate a geometric programming framework to compute the optimal solution at low complexity. We carry out simulation experiments to assess the performance of the proposed system using actual 6DOF navigation traces from multiple mobile VR users that we collected. Our results demonstrate that our system considerably advances the traditional state of the art and enables streaming of 8K-120 frames-per-second (fps) 6DOF content at high fidelity.
- [1] . 2017. Drone networks for virtual human teleportation. In Proc. ACM Workshop on Micro Aerial Vehicle Networks, Systems, and Applications. 21–26.Google Scholar
- [2] . 2012. The road to immersive communication. Proc. IEEE 100, 4 (
April 2012), 974–990.Google ScholarCross Ref
- [3] . 2019. UAV-IoT for next generation virtual reality. IEEE Transactions on Image Processing 28, 12 (2019), 5977–5990.Google Scholar
Cross Ref
- [4] . 2016. Quality requirements for VR. In Proc. 116th MPEG Meeting of ISO/IEC JTC1/SC29/WG11 MPEG, vol. 116, p. m39532.Google Scholar
- [5] . 2015. Principles of LED Light Communications: Towards Networked Li-Fi. Cambridge University Press.Google Scholar
Cross Ref
- [6] . 2019. 6G: The next frontier: From holographic messaging to artificial intelligence using subterahertz and visible light communication. IEEE Vehicular Technology Magazine 14, 3 (2019), 42–50.
DOI: Google ScholarCross Ref
- [7] . 2018. FSO-VR: Steerable free space optics link for virtual reality headsets. In Proc. ACM Workshop on Wearable Systems and Applications. 11–15.Google Scholar
- [8] . 2020. A cell-free networking system with visible light. IEEE/ACM Transactions on Networking 28, 2 (2020), 461–476.
DOI: Google ScholarDigital Library
- [9] . 2019. Multi-user hybrid MIMO at 60 GHz using 16-antenna transmitters. IEEE Transactions on Circuits and Systems I: Regular Papers 66, 2 (2019), 848–858.
DOI: Google ScholarCross Ref
- [10] . 2008. Distributed collaboration for enhanced sender-driven video streaming. IEEE Trans. Multimedia 10, 5 (
Aug. 2008), 858–870.Google ScholarDigital Library
- [11] . 2016. Why the Internet pipes will burst when virtual reality takes off. Forbes Magazine.Google Scholar
- [12] . 2018. Viewport-driven rate-distortion optimized 360\(^\circ\) video streaming. In Proc. IEEE Int’l Conf. Communications.Google Scholar
- [13] . 2017. Viewport-adaptive navigable 360-degree video delivery. In Proc. Int’l Conf. Communications. IEEE.Google Scholar
- [14] . 2016. Adaptive 360 VR video streaming: Divide and conquer! In Proc. IEEE Int’l Symp. Multimedia.Google Scholar
- [15] . 2018. Creating the perfect illusion: What will it take to create life-like virtual reality headsets? In Proc. HotMobile. ACM, 7–12.Google Scholar
- [16] . 2017. On building a programmable wireless high-quality virtual reality system using commodity hardware. In Proceedings of the 8th Asia-Pacific Workshop on Systems. ACM, 7.Google Scholar
Digital Library
- [17] . 2017. Enabling high-quality untethered virtual reality. In Proc. Symposium on Networked Systems Design and Implementation. 531–544.Google Scholar
- [18] . 2019. Visible light communication for next generation untethered virtual reality systems. In Proc. IEEE International Conference on Communications Workshops. 1–6.Google Scholar
Cross Ref
- [19] . 2020. Viewport-adaptive scalable multi-user virtual reality mobile-edge streaming. IEEE Communications Magazine 58, 1 (
Jan. 2020), 93–99.Google ScholarDigital Library
- [20] . 2021. Point cloud video streaming: Challenges and solutions. IEEE Network Magazine 35, 5 (
Sept. 2021), 202–209.Google ScholarDigital Library
- [21] . 2006. Rate-distortion optimized distributed packet scheduling of multiple video streams over shared communication resources. IEEE Trans. Multimedia 8, 2 (
April 2006), 207–218.Special issue on Distributed Media Technologies and Applications. Google ScholarDigital Library
- [22] . 2010. Distortion optimized multi-service scheduling for next generation wireless mesh networks. In Proc. IEEE INFOCOM Int’l Workshop on Carrier-grade Wireless Mesh Networks. USA.Google Scholar
- [23] . 2004. Distortion chains for predicting the video distortion for general packet loss patterns. In Proc. Int’l Conf. Acoustics, Speech, and Signal Processing, Vol. 5. IEEE, 1001–1004.Google Scholar
- [24] . 2005. Rate-distortion hint tracks for adaptive video streaming. IEEE Trans. Circuits and Systems for Video Technology 15, 10 (
Oct. 2005), 1257–1269.Special issue on Analysis and Understanding for Video Adaptation. Google ScholarDigital Library
- [25] . 2006. RaDiO edge: Rate-distortion optimized proxy-driven streaming from the network edge. IEEE/ACM Trans. Networking 14, 6 (
Dec. 2006), 1302–1312.Google ScholarDigital Library
- [26] . 2004. Application layer error correction coding for rate-distortion optimized streaming to wireless clients. IEEE Trans. Communications 52, 10 (
Oct. 2004), 1675–1687.Google ScholarCross Ref
- [27] . 2015. 10 Gbps mobile visible light communication system employing angle diversity, imaging receivers, and relay nodes. OSA Journal of Optical Communications and Networking 7, 8 (2015), 718–735.Google Scholar
Cross Ref
- [28] NVIDIA GeForce GTX 980. n.d. https://www.techpowerup.com/gpu-specs/geforce-gtx-980.c2621.Google Scholar
- [29] GPU Systems. n.d. https://www.supermicro.com/en/products/GPU/.Google Scholar
- [30] NVIDIA Video Encode and Decode GPU Support Matrix. n.d. https://developer.nvidia.com/video-encode-and-decode-gpu-support-matrix-new.Google Scholar
- [31] . 2015. Overview of SHVC: Scalable extensions of the high efficiency video coding standard. IEEE Transactions on Circuits and Systems for Video Technology 26, 1 (2015), 20–34.Google Scholar
Digital Library
- [32] . 2020. Viewport-adaptive scalable multi-user virtual reality mobile-edge streaming. IEEE Trans. Image Processing 29 (2020), 6330–6342.Google Scholar
Cross Ref
- [33] QualComm Adreno 650 GPU. n.d. https://www.qualcomm.com/snapdragon/processors/comparison.Google Scholar
- [34] . 2014. Global optimization of signomial geometric programming problems. European Journal of Operational Research 233, 3 (2014), 500–510.Google Scholar
Cross Ref
- [35] . 2004. Convex Optimization. Cambridge University Press.Google Scholar
Digital Library
- [36] . 2002. Free Space Optics: Enabling Optical Connectivity in Today’s Networks. SAMS Publishing.Google Scholar
- [37] . May 2019. Angle diversity receiver in LiFi cellular networks. In Proc. IEEE International Conference on Communications. 1–6.Google Scholar
Cross Ref
- [38] . 2003. Linear diversity combining techniques. Proc. IEEE 91, 2 (2003), 331–356.Google Scholar
Cross Ref
- [39] . 2014. Quasi-deterministic approach to mmWave channel modeling in a non-stationary environment. In 2014 IEEE Globecom Workshops (GC Wkshps’14). IEEE, 966–971.Google Scholar
- [40] . 1994. Improved complexity bound for the maximum cardinality bottleneck bipartite matching problem. Discrete Applied Mathematics 55, 1 (1994), 91–93.Google Scholar
Digital Library
- [41] . 2022. Markov multi-beamtracking on 60 GHz mobile channel measurements. IEEE Open Journal of Vehicular Technology 3 (2022), 26–39.
DOI: Google ScholarCross Ref
- [42] Virtual Museum. n.d. https://assetstore.unity.com/packages/3d/environments/museum-117927.Google Scholar
- [43] SteamVR SDK. n.d. https://github.com/ValveSoftware/openvr.Google Scholar
- [44] Opentrack tracking. n.d. https://github.com/opentrack/opentrack.Google Scholar
- [45] . n.d. NJIT 6DOF VR Navigation Dataset. https://www.jakov.org.Google Scholar
- [46] . n.d. Q-D realization software. https://github.com/wigig-tools/qd-realization.Google Scholar
- [47] . 2019. Enhancing the Ns-3 IEEE 802.11ad model fidelity: Beam codebooks, multi-antenna beamforming training, and quasi-deterministic mmwave channel. In Proceedings of the 2019 Workshop on Ns-3 (WNS3’19). Association for Computing Machinery, New York, NY, 33–40.
DOI: Google ScholarDigital Library
- [48] . 2019. High fidelity simulation of IEEE 802.11ad in ns-3 using a quasi-deterministic channel model. In Proceedings of the 2019 Workshop on Next-Generation Wireless with ns-3. 22–25.Google Scholar
Digital Library
- [49] . 2017. Improving virtual reality streaming using HTTP/2. In Proc. Multimedia Systems Conference. ACM, 225–228.Google Scholar
Digital Library
Index Terms
Millimeter Wave and Free-space-optics for Future Dual-connectivity 6DOF Mobile Multi-user VR Streaming
Recommendations
Wifi-VLC dual connectivity streaming system for 6DOF multi-user virtual reality
NOSSDAV '21: Proceedings of the 31st ACM Workshop on Network and Operating Systems Support for Digital Audio and VideoWe investigate a future WiFi-VLC dual connectivity streaming system for 6DOF multi-user virtual reality that enables reliable high-fidelity remote scene immersion. The system integrates an edge server that uses scalable 360° tiling to adaptively split ...
Future Standards for Immersive VR: Report on the IEEE Virtual Reality 2007 Workshop
The Future Standards for Immersive VR Workshop at the IEEE Virtual Reality 2007 Conference aimed to foster an understanding between the VR and Web3D communities. Through multiple tool and application examples, contributors looked at the challenges and ...
Effective stereoscopic rendering for mobile VR
SA '16: SIGGRAPH ASIA 2016 PostersLately, the VR market is expanding rapidly with the advent of mobile phone based VR such as Google Cardboard, Samsung Gear VR, LG 360VR starting with VR(Virtual Reality) HMD(Head Mount Device) PC based VR using Stereoscopic rendering such as Oculus Rift,...






Comments