Abstract
To design an optimal adaptive video streaming method, video service providers need to consider both the efficiency and the fairness of the Quality of Experience (QoE) of their users. In Reference [8], we proposed a server-side QoE-fair rate adaptation method that considers both efficiency and fairness of the QoE. The server uses Reinforcement Learning (RL) to select a bitrate for each client sharing the same bottleneck link to the server in a way that achieves fairness among concurrent DASH clients and imposes that bitrate by dynamically modifying the client’s Media Presentation Description (MPD) file. In this article, we extend that work to minimize the number of actions the server needs to take to keep the system in its equilibrium state. By incorporating a Recurrent Neural Network, specifically an LSTM model, we modify the server’s training algorithm to achieve improvements in both the quality and the quantity of actions the server takes to guide the client. Performance evaluation of the modified algorithm for clients running both homogeneous and heterogeneous adaptation algorithms showed that the number of server actions dropped by 14% and 22%, respectively, while QoE-fairness improved by at least 6% and 10%, respectively.
- 2012. ITU-T Recommendation G.1070 Opinion Model for Video Applications.Google Scholar
- 2014. ISO/IEC 23009-1, Information Technology - Dynamic Adaptive Streaming Over HTTP (DASH) - Part 1: Media Presentation Description and Segment Formats.Google Scholar
- 2020. Global Internet Phenomena Report. Sandvine, September 2019. Retrieved March 29, 2020 from https://www.sandvine.com/press-releases/sandvine-releases-2019-global-internet-phenomena-report.Google Scholar
- 2020. The Mobile Internet Phenomena Report. Sandvine. Retrieved March 29, 2020 from https://www.sandvine.com/press-releases/sandvine-releases-2020-mobile-internet-phenomena-report-youtube-is-over-25-of-all-mobile-traffic.Google Scholar
- Saamer Akhshabi, Lakshmi Anantakrishnan, Ali C. Begen, and Constantine Dovrolis. 2012. What happens when HTTP adaptive streaming players compete for bandwidth? In Proceedings of the 22nd International Workshop on Network and Operating System Support for Digital Audio and Video (NOSSDAV’12). ACM, New York, NY, 9--14. DOI:https://doi.org/10.1145/2229087.2229092Google Scholar
Digital Library
- Saamer Akhshabi, Lakshmi Anantakrishnan, Constantine Dovrolis, and Ali Begen. 2013. Server-based traffic shaping for stabilizing oscillating adaptive streaming players. In Proceeding of the 23rd ACM Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV’13). ACM, 19--24.Google Scholar
Digital Library
- Zahaib Akhtar, Yun Seong Nam, Ramesh Govindan, Sanjay Rao, Jessica Chen, Ethan Katz-Bassett, Bruno Ribeiro, Jibin Zhan, and Hui Zhang. 2018. Oboe: Auto-tuning video ABR algorithms to network conditions. In Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication (SIGCOMM’18). CM, New York, NY, 44--58. DOI:https://doi.org/10.1145/3230543.3230558Google Scholar
Digital Library
- Sa’di Altamimi and Shervin Shirmohammadi. 2019. Client-server cooperative and fair DASH video streaming. In Proceedings of the 29th ACM Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV’19). ACM, New York, NY, USA, 1--6. DOI:https://doi.org/10.1145/3304112.3325604Google Scholar
Digital Library
- Abdelhak Bentaleb, Ali C. Begen, and Roger Zimmermann. 2018. ORL-SDN: Online reinforcement learning for SDN-enabled HTTP adaptive streaming. ACM Trans. Multimedia Comput. Commun. Appl. 14, 3, Article 71 (Aug. 2018), 28 pages. DOI:https://doi.org/10.1145/3219752Google Scholar
Digital Library
- Tyler Chesebro and Aleks Kamko. 2016. Learning Atari: An Exploration of the A3C Reinforcement Learning Method. Available: https://bcourses.berkeley.edu/files/70573736/download?download_frd=1.Google Scholar
- Federal Communications Commission. [n.d.]. Raw Data - Measuring Broadband America. Retrieved from https://www.fcc.gov/reports-research/reports.Google Scholar
- M. Gadaleta, F. Chiariotti, M. Rossi, and A. Zanella. 2017. D-DASH: A deep Q-learning framework for DASH video streaming. IEEE Trans. Cogn. Commun. Netw. 3, 4 (2017), 703--718.Google Scholar
- Mahdi Hemmati. 2017. New Bandwidth Allocation Methods to Provide Quality-of-Experience Fairness for Video Streaming Services. Ph.D. Dissertation. University of Ottawa, School of Electrical Engineering 8 Computer Science.Google Scholar
- Mahdi Hemmati, Abdulsalam Yassine, and Shervin Shirmohammadi. 2015. A Dec-POMDP model for congestion avoidance and fair allocation of network bandwidth in rate-adaptive video streaming. In Proceedings of the 2015 IEEE Symposium Series on Computational Intelligence. 1182--1189. DOI:https://doi.org/10.1109/SSCI.2015.170Google Scholar
Cross Ref
- T. Hoßfeld, L. Skorin-Kapov, P. E. Heegaard, and M. Varela. 2017. Definition of QoE fairness in shared systems. IEEE Commun. Lett. 21, 1 (2017), 184--187. https://ieeexplore.ieee.org/document/7588099.Google Scholar
Cross Ref
- T. Huang, X. Yao, C. Wu, R. Zhang, Z. Pang, and L. Sun. 2019. Tiyuntsong: A self-play reinforcement learning approach for ABR video streaming. In Proceedings of the 2019 IEEE International Conference on Multimedia and Expo (ICME’19). 1678--1683.Google Scholar
- Tianchi Huang, Chao Zhou, Rui-Xiao Zhang, Chenglei Wu, Xin Yao, and Lifeng Sun. 2019. Comyco: Quality-aware adaptive video streaming via imitation learning. In Proceedings of the 27th ACM International Conference on Multimedia (MM’19). ACM, New York, NY, 429--437. DOI:https://doi.org/10.1145/3343031.3351014Google Scholar
Digital Library
- Te-Yuan Huang, Nikhil Handigol, Brandon Heller, Nick McKeown, and Ramesh Johari. 2012. Confused, timid, and unstable: Picking a video streaming rate is hard. In Proceedings of the 2012 Internet Measurement Conference (IMC’12). ACM, New York, NY, 225--238. DOI:https://doi.org/10.1145/2398776.2398800Google Scholar
Digital Library
- R. Jain, D.-M. Chiu, W. R. Hawe, T. Zinner, T. Hoßfeld, and P. Tran-Gia. 1984. A quantitative measure of fairness and discrimination for resource allocation in shared computer system. Eastern Res. Lab. (1984), Hudson, MA.Google Scholar
- Junchen Jiang, Vyas Sekar, and Hui Zhang. 2014. Improving fairness, efficiency, and stability in HTTP-based adaptive video streaming with festive. IEEE/ACM Trans. Netw. 22, 1 (Feb. 2014), 326--340. DOI:https://doi.org/10.1109/TNET.2013.2291681Google Scholar
Digital Library
- Vijay Konda and John Tsitsiklis. 2003. On actor-critic algorithms. SIAM J. Contr. Optim. 42, 4 (2003), 1143--1166.Google Scholar
Digital Library
- Stefan Lederer, Christopher Muller, and Christian Timmerer. 2012. Dynamic adaptive streaming over HTTP dataset. In Proceedings of the 3rd Multimedia Systems Conference (MMSys’12). ACM, New York, NY, 89--94. DOI:https://doi.org/10.1145/2155555.2155570Google Scholar
Digital Library
- De Cicco Luca, Saverio Mascolo, and Vittorio Palmisano. 2011. Feedback control for adaptive live video streaming. In Proceedings of the 2nd Annual ACM Conference on Multimedia Systems (MMSys’11). ACM, 145--156.Google Scholar
- Hongzi Mao, Ravi Netravali, and Mohammad Alizadeh. 2017. Neural adaptive video streaming with pensieve. Proceedings of the ACM Special Interest Group on Data Communication (SIGCOMM’17).Google Scholar
Digital Library
- Volodymyr Mnih, Koray Kavukcuoglu, David Silver, et al. 2015. Human-level control through deep reinforcement learning. Nature 25, 2, 529--533. http://dx.doi.org/10.1038/518486aGoogle Scholar
Cross Ref
- A. Nadembega, A. Hafid, and T. Taleb. 2014. An integrated predictive mobile-oriented bandwidth-reservation framework to support mobile multimedia streaming. IEEE Trans. Wireless Commun. 13, 12 (2014), 6863--6875.Google Scholar
Cross Ref
- Vikram Nathan, Vibhaalakshmi Sivaraman, Ravichandra Addanki, Mehrdad Khani, Prateesh Goyal, and Mohammad Alizadeh. 2019. End-to-end transport for video QoE fairness. In Proceedings of the ACM Special Interest Group on Data Communication (SIGCOMM’19). ACM, New York, NY, 408--423. DOI:https://doi.org/10.1145/3341302.3342077Google Scholar
Digital Library
- Haakon Riiser, Paul Vigmostad, Carsten Griwodz, and Halvorsen Pal. 2013. Commute path bandwidth traces from 3G networks: Analysis and applications. In Proceedings of the 4th ACM Multimedia Systems Conference (MMSys’13). ACM, New York, NY, 114--118. DOI:https://doi.org/10.1145/2483977.2483991Google Scholar
Digital Library
- J. Padhye S. Floyd, M. Handley and J. Widmer. 2008. TCP Friendly Rate Control (TFRC): Protocol Specification (Proposed Standard). Retrieved from https://tools.ietf.org/html/rfc5348.Google Scholar
- M. Seufert, S. Egger, M. Slanina, T. Zinner, T. Hoßfeld, and P. Tran-Gia. 2015. A survey on quality of experience of HTTP adaptive streaming. IEEE Commun. Surv. Tutor. 17, 1 (2015), 469--492.Google Scholar
Digital Library
- M. Seufert, N. Wehner, and P. Casas. 2019. A fair share for all: TCP-inspired adaptation logic for QoE fairness among heterogeneous HTTP adaptive video streaming clients. IEEE Trans. Netw. Serv. Manag. 16, 2 (2019), 475--488.Google Scholar
Cross Ref
- Ashkan Sobhani, Abdulsalam Yassine, and Shervin Shirmohammadi. 2017. A video bitrate adaptation and prediction mechanism for HTTP adaptive streaming. ACM Trans. Multimedia Comput. Commun. Appl. 13, 2, (Mar. 2017), 25 pages. DOI:https://doi.org/10.1145/3052822Google Scholar
Digital Library
- Kevin Spiteri, Rahul Urgaonkar, and Ramesh K. Sitaraman. 2016. BOLA: Near-optimal bitrate adaptation for online videos. In Proceedings of the 35th Annual IEEE International Conference on Computer Communications (INFOCOM’16). 1--9. DOI:https://doi.org/10.1109/INFOCOM.2016.7524428Google Scholar
- Richard Sutton and Andrew Barto. 1998. Reinforcement Learning: An Introduction. MIT Press.Google Scholar
Digital Library
- The PyTorch team. 2020. PyTorch Neural Network Framework. Retrieved from https://pytorch.org/docs/stable/nn.html#lstm.Google Scholar
- Xiaoqi Yin, Abhishek Jindal, Vyas Sekar, and Bruno Sinopoli. 2015. A control-theoretic approach for dynamic adaptive video streaming over HTTP. In Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication (SIGCOMM’15). ACM, New York, NY, 325--338. DOI:https://doi.org/10.1145/2785956.2787486Google Scholar
Digital Library
Index Terms
QoE-Fair DASH Video Streaming Using Server-side Reinforcement Learning
Recommendations
Client-server cooperative and fair DASH video streaming
NOSSDAV '19: Proceedings of the 29th ACM Workshop on Network and Operating Systems Support for Digital Audio and VideoAdaptive video streaming over HTTP, such as the MPEG-DASH standard, is now widely used by video service provides to stream their videos to users. But DASH and similar methods are known to suffer from two practical challenges: on the one hand, clients ...
DQ-DASH: A Queuing Theory Approach to Distributed Adaptive Video Streaming
The significant popularity of HTTP adaptive video streaming (HAS), such as Dynamic Adaptive Streaming over HTTP (DASH), over the Internet has led to a stark increase in user expectations in terms of video quality and delivery robustness. This situation ...
Factor Selection for Reinforcement Learning in HTTP Adaptive Streaming
MMM 2014: Proceedings of the 20th Anniversary International Conference on MultiMedia Modeling - Volume 8325At present, HTTP Adaptive Streaming (HAS) is developing into a key technology for video delivery over the Internet. In this delivery strategy, the client proactively and adaptively requests a quality version of chunked video segments based on its ...






Comments