Stefanos Antaris
ABSTRACT
In this study, we present a meta-learning model to adapt the predictions of the network's capacity between viewers who participate in a live video streaming event. We propose the MELANIE model, where an event is formulated as a Markov Decision Process, performing meta-learning on reinforcement learning tasks. By considering a new event as a task, we design an actor-critic learning scheme to compute the optimal policy on estimating the viewers' high-bandwidth connections. To ensure fast adaptation to new connections or changes among viewers during an event, we implement a prioritized replay memory buffer based on the Kullback-Leibler divergence of the reward/throughput of the viewers' connections. Moreover, we adopt a model-agnostic meta-learning framework to generate a global model from past events. As viewers scarcely participate in several events, the challenge resides on how to account for the low structural similarity of different events. To combat this issue, we design a graph signature buffer to calculate the structural similarities of several streaming events and adjust the training of the global model accordingly. We evaluate the proposed model on the link weight prediction task on three real-world datasets of live video streaming events. Our experiments demonstrate the effectiveness of our proposed model, with an average relative gain of 25% against state-of-the-art strategies. For reproduction purposes, our evaluation datasets and implementation are publicly available at https://github.com/stefanosantaris/melanie
- "Gauging demand for enterprise streaming - 2020 - investment trends in times of global change," https://www.ibm.com/downloads/cas/DEAKXQ5P, 2020, [Online; accessed 29-January-2021].Google Scholar
- "Bringing down the walls," https://www.akamai.com/us/en/multimedia/documents/ebooks/broadcast-over-the-internet-book-one-bringing-down-the-walls-ebook.pdf, 2017, [Online; accessed 29-January-2021].Google Scholar
- S. Antaris and D. Rafailidis, "Vstreamdrls: Dynamic graph representation learning with self-attention for enterprise distributed video streaming solutions," in ASONAM, 2020.Google Scholar
- R. Roverso, R. Reale, S. El-Ansary, and S. Haridi, "Smoothcache 2.0: Cdn-quality adaptive http live streaming on peer-to-peer overlays," in MMSys, 2015, p. 61--72.Google ScholarDigital Library
- Y. Ma, Z. Guo, Z. Ren, J. Tang, and D. Yin, "Streaming graph neural networks," in SIGIR, 2020, p. 719--728.Google ScholarDigital Library
- S. Kumar, X. Zhang, and J. Leskovec, "Predicting dynamic embedding trajectory in temporal interaction networks," in KDD, 2019, p. 1269--1278.Google Scholar
- A. Pareja, G. Domeniconi, J. Chen, T. Ma, T. Suzumura, H. Kanezashi, T. Kaler, T. B. Schardl, and C. E. Leiserson, "Evolvegcn: Evolving graph convolutional networks for dynamic graphs," in AAAI, 2020, pp. 5363--5370.Google ScholarCross Ref
- A. Sankar, Y. Wu, L. Gou, W. Zhang, and H. Yang, "Dysat: Deep neural representation learning on dynamic graphs via self-attention networks," in WSDM, 2020, p. 519--527.Google ScholarDigital Library
- S. Antaris, D. Rafailidis, and S. Girdzijauskas, "A deep graph reinforcement learning model for improving user experience in live video streaming," 2021.Google ScholarCross Ref
- S. Antaris, D. Rafailidis, and S. Girdzijauskas, "Egad: Evolving graph representation learning with self-attention and knowledge distillation for live video streaming events," in Big Data, 2020, pp. 1455--1464.Google Scholar
- "Supplementary Material," https://github.com/stefanosantaris/melanie/blob/main/supplementary/supplementary.pdf, 2021, [Online; accessed 14-July-2021].Google Scholar
- A. J. Bose, A. Jain, P. Molino, and W. L. Hamilton, "Meta-graph: Few shot link prediction via meta learning," 2020.Google Scholar
- C. Finn, P. Abbeel, and S. Levine, "Model-agnostic meta-learning for fast adaptation of deep networks," in ICML, 2017, pp. 1126--1135.Google Scholar
- Y. Lu, Y. Fang, and C. Shi, "Meta-learning on heterogeneous information networks for cold-start recommendation," in SIGKDD, 2020, p. 1563--1573.Google Scholar
- S. Khodadadeh, L. Boloni, and M. Shah, "Unsupervised meta-learning for few-shot image classification," in NeurIPS, 2019, pp. 10 132--10 142.Google Scholar
- Y. Zhu, C. Liu, and S. Jiang, "Multi-attention meta learning for few-shot fine-grained image recognition," in IJCAI, 2020, pp. 1090--1096.Google Scholar
- X. S. Wei, P. Wang, L. Liu, C. Shen, and J. Wu, "Piecewise classifier mappings: Learning fine-grained learners for novel categories with few examples," IEEE Transactions on Image Processing, vol. 28, no. 12, pp. 6116--6125, 2019.Google ScholarDigital Library
- M. Vartak, A. Thiagarajan, C. Miranda, J. Bratman, and H. Larochelle, "A meta-learning perspective on cold-start recommendations for items," in NeurIPS, 2017, pp. 6904--6914.Google Scholar
- M. Brockschmidt, "Gnn-film: Graph neural networks with feature-wise linear modulation," in ICML, 2020, pp. 1144--1152.Google Scholar
- S. Hochreiter, A. S. Younger, and P. R. Conwell, "Learning to learn using gradient descent," in ICANN, 2001, pp. 87--94.Google Scholar
- T. Schaul, J. Quan, I. Antonoglou, and D. Silver, "Prioritized experience replay," in ICLR, 2016.Google Scholar
- H. Lee, J. Im, S. Jang, H. Cho, and S. Chung, "Melu: Meta-learned user preference estimator for cold-start recommendation," in SIGKDD, 2019, p. 1073--1082.Google Scholar
- P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio, "Graph Attention Networks," in ICLR, 2018.Google Scholar
- R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. A Bradford Book, 2018.Google Scholar
- D. Silver, G. Lever, N. Heess, T. Degris, D. Wierstra, and M. Riedmiller, "Deterministic policy gradient algorithms," in ICML, 2014, p. I-387--I-395.Google ScholarDigital Library
- W. Fedus, P. Ramachandran, R. Agarwal, Y. Bengio, H. Larochelle, M. Rowland, and W. Dabney, "Revisiting fundamentals of experience replay," in ICML, 2020, pp. 3061--3071.Google Scholar
- S. Kullback and R. A. Leibler, "On information and sufficiency," Ann. Math. Statist., vol. 22, pp. 79--86, 1951.Google ScholarCross Ref
- K.-H. Lai, D. Zha, K. Zhou, and X. Hu, "Policy-gnn: Aggregation optimization for graph neural networks," in KDD, 2020, p. 461--471.Google ScholarDigital Library
- F. Dobrian, V. Sekar, A. Awan, I. Stoica, D. Joseph, A. Ganjam, J. Zhan, and H. Zhang, "Understanding the impact of video quality on user engagement," in SIGCOMM, 2011, p. 362--373.Google ScholarDigital Library
- T. Huang, R.-X. Zhang, C. Zhou, and L. Sun, "Qarc: Video quality aware rate control for real-time video streaming based on deep reinforcement learning," in MM, 2018, p. 1208--1216.Google Scholar
Index Terms
(auto-classified)Meta-reinforcement learning via buffering graph signatures for live video streaming events
Recommendations
MetaLive: Meta-Reinforcement Learning Based Collective Bitrate Adaptation for Multi-Party Live Streaming
Euro-Par 2023: Parallel ProcessingAbstractMulti-party interactive live video streaming applications have attracted millions of daily active users and are anticipated a blooming market in the next few years. A fundamental research problem in live streaming is bitrate coordination, which ...
Reinforcement learning-based rate adaptation in dynamic video streaming
AbstractVideo streaming stands out as the most significant traffic type consumed by mobile devices. This increased demand has been a major driver for research on bitrate adaptation algorithms. Bitrate adaptation ensures high user-perceived quality, which, ...
Multi-task Learning for User Engagement and Adoption in Live Video Streaming Events
Machine Learning and Knowledge Discovery in Databases. Applied Data Science TrackAbstractNowadays, live video streaming events have become a mainstay in viewer’s communication in large international enterprises. Provided that viewers are distributed worldwide, the main challenge resides on how to schedule the optimal event’s time so ...
Comments