Abstract
The fundamental conflict between the enormous space of adaptive streaming videos and the limited capacity for subjective experiment casts significant challenges to objective Quality-of-Experience (QoE) prediction. Existing objective QoE models either employ pre-defined parametrization or exhibit complex functional form, achieving limited generalization capability in diverse streaming environments. In this study, we propose an objective QoE model, namely, the Bayesian streaming quality index (BSQI), to integrate prior knowledge on the human visual system and human annotated data in a principled way. By analyzing the subjective characteristics towards streaming videos from a corpus of subjective studies, we show that a family of QoE functions lies in a convex set. Using a variant of projected gradient descent, we optimize the objective QoE model over a database of training videos. The proposed BSQI demonstrates strong prediction accuracy in a broad range of streaming conditions, evident by state-of-the-art performance on four publicly available benchmark datasets and a novel analysis-by-synthesis visual experiment.
- [1] . 2017. Learning to predict streaming video QoE: Distortions, rebuffering and memory. ArXiv preprint arXiv:1703.00633 (
Mar. 2017).Google Scholar - [2] . 2018. A simple prediction fusion improves data-driven full-reference video quality assessment models. In Picture Coding Symposium. IEEE, 298–302.Google Scholar
- [3] . 2017. Continuous prediction of streaming video QoE using dynamic networks. IEEE Sig. Process. Lett. 24, 7 (
Jul. 2017), 1083–1087.Google ScholarCross Ref
- [4] . 2018. Towards perceptually optimized end-to-end adaptive video streaming. ArXiv preprint arXiv:1808.03898 (
Aug. 2018).Google Scholar - [5] . 2017. Study of temporal effects on subjective video quality of experience. IEEE Trans. Image Process. 26, 11 (
Nov. 2017), 5217–5231.Google ScholarDigital Library
- [6] . 2016. SDNDASH: Improving QoE of HTTP adaptive streaming using software defined networking. In ACM International Conference on Multimedia. ACM, 1296–1305.Google Scholar
Digital Library
- [7] . 2006. Pattern Recognition and Machine Learning. Springer-Verlag, Berlin. Google Scholar
Digital Library
- [8] . 2011. Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends® Mach. Learn. 3, 1 (
July 2011), 1–122.Google ScholarDigital Library
- [9] . 2001. Random forests. Mach. Learn. 45, 1 (
Oct. 2001), 5–32.Google ScholarDigital Library
- [10] . 2017. Cisco Visual Networking Index: Global Mobile Data Traffic Forecast Update, 2016–2021 White Paper. Retrieved from https://www.cisco.com/c/en/us/solutions/collateral/service-provider/visual-networking-index-vni/mobile-white-paper-c11-520862.html.Google Scholar
- [11] . 2016. Complexity-based consistent-quality encoding in the cloud. In IEEE International Conference on Image Processing.IEEE, 1484–1488.Google Scholar
- [12] . 2011. Understanding the impact of video quality on user engagement. ACM SIGCOMM Comput. Commun. Rev. 41, 4 (
Aug. 2011), 362–373.Google ScholarDigital Library
- [13] . 2000. A unified bias-variance decomposition. In International Conference on Machine Learning. 231–238.Google Scholar
- [14] . 2019. Waterloo Streaming Quality-of-Experience Database IV. Retrieved from http://ece.uwaterloo.ca/zduanmu/waterloosqoe4.Google Scholar
- [15] . 2019. Pairwise Comparison of Objective QoE Models via Analysis-by-synthesis. Retrieved from http://ivc.uwaterloo.ca/research/KSQI/demo/.Google Scholar
- [16] . 2020. Assessing the quality-of-experience of adaptive bitrate video streaming. arXiv preprint arXiv:2008.08804 (2020).Google Scholar
- [17] . 2021. To Appear. Quantifying visual image quality: A Bayesian view. Ann. Rev. Vis. Sci. (
Sep. 2021, To Appear).Google ScholarCross Ref
- [18] . 2017. Quality-of-experience of adaptive video streaming: Exploring the space of adaptations. In ACM International Conference on Multimedia. ACM, 1752–1760.Google Scholar
Digital Library
- [19] . 2018. Quality-of-experience for adaptive streaming videos: An expectation confirmation theory motivated approach. IEEE Trans. Image Process. 27, 12 (
Dec. 2018), 6135–6146.Google ScholarCross Ref
- [20] . 2018. A quality-of-experience database for adaptive video streaming. IEEE Trans. Broadcast. 64, 2 (
June 2018), 474–487.Google ScholarCross Ref
- [21] . 2017. A quality-of-experience index for streaming video. IEEE J. Select. Topics Sig. Process. 11, 1 (
Sep. 2017), 154–166.Google ScholarCross Ref
- [22] . 2019. Bidirectional LSTM for continuously predicting QoE in HTTP adaptive streaming. In International Conference on Information Science and Systems. 156–160.Google Scholar
Digital Library
- [23] . 2016. Microsoft Smooth Streaming. Retrieved from https://www.encoding.com/microsoft-smooth-streaming/.Google Scholar
- [24] . 2020. Streaming video QoE modeling and prediction: A long short-term memory approach. IEEE Trans. Circ. Syst. Video Technol. 30, 3 (
Mar. 2020), 661–673.Google ScholarDigital Library
- [25] . 2014. A survey on concept drift adaptation. ACM Comput. Surv. 46, 4 (
Apr. 2014), 1–37.Google ScholarDigital Library
- [26] . 2017. A subjective and objective study of stalling events in mobile streaming videos. IEEE Trans. Circ. Syst. Video Technol. 29, 1 (2017), 183–197.Google Scholar
Digital Library
- [27] . 2018. Learning a continuous-time streaming video QoE model. IEEE Trans. Image Process. 27, 5 (
May 2018), 2257–2271.Google ScholarCross Ref
- [28] . 2013. Representation switch smoothing for adaptive HTTP streaming. In IEEE International Workshop on Perceptual Quality of Systems. ISCA/DEGA, 178–183.Google Scholar
- [29] . 1996. Neural Network Design. Vol. 20. PWS Pub. Boston.Google Scholar
Digital Library
- [30] . 1989. Multilayer feedforward networks are universal approximators. Neural Netw. 2, 5 (
Jan. 1989), 359–366.Google ScholarDigital Library
- [31] . 2013. Internet video delivery in YouTube: From traffic measurements to quality of experience. In Data Traffic Monitoring and Analysis. Springer, Berlin, 264–301.Google Scholar
Cross Ref
- [32] . 2011. Quantification of YouTube QoE via crowdsourcing. In IEEE International Symosium on Multimedia. IEEE, 494–499.Google Scholar
- [33] . 2019. Comyco: Quality-aware adaptive video streaming via imitation learning. In ACM International Conference on Multimedia. 429–437.Google Scholar
Digital Library
- [34] . 2015. A buffer-based approach to rate adaptation: Evidence from a large video streaming service. ACM SIGCOMM Comput. Commun. Rev. 44, 4 (
Feb. 2015), 187–198.Google ScholarDigital Library
- [35] . 2010. 2000 fps real-time vision system with high-frame-rate video recording. In IEEE International Conference on Robotics and Automation. IEEE, 1536–1541.Google Scholar
Cross Ref
- [36] . 1993. Recommendation: Methodology for the Subjective Assessment of the Quality of Television Pictures.Google Scholar
- [37] . 2014. Improving fairness, efficiency, and stability in HTTP-based adaptive video streaming with FESTIVE. IEEE/ACM Trans. Netw. 22, 1 (
Feb. 2014), 326–340.Google ScholarCross Ref
- [38] . 2003. High dynamic range video. ACM Trans. Graph. 22, 3 (
July 2003), 319–325.Google ScholarDigital Library
- [39] . 2012. QoE assessment model for video streaming service using QoS parameters in wired-wireless network. In IEEE International Conference on Advanced Communications Technology. 459–464.Google Scholar
- [40] . 2007. GPAC: Open source multimedia framework. In ACM International Conference on Multimedia. ACM, 1009–1012.Google Scholar
Digital Library
- [41] . 2015. Optimal Adaptive Streaming Formats MPEG-DASH & HLS Segment Length. Retrieved from https://bitmovin.com/mpeg-dash-hls-segment-length/.Google Scholar
- [42] . 2016. Toward a Practical Perceptual Video Quality Metric. Retrieved from http://techblog.netflix.com/2016/06/toward-practical-perceptual-video.html.Google Scholar
- [43] . To Appear. AVC, HEVC, VP9, AVS2, or AV1? - A comparative study of state-of-the-art video encoders on 4K videos. In International Conference on Image Analysis and Recognition. AIMI.Google Scholar
- [44] . 2014. Probe and adapt: Rate adaptation for HTTP video streaming at scale. IEEE J. Select. Areas Commun. 32, 4 (
Apr. 2014), 719–733.Google ScholarCross Ref
- [45] . 2018. End-to-end blind quality assessment of compressed videos using deep neural networks. In ACM International Conference on Multimedia. ACM, 546–554.Google Scholar
Digital Library
- [46] . 2012. A case for a coordinated internet video control plane. ACM SIGCOMM Comput. Commun. Rev. 42, 4 (
Sep. 2012), 359–370.Google ScholarDigital Library
- [47] . 2015. Deriving and validating user experience model for DASH video streaming. IEEE Trans. Broadcast. 61, 4 (
Dec. 2015), 651–665.Google ScholarCross Ref
- [48] . 2019. Group maximum differentiation competition: Model comparison with few samples. IEEE Trans. Patt. Anal. Mach. Intell. (2019),
DOI: 10.1109/TPAMI.2018.2889948Google Scholar - [49] . 2017. Neural adaptive video streaming with pensieve. In ACM SIGCOMM. ACM, 197–210.Google Scholar
- [50] . 2001. Quality of Service in Communications Networks. Wiley.Google Scholar
- [51] . 2012. QDASH: A QoE-aware DASH system. In ACM Conference on Multimedia Systems. ACM, 11–22.Google Scholar
Digital Library
- [52] . 2012. Video quality assessment on mobile devices: Subjective, behavioral and objective studies. IEEE J. Select. Topics Sig. Process. 6, 6 (
Oct. 2012), 652–671.Google ScholarCross Ref
- [53] . 2015. Perceptual quality assessment of high frame rate video. In IEEE International Conference on Multimedia and Signal Processing. IEEE, 1–6.Google Scholar
Cross Ref
- [54] 2015. Per-title Encode Optimization. Retrieved from http://techblog.netflix.com/2015/12/per-title-encode-optimization.html.Google Scholar
- [55] . 2005. Light Field Photography with a Hand-held Plenoptic Camera. Computer Science Technical Report.Google Scholar
- [56] . 2011. Flicker effects in adaptive video streaming to handheld devices. In ACM International Conference on Multimedia. ACM, 463–472.Google Scholar
Digital Library
- [57] . 2014. The impact of network impairment on quality of experience (QoE) in H.265/HEVC video streaming. IEEE Trans. Consum. Electron. 60, 2 (2014), 242–250.Google Scholar
Cross Ref
- [58] . 1980. A cognitive model of the antecedents and consequences of satisfaction decisions. J. Market. Res. 17, 4 (
Nov. 1980), 460–469.Google ScholarCross Ref
- [59] . 2017. Parametric Bitstream-based Quality Assessment of Progressive Download and Adaptive Audiovisual Streaming Services Over Reliable Transport. Retrieved from https://www.itu.int/rec/dologin_pub.asp?lang=e&id=T-REC-P.1203-201710-I!!PDF-E&type=items.Google Scholar
- [60] . 2004. Sporadic frame dropping impact on quality perception. In Human Vision and Electronic Imaging IX. SPIE, 182–194.Google Scholar
- [61] . 2016. Impact of video content and transmission impairments on Quality of Experience. Multim. Tools Applic. 75, 23 (2016), 16461–16485.Google Scholar
Digital Library
- [62] . 2015. Image database TID2013: Peculiarities, results and perspectives. Sig. Process.: Image Commun. 30 (
Jan. 2015), 57–77.Google ScholarDigital Library
- [63] . 2013. Perceptual experience of time-varying video quality. In IEEE International Workshop Quality of Multimedia Experience. IEEE, 218–223.Google Scholar
- [64] . 2015. Display device-adapted video quality-of-experience assessment. In SPIE. SPIE, 939406.1–939406.11.Google Scholar
- [65] . 2013. Commute path bandwidth traces from 3G networks: Analysis and applications. In ACM Conference on Multimedia Systems. ACM, 114–118.Google Scholar
Digital Library
- [66] . 2014. The impact of video-quality-level switching on user quality of experience in dynamic adaptive streaming over HTTP. EURASIP J. Wirel. Commun. Netw. 2014, 1 (2014), 1–15.Google Scholar
Cross Ref
- [67] . 2006. A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Trans. Image Process. 15, 11 (
Nov. 2006), 3440–3451.Google ScholarDigital Library
- [68] . 2012. Quality of experience estimation for adaptive HTTP/TCP video streaming using H.264/AVC. In IEEE Consumer Communications & Networking Conference. IEEE, 1–6.Google Scholar
Cross Ref
- [69] . 2016. BOLA: Near-optimal bitrate adaptation for online videos. In IEEE International Conference on Computer Communications.IEEE, 1–9.Google Scholar
Digital Library
- [70] . 2017. OSQP: An operator splitting solver for quadratic programs. ArXiv preprint arXiv:1711.08013 (
Nov. 2017).Google Scholar - [71] . 2015. Optimal selection of adaptive streaming representations. ACM Trans. Multim. Comput., Commun., Applic. 11, 2s (
Feb. 2015), 1–43.Google ScholarDigital Library
- [72] . 2011. How to Analyze Paired Comparison Data. University of Washington, Technical Report. UWEETR-2011-0004.Google Scholar
- [73] . 2016. HTTP/2-based adaptive streaming of HEVC video over 4G/LTE networks. IEEE Commun. Lett. 20, 11 (
Aug. 2016), 2177–2180.Google ScholarCross Ref
- [74] . 2000. Final Report from the Video Quality Experts Group on the Validation of Objective Models of Video Quality Assessment. Technical Report. Retrieved from http://www.vqeg.org/.Google Scholar
- [75] . 2001. Internet QoS: Architectures and Mechanisms for Quality of Service. Morgan Kaufmann.Google Scholar
- [76] . 2004. Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Process. 13, 4 (
Apr. 2004), 600–612.Google ScholarDigital Library
- [77] . 2009. Mean squared error: Love it or leave it? A new look at signal fidelity measures. IEEE Sig. Process. Mag. 26, 1 (
Jan. 2009), 98–117.Google ScholarCross Ref
- [78] . 2017. Method and system for automatic user quality-of-experience measurement of streaming video.
US Patent WO/2017/152274. Google Scholar - [79] . 2017. Begin with the end in mind: A unified end-to-end quality-of-experience monitoring, optimization and management framework. In SMPTE Annual Technical Conference and Exhibition. SMPTE, 1–11.Google Scholar
Cross Ref
- [80] . 2008. Maximum differentiation (MAD) competition: A methodology for comparing computational models of perceptual quantities. J. Vis. 8, 12 (2008), 8–8.Google Scholar
Cross Ref
- [81] . 2016. Method and system for smart adaptive video streaming driven by perceptual quality-of-experience estimations.
US Patent WO/2016/123721. Google Scholar - [82] . 2007. Objective video quality assessment method for evaluating effects of freeze distortion in arbitrary video scenes. In Image Quality and System Performance IV, Vol. 64940P. SPIE, 1–8.Google Scholar
- [83] . 2014. Assessing quality of experience for adaptive HTTP video streaming. In IEEE International Conference on Multimedia and Expo Workshop. IEEE, 1–6.Google Scholar
- [84] . 2015. A control-theoretic approach for dynamic adaptive video streaming over HTTP. ACM SIGCOMM Comput. Commun. Rev. 45, 4 (
Apr. 2015), 325–338.Google ScholarDigital Library
Index Terms
A Bayesian Quality-of-Experience Model for Adaptive Streaming Videos
Recommendations
Quality of Experience of adaptive video streaming
The usage of HTTP adaptive streaming (HAS) has become widely spread in multimedia services. Because it allows the service providers to improve the network resource utilization and user's Quality of Experience (QoE). Using this technology, the video ...
Feedback control for adaptive live video streaming
MMSys '11: Proceedings of the second annual ACM conference on Multimedia systemsMultimedia content feeds an ever increasing fraction of the Internet traffic. Video streaming is one of the most important applications driving this trend. Adaptive video streaming is a relevant advancement with respect to classic progressive download ...
Quality-of-Experience of Adaptive Video Streaming: Exploring the Space of Adaptations
MM '17: Proceedings of the 25th ACM international conference on MultimediaWith the remarkable growth of adaptive streaming media applications, especially the wide usage of dynamic adaptive streaming schemes over HTTP (DASH), it becomes ever more important to understand the perceptual quality-of-experience (QoE) of end users, ...






Comments