Abstract
The article presents a novel affective content-aware adaptation scheme (ACAA) to optimize Quality of Experience (QoE) for dynamic adaptive video streaming over HTTP (DASH). Most of the existing DASH adaptation schemes conduct video bit-rate adaptation based on an estimation of available network resources, which ignore user preference on affective content (AC) embedded in video data streaming over the network. Since the personal demands to AC is very different among all viewers, to satisfy individual affective demand is critical to improve the QoE in commercial video services. However, the results of video affective analysis cannot be applied into a current adaptive streaming scheme directly. Correlating the AC distributions in user's viewing history to each being streamed segment, the affective relevancy can be inferred as an affective metric for the AC related segment. Further, we have proposed an ACAA scheme to optimize QoE for user desired affective content while taking into account both network status and affective relevancy. We have implemented the ACAA scheme over a realistic trace-based evaluation and compared its performance in terms of network performance, QoE with that of Probe and Adaptation (PANDA), buffer-based adaptation (BBA), and Model Predictive Control (MPC). Experimental results show that ACAA can preserve available buffer time for future being delivered affective content preferred by viewer's individual preference to achieve better QoE in affective contents than those normal contents while remain the overall QoE to be satisfactory.
- Christian Timmerer and Ali C. Begen. 2014. Over-the-top content delivery: State of the art and challenges ahead. In Proceedings of the 22nd ACM International Conference on Multimedia (MM’14). ACM, New York, NY, 1231–1232. DOI:https://doi.org/10.1145/2647868.2654849Google Scholar
- Cisco. 2019. Cisco Visual Net-working Index: Forecast and Methodology 2017-2022. Retrieved January 12, 2019 from https://www.cisco.com/c/en/us/solutions/collateral/service-provider/visual-networking-index-vni/white-paper-c11-741490.pdf.Google Scholar
- Michael Seufert, Sebastian Egger, Martin Slanina, Thomas Zinner, Tobias Hoßfeld, and Phuoc Tran-Gia. 2015. A survey on quality of experience of HTTP adaptive streaming. IEEE Commun. Surv. Tutor. 17, 1 (2015), 469–492. DOI:http://dx.doi.org/10.1109/COMST.2014.2360940Google Scholar
Digital Library
- J. Kua, G. Armitage, and F. Branch. 2017. A survey of rate adaptation techniques for dynamic adaptive streaming over HTTP. IEEE Commun. Surv. Tutor. 19, 3 (2017), 1842–1866. DOI:http://dx.doi.org/10.1109/COMST.2017.2685630Google Scholar
Digital Library
- Z. Li, X. Zhu, and J. Gahm. 2014. Probe and adapt: Rate adaptation for HTTP video streaming at scale. IEEE J. Select. Areas Commun. 32, 4 (2014), 719–733. DOI:http://dx.doi.org/10.1109/JSAC.2014.140405Google Scholar
Cross Ref
- T. C. Thang, H. T. Le, and A. T. Pham. 2014. An evaluation of bitrate adaptation methods for HTTP live streaming. IEEE J. Select. Areas Commun. 32, 4 (2014), 693–705. DOI:http://dx.doi.org/10.1109/JSAC.2014.140403Google Scholar
Cross Ref
- Te-Yuan Huang, Ramesh Johari, Nick McKeown, Matthew Trunnell, and Mark Watson. 2014. A buffer-based approach to rate adaptation: Evidence from a large video streaming service. SIGCOMM Comput. Commun. Rev. 44, 4 (2014), 187–198. DOI:https://doi.org/10.1145/2740070.2626296Google Scholar
Digital Library
- Xiaoqi Yin, Abhishek Jindal, Vyas Sekar, and Bruno Sinopoli. 2015. A control-theoretic approach for dynamic adaptive video streaming over HTTP. SIGCOMM Comput. Commun. Rev. 45, 4 (2015), 325–338. DOI:https://doi.org/10.1145/2829988.2787486Google Scholar
Digital Library
- Hongzi Mao, Ravi Netravali, and Mohammad Alizadeh. 2017. Neural adaptive video streaming with pensieve. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication (SIGCOMM’17). ACM, New York, NY, 197–210. DOI:https://doi.org/10.1145/3098822.3098843Google Scholar
Digital Library
- S. Hu, L. Sun, C. Gui, E. Jammeh, and I.-H. Mkwawa. 2014. Content-aware adaptation scheme for QoE optimized DASH applications. In Proceedings of the IEEE Global Communications Conference (GLOBECOM’14). 1336–1341. DOI:http://dx.doi.org/10.1109/GLOCOM.2014.7036993Google Scholar
Cross Ref
- G. Gao, H. Zhang, H. Hu, and Y. Wen. 2018. Optimizing quality of experience for adaptive bitrate streaming via viewer interest inference. IEEE Trans. Multimedia 20, 12 (2018), 3399–3413. DOI:http://dx.doi.org/10.1109/TMM.2018.2838330Google Scholar
Digital Library
- Maarten Wijnants, Gustavo Rovelo, Peter Quax, and Wim Lamotte. 2016. A pragmatically designed adaptive and web-compliant object-based video streaming methodology: Implementation and subjective evaluation. In Proceedings of the 24th ACM International Conference on Multimedia (MM’16). ACM, New York, NY, 1267–1276. DOI:https://doi.org/10.1145/2964284.2964300Google Scholar
Digital Library
- Shenghong Hu, Lingfen Sun, Chunxia Xiao, and Chao Gui. 2017. Semantic-aware adaptation scheme for soccer video over MPEG-DASH. In Proceedings of the IEEE International Conference on Multimedia 8 Expo (ICME’17). 493–498. DOI:http://dx.doi.org/10.1109/ICME.2017.8019541Google Scholar
Cross Ref
- P. Ekman. 1982. Emotion in the Human Face (2nd ed). Cambridge University Press, Cambridge, UK.Google Scholar
- M. Xu, J. Wang, X. He, J. S. Jin, S. Luo, and H. Lu. 2014. A three-level framework for affective content analysis and its case studies. Multimedia Tools Appl. 70, 2 (2014), 757–779.Google Scholar
Digital Library
- Min Xu, Jesse S. Jin, Suhuai Luo, and Lingyu Duan. 2008. Hierarchical movie affective content analysis based on arousal and valence features. In Proceedings of the 16th ACM International Conference on Multimedia (MM’08). ACM, New York, NY, 677–680. DOI:https://doi.org/10.1145/1459359.1459457Google Scholar
Digital Library
- H. Zhang and M. Xu. 2018. Recognition of emotions in user-generated videos with kernelized features. IEEE Trans. Multimedia 20, 10 (2018), 2824–2835. DOI:http://dx.doi.org/10.1109/TMM.2018.2808760Google Scholar
Cross Ref
- C. Chen, Z. Wu, and Y. G. Jiang. 2016. Emotion in context: deep semantic feature fusion for video emotion recognition. In Proceedings of the ACM International Conference on Multimedia. 127–131. DOI:http://dx.doi.org/10.1145/2964284.2967196Google Scholar
- K. Peng, T. Chen, A. Sadovnik, and A. Gallagher. 2015. A mixed bag of emotions: Model, predict, and transfer emotion distributions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’15). 860–868. DOI:http://dx.doi.org/10.1109/CVPR.2015.7298687Google Scholar
- S. Zhao, G. Amir, G. Ding, Y. Gao, J. Han, and K. Keutzer. 2019. Personalized emotion recognition by personality-aware high-order learning of physiological signals. ACM Trans. Multimedia Comput. Commun. Appl. 15, 1s (2019), 14:1–14:18. DOI:http://dx.doi.org/10.1145/3233184Google Scholar
Digital Library
- S. Wang and Q. Ji. 2015. Video affective content analysis: A survey of state-of-the-art methods. IEEE Trans. Affect. Comput. 6, 4 (2015), 410–430. DOI:http://dx.doi.org/10.1109/TAFFC.2015.2432791Google Scholar
Digital Library
- S. Zhao, G. Ding, Y. Gao, and J. Han. 2017. Approximating discrete probability distribution of image emotions by multi-modal features fusion. In Proceedings of the 26th International Joint Conference on Artificial Intelligence. 4669–4675. DOI:http://dx.doi.org/10.24963/ijcai.2017/651Google Scholar
- S. Zhao, G. Ding, Y. Gao, and J. Han. 2017. Learning visual emotion distributions via multi-modal features fusion. In Proceedings of the 2017 ACM on Multimedia Conference. 369--377. DOI:http://doi.acm.org/10.1145/3123266.3130858Google Scholar
- S. Zhao, H. Yao, Y. Gao, G. Ding, and T.-S. Chua. 2018. Predicting personalized image emotion perceptions in social networks. IEEE Trans. Affect. Comput. 9, 4 (2018), 526–540. DOI:http://dx.doi.org/10.1109/TAFFC.2016.2628787Google Scholar
Digital Library
- J. He, Z. Xue, D. Wu, D. O. Wu, and Y. Wen. 2014. CBM: Online strategies on cost-aware buffer management for mobile video streaming. IEEE Trans. Multimedia 16, 1 (2014), 242–252. DOI:10.1145/2413176.2413189Google Scholar
Digital Library
- Y. Xu, E. Altman, R. El-Azouzi, M. Haddad, S. Elayoubi, and T. Jimenez. 2014. Analysis of buffer starvation with application to objective qoe optimization of streaming services. IEEE Trans. Multimedia 16, 3 (2014), 813–827. DOI:http://dx.doi.org/10.1145/2413176.2413189Google Scholar
Digital Library
- S. Tavakoli, S. Egger, M. Seufert, R. Schatz, K. Brunnström, and N. García. 2016. Perceptual quality of HTTP adaptive streaming strategies: Cross-experimental analysis of multi-laboratory and crowdsourced subjective studies. IEEE J. Select. Areas Commun. 34, 8 (2016), 2141–2153. DOI:http://dx.doi.org/10.1109/JSAC.2016.2577361Google Scholar
Digital Library
- S. F. Chang and A. Vetro. 2005. Video adaptation: Concepts, technologies and open issues. Proc. IEEE 93, 1 (2005), 148–158. DOI:http://dx.doi.org/10.1109/JPROC.2004.839600Google Scholar
Cross Ref
- K. Asiya, S. Lingfen, and I. Emmanuel. 2012. QoE prediction model and its application in video quality adaptation over UMTS networks. IEEE Trans. Multimedia 14, 2 (2012), 431–442. DOI:http://dx.doi.org/10.1109/TMM.2011.2176324Google Scholar
Digital Library
- M. Xu, J. Li, L. T. Chia, Y. Hu, B.-S. Lee, D. Rajan, and J. S. Jin. 2006. Event on demand with MPEG-21 video adaptation system. In Proceedings of the 14th Annual ACM International Conference on Multimedia (MM’06). 921–930. DOI:http://dx.doi.org/10.1145/1180639.1180843Google Scholar
- B. Ciubotaru, G. Ghinea, and G.-M. Muntean. 2014. Subjective assessment of region of interest-aware adaptive multimedia streaming quality. IEEE Trans. Broadcast. 60, 1 (2014), 50–60. DOI:http://dx.doi.org/10.1109/TBC.2013.2290238Google Scholar
Cross Ref
- X. Chen, J.-N. Hwang, and D. Meng. 2017. A quality-of-content-based joint source and channel coding for human detections in a mobile surveillance cloud. IEEE Trans. Circ. Syst. Vid. Technol. 27, 1 (2017), 19–31. DOI:http://dx.doi.org/10.1109/TCSVT.2016.2539758Google Scholar
Digital Library
- Kevin Spiteri, Ramesh Sitaraman, and Daniel Sparacio. 2018. From theory to practice: Improving bitrate adaptation in the DASH reference player. In Proceedings of the 9th ACM Multimedia Systems Conference (MMSys’18). ACM, New York, NY, 123–137. DOI:https://doi.org/10.1145/3204949.3204953Google Scholar
Digital Library
- Blender Foundation. 2008. BigBuckBunny. Retrieved January 2, 2019 from https://peach.blender.org/.Google Scholar
- R. Rassool. 2017. VMAF reproducibility: Validating a perceptual practical video quality metric. In Proceedings of the IEEE International Symposium in Broadband Multimedia Systems and Broadcasting (BMSB’17). 1–2. DOI:http://dx.doi.org/10.1109/ BMSB.2017.7986143Google Scholar
Cross Ref
- H. Riiser, P. Vigmostad, C. Griwodz, and Halvorsen Pål. 2013. Commute path bandwidth traces from 3G networks: analysis and applications. In Proceedings of the International Conference on Multimedia Systems (MMSys’13). 114–118. DOI:http://dx.doi.org/10.1145/2483977.2483991Google Scholar
Digital Library
- Y. Nie, C. Xiao, H. Sun, and P. Li. 2013. Compact video synopsis via global spatiotemporal optimization. IEEE Trans. Vis. Comput. Graph. 19, 10 (2013), 1664–1676. DOI:http://dx.doi.org/10.1109/TVCG.2012.176Google Scholar
Cross Ref
- L. Zhang, Q. Yan, Z. Liu, H. Zou, and C. Xiao. 2017. Illumination decomposition for photograph with multiple light sources. IEEE Trans. Image Process. 26, 9 (2017), 4114–4127. DOI:http://dx.doi.org/10.1109/TIP.2017.2712283Google Scholar
Cross Ref
Index Terms
Affective Content-aware Adaptation Scheme on QoE Optimization of Adaptive Streaming over HTTP
Recommendations
Multipath-based transmission scheme for improving the QoE of HTTP adaptive streaming
Highlights- The drawbacks of existing multipath-based transmission schemes are analyzed.
- ...
AbstractThe number of users using multimedia streaming services in various environments is increasing. Accordingly, HyperText Transfer Protocol (HTTP) adaptive streaming has attracted attention. Recently, there have been many studies on ...
On the impact of redirection on HTTP adaptive streaming services in federated CDNs
AIMS'13: Proceedings of the 7th IFIP WG 6.6 international conference on Autonomous Infrastructure, Management, and Security: emerging management mechanisms for the future internet - Volume 7943HTTP Adaptive Streaming (HAS) refers to a set of novel streaming services that allow clients to adapt video quality based on current network conditions. Their use of existing HTTP delivery infrastructure makes them perfectly suited for deployment on ...
A Survey on Quality of Experience of HTTP Adaptive Streaming
Changing network conditions pose severe problems to video streaming in the Internet. HTTP adaptive streaming (HAS) is a technology employed by numerous video services that relieves these issues by adapting the video to the current network conditions. It ...






Comments