skip to main content
research-article

Affective Content-aware Adaptation Scheme on QoE Optimization of Adaptive Streaming over HTTP

Authors Info & Claims
Published:07 December 2019Publication History
Skip Abstract Section

Abstract

The article presents a novel affective content-aware adaptation scheme (ACAA) to optimize Quality of Experience (QoE) for dynamic adaptive video streaming over HTTP (DASH). Most of the existing DASH adaptation schemes conduct video bit-rate adaptation based on an estimation of available network resources, which ignore user preference on affective content (AC) embedded in video data streaming over the network. Since the personal demands to AC is very different among all viewers, to satisfy individual affective demand is critical to improve the QoE in commercial video services. However, the results of video affective analysis cannot be applied into a current adaptive streaming scheme directly. Correlating the AC distributions in user's viewing history to each being streamed segment, the affective relevancy can be inferred as an affective metric for the AC related segment. Further, we have proposed an ACAA scheme to optimize QoE for user desired affective content while taking into account both network status and affective relevancy. We have implemented the ACAA scheme over a realistic trace-based evaluation and compared its performance in terms of network performance, QoE with that of Probe and Adaptation (PANDA), buffer-based adaptation (BBA), and Model Predictive Control (MPC). Experimental results show that ACAA can preserve available buffer time for future being delivered affective content preferred by viewer's individual preference to achieve better QoE in affective contents than those normal contents while remain the overall QoE to be satisfactory.

References

  1. Christian Timmerer and Ali C. Begen. 2014. Over-the-top content delivery: State of the art and challenges ahead. In Proceedings of the 22nd ACM International Conference on Multimedia (MM’14). ACM, New York, NY, 1231–1232. DOI:https://doi.org/10.1145/2647868.2654849Google ScholarGoogle Scholar
  2. Cisco. 2019. Cisco Visual Net-working Index: Forecast and Methodology 2017-2022. Retrieved January 12, 2019 from https://www.cisco.com/c/en/us/solutions/collateral/service-provider/visual-networking-index-vni/white-paper-c11-741490.pdf.Google ScholarGoogle Scholar
  3. Michael Seufert, Sebastian Egger, Martin Slanina, Thomas Zinner, Tobias Hoßfeld, and Phuoc Tran-Gia. 2015. A survey on quality of experience of HTTP adaptive streaming. IEEE Commun. Surv. Tutor. 17, 1 (2015), 469–492. DOI:http://dx.doi.org/10.1109/COMST.2014.2360940Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. J. Kua, G. Armitage, and F. Branch. 2017. A survey of rate adaptation techniques for dynamic adaptive streaming over HTTP. IEEE Commun. Surv. Tutor. 19, 3 (2017), 1842–1866. DOI:http://dx.doi.org/10.1109/COMST.2017.2685630Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Z. Li, X. Zhu, and J. Gahm. 2014. Probe and adapt: Rate adaptation for HTTP video streaming at scale. IEEE J. Select. Areas Commun. 32, 4 (2014), 719–733. DOI:http://dx.doi.org/10.1109/JSAC.2014.140405Google ScholarGoogle ScholarCross RefCross Ref
  6. T. C. Thang, H. T. Le, and A. T. Pham. 2014. An evaluation of bitrate adaptation methods for HTTP live streaming. IEEE J. Select. Areas Commun. 32, 4 (2014), 693–705. DOI:http://dx.doi.org/10.1109/JSAC.2014.140403Google ScholarGoogle ScholarCross RefCross Ref
  7. Te-Yuan Huang, Ramesh Johari, Nick McKeown, Matthew Trunnell, and Mark Watson. 2014. A buffer-based approach to rate adaptation: Evidence from a large video streaming service. SIGCOMM Comput. Commun. Rev. 44, 4 (2014), 187–198. DOI:https://doi.org/10.1145/2740070.2626296Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Xiaoqi Yin, Abhishek Jindal, Vyas Sekar, and Bruno Sinopoli. 2015. A control-theoretic approach for dynamic adaptive video streaming over HTTP. SIGCOMM Comput. Commun. Rev. 45, 4 (2015), 325–338. DOI:https://doi.org/10.1145/2829988.2787486Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Hongzi Mao, Ravi Netravali, and Mohammad Alizadeh. 2017. Neural adaptive video streaming with pensieve. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication (SIGCOMM’17). ACM, New York, NY, 197–210. DOI:https://doi.org/10.1145/3098822.3098843Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. S. Hu, L. Sun, C. Gui, E. Jammeh, and I.-H. Mkwawa. 2014. Content-aware adaptation scheme for QoE optimized DASH applications. In Proceedings of the IEEE Global Communications Conference (GLOBECOM’14). 1336–1341. DOI:http://dx.doi.org/10.1109/GLOCOM.2014.7036993Google ScholarGoogle ScholarCross RefCross Ref
  11. G. Gao, H. Zhang, H. Hu, and Y. Wen. 2018. Optimizing quality of experience for adaptive bitrate streaming via viewer interest inference. IEEE Trans. Multimedia 20, 12 (2018), 3399–3413. DOI:http://dx.doi.org/10.1109/TMM.2018.2838330Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Maarten Wijnants, Gustavo Rovelo, Peter Quax, and Wim Lamotte. 2016. A pragmatically designed adaptive and web-compliant object-based video streaming methodology: Implementation and subjective evaluation. In Proceedings of the 24th ACM International Conference on Multimedia (MM’16). ACM, New York, NY, 1267–1276. DOI:https://doi.org/10.1145/2964284.2964300Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Shenghong Hu, Lingfen Sun, Chunxia Xiao, and Chao Gui. 2017. Semantic-aware adaptation scheme for soccer video over MPEG-DASH. In Proceedings of the IEEE International Conference on Multimedia 8 Expo (ICME’17). 493–498. DOI:http://dx.doi.org/10.1109/ICME.2017.8019541Google ScholarGoogle ScholarCross RefCross Ref
  14. P. Ekman. 1982. Emotion in the Human Face (2nd ed). Cambridge University Press, Cambridge, UK.Google ScholarGoogle Scholar
  15. M. Xu, J. Wang, X. He, J. S. Jin, S. Luo, and H. Lu. 2014. A three-level framework for affective content analysis and its case studies. Multimedia Tools Appl. 70, 2 (2014), 757–779.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Min Xu, Jesse S. Jin, Suhuai Luo, and Lingyu Duan. 2008. Hierarchical movie affective content analysis based on arousal and valence features. In Proceedings of the 16th ACM International Conference on Multimedia (MM’08). ACM, New York, NY, 677–680. DOI:https://doi.org/10.1145/1459359.1459457Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. H. Zhang and M. Xu. 2018. Recognition of emotions in user-generated videos with kernelized features. IEEE Trans. Multimedia 20, 10 (2018), 2824–2835. DOI:http://dx.doi.org/10.1109/TMM.2018.2808760Google ScholarGoogle ScholarCross RefCross Ref
  18. C. Chen, Z. Wu, and Y. G. Jiang. 2016. Emotion in context: deep semantic feature fusion for video emotion recognition. In Proceedings of the ACM International Conference on Multimedia. 127–131. DOI:http://dx.doi.org/10.1145/2964284.2967196Google ScholarGoogle Scholar
  19. K. Peng, T. Chen, A. Sadovnik, and A. Gallagher. 2015. A mixed bag of emotions: Model, predict, and transfer emotion distributions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’15). 860–868. DOI:http://dx.doi.org/10.1109/CVPR.2015.7298687Google ScholarGoogle Scholar
  20. S. Zhao, G. Amir, G. Ding, Y. Gao, J. Han, and K. Keutzer. 2019. Personalized emotion recognition by personality-aware high-order learning of physiological signals. ACM Trans. Multimedia Comput. Commun. Appl. 15, 1s (2019), 14:1–14:18. DOI:http://dx.doi.org/10.1145/3233184Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. S. Wang and Q. Ji. 2015. Video affective content analysis: A survey of state-of-the-art methods. IEEE Trans. Affect. Comput. 6, 4 (2015), 410–430. DOI:http://dx.doi.org/10.1109/TAFFC.2015.2432791Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. S. Zhao, G. Ding, Y. Gao, and J. Han. 2017. Approximating discrete probability distribution of image emotions by multi-modal features fusion. In Proceedings of the 26th International Joint Conference on Artificial Intelligence. 4669–4675. DOI:http://dx.doi.org/10.24963/ijcai.2017/651Google ScholarGoogle Scholar
  23. S. Zhao, G. Ding, Y. Gao, and J. Han. 2017. Learning visual emotion distributions via multi-modal features fusion. In Proceedings of the 2017 ACM on Multimedia Conference. 369--377. DOI:http://doi.acm.org/10.1145/3123266.3130858Google ScholarGoogle Scholar
  24. S. Zhao, H. Yao, Y. Gao, G. Ding, and T.-S. Chua. 2018. Predicting personalized image emotion perceptions in social networks. IEEE Trans. Affect. Comput. 9, 4 (2018), 526–540. DOI:http://dx.doi.org/10.1109/TAFFC.2016.2628787Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. J. He, Z. Xue, D. Wu, D. O. Wu, and Y. Wen. 2014. CBM: Online strategies on cost-aware buffer management for mobile video streaming. IEEE Trans. Multimedia 16, 1 (2014), 242–252. DOI:10.1145/2413176.2413189Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Y. Xu, E. Altman, R. El-Azouzi, M. Haddad, S. Elayoubi, and T. Jimenez. 2014. Analysis of buffer starvation with application to objective qoe optimization of streaming services. IEEE Trans. Multimedia 16, 3 (2014), 813–827. DOI:http://dx.doi.org/10.1145/2413176.2413189Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. S. Tavakoli, S. Egger, M. Seufert, R. Schatz, K. Brunnström, and N. García. 2016. Perceptual quality of HTTP adaptive streaming strategies: Cross-experimental analysis of multi-laboratory and crowdsourced subjective studies. IEEE J. Select. Areas Commun. 34, 8 (2016), 2141–2153. DOI:http://dx.doi.org/10.1109/JSAC.2016.2577361Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. S. F. Chang and A. Vetro. 2005. Video adaptation: Concepts, technologies and open issues. Proc. IEEE 93, 1 (2005), 148–158. DOI:http://dx.doi.org/10.1109/JPROC.2004.839600Google ScholarGoogle ScholarCross RefCross Ref
  29. K. Asiya, S. Lingfen, and I. Emmanuel. 2012. QoE prediction model and its application in video quality adaptation over UMTS networks. IEEE Trans. Multimedia 14, 2 (2012), 431–442. DOI:http://dx.doi.org/10.1109/TMM.2011.2176324Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. M. Xu, J. Li, L. T. Chia, Y. Hu, B.-S. Lee, D. Rajan, and J. S. Jin. 2006. Event on demand with MPEG-21 video adaptation system. In Proceedings of the 14th Annual ACM International Conference on Multimedia (MM’06). 921–930. DOI:http://dx.doi.org/10.1145/1180639.1180843Google ScholarGoogle Scholar
  31. B. Ciubotaru, G. Ghinea, and G.-M. Muntean. 2014. Subjective assessment of region of interest-aware adaptive multimedia streaming quality. IEEE Trans. Broadcast. 60, 1 (2014), 50–60. DOI:http://dx.doi.org/10.1109/TBC.2013.2290238Google ScholarGoogle ScholarCross RefCross Ref
  32. X. Chen, J.-N. Hwang, and D. Meng. 2017. A quality-of-content-based joint source and channel coding for human detections in a mobile surveillance cloud. IEEE Trans. Circ. Syst. Vid. Technol. 27, 1 (2017), 19–31. DOI:http://dx.doi.org/10.1109/TCSVT.2016.2539758Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Kevin Spiteri, Ramesh Sitaraman, and Daniel Sparacio. 2018. From theory to practice: Improving bitrate adaptation in the DASH reference player. In Proceedings of the 9th ACM Multimedia Systems Conference (MMSys’18). ACM, New York, NY, 123–137. DOI:https://doi.org/10.1145/3204949.3204953Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Blender Foundation. 2008. BigBuckBunny. Retrieved January 2, 2019 from https://peach.blender.org/.Google ScholarGoogle Scholar
  35. R. Rassool. 2017. VMAF reproducibility: Validating a perceptual practical video quality metric. In Proceedings of the IEEE International Symposium in Broadband Multimedia Systems and Broadcasting (BMSB’17). 1–2. DOI:http://dx.doi.org/10.1109/ BMSB.2017.7986143Google ScholarGoogle ScholarCross RefCross Ref
  36. H. Riiser, P. Vigmostad, C. Griwodz, and Halvorsen Pål. 2013. Commute path bandwidth traces from 3G networks: analysis and applications. In Proceedings of the International Conference on Multimedia Systems (MMSys’13). 114–118. DOI:http://dx.doi.org/10.1145/2483977.2483991Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Y. Nie, C. Xiao, H. Sun, and P. Li. 2013. Compact video synopsis via global spatiotemporal optimization. IEEE Trans. Vis. Comput. Graph. 19, 10 (2013), 1664–1676. DOI:http://dx.doi.org/10.1109/TVCG.2012.176Google ScholarGoogle ScholarCross RefCross Ref
  38. L. Zhang, Q. Yan, Z. Liu, H. Zou, and C. Xiao. 2017. Illumination decomposition for photograph with multiple light sources. IEEE Trans. Image Process. 26, 9 (2017), 4114–4127. DOI:http://dx.doi.org/10.1109/TIP.2017.2712283Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Affective Content-aware Adaptation Scheme on QoE Optimization of Adaptive Streaming over HTTP

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM Transactions on Multimedia Computing, Communications, and Applications
          ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 15, Issue 3s
          Special Issue on Face Analysis for Applications and Special Issue on Affective Computing for Large-Scale Heterogeneous Multimedia Data
          November 2019
          304 pages
          ISSN:1551-6857
          EISSN:1551-6865
          DOI:10.1145/3368027
          Issue’s Table of Contents

          Copyright © 2019 ACM

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 7 December 2019
          • Revised: 1 April 2019
          • Accepted: 1 April 2019
          • Received: 1 February 2019
          Published in tomm Volume 15, Issue 3s

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article
          • Research
          • Refereed

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        HTML Format

        View this article in HTML Format .

        View HTML Format
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!