skip to main content
research-article

Measuring Individual Video QoE: A Survey, and Proposal for Future Directions Using Social Media

Authors Info & Claims
Published:01 May 2018Publication History
Skip Abstract Section

Abstract

The next generation of multimedia services have to be optimized in a personalized way, taking user factors into account for the evaluation of individual experience. Previous works have investigated the influence of user factors mostly in a controlled laboratory environment which often includes a limited number of users and fails to reflect real-life environment. Social media, especially Facebook, provide an interesting alternative for Internet-based subjective evaluation. In this article, we develop (and open-source) a Facebook application, named YouQ1, as an experimental platform for studying individual experience for videos. Our results show that subjective experiments based on YouQ can produce reliable results as compared to a controlled laboratory experiment. Additionally, YouQ has the ability to collect user information automatically from Facebook, which can be used for modeling individual experience.

References

  1. Cisco Visual Networking Index. 2013. The zettabyte era--trends and analysis. Cisco white paper (2013).Google ScholarGoogle Scholar
  2. Conviva. 2015. Experience report. Retrieved from http://www.conviva.com/conviva-viewer-experience-report/vxr-2015/.Google ScholarGoogle Scholar
  3. P. Le Callet, S. Möller, and A. Perkis. 2012. Qualinet White Paper on definitions of quality of experience. ENQEMSS (COST Action IC 1003) (2012).Google ScholarGoogle Scholar
  4. Y. Zhu, A. Hanjalic, and J. A. Redi. 2016. QoE prediction for enriched assessment of individual video viewing experience. ACM Mutimedia, Amsterdam. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. J. A. Redi, Y. Zhu, H. de Ridder, and I. Heynderickx. 2015. How passive image viewers became active multimedia users. Springer.Google ScholarGoogle Scholar
  6. ITU-R BT.500-11, Methodology for the subjective assessment of the quality of television pictures. International Telecommunication Union, Geneva, Switzerland, 4 (2002), 2.Google ScholarGoogle Scholar
  7. W. Lin and C.-C. Jay Kuo. 2011. Perceptual visual quality metrics: A survey. JVCI 22, 4 (2011), 297--312. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. R. Hu and P. Pu. 2011. Enhancing collaborative filtering systems with personality information. ACM Recsys. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. W. Wu, L. Chen, and L. He. 2013. Using personality to adjust diversity in recommender systems. ACM Hypertext. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. M. Swan. 2012. Health 2050: The realization of personalized medicine through crowdsourcing, the quantified self, and the participatory biocitizen. J. Personalized Medicine 2, 3 (2012), 93--118.Google ScholarGoogle ScholarCross RefCross Ref
  11. Y. Zhu, I. Heynderickx, and J. A. Redi. 2015. Understanding the role of social context and user factors in video quality of experience. Computers in Human Behavior 49 (2015), 412--426. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. M. J. Scott, S. C. Guntuku, Y. Huan, W. Lin, and G. Ghinea. 2015. October. Modelling human factors in perceptual multimedia quality: On the role of personality and culture. ACM Multimedia. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. S. Möller and A. Raake. 2014. Quality of experience - advanced concepts, applications and methods. Springer International Publishing Switzerland 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. T. Hoßfeld, C. Keimel, M. Hirth, B. Gardlo, J. Habigt, K. Diepold, and P. Tran-Gia. 2014. Best practices for QoE crowdtesting: QoE assessment with crowdsourcing. IEEE TMM 16, 2 (2014), 541--558. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. S. D. Gosling and W. Mason. 2015. Internet research in psychology. AR Psychology 66 (2015), 877--902.Google ScholarGoogle Scholar
  16. L. J. Skitka and E. G. Sargis. 2006. The internet as psychological laboratory. AR Psychology 57 (2006), 529--555.Google ScholarGoogle Scholar
  17. M. Kosinski, S. C. Matz, S. D. Gosling, V. Popov, and D. Stillwell. 2015. Facebook as a research tool for the social sciences: Opportunities, challenges, ethical considerations, and practical guidelines. American Psychologist 70, 6 (2015), 543.Google ScholarGoogle ScholarCross RefCross Ref
  18. A. Doan, R. Ramakrishnan, and A. Y. Halevy. 2011. Crowdsourcing systems on the world-wide web. Communications of the ACM 54, 4 (2011), 86--96. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. J. Redi, E. Siahaan, P. Korshunov, J. Habigt, and T. Hossfeld. 2015. When the crowd challenges the lab: lessons learnt from subjective studies on image aesthetic appeal. CrowdMM 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. K. Casler, L. Bickel, and E. Hackett. 2013. Separate but equal? A comparison of participants and data gathered via Amazon's MTurk, social media, and face-to-face behavioral testing. CHB 29, 6 (2013), 2156--2160. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. R. E. Wilson, S. D. Gosling, and L. T. Graham. 2012. A review of Facebook research in the social sciences. Perspectives on Psychological Science 7, 3 (2012), 203--220.Google ScholarGoogle ScholarCross RefCross Ref
  22. F. Delogu, M. Franetovic, and L. Shamir. 2015. Keep me posted!: Human and machine learning analysis of facebook updates. Mobile MM Communications, 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. M. D. Back, J. M. Stopfer, S. Vazire, S. Gaddis, S. C. Schmukle, B. Egloff, and S. D. Gosling. 2010. Facebook profiles reflect actual personality, not self-idealization. Psychological Science (2010).Google ScholarGoogle Scholar
  24. L. Gou, M. X. Zhou, and H. Yang. 2014. KnowMe and shareme: Understanding automatically discovered personality traits from social media and user sharing preferences. ACM CHI, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. N. Seitz. 2003. ITU-T QoS standards for IP-based networks. IEEE Communications Magazine (2003), 82--89. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. P. Brooks and B. Hestnes. 2010. User measures of quality of experience: Why being objective and quantitative is important. Network, IEEE 24, 2 (2010), 8--13. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. F. Dobrian, V. Sekar, A. Awan, I. Stoica, D. Joseph, A. Ganjam, J. Zhan, and H. Zhang. 2011. Understanding the impact of video quality on user engagement. ACM Computer Communication Review (2011), 362--373. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. G. Ghinea and J. Thomas. 2005. Quality of perception: user quality of service in multimedia presentations. IEEE Transactions on Multimedia 7, 4 (2005), 786--789. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. S. Ickin, K. Wac, M. Fiedler, L. Janowski, J.-H. Hong, and A. K. Dey. 2012. Factors influencing quality of experience of commonly used mobile applications. IEEE Communications Magazine 50, 4 (2012), 48--56.Google ScholarGoogle ScholarCross RefCross Ref
  30. A. Balachandran, V. Sekar, A. Akella, S. Seshan, I. Stoica, and H. Zhang. 2012. A quest for an internet video quality-of-experience metric. In Proceedings of the 11th ACM Workshop on Hot Topics in Networks. 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Q. Huynh-Thu and M. Ghanbari. 2008. Temporal aspect of perceived quality in mobile video broadcasting. IEEE Transactions on Broadcasting 54, 3 (2008), 641--651.Google ScholarGoogle ScholarCross RefCross Ref
  32. J. H. Westerink and J. A. Roufs. 1989. Subjective image quality as a function of viewing distance, resolution, and picture size. SMPTE Journal 98, 2 (1989), 113--119.Google ScholarGoogle ScholarCross RefCross Ref
  33. N. Staelens, S. Moens, W. Van den Broeck, I. Marien, B. Vermeulen, P. Lambert, R. Van de Walle, and P. Demeester. 2010. Assessing quality of experience of IPTV and video on demand services in real-life environments. IEEE Transactions on Broadcasting 56, 4 (2010), 458--466.Google ScholarGoogle ScholarCross RefCross Ref
  34. K. Yamori and Y. Tanaka. 2004. Relation between willingness to pay and guaranteed minimum bandwidth in multiple-priority service. Communications, 2004.Google ScholarGoogle Scholar
  35. C. Owsley, R. Sekuler, and D. Siemsen. 1983. Contrast sensitivity throughout adulthood. Vision Research, 1983.Google ScholarGoogle Scholar
  36. M. S. El-Nasr and S. Yan. 2006. 3dVisual attention in 3D video games. Workshop on CST, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. P. Orero and A. Remael. 2007. Media for all: Subtitling for the deaf, audio description, and sign language. Rodopi, 2007.Google ScholarGoogle Scholar
  38. R. E. Nisbett and Y. Miyamoto. 2005. The influence of culture: holistic versus analytic perception. Trends in Cognitive Sciences 9, 10 (2005), 467--473.Google ScholarGoogle ScholarCross RefCross Ref
  39. E. Balcetis and G. D. Lassiter. 2010. Social Psychology of Visual Perception. Psychology Press, 2010.Google ScholarGoogle ScholarCross RefCross Ref
  40. A. Acar, T. Taura, E. Yamamoto, and N. F. M. Yusof. 2011. Object vs. relation: Understanding the link between culture and cognition with the help of wordnet. Int. J. of Asian Lang. Proc 21, 4 (2011), 199--208.Google ScholarGoogle Scholar
  41. I. Wechsung, M. Schulz, K.-P. Engelbrecht, J. Niemann, and S. Möller. 2011. All users are (not) equal-the influence of user characteristics on perceived quality, modality choice and performance. In Proceedings of the Paralinguistic Information and its Integration in Spoken Dialogue Systems Workshop. Springer, New York, NY.Google ScholarGoogle Scholar
  42. A. B. Naumann, I. Wechsung, and J. Hurtienne. 2010. Multimodal interaction: A suitable strategy for including older users? Interacting with Computers 22, 6 (2010), 465--474. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. P. J. Silvia. 2008. Interest—The curious emotion. Curr. Dir. Psychological Science 17, 1 (2008), 57--60.Google ScholarGoogle ScholarCross RefCross Ref
  44. H. L. O'Brien and E. G. Toms. 2008. What is user engagement? A conceptual framework for defining user engagement with technology. JAIST 59, 6 (2008), 938--955. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. P. Kortum and M. Sullivan. 2010. The effect of content desirability on subjective video quality ratings. Human Factors: The Journal of the Human Factors and Ergonomics Society 52, 1 (2010), 105--118.Google ScholarGoogle ScholarCross RefCross Ref
  46. J. Palhais, R. S. Cruz, and M. S. Nunes. 2011. Quality of experience assessment in internet TV. In International Conference on Mobile Networks and Management. Springer, Berlin, Heidelberg, 261--274.Google ScholarGoogle Scholar
  47. A. Sackl, R. Schatz, and A. Raake. 2017. More than I ever wanted or just good enough? User expectations and subjective quality perception in the context of networked multimedia services. QUEx 2, 1 (2017), 3.Google ScholarGoogle Scholar
  48. A. Sackl and R. Schatz. 2014. Got what you want? Modeling expectations to enhance web QoE prediction. Quality of Multimedia Experience (QoMEX), IEEE, 2014.Google ScholarGoogle Scholar
  49. A. Sackl and R. Schatz. 2014. Evaluating the influence of expectations, price and content selection on video quality perception. Quality of Multimedia Experience (QoMEX), IEEE, 2014.Google ScholarGoogle Scholar
  50. P. A. Kara, L. Bokor, A. Sackl, and M. Mourão. 2015. What your phone makes you see: Investigation of the effect of end-user devices on the assessment of perceived multimedia quality. Quality of Multimedia Experience (QoMEX), IEEE, 2015.Google ScholarGoogle Scholar
  51. P. G. Engeldrum. 2000. Psychometric Scaling: A Toolkit for Imaging Systems Development. Imcotek Press, 2000.Google ScholarGoogle Scholar
  52. A. G. Greenwald, D. E. McGhee, and J. L. Schwartz. 1998. Measuring individual differences in implicit cognition: the implicit association test. Journal of Personality and Social Psychology 74, 6 (1998), 1464.Google ScholarGoogle ScholarCross RefCross Ref
  53. C. Keimel, J. Habigt, and K. Diepold. 2012. Challenges in crowd-based video quality assessment. Quality of Multimedia Experience (QoMEX), IEEE, 2012.Google ScholarGoogle Scholar
  54. T. Hoßfeld, M. Hirth, J. Redi, F. Mazza, P. Korshunov, B. Naderi, M. Seufert, B. Gardlo, S. Egger, and C. Keimel. 2014. Best practices and recommendations for crowdsourced QoE-lessons learned from the qualinet task force. Crowdsourcing (2014).Google ScholarGoogle Scholar
  55. M. Weisbuch, Z. Ivcevic, and N. Ambady. 2009. On being liked on the web and in the “real world”: Consistency in first impressions across personal webpages and spontaneous behavior. JESP 45, 3 (2009), 573--576.Google ScholarGoogle ScholarCross RefCross Ref
  56. M. Burke, C. Marlow, and T. Lento. 2009. Feed me: Motivating newcomer contribution in social network sites. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. H. Li, H. Wang, J. Liu, and K. Xu. 2012. Video sharing in online social networks: measurement and analysis. In Proceedings of the International Workshop on Network and Operating System Support for Digital Audio and Video. ACM, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. L. T. Graham, C. J. Sandy, and S. D. Gosling. 2011. Manifestations of individual differences in physical and virtual environments. The Wiley-Blackwell Handbook of Individual Differences (2011), 773--800.Google ScholarGoogle Scholar
  59. D. Garcia and S. Sikström. 2014. The dark side of facebook: Semantic representations of status updates predict the dark triad of personality. Personality and Individual Differences 67 (2014), 92--96.Google ScholarGoogle ScholarCross RefCross Ref
  60. Y. Bachrach, M. Kosinski, T. Graepel, P. Kohli, and D. Stillwell. 2012. Personality and patterns of Facebook usage. In Proceedings of the 4th Annual ACM Web Science Conference. ACM, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. A. Eftekhar, C. Fullwood, and N. Morris. 2014. Capturing personality from facebook photos and photo-related activities: How much exposure do you need? Computers in Human Behavior 37 (2014), 162--170. Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. M. Kosinski, D. Stillwell, and T. Graepel. 2013. Private traits and attributes are predictable from digital records of human behavior. Proceedings of the National Academy of Sciences 110, 15 (2013), 5802--5805.Google ScholarGoogle Scholar
  63. M. Park, M. Naaman, and J. Berger. 2016. A data-driven study of view duration on Youtube. arXiv:1603.08308 (2016).Google ScholarGoogle Scholar
  64. X. Yi, L. Hong, E. Zhong, N. N. Liu, and S. Rajan. 2014. Beyond clicks: Dwell time for personalization. In Proceedings of the 8th ACM Conference on Recommender Systems. ACM, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  65. B. Gardlo, M. Ries, T. Hossfeld, and R. Schatz. 2012. Microworkers vs. facebook: The impact of crowdsourcing platform choice on experimental results. Quality of Multimedia Experience (QoMEX), IEEE.Google ScholarGoogle Scholar
  66. D. J. Stillwell and M. Kosinski. 2004. Mypersonality project: Example of successful utilization of online social networks for large-scale social research. American Psychologist 59, 2 (2004), 93--104.Google ScholarGoogle Scholar
  67. H. A. Schwartz, J. Eichstaedt, and M. E. Seligman. 2013. Personality, gender, and age in the language of social media: The open-vocabulary approach. PloS one 8, 9 (2013), e73791.Google ScholarGoogle ScholarCross RefCross Ref
  68. D. Preoţiuc-Pietro, S. Volkova, V. Lampos, Y. Bachrach, and N. Aletras. 2015. Studying user income through language, behaviour and affect in social media. PloS one 10, 9 (2015), e0138717.Google ScholarGoogle ScholarCross RefCross Ref
  69. L. Zwarun and A. Hall. 2014. What's going on? Age, distraction, and multitasking during online survey taking. Computers in Human Behavior 41 (2014), 236--244. Google ScholarGoogle ScholarDigital LibraryDigital Library
  70. S. R. Gulliver and G. Ghinea. 2006. Defining user perception of distributed multimedia quality. ACM Transactions on Multimedia Computing, Communications, and Applications 2, 4 (2006), 241--257. Google ScholarGoogle ScholarDigital LibraryDigital Library
  71. S. D. Gosling, P. J. Rentfrow, and W. B. Swann. 2003. A very brief measure of the Big-Five personality domains. Journal of Research in Personality 37, 6 (2003), 504--528.Google ScholarGoogle ScholarCross RefCross Ref
  72. W. Robitza, M. N. Garcia, and A. Raake. 2015. At home in the lab: Assessing audiovisual quality of HTTP-based adaptive streaming with an immersive test paradigm. Quality of Multimedia Experience (QoMEX) IEEE, 2015.Google ScholarGoogle Scholar
  73. A. Hanjalic and L.-Q. Xu. 2005. Affective video content representation and modeling. TMM, IEEE (2005). Google ScholarGoogle ScholarDigital LibraryDigital Library
  74. D. Rao, D. Yarowsky, A. Shreevats, and M. Gupta. 2010. Classifying latent user attributes in Twitter. In Proceedings of the 2nd International Workshop on Search and Mining User-Generated Contents. ACM, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  75. J. D. Burger, J. Henderson, G. Kim, and G. Zarrella. 2011. Discriminating gender on Twitter. Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  76. S. C. Guntuku, D. B. Yaden, M. L. Kern, L. H. Ungar, and J. C. Eichstaedt. 2017. Detecting depression and mental illness on social media: an integrative review. CoBeHa 18, 43--49.Google ScholarGoogle Scholar
  77. S. C. Guntuku, W. Lin, M. J. Scott, and G. Ghinea. 2015. Modelling the influence of personality and culture on affect and enjoyment in multimedia. In Proceedings of the 2015 International Conference on Affective Computing and Intelligent Interaction (ACII). IEEE, 236--242. Google ScholarGoogle ScholarDigital LibraryDigital Library
  78. S. C. Guntuku, M. J. Scott, H. Yang, G. Ghinea, and W. Lin. 2015. The CP-QAE-I: A video dataset for exploring the effect of personality and culture on perceived quality and affect in multimedia. In Proceedings of the 2015 7th International Workshop on Quality of Multimedia Experience (QoMEX). IEEE, 1--7.Google ScholarGoogle Scholar
  79. S. C. Guntuku, J. R. Ramsay, R. M. Merchant, and L. H. Ungar. 2017. Language of ADHD in adults on social media. Journal of Attention Disorders, 1087054717738083.Google ScholarGoogle ScholarCross RefCross Ref
  80. S. C. Guntuku, W. Lin, J. Carpenter, W. K. Ng, L. H. Ungar, and D. Preoţiuc-Pietro. 2017. Studying personality through the content of posted and liked images on Twitter. In Proceedings of the 2017 ACM on Web Science Conference. ACM, 223--227. Google ScholarGoogle ScholarDigital LibraryDigital Library
  81. S. C. Guntuku, J. T. Zhou, S. Roy, L. Weisi, and I. W. Tsang. 2016. Who likes what, and why? Insights into personality modeling based on imagelikes. IEEE Transactions on Affective Computing. Google ScholarGoogle ScholarDigital LibraryDigital Library
  82. S. C. Guntuku, S. Roy, W. Lin, K. Ng, N. W. Keong, and V. Jakhetiya. 2016. Personalizing user interfaces for improving quality of experience in VoD recommender systems. In Proceedings of the 2016 8th International Conference on Quality of Multimedia Experience (QoMEX). IEEE, 1--6.Google ScholarGoogle Scholar
  83. S. C. Guntuku, L. Qiu, S. Roy, W. Lin, and V. Jakhetiya. 2015. Do others perceive you as you want them to?: Modeling personality based on selfies. In Proceedings of the 1st International Workshop on Affect 8 Sentiment In Multimedia. ACM, 21--26. Google ScholarGoogle ScholarDigital LibraryDigital Library
  84. M. J. Scott, S. C. Guntuku, W. Lin, and G. Ghinea. 2016. Do personality and culture influence perceived video quality and enjoyment? IEEE Transactions on Multimedia 18, 9, 1796--1807. Google ScholarGoogle ScholarDigital LibraryDigital Library
  85. G. Ghinea and S. Y. Chen. 2006. Perceived quality of multimedia educational content: A cognitive style approach. Multimedia Systems 11, 3, 271--279. Google ScholarGoogle ScholarDigital LibraryDigital Library
  86. S. R. Gulliver and G. Ghinea. 2010. Cognitive style and personality: Impact on multimedia perception. Online Information Review 34, 1, 39--58.Google ScholarGoogle ScholarCross RefCross Ref
  87. S. C. Guntuku, M. J. Scott, G. Ghinea, and W. Lin. 2016. Personality, culture, and system factors - Impact on affective response to multimedia. arXiv Preprint arXiv:1606.06873.Google ScholarGoogle Scholar

Index Terms

  1. Measuring Individual Video QoE: A Survey, and Proposal for Future Directions Using Social Media

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM Transactions on Multimedia Computing, Communications, and Applications
          ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 14, Issue 2s
          April 2018
          287 pages
          ISSN:1551-6857
          EISSN:1551-6865
          DOI:10.1145/3210485
          Issue’s Table of Contents

          Copyright © 2018 ACM

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 1 May 2018
          • Accepted: 1 January 2018
          • Revised: 1 November 2017
          • Received: 1 June 2017
          Published in tomm Volume 14, Issue 2s

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article
          • Research
          • Refereed

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!