Abstract
This article proposes the integration of multisensorial stimuli and multimodal interaction components into a sports multimedia asset under two dimensions: immersion and interaction. The first dimension comprises a binaural audio system and a set of sensory effects synchronized with the audiovisual content, whereas the second explores interaction through the insertion of interactive 3D objects into the main screen and on-demand presentation of additional information in a second touchscreen. We present an end-to-end solution integrating these components into a hybrid (internet-broadcast) television system using current 3DTV standards. Results from an experimental study analyzing the perceived quality of these stimuli and their influence on the Quality of Experience are presented.
Supplemental Material
Available for Download
Supplemental movie, appendix, image and software files for, Integration of Multisensorial Stimuli and Multimodal Interaction in a Hybrid 3DTV System
- A. T. Beck. 1961. An inventory for measuring depression. Archives Gen. Psychiatry 4, 6, 561.Google Scholar
Cross Ref
- C. Bracken, G. Pettey, and M. Wu. 2011. Telepresence and attention: Secondary task reaction time and media form. In Proceedings of the International Society for Presence Research Annual Conference.Google Scholar
- J. Cha, M. Eid, and A. E. Saddik. 2009. Touchable 3D video system. ACM Trans. Multimedia Comput. Commun. Appl. 5, 4, 1--25. Google Scholar
Digital Library
- M. Cobos, J. J. Lopez, J. M. Navarro, and G. Ramos. 2013. Subjective quality assessment of multichannel audio accompanied with video in representative broadcasting genres. Multimedia Syst. Google Scholar
Digital Library
- B. De Ruyter and E. Aarts. 2004. Ambient intelligence. In Proceedings of the Working Conference on Advanced Visual Interfaces (AVI'04). ACM Press, New York, New York, 203. Google Scholar
Digital Library
- ETSI. 2006. Digital video broadcasting (DVB); Multimedia home platform (MHP). European Telecomunications Standards Institute (ETSI) Technical Specification TS 101 812 v1.0.2.Google Scholar
- ETSI. 2010a. Digital video broadcasting (DVB); DVB-HN (home network) reference model phase 1. Technical Specification TS 102 905 Ver. 1.1.1. European Telecomunications Standards Institute (ETSI).Google Scholar
- ETSI. 2010b. Digital video broadcasting (DVB); Signalling and carriage of interactive applications and services in hybrid broadcast/broadband environments. Technical Specification TS 102 809 v1.1.1. European Telecomunications Standards Institute (ETSI).Google Scholar
- ETSI. 2012a. Digital video broadcasting (DVB); Plano-stereoscopic 3DTV; Part 2: Frame compatible plano-stereoscopic 3DTV. Technical Specification TS 101 547-2 V1.2.1. European Telecomunications Standards Institute (ETSI).Google Scholar
- ETSI. 2012b. HbbTV (hybrid broadcast broadband TV) technical specification. Technical Specification TS 102 796 v1.2.1. European Telecomunications Standards Institute (ETSI).Google Scholar
- I. Galloso, F. P. Luque, L. Piovano, D. Garrido, E. Sánchez, and C. A. Feijoo. 2012. Foundations of a new interaction paradigm for immersive 3D multimedia. In Proceedings of the NEM Summit: Implementing Future Media Internet towards New Horizons.Google Scholar
- G. Ghinea and O. Ademoye. 2012. The sweet smell of success: Enhancing multimedia applications with olfaction. ACM Trans. Multimedia Comput. Commun. Appl. 8, 1, 1--17. Google Scholar
Digital Library
- E. B. Goldstein. 2010. Sensation and Perception. Cengage Learning.Google Scholar
- D. W. Heightman. 1975. Digital video broadcasting (DVB); DVB-HN (home network) reference model phase 1. European Telecomunications Standards Institute (ETSI) Technical Specification TS 102 905 Ver. 1.1.1. Radio Electron. Eng. 45, 10, 559--569.Google Scholar
- H. Hirschmüller. 2008. Stereo processing by semiglobal matching and mutual information. IEEE Trans. Patt. Anal. Mach. Intell. 30, 2, 328--41. Google Scholar
Digital Library
- INSPIRA. 2006. Informe sobre el uso de la televisión digital interactiva por parte del público español. Coordinated by G@TV-UPM. Sponsored by Spanish Ministry of Industry, Tourism and Trade.Google Scholar
- ISO/IEC. 1998. Information technology—Coding of multimedia and hypermedia information—Part 6: Support for enhanced interactive applications. International Standard (ISO/IEC 13522-6). International Organization for Standardization.Google Scholar
- ISO/IEC. 2000. Generic coding of moving pictures and associated audio information: Systems. International Standard (ISO/IEC 13818-1). International Organization for Standardization.Google Scholar
- ISO/IEC. 2007. Home electronic system (HES) architecture Part 3--5. International Standard (ISO/IEC 14543-3). International Organization for Standardization.Google Scholar
- ISO/IEC. 2013. Information technology—Media context and control—Part 3: Sensory Information. International Standard (ISO/IEC 23005-3). Organization for Standardization.Google Scholar
- ITU-R. 2000. Subjective assessment of stereoscopic television pictures. International Telecommunication Union.Google Scholar
- ITU-R. 2002. 500-11. Methodology for the subjective assessment of the quality of television pictures. International Telecommunication Union.Google Scholar
- ITU-R. 2012. Methodology for the subjective assessment of the quality of television pictures BT Series Broadcasting service. International Telecommunication Union.Google Scholar
- ITU-T. 1998. P.911: Subjective audiovisual quality assessment methods for multimedia applications. International Telecommunication Union.Google Scholar
- B. R. Jones, H. Benko, E. Ofek, and A. D. Wilson. 2013. IllumiRoom. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'13). ACM Press, New York, New York, 869. Google Scholar
Digital Library
- S. Jumisko-Pyykkö. 2011. User-centered quality of experience and its evaluation methods for mobile television. Master's Thesis, Tampere University of Technology.Google Scholar
- J. Kowalczuk, E. Psota, and L. Perez. 2012. Real-time stereo matching on CUDA using an iterative refinement method for adaptive support-weight correspondences. IEEE Trans. Circuits Syst. Video Technol. 23, 1, 94--104. Google Scholar
Digital Library
- P. Le Callet, S. Möller, and A. Perkis, eds. 2012. Qualinet white paper on definitions of quality of experience Qualinet.Google Scholar
- P. Mills, A. Sheikh, G. Thomas, and P. Debenham. 2011. BBC R&D white paper WHP208. BBC.Google Scholar
- N. Murray, Y. Qiao, B. Lee, A. K. Karunakar, and G.-M. Muntean. 2013. Subjective evaluation of olfactory and visual media synchronization. In Proceedings of the 4th ACM Multimedia Systems Conference (MMSys'13). 162--171. Google Scholar
Digital Library
- R. L. Nabi and M. Krcmar. 2004. Conceptualizing media enjoyment as attitude: Implications for mass media effects research. Commun. Theory 14, 4, 288--310.Google Scholar
Cross Ref
- D. R. Olsen, D. Bunn, T. Boulter, and R. Walz. 2012. Interactive television news. ACM Trans. Multimedia Comput. Commun. Appl. 8, 2, 1--20. Google Scholar
Digital Library
- G. Ramos and M. Cobos. 2013. Parametric head-related transfer function modeling and interpolation for cost-efficient binaural sound applications. J. Acoustical Soci. Am. 134, 3, 1735--8.Google Scholar
Cross Ref
- A. A. Raney and J. Bryant. 2002. Moral judgment and crime drama: An integrated theory of enjoyment. J. Commun. 52, 2, 402--415.Google Scholar
Cross Ref
- M. J. Schuemie, P. van der Straaten, M. Krijn, and C. van der Mast. 2001. Research on presence in virtual reality: A survey. Cyberpsychol. Behav. Impact Internet Multimedia Virtual Reality Behav. Soci. 4, 2, 183--201.Google Scholar
Cross Ref
- H. Schulzrinne, S. Casner, R. Frederick, and V. Jacobson. 2003. RTP: A transport protocol for real-time applications. Internet Engineering Task Force, RFC 3550. Google Scholar
Digital Library
- M. Slater and S. Wilbur. 1997. A framework for immersive virtual environments (FIVE): Speculations on the role of presence in virtual environments. Presence-Teleoperators Virtual Environ. 6, 6, 603--616.Google Scholar
Digital Library
- M. D. Slater. 2003. Alienation, aggression, and sensation seeking as predictors of adolescent use of violent film, computer, and website content. J. Commun. 53, 1, 105--121.Google Scholar
Cross Ref
- C. Timmerer, M. Waltl, B. Rainer, and H. Hellwagner. 2012. Assessing the quality of sensory experience for multimedia presentations. Signal Process. Image Commun. 27, 8, 909--916. Google Scholar
Digital Library
- M. Waltl, B. Rainer, C. Timmerer, and H. Hellwagner. 2013. An end-to-end tool chain for sensory experience based on MPEG-V. Signal Process. Image Commun. 28, 2, 136--150. Google Scholar
Digital Library
- M. Waltl, C. Timmerer, and H. Hellwagner. 2009. A test-bed for quality of multimedia experience evaluation of sensory effects. In Proceedings of the International Workshop on Quality of Multimedia Experience. IEEE, 145--150.Google Scholar
- M. Waltl, C. Timmerer, and H. Hellwagner. 2010. Increasing the user experience of multimedia presentations with sensory effects. In Proceedings of the 11th International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS). IEEE, 1--4.Google Scholar
- I. Wechsung, M. Schulz, K.-P. Engelbrecht, J. Niemann, and S. Möller. 2011. All users are (not) equal - The influence of user characteristics on perceived quality, modality choice and performance. In Proceedings of the Paralinguistic Information and its Integration in Spoken Dialogue Systems Workshop. Springer, New York, NY, 175--186.Google Scholar
- A. Weffers-Albu, S. de Waele, W. Hoogenstraaten, and C. Kwisthout. 2011. Immersive TV viewing with advanced Ambilight. In Proceedings of the IEEE International Conference on Consumer Electronics (ICCE). 753--754.Google Scholar
- D. Winer. 1999. XML remote procedure call (XML-RPC) specification. XML-RPC.Google Scholar
- Z. Yang, W. Wu, K. Nahrstedt, G. Kurillo, and R. Bajcsy. 2010. Enabling multi-party 3D tele-immersive environments with ViewCast. ACM Trans. Multimedia Comput. Commun. Appl. 6, 2, 1--30. Google Scholar
Digital Library
- K. Yoon. 2013. End-to-end framework for 4-D broadcasting based on MPEG-V standard. Signal Process. Image Commun. 28, 2, 127--135. Google Scholar
Digital Library
- D. Zillmann and P. Vorderer. 2000. Media Entertainment: The Psychology of Its Appeal. Lawrence Erlbaum Associates Publishers.Google Scholar
Cross Ref
Index Terms
Integration of Multisensorial Stimuli and Multimodal Interaction in a Hybrid 3DTV System
Recommendations
Reality Tales: Facilitating User-Character Interaction with Immersive Storytelling
CHI EA '21: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing SystemsReality Tales is a platform to facilitate interaction with fictional story characters for readers (13-20 years). The platform leverages binaural audio and multi-voice narration for an immersive story experience. Several paradigms for interactive ...
Using Interaction Data to Predict Engagement with Interactive Media
MM '21: Proceedings of the 29th ACM International Conference on MultimediaMedia is evolving from traditional linear narratives to personalised experiences, where control over information (or how it is presented) is given to individual audience members. Measuring and understanding audience engagement with this media is ...
Mobile phones as second screen for TV, enabling inter-audience interaction
ACE '11: Proceedings of the 8th International Conference on Advances in Computer Entertainment TechnologyDespite the ever expanding forms of digital entertainment, there are still TV shows and events that create an audience desire to be part of a mass shared experience. In the past direct inter-audience interaction of such events has been restricted to ...






Comments