skip to main content
research-article

Integration of Multisensorial Stimuli and Multimodal Interaction in a Hybrid 3DTV System

Published:01 October 2014Publication History
Skip Abstract Section

Abstract

This article proposes the integration of multisensorial stimuli and multimodal interaction components into a sports multimedia asset under two dimensions: immersion and interaction. The first dimension comprises a binaural audio system and a set of sensory effects synchronized with the audiovisual content, whereas the second explores interaction through the insertion of interactive 3D objects into the main screen and on-demand presentation of additional information in a second touchscreen. We present an end-to-end solution integrating these components into a hybrid (internet-broadcast) television system using current 3DTV standards. Results from an experimental study analyzing the perceived quality of these stimuli and their influence on the Quality of Experience are presented.

Skip Supplemental Material Section

Supplemental Material

References

  1. A. T. Beck. 1961. An inventory for measuring depression. Archives Gen. Psychiatry 4, 6, 561.Google ScholarGoogle ScholarCross RefCross Ref
  2. C. Bracken, G. Pettey, and M. Wu. 2011. Telepresence and attention: Secondary task reaction time and media form. In Proceedings of the International Society for Presence Research Annual Conference.Google ScholarGoogle Scholar
  3. J. Cha, M. Eid, and A. E. Saddik. 2009. Touchable 3D video system. ACM Trans. Multimedia Comput. Commun. Appl. 5, 4, 1--25. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. M. Cobos, J. J. Lopez, J. M. Navarro, and G. Ramos. 2013. Subjective quality assessment of multichannel audio accompanied with video in representative broadcasting genres. Multimedia Syst. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. B. De Ruyter and E. Aarts. 2004. Ambient intelligence. In Proceedings of the Working Conference on Advanced Visual Interfaces (AVI'04). ACM Press, New York, New York, 203. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. ETSI. 2006. Digital video broadcasting (DVB); Multimedia home platform (MHP). European Telecomunications Standards Institute (ETSI) Technical Specification TS 101 812 v1.0.2.Google ScholarGoogle Scholar
  7. ETSI. 2010a. Digital video broadcasting (DVB); DVB-HN (home network) reference model phase 1. Technical Specification TS 102 905 Ver. 1.1.1. European Telecomunications Standards Institute (ETSI).Google ScholarGoogle Scholar
  8. ETSI. 2010b. Digital video broadcasting (DVB); Signalling and carriage of interactive applications and services in hybrid broadcast/broadband environments. Technical Specification TS 102 809 v1.1.1. European Telecomunications Standards Institute (ETSI).Google ScholarGoogle Scholar
  9. ETSI. 2012a. Digital video broadcasting (DVB); Plano-stereoscopic 3DTV; Part 2: Frame compatible plano-stereoscopic 3DTV. Technical Specification TS 101 547-2 V1.2.1. European Telecomunications Standards Institute (ETSI).Google ScholarGoogle Scholar
  10. ETSI. 2012b. HbbTV (hybrid broadcast broadband TV) technical specification. Technical Specification TS 102 796 v1.2.1. European Telecomunications Standards Institute (ETSI).Google ScholarGoogle Scholar
  11. I. Galloso, F. P. Luque, L. Piovano, D. Garrido, E. Sánchez, and C. A. Feijoo. 2012. Foundations of a new interaction paradigm for immersive 3D multimedia. In Proceedings of the NEM Summit: Implementing Future Media Internet towards New Horizons.Google ScholarGoogle Scholar
  12. G. Ghinea and O. Ademoye. 2012. The sweet smell of success: Enhancing multimedia applications with olfaction. ACM Trans. Multimedia Comput. Commun. Appl. 8, 1, 1--17. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. E. B. Goldstein. 2010. Sensation and Perception. Cengage Learning.Google ScholarGoogle Scholar
  14. D. W. Heightman. 1975. Digital video broadcasting (DVB); DVB-HN (home network) reference model phase 1. European Telecomunications Standards Institute (ETSI) Technical Specification TS 102 905 Ver. 1.1.1. Radio Electron. Eng. 45, 10, 559--569.Google ScholarGoogle Scholar
  15. H. Hirschmüller. 2008. Stereo processing by semiglobal matching and mutual information. IEEE Trans. Patt. Anal. Mach. Intell. 30, 2, 328--41. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. INSPIRA. 2006. Informe sobre el uso de la televisión digital interactiva por parte del público español. Coordinated by G@TV-UPM. Sponsored by Spanish Ministry of Industry, Tourism and Trade.Google ScholarGoogle Scholar
  17. ISO/IEC. 1998. Information technology—Coding of multimedia and hypermedia information—Part 6: Support for enhanced interactive applications. International Standard (ISO/IEC 13522-6). International Organization for Standardization.Google ScholarGoogle Scholar
  18. ISO/IEC. 2000. Generic coding of moving pictures and associated audio information: Systems. International Standard (ISO/IEC 13818-1). International Organization for Standardization.Google ScholarGoogle Scholar
  19. ISO/IEC. 2007. Home electronic system (HES) architecture Part 3--5. International Standard (ISO/IEC 14543-3). International Organization for Standardization.Google ScholarGoogle Scholar
  20. ISO/IEC. 2013. Information technology—Media context and control—Part 3: Sensory Information. International Standard (ISO/IEC 23005-3). Organization for Standardization.Google ScholarGoogle Scholar
  21. ITU-R. 2000. Subjective assessment of stereoscopic television pictures. International Telecommunication Union.Google ScholarGoogle Scholar
  22. ITU-R. 2002. 500-11. Methodology for the subjective assessment of the quality of television pictures. International Telecommunication Union.Google ScholarGoogle Scholar
  23. ITU-R. 2012. Methodology for the subjective assessment of the quality of television pictures BT Series Broadcasting service. International Telecommunication Union.Google ScholarGoogle Scholar
  24. ITU-T. 1998. P.911: Subjective audiovisual quality assessment methods for multimedia applications. International Telecommunication Union.Google ScholarGoogle Scholar
  25. B. R. Jones, H. Benko, E. Ofek, and A. D. Wilson. 2013. IllumiRoom. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'13). ACM Press, New York, New York, 869. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. S. Jumisko-Pyykkö. 2011. User-centered quality of experience and its evaluation methods for mobile television. Master's Thesis, Tampere University of Technology.Google ScholarGoogle Scholar
  27. J. Kowalczuk, E. Psota, and L. Perez. 2012. Real-time stereo matching on CUDA using an iterative refinement method for adaptive support-weight correspondences. IEEE Trans. Circuits Syst. Video Technol. 23, 1, 94--104. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. P. Le Callet, S. Möller, and A. Perkis, eds. 2012. Qualinet white paper on definitions of quality of experience Qualinet.Google ScholarGoogle Scholar
  29. P. Mills, A. Sheikh, G. Thomas, and P. Debenham. 2011. BBC R&D white paper WHP208. BBC.Google ScholarGoogle Scholar
  30. N. Murray, Y. Qiao, B. Lee, A. K. Karunakar, and G.-M. Muntean. 2013. Subjective evaluation of olfactory and visual media synchronization. In Proceedings of the 4th ACM Multimedia Systems Conference (MMSys'13). 162--171. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. R. L. Nabi and M. Krcmar. 2004. Conceptualizing media enjoyment as attitude: Implications for mass media effects research. Commun. Theory 14, 4, 288--310.Google ScholarGoogle ScholarCross RefCross Ref
  32. D. R. Olsen, D. Bunn, T. Boulter, and R. Walz. 2012. Interactive television news. ACM Trans. Multimedia Comput. Commun. Appl. 8, 2, 1--20. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. G. Ramos and M. Cobos. 2013. Parametric head-related transfer function modeling and interpolation for cost-efficient binaural sound applications. J. Acoustical Soci. Am. 134, 3, 1735--8.Google ScholarGoogle ScholarCross RefCross Ref
  34. A. A. Raney and J. Bryant. 2002. Moral judgment and crime drama: An integrated theory of enjoyment. J. Commun. 52, 2, 402--415.Google ScholarGoogle ScholarCross RefCross Ref
  35. M. J. Schuemie, P. van der Straaten, M. Krijn, and C. van der Mast. 2001. Research on presence in virtual reality: A survey. Cyberpsychol. Behav. Impact Internet Multimedia Virtual Reality Behav. Soci. 4, 2, 183--201.Google ScholarGoogle ScholarCross RefCross Ref
  36. H. Schulzrinne, S. Casner, R. Frederick, and V. Jacobson. 2003. RTP: A transport protocol for real-time applications. Internet Engineering Task Force, RFC 3550. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. M. Slater and S. Wilbur. 1997. A framework for immersive virtual environments (FIVE): Speculations on the role of presence in virtual environments. Presence-Teleoperators Virtual Environ. 6, 6, 603--616.Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. M. D. Slater. 2003. Alienation, aggression, and sensation seeking as predictors of adolescent use of violent film, computer, and website content. J. Commun. 53, 1, 105--121.Google ScholarGoogle ScholarCross RefCross Ref
  39. C. Timmerer, M. Waltl, B. Rainer, and H. Hellwagner. 2012. Assessing the quality of sensory experience for multimedia presentations. Signal Process. Image Commun. 27, 8, 909--916. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. M. Waltl, B. Rainer, C. Timmerer, and H. Hellwagner. 2013. An end-to-end tool chain for sensory experience based on MPEG-V. Signal Process. Image Commun. 28, 2, 136--150. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. M. Waltl, C. Timmerer, and H. Hellwagner. 2009. A test-bed for quality of multimedia experience evaluation of sensory effects. In Proceedings of the International Workshop on Quality of Multimedia Experience. IEEE, 145--150.Google ScholarGoogle Scholar
  42. M. Waltl, C. Timmerer, and H. Hellwagner. 2010. Increasing the user experience of multimedia presentations with sensory effects. In Proceedings of the 11th International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS). IEEE, 1--4.Google ScholarGoogle Scholar
  43. I. Wechsung, M. Schulz, K.-P. Engelbrecht, J. Niemann, and S. Möller. 2011. All users are (not) equal - The influence of user characteristics on perceived quality, modality choice and performance. In Proceedings of the Paralinguistic Information and its Integration in Spoken Dialogue Systems Workshop. Springer, New York, NY, 175--186.Google ScholarGoogle Scholar
  44. A. Weffers-Albu, S. de Waele, W. Hoogenstraaten, and C. Kwisthout. 2011. Immersive TV viewing with advanced Ambilight. In Proceedings of the IEEE International Conference on Consumer Electronics (ICCE). 753--754.Google ScholarGoogle Scholar
  45. D. Winer. 1999. XML remote procedure call (XML-RPC) specification. XML-RPC.Google ScholarGoogle Scholar
  46. Z. Yang, W. Wu, K. Nahrstedt, G. Kurillo, and R. Bajcsy. 2010. Enabling multi-party 3D tele-immersive environments with ViewCast. ACM Trans. Multimedia Comput. Commun. Appl. 6, 2, 1--30. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. K. Yoon. 2013. End-to-end framework for 4-D broadcasting based on MPEG-V standard. Signal Process. Image Commun. 28, 2, 127--135. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. D. Zillmann and P. Vorderer. 2000. Media Entertainment: The Psychology of Its Appeal. Lawrence Erlbaum Associates Publishers.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Integration of Multisensorial Stimuli and Multimodal Interaction in a Hybrid 3DTV System

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!