skip to main content
research-article

Audio Masking Effect on Inter-Component Skews in Olfaction-Enhanced Multimedia Presentations

Published:03 August 2016Publication History
Skip Abstract Section

Abstract

Media-rich content plays a vital role in consumer applications today, as these applications try to find new and interesting ways to engage their users. Video, audio, and the more traditional forms of media content continue to dominate with respect to the use of media content to enhance the user experience. Tactile interactivity has also now become widely popular in modern computing applications, while our olfactory and gustatory senses continue to have a limited role. However, in recent times, there have been significant advancements regarding the use of olfactory media content (i.e., smell), and there are a variety of devices now available to enable its computer-controlled emission. This paper explores the impact of the audio stream on user perception of olfactory-enhanced video content in the presence of skews between the olfactory and video media. This research uses the results from two experimental studies of user-perceived quality of olfactory-enhanced multimedia, where audio was present and absent, respectively. Specifically, the paper shows that the user Quality of Experience (QoE) is generally higher in the absence of audio for nearly perfect synchronized olfactory-enhanced multimedia presentations (i.e., an olfactory media skew of between {−10,+10s}); however, for greater olfactory media skews (ranging between {−30s;−10s} and {+10s, +30s}) user QoE is higher when the audio stream is present. It can be concluded that the presence of the audio has the ability to mask larger synchronization skews between the other media components in olfaction-enhanced multimedia presentations.

References

  1. O. A. Ademoye and G. Ghinea. 2009. Synchronization of olfaction-enhanced multimedia. IEEE Transactions on Multimedia 11, 3, 561--565. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. O. A. Ademoye and G. Ghinea. 2013. Information recall task impact in olfaction-enhanced multimedia. ACM Transactions on Multimedia Computing, Communications and Applications 9, 3, Article 17. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. G. Blakowski and R. Steinmetz.1996. A media synchronization survey: Reference model, specification, and case studies. IEEE Journal on Selected Areas in Communications 14, 5--35. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. B. R. Brkic, A. Chalmers, K. Boulanger, S. Pattanaik, and J. Covington. 2009. Cross-modal effects of smell on the real-time rendering of grass. In Proceedings of the Spring Conference on Computer Graphics (SCCG). 161--166. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. J. Cha, M. Eid, A. Barghout, A. M. Rahman, and A. El Saddik. 2009. HugMe: Synchronous haptic teleconferencing. In Proceedings of the 17th ACM International Conference on Multimedia (MM’09). 1135. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. C. Concolato, S. Thomas, R. Bouqueau, and J. Le Feuvre. 2012. Synchronized delivery of multimedia content over uncoordinated broadcast broadband networks. In Proceedings of the 3rd Multimedia Systems Conference (MMSys’12). 227--232. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. H. Q. Dinh, N. Walker, L. F. Hodges, C. S. C. Song, and A. Kobayashi. 1999. Evaluating the importance of multi-sensory input on memory and the sense of presence in virtual environments. In Proceedings of the IEEE Virtual Reality Symposium. 222--228. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. M. Fernstrom and L. Bannon. 1997. Enabling technology for museum visitors: Issues and experiences. In Proceedings of the International Conference on Museums and the Web. 191--199.Google ScholarGoogle Scholar
  9. G. Ghinea, S. R. Gulliver, and F. Andres. 2011. Multiple Sensorial Media Advances and Applications: New Developments in Mulsemedia. IGI Global, Hershey, PA.Google ScholarGoogle Scholar
  10. G. Ghinea and O. A. Ademoye. 2011. Olfaction-enhanced multimedia: Perspectives and challenges. Multimedia Tools and Applications 55, 3, 601--626. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. G. Ghinea and O. A. Ademoye. 2012. The sweet smell of success. ACM Transactions on Multimedia Computing Communications and Applications 8, 1, 2012, 1--17. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. G. Ghinea and O. A. Ademoye. 2014. Perceived synchronization of olfactory multimedia. IEEE Transactions on Systems, Man, and Cybernetics: Systems 40, 4, 2014, 657--663. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. G. Ghinea, C. Timmerer, W. Lin, and S. R. Gulliver. 2014. Mulsemedia: State of the art, perspectives, and challenges. ACM Transactions on Multimedia Computing Communications and Applications 11, 1s, 1--23. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. T. Hall and L. Bannon. 2006. Designing ubiquitous computing to enhance children's learning in museums. Journal of Computer Assisted Learning 22, 231--243.Google ScholarGoogle ScholarCross RefCross Ref
  15. M. Huadong, K. G. Shin. 2004. Checking consistency in multimedia synchronization constraints. IEEE Transactions on Multimedia 6, 4, 565--574 Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Z. Huang, K. Nahrstedt, and R. Steinmetz. 2013. Evolution of temporal multimedia synchronization principles: a historical viewpoint. ACM Transactions on Multimedia Computing Communications and Applications 9, 1--23. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. C. E. Hughes, C. B. Stapleton, D. E. Hughes, and E. M. Smith. 2005. Mixed reality in education, entertainment, and training. IEEE Computer Graphics and Applications 25, 24--30. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. ITU-T R. P.913 2014. Methods for the subjective assessment of video quality, audio quality and audiovisual quality of Internet video and distribution quality television in any environment.Google ScholarGoogle Scholar
  19. P. T. Kovacs, N. Murray, G. Rozinaj, Y. Sulema, and R. Rybarova. 2015. Application of immersive technologies for education: State of the art. In Proceedings of the International Conference on Interactive Mobile Communication, Technologies and Learning (IMCL’15). 283--288.Google ScholarGoogle Scholar
  20. D. G. Laing. 1983. Natural sniffing gives optimum odour perception for humans. Perception 12, 2, 99--117.Google ScholarGoogle ScholarCross RefCross Ref
  21. H. W. Ludvigson and T. R. Rottman. 1989. Effects of ambient odors of lavender and cloves on cognition, memory, affect and mood. Chemical Senses 14, 525--536.Google ScholarGoogle ScholarCross RefCross Ref
  22. R. Mekuria, P. Cesar, and D. Bulterman. 2012. Digital TV: The effect of delay when watching football. In Proceedings of the 10th European conference on Interactive TV and Video (EuroiTV’12). 71--74. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. T. A. Mikropoulos and A. Natsis. 2011. Educational virtual environments: A ten-year review of empirical research (1999-2009). Computers and Education 56, 769--780. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. M. Montagud, F. Boronat, and H. Stokking. 2013. Early event-driven (EED) RTCP feedback for rapid IDMS. In Proceedings of the 21st ACM International Conference on Multimedia (MM’13). 323--332. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. N. Murray, Y. Qiao, A. K. Karunakar, B. Lee, and G. M. Muntean. 2013a. Age and gender influence on perceived olfactory & visual media synchronization. In Proceedings of IEEE International Conference on Multimedia & Expo. 1--6. DOI:10.1109/ICME.2013.6607467Google ScholarGoogle Scholar
  26. N. Murray, Y. Qiao, B. Lee, A. K. Karunakar, and G. M. Muntean. 2013b. Subjective evaluation of olfactory and visual media synchronization. In Proceedings of the 4th ACM Multimedia Systems Conference. 162--171. DOI:10.1145/2483977.2483999 Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. N. Murray, Y. Qiao, B. Lee, and G. M. Muntean, 2014a. User-profile-based perceived olfactory and visual media synchronization. ACM Trans. Multimed. Comput. Commun. Appl. 10, 1s, article no. 11. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. N. Murray, B. Lee, Y. Qiao, and G. M. Muntean. 2014b. Multiple-scent enhanced multimedia synchronization. ACM Trans. Multimed. Comput. Commun. Appl. 11, 1s, article no. 12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. N. Murray, B. Lee, Y. Qiao, and G. M. Muntean. 2016a. Olfaction-enhanced multimedia: A survey of application domains, displays, and research challenges. ACM Comput. Surv. 48, 4, Article 56 (May 2016), 34 pages. DOI:http://dx.doi.org/10.1145/2816454 Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. N. Murray, B. Lee, Y. Qiao, and G. M. Muntean. 2016b. The impact of scent type on olfaction-enhanced multimedia quality of experience. In IEEE Transactions on Systems, Man, and Cybernetics: Systems PP, 99 (2016), 1--13. DOI:10.1109/TSMC.2016.2531654Google ScholarGoogle Scholar
  31. H. T. Nagle, S. S. Schiffman, and R. Gutierrez-Osuna. 1998. How and why of electronic noses. IEEE Spectrum 35, 22--34. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. T. Narumi, S. Nishizaka, T. Kajinami, T. Tanikawa, and M. Hirose. 2011. Augmented reality flavors: Gustatory display based on edible marker and cross-modal interaction. In Proceedings of the 2011 Annual Conference on Human Factors in Computing Systems (CHI’11). 93--102. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. S. Pan and C. Ryan. 2009. Tourism sense-making: The role of the senses and travel journalism. Journal of Travel and Tourism Marketing 26, 7, 625--639.Google ScholarGoogle ScholarCross RefCross Ref
  34. Picture Exchange Communication System. Retrieved from http://www.pecs.com/.Google ScholarGoogle Scholar
  35. L. Schauble, M. Gleason, R. Lehrer, K. Bartlett, A. Petrosino, A. Allen, K. Clinton, E. Ho, M. Jones, Y. S. Yee, J. E. Phillips, J. Siegler, and J. Street. 2002. Supporting science learning in 266 museums. In Learning Conversations in Museums. 425--452.Google ScholarGoogle Scholar
  36. M. J. Smith and P. J. Kiger. 2006. The lingering reek of smell-o-vision. Los Angeles Times, Feb-2006.Google ScholarGoogle Scholar
  37. R. Steinmetz. 1996. Human perception of jitter and media Synchronization. IEEE Journal on Selected Areas in Communication 14, 1, 61--72. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. J. N. Kaye. 2001. Symbolic Olfactory Display. Massachusetts Institute of Technology.Google ScholarGoogle Scholar
  39. E. Richard, A. Tijou, P. Richard, and J. L. Ferrier. 2006a. Multi-modal virtual environments for education with haptic and olfactory feedback. Virtual Reality 10, 3--4, 207--225.Google ScholarGoogle ScholarCross RefCross Ref
  40. E. Richard, A. Tijou, and P. Richard. 2006b. Multi-modal virtual environments for education: From illusion to immersion. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) LNCS, 3942, 1274--1279. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. L. A. Rowe and R. Jain. 2005. ACM SIGMM retreat report on future directions in multimedia research. ACM Transactions on Multimedia Computing, Communications, and Applications 1, 3--13. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. SBi4 radio v2, Exhalia. Retrieved fromhttp://exhalia.com/fr/.Google ScholarGoogle Scholar
  43. A. Tijou, E. Richard, and P. Richard. 2006. Using olfactive virtual environments for learning organic molecules. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), LNCS, 3942, 1223--1233. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. C. Timmerer, M. Waltl, B. Rainer, and N. Murray. 2014. Sensory experience: quality of experience beyond audio-visual. In Quality of Experience, Ed: Springer, 351--365.Google ScholarGoogle Scholar
  45. Vortex Activ USB. 2014. Dale Air. Retrieved from http://www.daleair.com/dispensers/machines/vortex-activ.Google ScholarGoogle Scholar
  46. D. A. Washburn, L. M. Jones, R. V. Satya, C. A. Bowers, and A. Cortes. 2003. Olfactory use in virtual environment training. Modeling & Simulation 2, 3, 19--25.Google ScholarGoogle Scholar
  47. Z. Yuan, G. Ghinea, and G M. Muntean. 2015 Beyond multimedia adaptation: quality of experience-aware multi-sensorial media. IEEE Transactions on Multimedia 17, 1, 104--117.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Audio Masking Effect on Inter-Component Skews in Olfaction-Enhanced Multimedia Presentations

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          • Published in

            cover image ACM Transactions on Multimedia Computing, Communications, and Applications
            ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 12, Issue 4
            August 2016
            219 pages
            ISSN:1551-6857
            EISSN:1551-6865
            DOI:10.1145/2983297
            Issue’s Table of Contents

            Copyright © 2016 ACM

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 3 August 2016
            • Revised: 1 May 2016
            • Accepted: 1 May 2016
            • Received: 1 January 2016
            Published in tomm Volume 12, Issue 4

            Permissions

            Request permissions about this article.

            Request Permissions

            Check for updates

            Qualifiers

            • research-article
            • Research
            • Refereed

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!