Abstract
Media-rich content plays a vital role in consumer applications today, as these applications try to find new and interesting ways to engage their users. Video, audio, and the more traditional forms of media content continue to dominate with respect to the use of media content to enhance the user experience. Tactile interactivity has also now become widely popular in modern computing applications, while our olfactory and gustatory senses continue to have a limited role. However, in recent times, there have been significant advancements regarding the use of olfactory media content (i.e., smell), and there are a variety of devices now available to enable its computer-controlled emission. This paper explores the impact of the audio stream on user perception of olfactory-enhanced video content in the presence of skews between the olfactory and video media. This research uses the results from two experimental studies of user-perceived quality of olfactory-enhanced multimedia, where audio was present and absent, respectively. Specifically, the paper shows that the user Quality of Experience (QoE) is generally higher in the absence of audio for nearly perfect synchronized olfactory-enhanced multimedia presentations (i.e., an olfactory media skew of between {−10,+10s}); however, for greater olfactory media skews (ranging between {−30s;−10s} and {+10s, +30s}) user QoE is higher when the audio stream is present. It can be concluded that the presence of the audio has the ability to mask larger synchronization skews between the other media components in olfaction-enhanced multimedia presentations.
- O. A. Ademoye and G. Ghinea. 2009. Synchronization of olfaction-enhanced multimedia. IEEE Transactions on Multimedia 11, 3, 561--565. Google Scholar
Digital Library
- O. A. Ademoye and G. Ghinea. 2013. Information recall task impact in olfaction-enhanced multimedia. ACM Transactions on Multimedia Computing, Communications and Applications 9, 3, Article 17. Google Scholar
Digital Library
- G. Blakowski and R. Steinmetz.1996. A media synchronization survey: Reference model, specification, and case studies. IEEE Journal on Selected Areas in Communications 14, 5--35. Google Scholar
Digital Library
- B. R. Brkic, A. Chalmers, K. Boulanger, S. Pattanaik, and J. Covington. 2009. Cross-modal effects of smell on the real-time rendering of grass. In Proceedings of the Spring Conference on Computer Graphics (SCCG). 161--166. Google Scholar
Digital Library
- J. Cha, M. Eid, A. Barghout, A. M. Rahman, and A. El Saddik. 2009. HugMe: Synchronous haptic teleconferencing. In Proceedings of the 17th ACM International Conference on Multimedia (MM’09). 1135. Google Scholar
Digital Library
- C. Concolato, S. Thomas, R. Bouqueau, and J. Le Feuvre. 2012. Synchronized delivery of multimedia content over uncoordinated broadcast broadband networks. In Proceedings of the 3rd Multimedia Systems Conference (MMSys’12). 227--232. Google Scholar
Digital Library
- H. Q. Dinh, N. Walker, L. F. Hodges, C. S. C. Song, and A. Kobayashi. 1999. Evaluating the importance of multi-sensory input on memory and the sense of presence in virtual environments. In Proceedings of the IEEE Virtual Reality Symposium. 222--228. Google Scholar
Digital Library
- M. Fernstrom and L. Bannon. 1997. Enabling technology for museum visitors: Issues and experiences. In Proceedings of the International Conference on Museums and the Web. 191--199.Google Scholar
- G. Ghinea, S. R. Gulliver, and F. Andres. 2011. Multiple Sensorial Media Advances and Applications: New Developments in Mulsemedia. IGI Global, Hershey, PA.Google Scholar
- G. Ghinea and O. A. Ademoye. 2011. Olfaction-enhanced multimedia: Perspectives and challenges. Multimedia Tools and Applications 55, 3, 601--626. Google Scholar
Digital Library
- G. Ghinea and O. A. Ademoye. 2012. The sweet smell of success. ACM Transactions on Multimedia Computing Communications and Applications 8, 1, 2012, 1--17. Google Scholar
Digital Library
- G. Ghinea and O. A. Ademoye. 2014. Perceived synchronization of olfactory multimedia. IEEE Transactions on Systems, Man, and Cybernetics: Systems 40, 4, 2014, 657--663. Google Scholar
Digital Library
- G. Ghinea, C. Timmerer, W. Lin, and S. R. Gulliver. 2014. Mulsemedia: State of the art, perspectives, and challenges. ACM Transactions on Multimedia Computing Communications and Applications 11, 1s, 1--23. Google Scholar
Digital Library
- T. Hall and L. Bannon. 2006. Designing ubiquitous computing to enhance children's learning in museums. Journal of Computer Assisted Learning 22, 231--243.Google Scholar
Cross Ref
- M. Huadong, K. G. Shin. 2004. Checking consistency in multimedia synchronization constraints. IEEE Transactions on Multimedia 6, 4, 565--574 Google Scholar
Digital Library
- Z. Huang, K. Nahrstedt, and R. Steinmetz. 2013. Evolution of temporal multimedia synchronization principles: a historical viewpoint. ACM Transactions on Multimedia Computing Communications and Applications 9, 1--23. Google Scholar
Digital Library
- C. E. Hughes, C. B. Stapleton, D. E. Hughes, and E. M. Smith. 2005. Mixed reality in education, entertainment, and training. IEEE Computer Graphics and Applications 25, 24--30. Google Scholar
Digital Library
- ITU-T R. P.913 2014. Methods for the subjective assessment of video quality, audio quality and audiovisual quality of Internet video and distribution quality television in any environment.Google Scholar
- P. T. Kovacs, N. Murray, G. Rozinaj, Y. Sulema, and R. Rybarova. 2015. Application of immersive technologies for education: State of the art. In Proceedings of the International Conference on Interactive Mobile Communication, Technologies and Learning (IMCL’15). 283--288.Google Scholar
- D. G. Laing. 1983. Natural sniffing gives optimum odour perception for humans. Perception 12, 2, 99--117.Google Scholar
Cross Ref
- H. W. Ludvigson and T. R. Rottman. 1989. Effects of ambient odors of lavender and cloves on cognition, memory, affect and mood. Chemical Senses 14, 525--536.Google Scholar
Cross Ref
- R. Mekuria, P. Cesar, and D. Bulterman. 2012. Digital TV: The effect of delay when watching football. In Proceedings of the 10th European conference on Interactive TV and Video (EuroiTV’12). 71--74. Google Scholar
Digital Library
- T. A. Mikropoulos and A. Natsis. 2011. Educational virtual environments: A ten-year review of empirical research (1999-2009). Computers and Education 56, 769--780. Google Scholar
Digital Library
- M. Montagud, F. Boronat, and H. Stokking. 2013. Early event-driven (EED) RTCP feedback for rapid IDMS. In Proceedings of the 21st ACM International Conference on Multimedia (MM’13). 323--332. Google Scholar
Digital Library
- N. Murray, Y. Qiao, A. K. Karunakar, B. Lee, and G. M. Muntean. 2013a. Age and gender influence on perceived olfactory & visual media synchronization. In Proceedings of IEEE International Conference on Multimedia & Expo. 1--6. DOI:10.1109/ICME.2013.6607467Google Scholar
- N. Murray, Y. Qiao, B. Lee, A. K. Karunakar, and G. M. Muntean. 2013b. Subjective evaluation of olfactory and visual media synchronization. In Proceedings of the 4th ACM Multimedia Systems Conference. 162--171. DOI:10.1145/2483977.2483999 Google Scholar
Digital Library
- N. Murray, Y. Qiao, B. Lee, and G. M. Muntean, 2014a. User-profile-based perceived olfactory and visual media synchronization. ACM Trans. Multimed. Comput. Commun. Appl. 10, 1s, article no. 11. Google Scholar
Digital Library
- N. Murray, B. Lee, Y. Qiao, and G. M. Muntean. 2014b. Multiple-scent enhanced multimedia synchronization. ACM Trans. Multimed. Comput. Commun. Appl. 11, 1s, article no. 12. Google Scholar
Digital Library
- N. Murray, B. Lee, Y. Qiao, and G. M. Muntean. 2016a. Olfaction-enhanced multimedia: A survey of application domains, displays, and research challenges. ACM Comput. Surv. 48, 4, Article 56 (May 2016), 34 pages. DOI:http://dx.doi.org/10.1145/2816454 Google Scholar
Digital Library
- N. Murray, B. Lee, Y. Qiao, and G. M. Muntean. 2016b. The impact of scent type on olfaction-enhanced multimedia quality of experience. In IEEE Transactions on Systems, Man, and Cybernetics: Systems PP, 99 (2016), 1--13. DOI:10.1109/TSMC.2016.2531654Google Scholar
- H. T. Nagle, S. S. Schiffman, and R. Gutierrez-Osuna. 1998. How and why of electronic noses. IEEE Spectrum 35, 22--34. Google Scholar
Digital Library
- T. Narumi, S. Nishizaka, T. Kajinami, T. Tanikawa, and M. Hirose. 2011. Augmented reality flavors: Gustatory display based on edible marker and cross-modal interaction. In Proceedings of the 2011 Annual Conference on Human Factors in Computing Systems (CHI’11). 93--102. Google Scholar
Digital Library
- S. Pan and C. Ryan. 2009. Tourism sense-making: The role of the senses and travel journalism. Journal of Travel and Tourism Marketing 26, 7, 625--639.Google Scholar
Cross Ref
- Picture Exchange Communication System. Retrieved from http://www.pecs.com/.Google Scholar
- L. Schauble, M. Gleason, R. Lehrer, K. Bartlett, A. Petrosino, A. Allen, K. Clinton, E. Ho, M. Jones, Y. S. Yee, J. E. Phillips, J. Siegler, and J. Street. 2002. Supporting science learning in 266 museums. In Learning Conversations in Museums. 425--452.Google Scholar
- M. J. Smith and P. J. Kiger. 2006. The lingering reek of smell-o-vision. Los Angeles Times, Feb-2006.Google Scholar
- R. Steinmetz. 1996. Human perception of jitter and media Synchronization. IEEE Journal on Selected Areas in Communication 14, 1, 61--72. Google Scholar
Digital Library
- J. N. Kaye. 2001. Symbolic Olfactory Display. Massachusetts Institute of Technology.Google Scholar
- E. Richard, A. Tijou, P. Richard, and J. L. Ferrier. 2006a. Multi-modal virtual environments for education with haptic and olfactory feedback. Virtual Reality 10, 3--4, 207--225.Google Scholar
Cross Ref
- E. Richard, A. Tijou, and P. Richard. 2006b. Multi-modal virtual environments for education: From illusion to immersion. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) LNCS, 3942, 1274--1279. Google Scholar
Digital Library
- L. A. Rowe and R. Jain. 2005. ACM SIGMM retreat report on future directions in multimedia research. ACM Transactions on Multimedia Computing, Communications, and Applications 1, 3--13. Google Scholar
Digital Library
- SBi4 radio v2, Exhalia. Retrieved fromhttp://exhalia.com/fr/.Google Scholar
- A. Tijou, E. Richard, and P. Richard. 2006. Using olfactive virtual environments for learning organic molecules. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), LNCS, 3942, 1223--1233. Google Scholar
Digital Library
- C. Timmerer, M. Waltl, B. Rainer, and N. Murray. 2014. Sensory experience: quality of experience beyond audio-visual. In Quality of Experience, Ed: Springer, 351--365.Google Scholar
- Vortex Activ USB. 2014. Dale Air. Retrieved from http://www.daleair.com/dispensers/machines/vortex-activ.Google Scholar
- D. A. Washburn, L. M. Jones, R. V. Satya, C. A. Bowers, and A. Cortes. 2003. Olfactory use in virtual environment training. Modeling & Simulation 2, 3, 19--25.Google Scholar
- Z. Yuan, G. Ghinea, and G M. Muntean. 2015 Beyond multimedia adaptation: quality of experience-aware multi-sensorial media. IEEE Transactions on Multimedia 17, 1, 104--117.Google Scholar
Cross Ref
Index Terms
Audio Masking Effect on Inter-Component Skews in Olfaction-Enhanced Multimedia Presentations
Recommendations
Olfaction-Enhanced Multimedia: A Survey of Application Domains, Displays, and Research Challenges
Recently, the concept of olfaction-enhanced multimedia applications has gained traction as a step toward further enhancing user quality of experience. The next generation of rich media services will be immersive and multisensory, with olfaction playing ...
Multiple-Scent Enhanced Multimedia Synchronization
Special Issue on Multiple Sensorial (MulSeMedia) Multimodal Media : Advances and ApplicationsThis study looked at users' perception of interstream synchronization between audiovisual media and two olfactory streams. The ability to detect skews and the perception and impact of skews on user Quality of Experience (QoE) is analyzed. The olfactory ...
Perceived synchronization of olfactory multimedia
The concept of synchronization is of fundamental importance in multimedia systems and applications. The focus of this this paper is on olfaction-enhanced multimedia, which concerns itself with associating computer-generated smell with other media. ...






Comments