Immersive 360° mobile video with an emotional perspective

ABSTRACT

By appealing to several senses and conveying very rich information, video has the potential for a strong emotional impact on viewers, their sense of presence and engagement. These capacities may be extended even further with multimedia sensing and the flexibility of mobility. Mobile devices are commonly used and increasingly incorporating a wide range of sensors and actuators with the potential to capture and display 360° and HD video and metadata and to support more powerful and immersive video user experien-ces. In this paper, we explore the immersion potential of mobile video augmented with visual, auditory and tactile perceptual sensing with an emotional perspective on the impact on users. Results confirmed advantages in using a multisensory approach to increase immersion and user satisfaction, and identified properties and parameters that worked better in different conditions.

References

1. Bleumers, L., Broeck, W., Lievens, B., Pierson, J. (2012) Seeing the Bigger Picture: A User Perspective on 360° TV. In Proc. of EuroiTV'12, 115--124.
2. Cardin, S., Thalmann, D., and Vexo, F. (2007) Head Mounted Wind. In Proc. of Computer Animation and Social Agents. Hasselt, Belgium, Jun 11--13. 101--108.
3. Courtois, C., D'heer, E. (2012) Second screen applications and tablet users: constellation, awareness, experience, and interest. EuroiTV'12, 153--156.
4. Dobler, D., and Stampfl, P. (2004) Enhancing Three-dimensional Vision with Three-dimensional Sound. In ACM Siggraph'04 Course Notes.
5. Douglas, Y., and Hargadon, A. (2000) The Pleasure Principle: Immersion, Engagement, Flow. ACM Hypertext'00, 153--160.
6. Ekman, P. (1992) Are there basic emotions? Psychological Review, 99(3):550--553.
7. Lehmann, A., Geiger, C., Wöldecke, B. and Stöcklein, J. (2009) Poster: Design and Evaluation of 3D Content with Wind Output. In Proc. of 3DUI.
8. Mendes, M. (2010) RTiVISS | Real-Time Video Inter-active Systems for Sustainability. Artech, 29--38.
9. Merer, A., Aramaki, M., Ystad, S., and Kronland-Martinet, R. (2013) Perceptual Characterization of Motion Evoked by Sounds for Synthesis Control Purposes. In Trans. On Applied Perception, 10(1), 1--23.
10. Moon, T., Kim, G. J., and Vexo, F. (2004) Design and evaluation of a wind display for virtual reality. In Proc. of ACM Symposium on Virtual Reality Software and Technology.
11. Mourão, A., Borges, P., Correia, N., Magalhães, J. Facial Expression Recognition by Sparse Reconstruction with Robust Features. In Image Analysis and Recognition. ICIAR'2013, LNCS, vol 7950, pp.107--115, 2013.
12. Namezi, M., and Gromala, D. (2012) Sound Design: A Procedural Communication Model for VE. In Proc. of 7th Audio Mostly Conference, Greece. 16--23.
13. Neng, L., and Chambel, T. (2010) Get Around 360° Hypervideo. In Proc. of MindTrek, Tampere, Finland. 119--122.
14. Noronha, G., Álvares, C., and Chambel, T. (2012) Sight Surfers: 360° Videos and Maps Navigation. In Proc.of Geo MM'12 ACM Multimedia'12. Nara, Japan. 19--22.
15. Oliveira, E., Martins, P., and Chambel, T. (2011) iFelt: Accessing Movies Through Our Emotions. In Proc. of EuroITV'2011, Lisbon, Portugal, Jun 29-Jul 1. 105--114.
16. Plutchik, R. (1980) Emotion: A psychoevolutionary synthesis Harper & Row New York.
17. Ramalho, J. and Chambel, T. (2013) Windy Sight Surfers: sensing and awareness of 360° immersive videos on the move. In Proc. of EuroITV 2013, Como, Italy. 107--116.
18. Slater, M., Lotto, B., Arnold, M.M., Sanchez-Vives, M.M. (2009) How we experience immersive virtual environments: the concept of presence and its measurement, Anuario de Psicología, 40(2), Fac. Psicologia, Univ. Barcelona, 193--210.
19. Slater, M. (1999) Measuring Presence: A Response to the Witmer and Singer Presence Questionnaire, Presence 8(5), Oct. 560--565.
20. Slater, M., & Wilbur, S. (1997) A framework for immersive virtual environments (five): Speculations on the role of presence in virtual environments. Presence,6, 603--616.
21. Visch, T., Tan, S., and Molenaar, D. (2010) The emotional and cognitive effect of immersion in film viewing. Cognition & Emotion, 24: 8,1439--1445.
22. Wirth, W., Hartmann, T., Bocking, S., Vorderer, P.,Klimmt, Ch., Schramm, H., et al. (2007) A process model of the formation of spatial presence experiences. Media Psychology, 9, 493--525.
23. Witmer, B., Singer, M.J. (1998) Measuring Presence in Virtual, Environments: A Presence Questionnaire. Presence, 7(3), Jun. (1998),MIT, 225--240.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision problems

        • Video segmentation

      • Computer vision tasks

        • Video summarization

      • Image and video acquisition

• Human-centered computing

  • Human computer interaction (HCI)

    • HCI design and evaluation methods

    • Interaction paradigms

      • Hypertext / hypermedia

  • Visualization

    • Visualization techniques