Auditory self-motion simulation is facilitated by haptic and vibrational cues suggesting the possibility of actual motion

Abstract

Sound fields rotating around stationary blindfolded listeners sometimes elicit auditory circular vection, the illusion that the listener is physically rotating. Experiment 1 investigated whether auditory circular vection depends on participants' situational awareness of “movability,” that is, whether they sense/know that actual motion is possible or not. While previous studies often seated participants on movable chairs to suspend the disbelief of self-motion, it has never been investigated whether this does, in fact, facilitate auditory vection. To this end, 23 blindfolded participants were seated on a hammock chair with their feet either on solid ground (“movement impossible”) or suspended (“movement possible”) while listening to individualized binaural recordings of two sound sources rotating synchronously at 60°/s. Although participants never physically moved, situational awareness of movability facilitated auditory vection. Moreover, adding slight vibrations like the ones resulting from actual chair rotation increased the frequency and intensity of vection. Experiment 2 extended these findings and showed that nonindividualized binaural recordings were as effective in inducing auditory circular vection as individualized recordings. These results have important implications both for our theoretical understanding of self-motion perception and for the applied field of self-motion simulations, where vibrations, nonindividualized binaural sound, and the cognitive/perceptual framework of movability can typically be provided at minimal cost and effort.

References

1. Andersen, G. J. and Braunstein, M. L. 1985. Induced self-motion in central vision. J. Exper. Psych. 11, 2, 122--132.
2. Banbury, S., Tremblay, S., and Banbury, S. 2004. A Cognitive Approach to Situation Awareness. Ashgate Publishing, Burlington, VT.
3. Begault, D. R. 1994. 3-D Sound for Virtual Reality and Multimedia. Academic Press, San Diego, CA.
4. Beierholm, U., Kording, K., Shams, L., and Ma, W. J. 2008. Comparing bayesian models for multisensory cue combination without mandatory integration. In Advances in Neural Information Processing Systems 20, J. Platt, D. Koller, Y. Singer, and S. Roweis, Eds. MIT Press, Cambridge, MA, 81--88.
5. Berthoz, A., Pavard, B., and Young, L. R. 1975. Perception of linear horizontal self-motion induced by peripheral vision (linear vection) -basic characteristics and visual-vestibular interactions. Exper. Brain Res. 23, 5, 471--489.
6. Blauert, J. and Allen, J. S. 1997. Spatial Hearing: The Psychophysics of Human Sound Localization. MIT Press, Cambridge, MA.
7. Bles, W. 1981. Stepping around: Circular vection and coriolis effects. In Attention and Performance IX, J. Long and A. Baddeley, Eds. Erlbaum, Hillsdale, NJ, 47--61.
8. Cohen, J. 1988. Statistical Power Analysis for the Behavioral Sciences. Erlbaum, Hillsdale, NJ.
9. Dichgans, J. and Brandt, T. 1978. Visual-vestibular interaction: Effects on self-motion perception and postural control. In Perception Handbook of Sensory Physiology, vol. VIII. Springer, Berlin, 756--804.
10. Dodge, R. 1923. Thresholds of rotation. J. Exper. Psychol. 6, 107--137.
11. Endsley, M. R., Bolté, B., Jones, D. G., and Betty Bolté, D. G. J. 2003. Designing for Situation Awareness. Taylor&Francis, London.
12. Ernst, M. O. and Banks, M. S. 2002. Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415, 429--433.
13. Ernst, M. O. and Bülthoff, H. H. 2004. Merging the senses into a robust percept. Trends Cognitive Sci. 8, 4, 162--169.
14. Feuereissen, D. 2008. Getting the reality part straight: Does jitter and suspension of the human body increase auditory circular vection? Bachelor's thesis, Department of Computer Science in Media, Furtwangen University, Germany. http://www.kyb.mpg.de/publication.html?publ = 5071.
15. Gekhman, B. 1991. Audiokinetic nystagmus. Sensornye Sistemy 5, 2, 71--78.
16. Hennebert, P. E. 1960. Audiokinetic nystagmus. J. Auditory Res. 1, 1, 84--87.
17. Hettinger, L. J. 2002. Illusory self-motion in virtual environments. In Handbook of Virtual Environments, K. M. Stanney, Ed. Lawrence Erlbaum, Hillsdale, NJ, 471--492.
18. Khasnis, A. and Gokula, R. M. 2003. Romberg's test. J. Postgrad. Med. 49, 2, 169--172. http://www.jpgmonline.com/text.asp?2003/49/2/169/894
19. Lackner, J. R. 1977. Induction of illusory self-rotation and nystagmus by a rotating sound-field. Aviat. Space Environ. Med. 48, 2, 129--131.
20. Langendijk, E. and Bronkhorst, A. 2002. Contribution of spectral cues to human sound localization. J. Acoust. Soc. Amer. 112, 4, 1583--1596.
21. Larsson, P., Västfjäll, D., and Kleiner, M. 2004. Perception of self-motion and presence in auditory virtual environments. In Proceedings of the 7th Annual Workshop of Presence. ACM, New York, 252--258.
22. Lepecq, J. C., Giannopulu, I., and Baudonniere, P. M. 1995. Cognitive effects on visually induced body motion in children. Perception 24, 4, 435--449.
23. Loomis, J., Klatzky, R. L., and Golledge, R. G. 1999. Auditory distance perception in real, virtual, and mixed environments. In Mixed Reality: Merging Real and Virtual Worlds, Ohta, Y, and Tamura, H. Springer, Berlin, 201--214.
24. Mach, E. 1875. Grundlinien der Lehre von der Bewegungsempfindung. Engelmann, Leipzig, Germany.
25. Marmekarelse, A. M. and Bles, W. 1977. Circular vection and human posture ii: Does the auditory-system play a role. Agressologie 18, 6, 329--333.
26. Mergner, T. and Becker, W. 1990. Perception of horizontal self-rotation: Multisensory and cognitive aspects. In Perception&Control of Self-Motion, R. Warren and A. H. Wertheim, Eds. Erlbaum, New Jersey, 219--263.
27. Moller, H., Sorensen, M., Jensen, C., and Hammershoi, D. 1996. Binaural technique: Do we need individual recordings? J. Audio Engin. Soc. 44, 6, 451--469.
28. Palmisano, S., Burke, D., and Allison, R. S. 2003. Coherent perspective jitter induces visual illusions of self-motion. Perception 32, 1, 97--110.
29. Palmisano, S., Gillam, B. J., and Blackburn, S. G. 2000. Global-perspective jitter improves vection in central vision. Perception 29, 1, 57--67.
30. Pavard, B. and Berthoz, A. 1977. Linear acceleration modifies perceived velocity of a moving visual scene. Perception 6, 5, 529--540.
31. Pick, H. L., Wagner, D., Rieser, J. J., and Garing, A. E. 1999. The recalibration of rotational locomotion. J. Exper. Psych. 25, 5, 1179--1188.
32. Riecke, B. E., Feuereissen, D., and Rieser, J. J. 2008a. Auditory self-motion illusions (“circular vection”) can be facilitated by vibrations and the potential for actual motion. In Proceedings of the ACM SIGGRAPH Symposium on Applied Perception in Graphics and Visualization (APGV'08). ACM, New York, 147--154.
33. Riecke, B. E., Feuereissen, D., and Rieser, J. J. 2008b. Contribution and interaction of auditory and biomechanical cues for self-motion illusions (“circular vection”). In Proceedings of the CyberWalk Workshop. Springer, Berlin.
34. Riecke, B. E., Schulte-Pelkum, J., Avraamides, M. N., von der Heyde, M., and Bülthoff, H. H. 2006. Cognitive factors can influence self-motion perception (vection) in virtual reality. ACM Trans. Appl. Perception 3, 3, 194--216.
35. Riecke, B. E., Schulte-Pelkum, J., and Caniard, F. 2006. Visually induced linear vection is enhanced by small physical accelerations. In Proceedings of the 7th International Multisensory Research Forum (IMRF). ACM, New York.
36. Riecke, B. E., Schulte-Pelkum, J., Caniard, F., and Bülthoff, H. H. 2005a. Influence of auditory cues on the visually-induced self-motion illusion (circular vection) in virtual reality. In Proceedings of 8th Annual Workshop Presence 2005. MIT, Cambridge, MA, 49--57.
37. Riecke, B. E., Schulte-Pelkum, J., Caniard, F., and Bülthoff, H. H. 2005b. Towards lean and elegant self-motion simulation in virtual reality. In Proceedings of IEEE Virtual Reality. IEEE, Los Alamitos, CA, 131--138.
38. Riecke, B. E., Västfjäll, D., Larsson, P., and Schulte-Pelkum, J. 2005. Top-down and multi-modal influences on self-motion perception in virtual reality. In Proceedings of the 11th Annual Internation Conference on Human-Computer Interaction (HCI'05). Springer, Berlin, 1--10.
39. Riecke, B. E., von der Heyde, M., and Bülthoff, H. H. 2005. Visual cues can be sufficient for triggering automatic, reflex-like spatial updating. ACM Trans. Appl. Perception 2, 3, 183--215.
40. Riecke, B. E., Väljamäe, A., and Schulte-Pelkum, J. 2009. Moving sounds enhance the visually-induced self-motion illusion (circular vection) in virtual reality. ACM Trans. Appl. Perception 6, 2.
41. Rieser, John, J., Pick, Herbert, L., Ashmead, Daniel, H., and Garing, Anne, E. 1995. Calibration of human locomotion and models of perceptual-motor organization. J. Exper. Psych. 21, 3, 480--497.
42. Sakamoto, S., Osada, Y., Suzuki, Y., and Gyoba, J. 2004. The effects of linearly moving sound images on self-motion perception. Acoust. Sci. Tech. 25, 100--102.
43. Schulte-Pelkum, J. 2008. Perception of self-motion: Vection experiments in multi-sensory virtual environments. Ph.D. thesis, Ruhr-Universität Bochum.
44. Schulte-Pelkum, J. and Riecke, B. E. 2009. An integrative approach to presence and self-motion perception research. In Immersed in Media Experiences: Presence Psychology and Design (Handbook of Presence). Lawrence Erlbaum, Hillsdale, NJ.
45. Schulte-Pelkum, J., Riecke, B. E., and Bülthoff, H. H. 2004. Vibrational cues enhance believability of ego-motion simulation. In Proceedings of the International Multisensory Research Forum (IMRF). www.kyb.mpg.de/publication.html?publ=2766.
46. Shilling, R. D. and Shinn-Cunningham, B. 2002. Virtual auditory displays. In Handbook of Virtual Environments, K. M. Stanney, Ed. Lawrence Erlbaum, 65--91.
47. Väljamäe, A., Larsson, P., Västfjäll, D., and Kleiner, M. 2004. Auditory presence, individualized head-related transfer functions, and illusory ego-motion in virtual environments. In Proceedings of 7th Annual Workshop of Presence. MIT, Cambridge, MA, 141--147.
48. Väljamäe, A., Larsson, P., Västfjäll, D., and Kleiner, M. 2006. Vibrotactile enhancement of auditory induced self-motion and spatial presence. J. Acoust. Engin. Soc. 54, 10, 954--963.
49. von Helmholtz, H. 1867. Handbuch der physiologischen Optik. Voss, Leipzig.
50. Väljamäe, A. 2005. Self-motion and presence in the perceptual optimization of a multisensory virtual reality environment. Licentiate dissertation, Chalmers University of Technology, Göteborg, Sweden.
51. Väljamäe, A. 2007. Sound for multi-sensory motion simulators. Ph.D. thesis, Chalmers University of Technology, Göteborg, Sweden.
52. Väljamäe, A., Larsson, P., Västfjäll, D., and Kleiner, M. 2009. Auditory landmarks enhance circular vection in multi-modal virtual reality. J. Acoust. Engin. Soc. To appear.
53. Warren, R. and Wertheim, A. H., Eds. 1990. Perception&Control of Self-Motion. Erlbaum, Hillsdale, NJ.
54. Wong, S. C. P. and Frost, B. J. 1981. The effect of visual-vestibular conflict on the latency of steady-state visually induced subjective rotation. Perception Psychophys. 30, 3, 228--236.
55. Wright, W. G., Dizio, P., and Lackner, J. R. 2006. Perceived self-motion in two visual contexts: Dissociable mechanisms underlie perception. J. Vestib. Res. 16, 1--2, 23--28.

Index Terms

• Applied computing

  • Law, social and behavioral sciences

• Computing methodologies

  • Computer graphics

    • Graphics systems and interfaces

      • Virtual reality

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction paradigms

      • Mixed / augmented reality

      • Virtual reality