Inconsistencies of Presence Questionnaires in Virtual Reality

ABSTRACT

Presence in virtual reality (VR) is typically assessed through questionnaires in the real world and after leaving an immersive experience. Previous research suggests that questionnaires in VR reduce biases caused by the real-world setup. However, it remains unclear whether presence questionnaires still provide valid results when subjects are being surveyed while the construct is perceived. In a user study with 36 participants, two standardized presence questionnaires (IPQ, SUSa) were either completed in the real lab, in a virtual lab scene, or in the actual scene after a virtual gaming experience. Our results show inconsistencies between the measurements and that main scores, as well as subscales of the presence measures are significantly affected by the subjects’ environment. As presence questionnaires have been designed to be answered after an immersive experience, we recommend revising those tools for measuring presence in VR.

References

1. Dmitry Alexandrovsky, Susanne Putze, Michael Bonfert, Sebastian Höffner, Pitt Michelmann, Dirk Wenig, Rainer Malaka, and Jan David Smeddinck. 2020. Examining Design Choices of Questionnaires in VR User Studies. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–21. https://doi.org/10.1145/3313831.3376260
2. Andrea Brogni, Mel Slater, and Anthony Steed. 2003. More breaks less presence. In Presence 2003: The 6th Annual International Workshop on Presence. 1–4.
3. Susanne Putze, Dmitry Alexandrovsky, Felix Putze, Sebastian Höffner, Jan David Smeddinck, and Rainer Malaka. 2020. Breaking The Experience: Effects of Questionnaires in VR User Studies. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–15. https://doi.org/10.1145/3313831.3376144
4. Georg Regal, Jan-Niklas Voigt-Antons, Steven Schmidt, Johann Schrammel, Tanja Kojić, Manfred Tscheligi, and Sebastian Möller. 2019. Questionnaires embedded in virtual environments: reliability and positioning of rating scales in virtual environments. Quality and User Experience 4 (12 2019). https://doi.org/10.1007/s41233-019-0029-1
5. Maria V. Sanchez-Vives and Mel Slater. 2005. From presence to consciousness through virtual reality.Nature reviews. Neuroscience 6, 4 (2005), 332–339. https://doi.org/10.1038/nrn1651
6. Thomas Schubert, Frank Friedmann, and Holger Regenbrecht. 2001. The Experience of Presence: Factor Analytic Insights. Presence: Teleoperators and Virtual Environments 10, 3(2001), 266–281. https://doi.org/10.1162/105474601300343603 arXiv:https://doi.org/10.1162/105474601300343603
7. Valentin Schwind, Pascal Knierim, Lewis Chuang, and Niels Henze. 2017. ”Where’s Pinky?”: The Effects of a Reduced Number of Fingers in Virtual Reality. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play(CHI PLAY ’17). ACM, New York, NY, USA, 507–515. https://doi.org/10.1145/3116595.3116596
8. Valentin Schwind, Pascal Knierim, Nico Haas, and Niels Henze. 2019. Using Presence Questionnaires in Virtual Reality. In CHI Conference on Human Factors in Computing Systems Proceedings (CHI 2019) (2019-05-04). ACM, Glasgow, Scotland Uk, 12. https://doi.org/10.1145/3290605.3300590
9. Valentin Schwind, Pascal Knierim, Cagri Tasci, Patrick Franczak, Nico Haas, and Niels Henze. 2017. ”These Are Not My Hands!”: Effect of Gender on the Perception of Avatar Hands in Virtual Reality. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems(CHI ’17). ACM, New York, NY, USA, 1577–1582. https://doi.org/10.1145/3025453.3025602
10. Richard Skarbez, Frederick P. Brooks, Jr., and Mary C. Whitton. 2017. A Survey of Presence and Related Concepts. ACM Comput. Surv. 50, 6, Article 96 (Nov. 2017), 39 pages. https://doi.org/10.1145/3134301
11. Mel Slater, Andrea Brogni, and Anthony Steed. 2003. Physiological responses to breaks in presence: A pilot study. 157 (10 2003).
12. Mel Slater, John McCarthy, and Francesco Maringelli. 1998. The Influence of Body Movement on Subjective Presence in Virtual Environments. Human Factors 40, 3 (1998), 469–477. https://doi.org/10.1518/001872098779591368
13. Mel Slater and Anthony Steed. 2000. A Virtual Presence Counter. Presence: Teleoperators and Virtual Environments 9, 5(2000), 413–434. https://doi.org/10.1162/105474600566925 arXiv:https://doi.org/10.1162/105474600566925
14. Mel Slater and Martin Usoh. 1993. Presence in immersive virtual environments. In Proceedings of IEEE Virtual Reality Annual International Symposium. 90–96. https://doi.org/10.1109/VRAIS.1993.380793
15. Martin Usoh, Ernest Catena, Sima Arman, and Mel Slater. 2000. Using Presence Questionnaires in Reality. Presence: Teleoperators and Virtual Environments 9, 5(2000), 497–503. https://doi.org/10.1162/105474600566989
16. Vinoba Vinayagamoorthy and Andrea Brogni. 2004. An Investigation of Presence Response across Variations in Visual Realism. Proceedings of Presence 2004: The 7th Annual International Workshop on Presence, 119–126.
17. Alexandra Voit, Sven Mayer, Valentin Schwind, and Niels Henze. 2019. Online, VR, AR, Lab, and In-Situ: Comparison of Research Methods to Evaluate Smart Artifacts. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems(CHI ’19). Association for Computing Machinery, New York, NY, USA, 12. https://doi.org/10.1145/3290605.3300737
18. Jacob O. Wobbrock, Leah Findlater, Darren Gergle, and James J. Higgins. 2011. The Aligned Rank Transform for Nonparametric Factorial Analyses Using Only Anova Procedures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’11). ACM, New York, NY, USA, 143–146. https://doi.org/10.1145/1978942.1978963