skip to main content
research-article
Open Access

Evaluating Tutorial-Based Instructions for Controllers in Virtual Reality Games

Published:06 October 2021Publication History
Skip Abstract Section

Abstract

Virtual reality (VR) has disrupted the gaming market and is rapidly becoming ubiquitous. Yet differences between VR and traditional mediums, such as controllers that are visible in the virtual world, enable entirely new approaches to instruction. In this paper, we present four studies, each using a different VR game. Within each study, we compared three different modalities of tutorials: Text (text-only), Text+Diagram (text with controller diagrams), and Text+Spatial (text with controller tooltips appearing on top of the player's virtual controllers). Data from our studies show that the importance of tutorial modality depends greatly on game type. In a third-person shooter, Text+Spatial led to significantly higher controls learnability than Text and Text+Diagram, and also led to significantly higher performance, player experience, and intrinsic motivation than Text. In a puzzle game, Text+Spatial led to significantly higher controls learnability and performance than Text. Additionally, Text+Diagram led to significantly higher controls learnability than Text. However, in a wave shooter and a rhythm game, differences between conditions were negligible on all measures. Our studies show that game type is an important factor to consider when designing tutorial modality.

Skip Supplemental Material Section

Supplemental Material

References

  1. Forecast augmented (AR) and virtual reality (VR) market size worldwide from 2016 to 2024: 2020. https://www.statista.com/statistics/591181/global-augmented-virtual-reality-market-size/.Google ScholarGoogle Scholar
  2. New Consumer Data Finds VR Headset Usage Expected To Increase In 2019, According To Greenlight Insights: 2018. https://greenlightinsights.com/new-consumer-data-finds-vr-headset-usage-expected-increase-2019-according-greenlight-insights/.Google ScholarGoogle Scholar
  3. Rose, T., Nam, C.S. and Chen, K.B. 2018. Immersion of virtual reality for rehabilitation - Review. Applied Ergonomics.Google ScholarGoogle Scholar
  4. Lawson, G., Salanitri, D. and Waterfield, B. 2016. Future directions for the development of virtual reality within an automotive manufacturer. Applied Ergonomics. 53, (2016), 323--330. DOI:https://doi.org/10.1016/j.apergo.2015.06.024.Google ScholarGoogle Scholar
  5. Jou, M. and Wang, J. 2013. Investigation of effects of virtual reality environments on learning performance of technical skills. Computers in Human Behavior. (2013). DOI:https://doi.org/10.1016/j.chb.2012.04.020.Google ScholarGoogle Scholar
  6. Bertram, J., Moskaliuk, J. and Cress, U. 2015. Virtual training: Making reality work? Computers in Human Behavior. 43, (2015), 284--292. DOI:https://doi.org/10.1016/j.chb.2014.10.032.Google ScholarGoogle Scholar
  7. Andersen, E., O'Rourke, E., Liu, Y.-E., Snider, R., Lowdermilk, J., Truong, D., Cooper, S. and Popovic, Z. 2012. The impact of tutorials on games of varying complexity. Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems - CHI '12. Pajitnov 1984 (2012), 59--68. DOI:https://doi.org/10.1145/2207676.2207687.Google ScholarGoogle Scholar
  8. Frommel, J., Fahlbusch, K., Brich, J. and Weber, M. 2017. The Effects of Context-Sensitive Tutorials in Virtual Reality Games. (2017), 367--375. DOI:https://doi.org/10.1145/3116595.3116610.Google ScholarGoogle Scholar
  9. Adams, E. 2014. Fundamentals of game design. Pearson Education.Google ScholarGoogle Scholar
  10. Burdea, G.C. and Coiffet, P. 2003. Virtual reality technology. John Wiley & Sons.Google ScholarGoogle Scholar
  11. Wann, J. and Mon-Williams, M. 1996. What does virtual reality NEED?: Human factors issues in the design of three-dimensional computer environments. International Journal of Human Computer Studies. 44, 6 (1996), 829--847. DOI:https://doi.org/10.1006/ijhc.1996.0035.Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Slater, M. and Sanchez-Vives, M. V. 2016. Enhancing our lives with immersive virtual reality. Frontiers Robotics AI.Google ScholarGoogle Scholar
  13. Ryan, R.M., Rigby, C.S. and Przybylski, A. 2006. The Motivational Pull of Video Games: A Self-Determination Theory Approach. Motivation and Emotion. 30, 4 (2006), 344--360. DOI:https://doi.org/10.1007/s11031-006--9051--8.Google ScholarGoogle ScholarCross RefCross Ref
  14. Frommel, J., Sonntag, S. and Weber, M. 2017. Effects of controller-based locomotion on player experience in a virtual reality exploration game. ACM International Conference Proceeding Series. Part F1301, August (2017). DOI:https://doi.org/10.1145/3102071.3102082.Google ScholarGoogle Scholar
  15. Ho, J.C.F. 2017. Practice in Reality for Virtual Reality Games: Making Players Familiar and Confident with a Game. IFIP Conference on Human-Computer Interaction. (2017), 147--162. DOI:https://doi.org/10.1007/978-0--387--35175--9.Google ScholarGoogle ScholarCross RefCross Ref
  16. Mayer, R.E. 2014. Cognitive theory of multimedia learning. The Cambridge Handbook of Multimedia Learning, Second Edition.Google ScholarGoogle Scholar
  17. Sweller, J. 1988. Cognitive load during problem solving: Effects on learning. Cognitive Science. (1988). DOI:https://doi.org/10.1016/0364-0213(88)90023--7.Google ScholarGoogle Scholar
  18. Clark, R.C. and Mayer, R.E. 2016. E-learning and the Science of Instruction important: Fourth Edition. Published by John Wiley & Sons, Inc., Hoboken, New Jersey. (2016).Google ScholarGoogle Scholar
  19. Clark, R.C. and Mayer, R.E. 2011. E-learning and the Science of Instruction?: Proven Guidelines for Consumers and Designers of Multimedia Learning. (2011).Google ScholarGoogle Scholar
  20. Butcher, K.R. 2006. Learning from text with diagrams: Promoting mental model development and inference generation. Journal of Educational Psychology. 98, 1 (2006), 182--197. DOI:https://doi.org/10.1037/0022-0663.98.1.182.Google ScholarGoogle Scholar
  21. Cuevas, H.M., Fiore, S.M. and Oser, R.L. 2002. Scaffolding cognitive and metacognitive processes in low verbal ability learners: Use of diagrams in computer-based training environments. Instructional Science. (2002). DOI:https://doi.org/10.1023/A:1020516301541.Google ScholarGoogle Scholar
  22. Yue, C.L., Bjork, E.L. and Bjork, R.A. 2013. Reducing verbal redundancy in multimedia learning: An undesired desirable difficulty? Journal of Educational Psychology. (2013). DOI:https://doi.org/10.1037/a0031971.Google ScholarGoogle Scholar
  23. Mayer, R.E. 1989. Models for Understanding. Review of Educational Research. (1989). DOI:https://doi.org/10.3102/00346543059001043.Google ScholarGoogle Scholar
  24. Mayer, R.E., Steinhoff, K., Bower, G. and Mars, R. 1995. A generative theory of textbook design: Using annotated illustrations to foster meaningful learning of science text. Educational Technology Research and Development. (1995). DOI:https://doi.org/10.1007/BF02300480.Google ScholarGoogle Scholar
  25. Moreno, R. and Mayer, R.E. 1999. Multimedia-supported metaphors for meaning making in mathematics. Cognition and Instruction. (1999). DOI:https://doi.org/10.1207/S1532690XCI1703_1.Google ScholarGoogle Scholar
  26. Chandler, P. and Sweller, J. 1991. Cognitive Load Theory and the Format of Instruction. Cognition and Instruction. (1991). DOI:https://doi.org/10.1207/s1532690xci0804_2.Google ScholarGoogle Scholar
  27. Pociask, F.D. and Morrison, G.R. 2008. Controlling split attention and redundancy in physical therapy instruction. Educational Technology Research and Development. (2008). DOI:https://doi.org/10.1007/s11423-007--9062--5.Google ScholarGoogle Scholar
  28. Erhel, S. and Jamet, E. 2006. Using pop-up windows to improve multimedia learning. Journal of Computer Assisted Learning. (2006). DOI:https://doi.org/10.1111/j.1365--2729.2006.00165.x.Google ScholarGoogle Scholar
  29. Ginns, P. 2006. Integrating information: A meta-analysis of the spatial contiguity and temporal contiguity effects. Learning and Instruction. (2006). DOI:https://doi.org/10.1016/j.learninstruc.2006.10.001.Google ScholarGoogle Scholar
  30. Wertheimer, M. 1938. Gestalt theory. (1938).Google ScholarGoogle Scholar
  31. Harp, S.F. and Mayer, R.E. 1998. How seductive details do their damage: A theory of cognitive interest in science learning. Journal of Educational Psychology. 90, 3 (1998), 414--434. DOI:https://doi.org/10.1037/0022-0663.90.3.414.Google ScholarGoogle ScholarCross RefCross Ref
  32. Mayer, R.E., Griffith, E., Jurkowitz, I.T.N. and Rothman, D. 2008. Increased Interestingness of Extraneous Details in a Multimedia Science Presentation Leads to Decreased Learning. Journal of Experimental Psychology: Applied. (2008). DOI:https://doi.org/10.1037/a0013835.Google ScholarGoogle Scholar
  33. Lin, L. and Atkinson, R.K. 2011. Using animations and visual cueing to support learning of scientific concepts and processes. Computers and Education. 56, 3 (2011), 650--658. DOI:https://doi.org/10.1016/j.compedu.2010.10.007.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Kelleher, C. and Pausch, R. 2005. Stencils-Based Tutorials: Design and Evaluation. Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '05. (2005), 541. DOI:https://doi.org/10.1145/1054972.1055047.Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Dillman, K.R., Mok, T.T.H., Tang, A., Oehlberg, L. and Mitchell, A. 2018. A visual interaction cue framework from video game environments for augmented reality. Conference on Human Factors in Computing Systems - Proceedings. 2018-April, April (2018). DOI:https://doi.org/10.1145/3173574.3173714.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Hu, X., Moore, A., Eubanks, J.C., Aiyaz, A. and Mcmahan, R.P. 2020. Evaluating Interaction Cue Purpose and Timing for Learning and Retaining Virtual Reality Training. Proceedings - SUI 2020: ACM Symposium on Spatial User Interaction. (2020). DOI:https://doi.org/10.1145/3385959.3418448.Google ScholarGoogle Scholar
  37. Oculus 2019. First Steps (https://www.oculus.com/experiences/quest/1863547050392688/). (2019).Google ScholarGoogle Scholar
  38. Klepsch, M., Schmitz, F. and Seufert, T. 2017. Development and validation of two instruments measuring intrinsic, extraneous, and germane cognitive load. Frontiers in psychology. 8, (2017), 1997.Google ScholarGoogle Scholar
  39. Lee, E.A.L. and Wong, K.W. 2008. A review of using virtual reality for learning. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). 5080 LNCS, (2008), 231--241. DOI:https://doi.org/10.1007/978--3--540--69744--2_18.Google ScholarGoogle Scholar
  40. Andersen, E., O'Rourke, E., Liu, Y.-E., Snider, R., Lowdermilk, J., Truong, D., Cooper, S. and Popovic, Z. 2012. The impact of tutorials on games of varying complexity. CHI (2012).Google ScholarGoogle Scholar
  41. Pallavicini, F. and Pepe, A. 2019. Comparing player experience in video games played in virtual reality or on desktop displays: Immersion, flow, and positive emotions. CHI PLAY 2019 - Extended Abstracts of the Annual Symposium on Computer-Human Interaction in Play. (2019), 195--210. DOI:https://doi.org/10.1145/3341215.3355736.Google ScholarGoogle Scholar
  42. Schrader, C. and Bastiaens, T.J. 2012. The influence of virtual presence: Effects on experienced cognitive load and learning outcomes in educational computer games. Computers in Human Behavior. (2012). DOI:https://doi.org/10.1016/j.chb.2011.11.011.Google ScholarGoogle Scholar
  43. Kao, D. 2020. Exploring Help Facilities in Game-Making Software. ACM Foundations of Digital Games (2020).Google ScholarGoogle Scholar
  44. Kao, D., Joshi, A., Mousas, C., Peddireddy, A., Gopi, A.K., Li, J., Springer, J., McGowan, B.S. and Reed, J.B. 2021. Fighting COVID-19 at Purdue University: Design and Evaluation of a Game for Teaching COVID-19 Hygienic Best Practices. ACM Foundations of Digital Games (2021).Google ScholarGoogle Scholar
  45. O'Rourke, E., Ballweber, C. and Popovi?, Z. 2014. Hint systems may negatively impact performance in educational games. [email protected] 2014 - Proceedings of the 1st ACM Conference on Learning at Scale. (2014), 51--60. DOI:https://doi.org/10.1145/2556325.2566248.Google ScholarGoogle Scholar
  46. Wauck, H. and Fu, W.-T. 2017. A Data-Driven, Multidimensional Approach to Hint Design in Video Games. (2017), 137--147. DOI:https://doi.org/10.1145/3025171.3025224.Google ScholarGoogle Scholar
  47. Rzayev, R., Mayer, S., Krauter, C. and Henze, N. 2019. Notification in VR: The effect of notification placement, task, and environment. CHI PLAY 2019 - Proceedings of the Annual Symposium on Computer-Human Interaction in Play. (2019), 199--211. DOI:https://doi.org/10.1145/3311350.3347190.Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Alexandrovsky, D., Putze, S., Bonfert, M., Höffner, S., Michelmann, P., Wenig, D., Malaka, R. and Smeddinck, J.D. 2020. Examining Design Choices of Questionnaires in VR User Studies. Conference on Human Factors in Computing Systems - Proceedings. (2020), 1--21. DOI:https://doi.org/10.1145/3313831.3376260.Google ScholarGoogle Scholar
  49. Kumaravel, B.T., Nguyen, C., DiVerdi, S. and Hartmann, B. 2019. TutoriVR: A video-based tutorial system for design applications in virtual reality. Conference on Human Factors in Computing Systems - Proceedings (2019).Google ScholarGoogle Scholar
  50. Yang, U. and Kim, G.J. 2002. Implementation and evaluation of ?just follow me": An immersive, VR-based, motion-training system. Presence: Teleoperators and Virtual Environments. (2002). DOI:https://doi.org/10.1162/105474602317473240.Google ScholarGoogle Scholar
  51. Oliveira, D.M., Cao, S.C., Hermida, X.F. and Rodríguez, F.M. 2007. Virtual reality system for industrial training. IEEE International Symposium on Industrial Electronics (2007), 1715--1720.Google ScholarGoogle Scholar
  52. Chittaro, L. and Buttussi, F. 2015. Assessing knowledge retention of an immersive serious game vs. A traditional education method in aviation safety. IEEE Transactions on Visualization and Computer Graphics. 21, 4 (2015), 529--538. DOI:https://doi.org/10.1109/TVCG.2015.2391853.Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Bowman, D.A., Hodges, L.F., Allison, D. and Wineman, J. 1999. The educational value of an information-rich virtual environment. Presence: Teleoperators and Virtual Environments. 8, 3 (Jun. 1999), 317--331. DOI:https://doi.org/10.1162/105474699566251.Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Galvan-Bobadilla, I., Ayala-Garcia, A., Rodriguez-Gallegos, E. and Arroyo-Figueroa, G. 2013. Virtual reality training system for the maintenance of underground lines in power distribution system. 2013 3rd International Conference on Innovative Computing Technology, INTECH 2013 (2013), 199--204.Google ScholarGoogle Scholar
  55. Corato, F., Frucci, M. and Di Baja, G.S. 2012. Virtual training of surgery staff for hand washing procedure. Proceedings of the Workshop on Advanced Visual Interfaces AVI (2012), 274--277.Google ScholarGoogle Scholar
  56. Carlson, P., Peters, A., Gilbert, S.B., Vance, J.M. and Luse, A. 2015. Virtual training: Learning transfer of assembly tasks. IEEE Transactions on Visualization and Computer Graphics. 21, 6 (2015), 770--782. DOI:https://doi.org/10.1109/TVCG.2015.2393871.Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. Cheng, A., Yang, L. and Andersen, E. 2017. Teaching Language and Culture with a Virtual Reality Game. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. (2017), 541--549. DOI:https://doi.org/10.1145/3025453.3025857.Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. Assarroudi, A., Heshmati Nabavi, F., Armat, M.R., Ebadi, A. and Vaismoradi, M. 2018. Directed qualitative content analysis: the description and elaboration of its underpinning methods and data analysis process. Journal of Research in Nursing. 23, 1 (2018), 42--55. DOI:https://doi.org/10.1177/1744987117741667.Google ScholarGoogle ScholarCross RefCross Ref
  59. Elo, S. and Kyngäs, H. 2008. The qualitative content analysis process. Journal of Advanced Nursing. (2008). DOI:https://doi.org/10.1111/j.1365--2648.2007.04569.x.Google ScholarGoogle Scholar
  60. Foxman, M., Leith, A.P., Beyea, D., Klebig, B., Chen, V.H.H. and Ratan, R. 2020. Virtual Reality Genres: Comparing Preferences in Immersive Experiences and Games. Extended Abstracts of the 2020 Annual Symposium on Computer-Human Interaction in Play (2020), 237--241.Google ScholarGoogle Scholar
  61. Sharma, H.N., Alharthi, S.A., Dolgov, I. and Toups, Z.O. 2017. A framework supporting selecting space to make place in spatial mixed reality play. CHI PLAY 2017 - Proceedings of the Annual Symposium on Computer-Human Interaction in Play. (2017), 83--100. DOI:https://doi.org/10.1145/3116595.3116612.Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. Mousas, C., Kao, D., Koilias, A. and Rekabdar, B. 2020. Real and Virtual Environment Mismatching Induces Arousal and Alters Movement Behavior. Proceedings - 2020 IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2020 (2020).Google ScholarGoogle ScholarCross RefCross Ref
  63. Head-Mounted Display Market by Technology (AR, VR), Application (Consumer, Commercial, Enterprise & Industry, Engineering & Design, Military, Medical), Product (Head Mounted, Eyewear), Component, Connectivity, and Geography.: 2019. https://www.marketsandmarkets.com/Market-Reports/head-mounted-display-hmd-market-729.html.Google ScholarGoogle Scholar
  64. Deci, E.L. and Ryan, R.M. 2012. Self-determination theory. (2012).Google ScholarGoogle Scholar
  65. Deterding, S. 2015. The Lens of Intrinsic Skill Atoms: A Method for Gameful Design. Human-Computer Interaction. 30, 3--4 (2015), 294--335. DOI:https://doi.org/10.1080/07370024.2014.993471.Google ScholarGoogle ScholarCross RefCross Ref
  66. Paas, F.G. 1992. Training strategies for attaining transfer of problem-solving skill in statistics: A cognitive-load approach. Journal of educational psychology. 84, 4 (1992), 429.Google ScholarGoogle ScholarCross RefCross Ref
  67. Sweller, J., Van Merrienboer, J.J.G. and Paas, F.G.W.C. 1998. Cognitive architecture and instructional design. Educational psychology review. 10, 3 (1998), 251--296.Google ScholarGoogle Scholar
  68. Deci, E.L. and Ryan, R.M. 2012. Motivation, personality, and development within embedded social contexts: An overview of self-determination theory. Oxford University Press.Google ScholarGoogle Scholar
  69. White, R.W. 1959. Motivation reconsidered: The concept of competence. Psychological Review. 66, 5 (1959), 297.Google ScholarGoogle ScholarCross RefCross Ref
  70. Baumeister, R.F. and Leary, M.R. 1995. The need to belong: Desire for interpersonal attachments as a fundamental human motivation. Psychological Bulletin. 117, 3 (1995), 497.Google ScholarGoogle ScholarCross RefCross Ref
  71. Chirkov, V., Ryan, R.M., Kim, Y. and Kaplan, U. 2003. Differentiating autonomy from individualism and independence: A self-determination theory perspective on internalization of cultural orientations and well-being. Journal of Personality and Social Psychology. 84, 1 (2003), 97.Google ScholarGoogle ScholarCross RefCross Ref
  72. McAuley, E., Duncan, T. and Tammen, V. V 1989. Psychometric properties of the Intrinsic Motivation Inventory in a competitive sport setting: A confirmatory factor analysis. Research Quarterly for Exercise and Sport. 60, 1 (1989), 48--58.Google ScholarGoogle Scholar
  73. Kennedy, R.S., Lane, N.E., Berbaum, K.S. and Lilienthal, M.G. 1993. Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness. The International Journal of Aviation Psychology. (1993). DOI:https://doi.org/10.1207/s15327108ijap0303_3.Google ScholarGoogle Scholar
  74. Witmer, B.G. and Singer, M.J. 1998. Measuring presence in virtual environments: A presence questionnaire. Presence: Teleoperators and Virtual Environments. (1998). DOI:https://doi.org/10.1162/105474698565686.Google ScholarGoogle Scholar
  75. Buhrmester, M., Kwang, T. and Gosling, S.D. 2011. Amazon's Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspectives on psychological science. 6, 1 (2011), 3--5.Google ScholarGoogle Scholar
  76. Chandler, J. and Shapiro, D. 2016. Conducting clinical research using crowdsourced convenience samples. Annual Review of Clinical Psychology. 12, (2016).Google ScholarGoogle Scholar
  77. Mason, W. and Suri, S. 2012. Conducting behavioral research on Amazon's Mechanical Turk. Behavior Research Methods. 44, 1 (2012), 1--23. DOI:https://doi.org/10.3758/s13428-011-0124--6.Google ScholarGoogle Scholar
  78. Kelly, J.W., Cherep, L.A., Lim, A.F., Doty, T. and Gilbert, S.B. 2021. Who Are Virtual Reality Headset Owners? A Survey and Comparison of Headset Owners and Non-Owners. (2021).Google ScholarGoogle Scholar
  79. Wai, J., Lubinski, D. and Benbow, C.P. 2009. Spatial Ability for STEM Domains: Aligning Over 50 Years of Cumulative Psychological Knowledge Solidifies Its Importance. Journal of Educational Psychology. (2009). DOI:https://doi.org/10.1037/a0016127.Google ScholarGoogle Scholar
  80. Ma, X., Cackett, M., Park, L., Chien, E. and Naaman, M. 2018. Web-Based VR Experiments Powered by the Crowd. (2018). DOI:https://doi.org/10.1145/3178876.3186034.Google ScholarGoogle Scholar
  81. Rice, M., Koh, R., Lui, Q., He, Q., Wan, M., Yeo, V., Ng, J. and Tan, W.P. 2013. Comparing avatar game representation preferences across three age groups. CHI '13 Extended Abstracts on Human Factors in Computing Systems on - CHI EA '13. (2013), 1161. DOI:https://doi.org/10.1145/2468356.2468564.Google ScholarGoogle ScholarDigital LibraryDigital Library
  82. Suznjevic, M., Mandurov, M. and Matijasevic, M. 2017. Performance and QoE assessment of HTC Vive and Oculus Rift for pick-and-place tasks in VR. 2017 9th International Conference on Quality of Multimedia Experience, QoMEX 2017 (2017).Google ScholarGoogle Scholar
  83. Lim, S. and Reeves, B. 2009. Being in the Game: Effects of Avatar Choice and Point of View on Psychophysiological Responses During Play. Media Psychology. 12, 4 (2009), 348--370. DOI:https://doi.org/10.1080/15213260903287242.Google ScholarGoogle ScholarCross RefCross Ref
  84. Tabachnick, B.G. and Fidell, L.S. 2007. Using multivariate statistics. Allyn & Bacon/Pearson Education.Google ScholarGoogle Scholar
  85. Tukey, J.W. 1949. Comparing individual means in the analysis of variance. Biometrics. 5, 2 (1949), 99--114.Google ScholarGoogle Scholar
  86. Cohen, J. 2013. Statistical power analysis for the behavioral sciences. Routledge.Google ScholarGoogle Scholar
  87. Miles, J. and Shevlin, M. 2001. Applying regression and correlation: A guide for students and researchers. Sage.Google ScholarGoogle Scholar
  88. Paivio, A. 1991. Dual coding theory: Retrospect and current status. Canadian Journal of Psychology/Revue canadienne de psychologie. 45, 3 (1991), 255.Google ScholarGoogle Scholar
  89. Mayer, R.E. and Moreno, R. 2003. Nine ways to reduce cognitive load in multimedia learning. Educational Psychologist. 38, 1 (Mar. 2003), 43--52. DOI:https://doi.org/10.1207/S15326985EP3801_6.Google ScholarGoogle Scholar
  90. Wouters, P., Paas, F. and van Merriënboer, J.J.G. 2008. How to optimize learning from animated models: A review of guidelines based on cognitive load. Review of Educational Research.Google ScholarGoogle Scholar
  91. Rogers, K., Ribeiro, G., Wehbe, R.R., Weber, M. and Nacke, L.E. 2018. Vanishing Importance. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI '18. (2018), 1--13. DOI:https://doi.org/10.1145/3173574.3173902.Google ScholarGoogle Scholar
  92. Van Delden, R., Moreno, A., Poppe, R., Reidsma, D. and Heylen, D. 2017. A thing of beauty: Steering behavior in an interactive playground. Conference on Human Factors in Computing Systems - Proceedings. 2017-May, (2017), 2462--2472. DOI:https://doi.org/10.1145/3025453.3025816.Google ScholarGoogle ScholarDigital LibraryDigital Library
  93. Alharthi, S.A., Raptis, G.E., Katsini, C., Dolgov, I., Nacke, L.E. and Toups, Z.O. 2018. Toward understanding the effects of cognitive styles on collaboration in multiplayer games. Proceedings of the ACM Conference on Computer Supported Cooperative Work, CSCW. (2018), 169--172. DOI:https://doi.org/10.1145/3272973.3274047.Google ScholarGoogle Scholar
  94. Drey, T., Jansen, P., Fischbach, F., Frommel, J. and Rukzio, E. 2020. Towards progress assessment for adaptive hints in educational virtual reality games. Conference on Human Factors in Computing Systems - Proceedings. (2020). DOI:https://doi.org/10.1145/3334480.3382789.Google ScholarGoogle Scholar
  95. Glassman, E.L., Lin, A., Cai, C.J. and Miller, R.C. 2016. Learnersourcing Personalized Hints. CSCW. (2016).Google ScholarGoogle Scholar
  96. Odili Uchidiuno, J., Hammer, J., Koedinger, K. and Ogan, A. 2021. Fostering Equitable Help-Seeking for K-3 Students in Low Income and Rural Contexts. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (2021), 1--14.Google ScholarGoogle ScholarDigital LibraryDigital Library
  97. Harpstead, E., Myers, B.A. and Aleven, V. 2013. In search of learning: Facilitating data analysis in educational games. Conference on Human Factors in Computing Systems - Proceedings (2013).Google ScholarGoogle ScholarDigital LibraryDigital Library
  98. Johnson, D., Nacke, L.E. and Wyeth, P. 2015. All about that Base: Differing Player Experiences in Video Game Genres and the Unique Case of MOBA Games. (2015).Google ScholarGoogle Scholar
  99. Schrier, K. 2014. Learning, education and games. Volume one: Curricular and design considerations. Carnegie Mellon University.Google ScholarGoogle Scholar
  100. Klopfer, E., Scheintaub, H., Huang, W., Wendel, D. and Roque, R. 2009. The simulation cycle: Combining games, simulations, engineering and science using StarLogo TNG. E-Learning and Digital Media. 6, 1 (2009), 71--96.Google ScholarGoogle ScholarCross RefCross Ref
  101. Tyack, A., Wyeth, P. and Klarkowski, M. 2018. Video game selection procedures for experimental research. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (2018).Google ScholarGoogle ScholarDigital LibraryDigital Library
  102. Cho, J.Y., Andersen, E. and Kizilcec, R.F. 2021. Delivery Ghost: Effects of Language Immersion and Interactivity in a Language Learning Game. Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (2021), 1--7.Google ScholarGoogle ScholarDigital LibraryDigital Library
  103. de Oliveira, A., Khamis, M. and Esteves, A. 2020. Using a VR field study to assess the effects of visual and haptic cues in "in-the-wild" locomotion. CEUR Workshop Proceedings (2020).Google ScholarGoogle Scholar
  104. Gerling, K.M., Klauser, M. and Niesenhaus, J. 2011. Measuring the impact of game controllers on player experience in FPS games. Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments, MindTrek 2011. (2011), 83--86. DOI:https://doi.org/10.1145/2181037.2181052.Google ScholarGoogle Scholar
  105. Birk, M. and Mandryk, R.L. 2013. Control your game-self. January 2014 (2013), 685. DOI:https://doi.org/10.1145/2470654.2470752.Google ScholarGoogle Scholar
  106. Márquez Segura, E., Waern, A., Moen, J. and Johansson, C. 2013. The design space of body games: technological, physical, and social design. Proceedings of the SIGCHI conference on Human Factors in computing systems (2013), 3365--3374.Google ScholarGoogle ScholarDigital LibraryDigital Library
  107. Xu, Y., Barba, E., Radu, I., Gandy, M., Shemaka, R., Schrank, B., MacIntyre, B. and Tseng, T. 2011. Pre-patterns for designing embodied interactions in handheld augmented reality games. 2011 IEEE International Symposium on Mixed and Augmented Reality-Arts, Media, and Humanities (2011), 19--28.Google ScholarGoogle Scholar
  108. Pillat, R., Nagendran, A. and Lindgren, R. 2012. Design requirements for using embodied learning and whole-body metaphors in a mixed reality simulation game. 2012 IEEE International Symposium on Mixed and Augmented Reality-Arts, Media, and Humanities (ISMAR-AMH) (2012), 105--106.Google ScholarGoogle ScholarCross RefCross Ref
  109. Kao, D. 2019. The Effects of Anthropomorphic Avatars vs. Non-Anthropomorphic Avatars in a Jumping Game. The Fourteenth International Conference on the Foundations of Digital Games (2019).Google ScholarGoogle ScholarDigital LibraryDigital Library
  110. Kao, D. and Harrell, D.F. 2015. Mazzy: A STEM Learning Game. Foundations of Digital Games. (2015).Google ScholarGoogle Scholar
  111. Bonsignore, E.M., Hansen, D.L., Toups, Z.O., Nacke, L.E., Salter, A. and Lutters, W. 2012. Mixed reality games. Proceedings of the ACM 2012 conference on computer supported cooperative work companion (2012), 7--8.Google ScholarGoogle ScholarDigital LibraryDigital Library
  112. Sobel, K., Bhattacharya, A., Hiniker, A., Lee, J.H., Kientz, J.A. and Yip, J.C. 2017. It wasn't really about the PokéMon: parents' perspectives on a location-based mobile game. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (2017), 1483--1496.Google ScholarGoogle ScholarDigital LibraryDigital Library
  113. Aleven, V., Dow, S., Christel, M., Stevens, S., Rosé, C., Koedinger, K., Myers, B., Flynn, J.B., Hintzman, Z., Harpstead, E. and others 2013. Supporting social-emotional development in collaborative inquiry games for K-3 science learning. Proceedings of the 9th Games+ Learning+ Society Conference-GLS (2013).Google ScholarGoogle Scholar
  114. Davis, K., Boss, J.A. and Meas, P. 2018. Playing in the virtual sandbox: Students' collaborative practices in Minecraft. International Journal of Game-Based Learning (IJGBL). 8, 3 (2018), 56--76.Google ScholarGoogle ScholarDigital LibraryDigital Library
  115. Buffum, P.S., Frankosky, M., Boyer, K.E., Wiebe, E.N., Mott, B.W. and Lester, J.C. 2016. Collaboration and gender equity in game-based learning for middle school computer science. Computing in Science & Engineering. 18, 2 (2016), 18--28.Google ScholarGoogle ScholarDigital LibraryDigital Library
  116. Yuan, Y., Cao, J., Wang, R. and Yarosh, S. 2021. Tabletop Games in the Age of Remote Collaboration: Design Opportunities for a Socially Connected Game Experience. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (2021), 1--14.Google ScholarGoogle ScholarDigital LibraryDigital Library
  117. DiSalvo, B.J., Crowley, K. and Norwood, R. 2008. Learning in Context: Digital Games and Young Black Men. Games and Culture. 3, 2 (Feb. 2008), 131--141. DOI:https://doi.org/10.1177/1555412008314130.Google ScholarGoogle ScholarCross RefCross Ref
  118. Richard, G.T. 2013. Gender and gameplay: Research and future directions. Playing with virtuality: Theories and methods of computer game studies. (2013), 269--284.Google ScholarGoogle Scholar
  119. Kumar, P., Vitak, J., Chetty, M., Clegg, T.L., Yang, J., McNally, B. and Bonsignore, E. 2018. Co-designing online privacy-related games and stories with children. Proceedings of the 17th ACM Conference on Interaction Design and Children (2018), 67--79.Google ScholarGoogle ScholarDigital LibraryDigital Library
  120. Easterday, M.W., Aleven, V., Scheines, R. and Carver, S.M. 2011. Using tutors to improve educational games. International Conference on Artificial Intelligence in Education (2011), 63--71.Google ScholarGoogle Scholar
  121. Kao, D. and Harrell, D.F. 2016. Exploring the Impact of Avatar Color on Game Experience in Educational Games. Proceedings of the 34th Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (CHI 2016). (2016).Google ScholarGoogle ScholarDigital LibraryDigital Library
  122. Kao, D. and Harrell, D.F. 2016. Exploring the Effects of Encouragement in Educational Games. Proceedings of the 34th Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (CHI 2016). (2016).Google ScholarGoogle ScholarDigital LibraryDigital Library
  123. Kao, D. 2020. The effects of juiciness in an action RPG. Entertainment Computing. (2020). DOI:https://doi.org/10.1016/j.entcom.2020.100359.Google ScholarGoogle Scholar
  124. Denny, P., Mcdonald, F., Empson, R., Kelly, P. and Petersen, A. 2018. Empirical Support for a Causal Relationship Between Gamification and Learning Outcomes. (2018), 1--13. DOI:https://doi.org/10.1145/3173574.3173885.Google ScholarGoogle Scholar
  125. Hamilton, M., DiSalvo, B. and Fullerton, T. 2021. Mindful Gaming: User Experiences with Headspace and Walden, a Game. International Conference on Human-Computer Interaction (2021), 3--19.Google ScholarGoogle Scholar
  126. Kao, D. 2019. Exploring How Preference and Perceived Performance Vary in Different Game Genres Across Time of Day. Proceedings of The Fourteenth International Conference on the Foundations of Digital Games (2019).Google ScholarGoogle ScholarDigital LibraryDigital Library
  127. Almeda, M.V., Kleinman, E., Jemmali, C., Ithier, C., Rowe, E. and El-Nasr, M.S. 2020. Labeling debugging in may's journey gameplay. Proceedings of the 51st ACM Technical Symposium on Computer Science Education (2020).Google ScholarGoogle ScholarDigital LibraryDigital Library
  128. Chaffin, A., Doran, K., Hicks, D. and Barnes, T. 2009. Experimental evaluation of teaching recursion in a video game. Proceedings of the 2009 ACM SIGGRAPH Symposium on Video Games - Sandbox '09. 1, 212 (2009), 79. DOI:https://doi.org/10.1145/1581073.1581086.Google ScholarGoogle ScholarDigital LibraryDigital Library
  129. Cooper, S., Khatib, F., Treuille, A., Barbero, J., Lee, J., Beenen, M., Leaver-Fay, A., Baker, D., Popovi?, Z. and Players, F. 2010. Predicting protein structures with a multiplayer online game. Nature. (2010). DOI:https://doi.org/10.1038/nature09304.Google ScholarGoogle Scholar
  130. Weintrop, D., Holbert, N., Horn, M.S. and Wilensky, U. 2016. Computational thinking in constructionist video games. International Journal of Game-Based Learning (IJGBL). 6, 1 (2016), 1--17.Google ScholarGoogle ScholarDigital LibraryDigital Library
  131. Lee, M.J. and Ko, A.J. 2014. A demonstration of gidget, a debugging game for computing education. 2014 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) (2014), 211--212.Google ScholarGoogle ScholarCross RefCross Ref
  132. Esteves, A., Shin, Y. and Oakley, I. 2020. Comparing selection mechanisms for gaze input techniques in head-mounted displays. International Journal of Human Computer Studies. (2020). DOI:https://doi.org/10.1016/j.ijhcs.2020.102414.Google ScholarGoogle Scholar
  133. Nielsen 2017. Games 360 U.S. Report.Google ScholarGoogle Scholar
  134. Steam 2021. Steam Top Sellers (VR Supported) (https://store.steampowered.com/search/?filter=topsellers&vrsupport=402). (2021).Google ScholarGoogle Scholar
  135. Kao, D., Mousas, C., Magana, A.J., Harrell, D.F., Ratan, R., Melcer, E.F., Sherrick, B., Parsons, P. and Gusev, D. 2020. Hack.VR: A Programming Game in Virtual Reality. ACM Foundations of Digital Games (2020).Google ScholarGoogle Scholar

Index Terms

  1. Evaluating Tutorial-Based Instructions for Controllers in Virtual Reality Games

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!