Evaluating the Onboarding Phase of Free-toPlay Mobile Games: A Mixed-Method Approach

ABSTRACT

The first few minutes of play, commonly referred to as the onboarding phase, of Free-to-Play mobile games typically display a substantial churn rate among new players. It is therefore vital for designers to effectively evaluate this phase to investigate its satisfaction of player expectations. This paper presents a study utilizing a lab-based mixed-methods approach in providing insights for evaluating the user experience of onboarding phases in mobile games. This includes an investigation into the contribution of physiological measures (Heart-Rate Variability and Galvanic Skin Conductance) as well as a range of self-reported proxy measures including: a) stimulated recall, engagement graphs, b) flow state survey and c) post-game experience questionnaire. These techniques were applied across 28 participants using three mobile Free-to-Play titles from different genres. This paper makes two important contributions to the games user research (GUR) domain: 1) evaluates different research techniques (e.g. physiological measures and experience graphs) in the context of mobile games; 2) provides an empirically based recommendation for design elements that result in high arousal.

References

1. Seufert, Eric. Freemium Economics - Leveraging Analytics and User Segmentation to Drive Revenue. Morgan Kaufmann, Burlington, MA, 2014.
2. Alha, Kati, Koskinen, Elina, Paavilainen, Janne, Hamari, Juho, and Kinnunen, Jani. Free-to-play games: Professionals' perspectives. In Proc DiGRA Nordic 2014 (2014).
3. ESA. Indystry Facts. http://essentialfacts.theesa.com/, 2016.
4. Hadiji, F, Sifa, S, Drachen, A, and Thurau, C. Predicting Player Churn in the Wild. Player Churn in the Wild. In Proceedings of the IEEE Computational Intelligence in Games (2014).
5. Chou, Yu-Kai. Actionable Gamification - Beyond Points, Badges, and Leaderboards. Lean Publishing, 2014.
6. Even, Alon. 2015/10/13/gaming-app-user-retentiononly-22-return-after-one-month/ (Oct. 13, 2015).
7. Sifa, R., Hadiji, F., Drachen, A., and Runge, J. Predicting Purchase Decisions in Mobile free to play Games. In AAAI Artificial Intelligence in Interactive Digital Entertainment (Santa Cruz, CA 2015).
8. White, Gareth R and McAllister, Graham. Video Game Development and User Experience. In Game User Experience. Springer International Publishing, Switzerland, 2015.
9. Pagulayan, R., Keeker, K., Wixon, D., Romero, R. L., and Fuller, T. User-centered design in games. In The Human-Computer Interaction Handbook. L. Erlbaum Associates Inc. , 2003.
10. Smeddinck, Jan David, Markus, Krause, and Kolja, Lubitz. Mobile Game User Research: The World as Your Lab? Proceedings at CHI'13, (April 27, 2013).
11. Korhonen, Hannu and Koivisto, Elina M, I. Playability heuristics for mobile games. In Proceedings of the 8th conference on Human-computer interaction with mobile devices and services (Helsinki 2006), ACM Digital Library, 9--16.
12. Duh, Henry Been-Lirn, Chen, Vivian Hsueh Hua, and Tan, Chee Boon. Playing Different Games on Different Phones: an empirical study on mobile gaming. In Proceedings of the 10th international conference on Human computer interaction with mobile devices and services (Amsterdam 2008), ACM Digital Library, 391394.
13. Stephan Engl, Lennart E. Nacke. Contextual influences on mobile player experience -- A game user experience model. Entertainment Computing, 4, 1 (2013), 83--91.
14. Paul, Sheila A, Jensen, Marianne, Wong, Chui Y, and Khong, Chee W. Socializing in mobile gaming. In Proceedings of the 3rd International Conference on Digital Interactive Media in Entertainment and Arts (New York 2008), ACM, 2--9.
15. Mandryk, Regan L, Atkins, M Stella, and Inkpen, Kori M. A Continuous and Objective Evaluation of Emotional Experience with Interactive Play Environments. CHI 2006 Proceedings - Novel Methods: Emotions, Gestures, Events (April 2006), 1027--1036.
16. Mirza-Babaei, Pejman, Long, Sebastian, Foley, Emma, and McAllister, Graham. Understanding the Contribution of Biometrics to Games User Research. (2011), DiGRA Conference.
17. Chalfoun, Pierre. Biometrics at Ubisoft Montreal (March 15, 2016).
18. Ambinder, M. Biofeedback in Gameplay: How Valve Measues Physiology to Enhance Gaming Experiences. In Game Developers Conference (San Francisco, CA 2011).
19. Mirza-Babaei, Pejman. Biometric Storyboards: A Games User Research Approach for Improving Qualitative Evaluations of Player Experience. Sussex, 2013.
20. Yao, Lin, Liu, Yanfang, Li, Wen, Zhou, Lei, Ge, Yan, Chai, Jing, and Sun, Xianghong. Using Physiological Measures to Evaluate Use Experience of Mobile Applications. In Engineering Psychology and Cognitive Ergonomics. Springer International Publishing, Switzerland, 2014.
21. Drachen, Anders, El-Nasr, Magy Seif, and Canossa, Alessandro. Game Analytics -- The Basics. In Game Analytics - Maximizing the Value of Player Data. Springer, London, 2013.
22. Alha, Kati, Koskinen, Elina, Paavilainen, Janne, and Hamari, Juho. Critical Acclaim and Commercial Success in Mobile Free-to. play Games. DiGRA/FDG'16 - Proceedings of the First International Joint Conference of DiGRA and FDG, 13, 1 (August 2016).
23. Korhonen, Hannu. Comparison of playtesting and expert review methods in mobile game evaluation. 10 Proceedings of the 3rd International Conference on Fun and Games (2010), 18--27.
24. Alha, K, Paavilainen, J, and Hamari, J. Domain-specific playability problems in social network games. In proceedings of DiGRA FDG Conference, Dundee, Scotland (2016).
25. White, Gareth R, Mirza-Babaei, Pejman, McAllister, Graham, and Good, Judith. Weak inter-rater reliability in heuristic evaluation of video games. In CHI'11 Extended Abstracts on Human Factors in Computing Systems ( 2011), ACM.
26. Jackson, Susan A. and Marsh, Herbert W. Development and Validation of a Scale of Measure Optimal Experience: The Flow State Scale. Journal of sport & exercise psychology (1996), 17--35.
27. Brockmeyer, Jeanne H., Fox, Christine M., Curtiss, Kathleen A., McBroom, Evan, Burkhart, Kimberly M., and Pidruzny, Jacquelyn N. The development of the Game Engagement Questionnaire. Journal of Experimental Social Psychology (2009), 624--635.
28. Nacke, Lennart E. Game user research and physiological evaluation. In Game user experience evaluation. Springer, Toulouse, 2015.
29. Bromley, Steve, Mirza-Babaei, Pejman, McAllister, Graha, and Napier, Jonathan. Playing to Win? In Quandt, Thorsten, ed., Multiplayer: The Social Aspects of Digital Gaming. Routledge, Oxon, 2013.
30. Silveira, Fernando, Eriksson, Brian, Sheth, Anmol, and Sheppard, Adam. Predicting Audience Responses to Movie Content from Electro-Dermal Activity Signals. UbiComp'13, ACM international joint conference on Pervasive and ubiquitous computing September and Behavior (September 8--12, 2013), 707--716.
31. Boucsein, Wolfram. Electrodermal Activity. Springer US, New York, 2012.
32. Lu, G., Yang, F., Taylor, J. A., and Stein, J. F. A comparison of photoplethysmography and ECG recording to analyse heart rate variability in healthy subjects. Journal of Medical Engineering & Technology (2009), 634--641.
33. Seif El-Nasr, M., Drachen, A., and Canossa, A. Game Analytics: Maximizing the Value of Player Data. Springer, London, 2013.
34. Mcalmont, T. Tips for a Great First Time User Experience (FTUE) in F2P Games (September 3, 2015), http://blog.gameanalytics.com/blog/tips-for-a-great-firsttime-user-experience-ftue-in-f2p-games.html.
35. Higgins, Krystal. First time user experiences in mobile apps (April 15, 2012), www.kryshiggins.com/first-timeuser-experiences-in-mobile-apps/.
36. Novik, David G and Ward, Karen. Why don't people read the manual? Departmental Papers (CS) (2006), Paper 15.
37. King Digital Entertainment. 2016. Candy Crush Jelly Saga. Game [mobile] (Jan 6, 2016) King Digital Entertainment, Dublin, Irland. Played January 2016.
38. Norsfell Games Inc. 2014. WinterForts: Exiled Kingdom. Game [mobile] (Oct 1, 2014). Execution Labs, Montréal, Canada.Played January 2016.
39. Norsfell Games Inc. 2015. Pogo Chick. Game [mobile] (Aug 13, 2015). Norsfell Games Inc, Montréal, Canada. Played January 2016.
40. Bordens, Kenneth S. and Abbott, Bruce B. Research Design and Methods A Process Approach. McGraw-Hill, New York, NY 10020, 2011.
41. Accessed on 2017-07--19, Wikidot. (2017). Retrieved from http://mgur.wikidot.com/start.
42. Khawaji, Ahmad , Zhou, Jianlong, Chen, Fang, and Marcus, Nadine. Using Galvanic Skin Response (GSR) to Measure Trust and Cognitive Load in the Text-Chat Environment. In SIGCHI Conference on Human Factors in Computing Systems (Crossings, Seoul, Korea 2015), ACM Digital Library, 420--423.
43. Lyle, John. Stimulated recall: a report on its use in naturalistic research. British Educational Research Journal , Volume 29, Issue 6 (2003), 861--878.
44. Poels, Karolien, de Kort, Yvonne, and Isselsteijn, Wijnand. Identification and Measurement of Post Game Experiences. Westminster paper (April 2009), 109 -129.
45. Nacke, Lennart and Lindley, Craig A. Flow and Immersion in First-Person Shooters: Measuring the player's gameplay experience. In Proceedings of the 2008 Conference on Future Play: Research (Toronto, Ontario, Canada 2008), ACM, 81--88.
46. Mirza-Babaei, Pejman, Wallner, Günter, McAllister, Graham, and Nacke, Lennart E. Unified visualization of quantitative and qualitative playtesting data. In CHI'14 Extended Abstracts on Human Factors in Computing Systems (New York 2014), ACM, 1363--1368.
47. Thomsen, Line Ebdrup, Petersen, Falko Weigert, Drachen, Anders, and Mirza-Babaei, Pejman. Identifying Onboarding Heuristics for Free-to-Play Mobile Games: A Mixed Methods Approach. In Entertainment Computing - ICEC 2016. Springer, 2016.
48. Kivikangas, Matias J., Ekman, Inger, Chanel, Guillaume et al. Review on psychophysiological methods in game research. In Nordic DiGRA 2010 (Stockholm 2010), Authors & Digital Games Research Association (DiGRA)., 181--199.
49. Ravaja, N, Turpeinen, M, Saari, T , Puttonen, S, and Keltikangas-Jarvinen, L. The psychophysiology of James Bond: Phasic emotional responses to violent video game events. Emotions vol. 8 (Feb 2008), 114120.
50. Cacioppo, John T., Tassinary, Louis G., and Berntson, Gary G. The Handbook of Psychophysiology. Cambridge University Press, New York, 2007.
51. Julkunen, J. 3 Things to KNow About Session-Length Restriction When Designing a Free2play Game (Februray 19, 2015), http://www.bluecloudsolutions.com/articles/mobilegaming-sessions-now-longer/.
52. Saari, Timo, Ravaja, Niklas, Salminen, Mikko, Laarni, Jari, and Kallinen, Kari. Phasic Emotional Reactions to Video Game Events: A Psychophysiological Investigation. Media psychology (2006), 343--367.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)