Towards gameplay analysis via gameplay metrics

ABSTRACT

User-oriented research in the game industry is undergoing a change from relying on informal user-testing methods adapted directly from productivity software development to integrating modern approaches to usability- and user experience testing. Gameplay metrics analysis form one of these techniques, being based on instrumentation methods in HCI. Gameplay metrics are instrumentation data about the user behavior and user-game interaction, and can be collected during testing, production and the live period of the lifetime of a digital game. The use of instrumentation data is relatively new to commercial game development, and remains a relatively unexplored method of user research. In this paper, the focus is on utilizing game metrics for informing the analysis of gameplay during commercial game production as well as in research contexts. A series of case studies are presented, focusing on the major commercial game titles Kane & Lynch and Fragile Alliance.

References

1. Bateman, C., Boon, R.: 21st Century Game Design. Charles River Media (2005)
2. Blythe, M. A., Overbeeke, K., Monk, A. F. Wright, P. C.: Funology -- from usability to enjoyment. Springer (2004)
3. Börner, K., Penumarthy, S.: Social Diffusion Patterns in Three-Dimensional Virtual Worlds. Information Visualization, 2 (2003), 182--198
4. Chittaro, L.&Ieronutti, L.: A visual tool for tracing users' behavior in virtual environments. In Proc. Working Conference on Advanced Visual Interfaces (2004), 40--47
5. Chittaro L., Ranon R., Ieronutti L.: VU-Flow: A Visualization Tool for Analyzing Navigation in Virtual Environments. IEEE Transactions on Visualization and Computer Graphics, 12, 6 (2006), 1475--1485
6. Davis, J.; Steury, K., Pagulayan, R.: A survey method for assessing perceptions of a game: The consumer playtest in game design. Game Studies: The International Journal of Computer Game Research, 5 (2005)
7. De Kort, Y., Ijsselssteijn, W: People, places and play: Player experience in a socio-spatial context. ACM Computers in Entertainment 6, 2, article no. 18 (2007)
8. DeRosa, P.: Tracking Player Feedback to Improve Game Design. Gamasutra, Aug. 7 (2007)
9. Desurvire, H., Caplan, M., Toth, J. A.: Using heuristics to evaluate the playability of games. In CHI '04 extended abstracts on Human factors in computing systems (2004)
10. Dumas, J. S. 2003. User-Based Evaluations. In: The HCI handbook. Lawrence Erlbaum Associates (2003), 1093--1117
11. Gilleade, K. M., Dix, A.: Using frustration in the design of adaptive videogames. In: Proceedings of the ACM SIGCHI International Conference on Advances in Computer Entertainment Technology, Singapore (2004), 228--232
12. Goetz, P.: Too Many Clicks! Unit-Based Interfaces Considered Harmful. Gamasutra, August 23 (2006)
13. Hilbert, D. M. and Redish, J. C.: A practical guide to usability testing. Intellect books (1999)
14. Hoobler, N., Humphreys, G., Agrawala, M.: Visualizing Competitive Behaviors in Multi-User Virtual Environments. In Proc. of the IEEE Visualization Conference (2004), 163--170
15. Hurst, A., Hudson, S. E., Mankoff, J.: Dynamic detection of novice vs. skilled use without a task model. In Proceedings of CHI 2007, ACM press (2007), 271--280
16. Ijsselsteijn, W., de Kort, Y., Poels, K., Jugelionis, A., Bellotti, F.: Characterizing and Measuring User Experiences in Digital Games. In: Proceedings of ACE, Toronto, Ontario (2007)
17. Isbister, K., Schaffer, N.: Game Usability: Advancing the Player Experience. Morgan Kaufman (2008)
18. Jørgensen, A. H.: Marrying HCI/Usability and Computer Games: A Preliminary Look, Proceedings of NordiCHI, Tampere, Finland (2004)
19. Kim, J. H., Gunn, D. V., Schuh, E., Phillips, B. C., Pagulayan, R. J., Wixon, D.: Tracking Real-Time User Experience (TRUE): A comprehensive instrumentation solution for complex systems. In Proceedings of CHI 2008, Florence, Italy (2008), 443--451
20. Kort, J., Steen, M. G. D., de Poot, H., ter Hofte, H., Mulder, I.: Studying usage of complex applications. In: Proceedings of Measuring Behavior. Noldus IT (2005), 266--269
21. Kuniavsky, M.: Observing the User Experience: A Practitioner's Guide to User Research. Morgan Kaufman (2003).
22. Longley, P.; Goodchild, M. F., Macquire, D.&Rhind, D.: Geographic Information Systems and Science. Wiley&sons (2005).
23. Mandryk, R., Atkins, M. S., Inkpen, K.: A continuous and objective evaluation of emotional experience with interactive play environments. In: Proceedings of CHI (2006.
24. Medlock, M. C., Wixon, D., Terrano, M., Romero, R. L., Fulton, B.: Using the RITE method to improve products: A definition and a case study. In: Proceedings of UPA (2002)
25. Mellon, L.: Metrics in MMP Development and Operations. Presentation at the GDC, Los Angeles, USA (2004)
26. Nørgaard, M., Rau, J.: User Testing in the Combat Zone. In: Proceedings of ACE, Toronto, Ontario (2007)
27. Pagulayan, R. J., Keeker, K., Wixon, D., Romero, R. L., Fuller, T.: User-centered design in games. In: The HCI handbook. Lawrence Erlbaum Associates (2003), 883--906
28. Pagulayan, R. J., Steury, K. R., Fulton, B., Romero, R. L.: Designing for fun: user-testing case studies. In: Blythe, M. A., Overbeeke, K., Monk, A. F. Wright, P. C. (Eds.): Funology: From usability to enjoyment. Human-Computer Interaction series. Kluwer Academic Publishers (2005).
29. Renaud, K. and Gray, P.: Making sense of low-level usage data to understand user activities. In: Proceedings of SAICSIT 2004, ACM Press (2004), 115--124.
30. Rouse, R. Game Design: Theory and Practice. Wordware publishing Inc. (2001).
31. Schultz, C. P.: Game Testing All in One. PTR (2005)
32. Swain, C. 2008. Master Metrics: The Science Behind the Art of Game Design. Presentation at NLGD Conference, Utrecht, Holland (2008)
33. Thompson, C.: Halo 3: How Microsoft Labs Invented a New Science of Play. Wired Magazine 15, 9 (2007)
34. Trappl, R., Petta, P.: Creating Personalities for Synthetic Actors: Towards Autonomous Personality Agents. Springer Publishers, Berlin (1997)
35. Tychsen, A., Canossa, A.: Defining Personas in Games Using Metrics. In: proceedings of Future Play 2008, Toronto, Canada. ACM publishers (2008), 73--80
36. Williams, D.; Yee, N., Caplan, S. E.: Who plays, how much, and why? Debunking the stereotypical gamer profile. Journal of Computer-Mediated Communication 13 (2005), 993--1018

Index Terms

• Applied computing

  • Computers in other domains

    • Personal computers and PC applications

      • Computer games

• Human-centered computing

  • Human computer interaction (HCI)

• Information systems

  • Information systems applications

    • Multimedia information systems

      • Massively multiplayer online games

• Software and its engineering

  • Software creation and management

    • Software verification and validation

      • Software defect analysis

        • Software testing and debugging

  • Software organization and properties

    • Contextual software domains

      • Virtual worlds software

        • Interactive games