skip to main content
research-article

Facial and Bodily Expressions of Emotional Engagement: How Dynamic Measures Reflect the Use of Game Elements and Subjective Experience of Emotions and Effort

Published:06 October 2021Publication History
Skip Abstract Section

Abstract

Users' emotional engagement in a task is important for performance and motivation. Non-intrusive, computerized process measures of engagement have the potential to provide fine-grained access to underlying affective states and processes. Thus, the current work brings together subjective measures (questionnaires) and objective process measures (facial expressions and head movements) of emotions to examine users' emotional engagement with respect to the absence or presence of game-elements. In particular, we randomly assigned 156 adult participants to either a spatial working memory task with or without game elements present, while their faces and head movements were recorded with a webcam during task execution. Positive and negative emotions were assessed before the task and twice during task execution using conventional questionnaires. We additionally examined whether perceived subjective effort, assumed to inherit a substantial affective component, manifests at a bodily expressive level alongside positive and negative emotions. Importantly, we explored the relationship between subjective and objective measures of emotions across the two tasks versions. We found a series of action units and head movements associated with the subjective experience of emotions as well as to subjective effort. Impacted by game elements, these associations often fit intuitively or lined up with findings from literature. As did a linear increase of blink (action unit 45) intensity relate to participants performing the task without game elements, presumably indicating disengagement in the more tedious task variant. On other occasions, associations between subjective and objective measures seemed indiscriminative or even contraindicated. Additionally, facial and bodily reactions and the resulting subjective-objective correspondences were rather consistent within, but not between the two task versions. Our work therefore both gains detailed access to automated emotion recognition and promotes its feasibility within research of game elements while highlighting the individuality and context dependency of emotional expressions.

References

  1. James J. Appleton, Sandra L. Christenson, and Michael J. Furlong. 2008. Student engagement with school: Critical conceptual and methodological issues of the construct. Psychol. Sch. 45, 5 (May 2008), 369--386. DOI:https://doi.org/10.1002/pits.20303Google ScholarGoogle ScholarCross RefCross Ref
  2. Isabelle Archambault, Michel Janosz, Jean-Sébastien Fallu, and Linda S. Pagani. 2009. Student engagement and its relationship with early high school dropout. J. Adolesc. 32, 3 (June 2009), 651--670. DOI:https://doi.org/10.1016/j.adolescence.2008.06.007Google ScholarGoogle ScholarCross RefCross Ref
  3. Katharina Bernecker and Manuel Ninaus. 2021. No Pain, no Gain? Investigating motivational mechanisms of game elements in cognitive tasks. Comput. Hum. Behav. 114, (January 2021), 106542. DOI:https://doi.org/10.1016/j.chb.2020.106542Google ScholarGoogle Scholar
  4. Nigel Bosch, Yuxuan Chen, and Sidney D'Mello. 2014. It's Written on Your Face: Detecting Affective States from Facial Expressions while Learning Computer Programming. In Intelligent Tutoring Systems (Lecture Notes in Computer Science), Springer International Publishing, Cham, 39--44. DOI:https://doi.org/10.1007/978--3--319-07221-0_5Google ScholarGoogle Scholar
  5. H. Boukricha, I. Wachsmuth, A. Hofstätter, and K. Grammer. 2009. Pleasure-arousal-dominance driven facial expression simulation. In 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops, 1--7. DOI:https://doi.org/10.1109/ACII.2009.5349579Google ScholarGoogle Scholar
  6. Elizabeth A. Boyle, Thomas Hainey, Thomas M. Connolly, Grant Gray, Jeffrey Earp, Michela Ott, Theodore Lim, Manuel Ninaus, Claudia Ribeiro, and João Pereira. 2016. An update to the systematic literature review of empirical evidence of the impacts and outcomes of computer games and serious games. Comput. Educ. 94, (2016), 178--192.Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. John T. Cacioppo and Wendi L. Gardner. 1999. EMOTION. Annu. Rev. Psychol. 50, 1 (February 1999), 191--214. DOI:https://doi.org/10.1146/annurev.psych.50.1.191Google ScholarGoogle ScholarCross RefCross Ref
  8. Tara M. Chaplin and Amelia Aldao. 2013. Gender differences in emotion expression in children: A meta-analytic review. Psychol. Bull. 139, 4 (2013), 735--765. DOI:https://doi.org/10.1037/a0030737Google ScholarGoogle ScholarCross RefCross Ref
  9. Giovanna Cilluffo, Gianluca Sottile, Stefania La Grutta, and Vito MR Muggeo. 2020. The Induced Smoothed lasso: A practical framework for hypothesis testing in high dimensional regression. Stat. Methods Med. Res. 29, 3 (March 2020), 765--777. DOI:https://doi.org/10.1177/0962280219842890Google ScholarGoogle ScholarCross RefCross Ref
  10. Mihaly Csikszentmihalyi. 1990. The psychology of optimal experience New York. Harper & Row.Google ScholarGoogle Scholar
  11. Nele Dael, Marcello Mortillaro, and Klaus R. Scherer. 2012. Emotion expression in body action and posture. Emotion 12, 5 (2012), 1085--1101. DOI:https://doi.org/10.1037/a0025737Google ScholarGoogle ScholarCross RefCross Ref
  12. Sidney D'Mello, Ed Dieterle, and Angela Duckworth. 2017. Advanced, Analytic, Automated (AAA) Measurement of Engagement During Learning. Educ. Psychol. 52, 2 (April 2017), 104--123. DOI:https://doi.org/10.1080/00461520.2017.1281747Google ScholarGoogle ScholarCross RefCross Ref
  13. Sidney D'Mello and Art Graesser. 2012. Dynamics of affective states during complex learning. Learn. Instr. 22, 2 (April 2012), 145--157. DOI:https://doi.org/10.1016/j.learninstruc.2011.10.001Google ScholarGoogle Scholar
  14. Paul Ekman. 1992. An argument for basic emotions. Cogn. Emot. 6, 3--4 (May 1992), 169--200. DOI:https://doi.org/10.1080/02699939208411068Google ScholarGoogle ScholarCross RefCross Ref
  15. Paul Ekman. 2002. Facial action coding system (FACS). Hum. Face (2002).Google ScholarGoogle Scholar
  16. Paul Ekman and E. Friesen. 1978. Facial action coding system: a technique for the measurement of facial movement. Palo Alto 3, 2 (1978), 5.Google ScholarGoogle Scholar
  17. Anna Marie Farrace-Di Zinno, Graham Douglas, Stephen Houghton, Vivienne Lawrence, John West, and Ken Whiting. 2001. Body movements of boys with attention deficit hyperactivity disorder (ADHD) during computer video game play. Br. J. Educ. Technol. 32, 5 (2001), 607--618.Google ScholarGoogle ScholarCross RefCross Ref
  18. Alexander L. Francis and Jordan Love. 2020. Listening effort: Are we measuring cognition or affect, or both? WIREs Cogn. Sci. 11, 1 (January 2020). DOI:https://doi.org/10.1002/wcs.1514Google ScholarGoogle ScholarCross RefCross Ref
  19. Jennifer Fredricks, Phyllis C Blumenfeld, and Alison H Paris. 2004. School Engagement: Potential of the Concept, State of the Evidence. Rev. Educ. Res. 74, 1 (March 2004), 59--109. DOI:https://doi.org/10.3102/00346543074001059Google ScholarGoogle ScholarCross RefCross Ref
  20. Jennifer Fredricks, Michael Filsecker, and Michael A. Lawson. 2016. Student engagement, context, and adjustment: Addressing definitional, measurement, and methodological issues. Learn. Instr. 43, (June 2016), 1--4. DOI:https://doi.org/10.1016/j.learninstruc.2016.02.002Google ScholarGoogle Scholar
  21. Jerome Friedman, Trevor Hastie, and Robert Tibshirani. 2010. Regularization Paths for Generalized Linear Models via Coordinate Descent. J. Stat. Softw. 33, 1 (2010), 1--22.Google ScholarGoogle ScholarCross RefCross Ref
  22. Nico H. Frijda. 1988. The laws of emotion. Am. Psychol. 43, 5 (1988), 349--358. DOI:https://doi.org/10.1037/0003-066X.43.5.349Google ScholarGoogle ScholarCross RefCross Ref
  23. Beatrice de Gelder. 2006. Towards the neurobiology of emotional body language. Nat. Rev. Neurosci. 7, 3 (March 2006), 242--249. DOI:https://doi.org/10.1038/nrn1872Google ScholarGoogle ScholarCross RefCross Ref
  24. Xiaowei Geng, Ziguang Chen, Wing Lam, and Quanquan Zheng. 2013. Hedonic Evaluation over Short and Long Retention Intervals: The Mechanism of the Peak-End Rule: Hedonic Evaluation over Retention Intervals. J. Behav. Decis. Mak. 26, 3 (July 2013), 225--236. DOI:https://doi.org/10.1002/bdm.1755Google ScholarGoogle ScholarCross RefCross Ref
  25. Patricia Goldberg, Ömer Sümer, Kathleen Stürmer, Wolfgang Wagner, Richard Göllner, Peter Gerjets, Enkelejda Kasneci, and Ulrich Trautwein. 2019. Attentive or Not? Toward a Machine Learning Approach to Assessing Students' Visible Engagement in Classroom Instruction. Educ. Psychol. Rev. (December 2019). DOI:https://doi.org/10.1007/s10648-019-09514-zGoogle ScholarGoogle Scholar
  26. Arthur C. Graesser and Brent A. Olde. 2003. How does one know whether a person understands a device? The quality of the questions the person asks when the device breaks down. J. Educ. Psychol. 95, 3 (2003), 524--536. DOI:https://doi.org/10.1037/0022-0663.95.3.524Google ScholarGoogle ScholarCross RefCross Ref
  27. Joseph F. Grafsgaard, Kristy Elizabeth Boyer, Eric N. Wiebe, and James C. Lester. 2012. Analyzing Posture and Affect in Task-Oriented Tutoring. In FLAIRS Conference, 438--443.Google ScholarGoogle Scholar
  28. Joseph F. Grafsgaard, Joseph B. Wiggins, Kristy Elizabeth Boyer, Eric N. Wiebe, and James C. Lester. 2013. Automatically Recognizing Facial Indicators of Frustration: A Learning-centric Analysis. In 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction, IEEE, Geneva, Switzerland, 159--165. DOI:https://doi.org/10.1109/ACII.2013.33Google ScholarGoogle Scholar
  29. Joseph Grafsgaard, Joseph B. Wiggins, Kristy Elizabeth Boyer, Eric N. Wiebe, and James Lester. 2013. Automatically recognizing facial expression: Predicting engagement and frustration. In Educational Data Mining 2013.Google ScholarGoogle Scholar
  30. Gillian Green, Jean Rhodes, Abigail Heitler Hirsch, Carola Suárez-Orozco, and Paul M. Camic. 2008. Supportive adult relationships and the academic engagement of Latin American immigrant youth. J. Sch. Psychol. 46, 4 (August 2008), 393--412. DOI:https://doi.org/10.1016/j.jsp.2007.07.001Google ScholarGoogle ScholarCross RefCross Ref
  31. Simon Greipl, Manuel Ninaus, and Korbinian Moeller. 2020. Potential and limits of game-based learning. Int. J. Technol. Enhanc. Learn. 12, 4 (2020), 363. DOI:https://doi.org/10.1504/IJTEL.2020.10028417Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Hatice Gunes, Caifeng Shan, Shizhi Chen, and YingLi Tian. 2015. Bodily Expression for Automatic Affect Recognition. In Emotion Recognition, Amit Konar and Aruna Chakraborty (eds.). John Wiley & Sons, Inc., Hoboken, NJ, USA, 343--377. DOI:https://doi.org/10.1002/9781118910566.ch14Google ScholarGoogle Scholar
  33. Curtis R. Henrie, Lisa R. Halverson, and Charles R. Graham. 2015. Measuring student engagement in technology-mediated learning: A review. Comput. Educ. 90, (December 2015), 36--53. DOI:https://doi.org/10.1016/j.compedu.2015.09.005Google ScholarGoogle Scholar
  34. Paul J. Hirschfield and Joseph Gasper. 2011. The Relationship Between School Engagement and Delinquency in Late Childhood and Early Adolescence. J. Youth Adolesc. 40, 1 (January 2011), 3--22. DOI:https://doi.org/10.1007/s10964-010--9579--5Google ScholarGoogle ScholarCross RefCross Ref
  35. Wouter M. van den Hoogen, Wijnand A. IJsselsteijn, and Yvonne AW de Kort. 2008. Exploring behavioral expressions of player experience in digital games. In Proceedings of the workshop on Facial and Bodily Expression for Control and Adaptation of Games ECAG, Citeseer, 11--19.Google ScholarGoogle Scholar
  36. Rob Hyndman, Yanfei Kang, Pablo Montero-Manso, Thiyanga Talagala, Earo Wang, Yangzhuoran Yang, and Mitchell O'Hara-Wild. 2020. tsfeatures: Time Series Feature Extraction. Retrieved from https://CRAN.R-project.org/package=tsfeaturesGoogle ScholarGoogle Scholar
  37. Herman Ilgen, Jacob Israelashvili, and Agneta Fischer. 2021. Personal Nonverbal Repertoires in facial displays and their relation to individual differences in social and emotional styles. Cogn. Emot. (January 2021), 1--10. DOI:https://doi.org/10.1080/02699931.2021.1877118Google ScholarGoogle Scholar
  38. Michael Inzlicht, Amitai Shenhav, and Christopher Y. Olivola. 2018. The Effort Paradox: Effort Is Both Costly and Valued. Trends Cogn. Sci. 22, 4 (April 2018), 337--349. DOI:https://doi.org/10.1016/j.tics.2018.01.007Google ScholarGoogle ScholarCross RefCross Ref
  39. Susanne M. Jaeggi, Martin Buschkuehl, John Jonides, and Walter J. Perrig. 2008. Improving fluid intelligence with training on working memory. Proc. Natl. Acad. Sci. 105, 19 (May 2008), 6829--6833. DOI:https://doi.org/10.1073/pnas.0801268105Google ScholarGoogle ScholarCross RefCross Ref
  40. Veronika Job, Carol S. Dweck, and Gregory M. Walton. 2010. Ego Depletion-Is It All in Your Head?: Implicit Theories About Willpower Affect Self-Regulation. Psychol. Sci. 21, 11 (November 2010), 1686--1693. DOI:https://doi.org/10.1177/0956797610384745Google ScholarGoogle Scholar
  41. B. Kort, R. Reilly, and R. W. Picard. 2001. An affective model of interplay between emotions and learning: reengineering educational pedagogy-building a learning companion. In Proceedings IEEE International Conference on Advanced Learning Technologies, 43--46. DOI:https://doi.org/10.1109/ICALT.2001.943850Google ScholarGoogle ScholarCross RefCross Ref
  42. Heinz W. Krohne, Boris Egloff, Carl-Walter Kohlmann, and Anja Tausch. 1996. Untersuchungen mit einer deutschen version der" positive and negative affect schedule"(PANAS). Diagn.-Gottingen- 42, (1996), 139--156.Google ScholarGoogle Scholar
  43. Richard N. Landers. 2014. Developing a Theory of Gamified Learning: Linking Serious Games and Gamification of Learning. Simul. Gaming 45, 6 (December 2014), 752--768. DOI:https://doi.org/10.1177/1046878114563660Google ScholarGoogle Scholar
  44. Richard N. Landers, Elena M. Auer, Andrew B. Collmus, and Michael B. Armstrong. 2018. Gamification Science, Its History and Future: Definitions and a Research Agenda. Simul. Gaming 49, 3 (June 2018), 315--337. DOI:https://doi.org/10.1177/1046878118774385Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Hao Lei, Yunhuo Cui, and Wenye Zhou. 2018. Relationships between student engagement and academic achievement: A meta-analysis. Soc. Behav. Personal. Int. J. 46, 3 (March 2018), 517--528. DOI:https://doi.org/10.2224/sbp.7054Google ScholarGoogle Scholar
  46. Shan Li and Weihong Deng. 2018. Deep Facial Expression Recognition: A Survey. ArXiv180408348 Cs (April 2018). Retrieved January 24, 2019 from http://arxiv.org/abs/1804.08348Google ScholarGoogle Scholar
  47. Gwen C. Littlewort, Marian S. Bartlett, Linda P. Salamanca, and Judy Reilly. 2011. Automated measurement of children's facial expressions during problem solving tasks. In Face and Gesture 2011, IEEE, Santa Barbara, CA, USA, 30--35. DOI:https://doi.org/10.1109/FG.2011.5771418Google ScholarGoogle ScholarCross RefCross Ref
  48. Meng Liu, Yaocong Duan, Robin A. A. Ince, Chaona Chen, Oliver G. B. Garrod, Philippe Schyns, and Rachael E. Jack. 2020. Facial Expressions of Emotion Categories are Embedded within a Dimensional Space of Valence-arousal. DOI:https://doi.org/10.31234/osf.io/pw5uhGoogle ScholarGoogle Scholar
  49. Konstantinos Makantasis, Antonios Liapis, and Georgios N. Yannakakis. 2021. The Pixels and Sounds of Emotion: General-Purpose Representations of Arousal in Games. ArXiv210110706 Cs (February 2021). Retrieved February 17, 2021 from http://arxiv.org/abs/2101.10706Google ScholarGoogle Scholar
  50. Regan L. Mandryk and M. Stella Atkins. 2007. A fuzzy physiological approach for continuously modeling emotion during interaction with play technologies. Int. J. Hum.-Comput. Stud. 65, 4 (April 2007), 329--347. DOI:https://doi.org/10.1016/j.ijhcs.2006.11.011Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Brais Martinez, Michel F. Valstar, Bihan Jiang, and Maja Pantic. 2019. Automatic Analysis of Facial Actions: A Survey. IEEE Trans. Affect. Comput. 10, 3 (July 2019), 325--347. DOI:https://doi.org/10.1109/TAFFC.2017.2731763Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Gerald Matthews, Joel S. Warm, and Andrew P. Smith. 2017. Task Engagement and Attentional Resources: Multivariate Models for Individual Differences and Stress Factors in Vigilance. Hum. Factors J. Hum. Factors Ergon. Soc. 59, 1 (February 2017), 44--61. DOI:https://doi.org/10.1177/0018720816673782Google ScholarGoogle ScholarCross RefCross Ref
  53. Bethany McDaniel, Sidney D'Mello, Brandon King, Patrick Chipman, Kristy Tapp, and Art Graesser. 2007. Facial features for affective state detection in learning environments. In Proceedings of the Annual Meeting of the Cognitive Science Society.Google ScholarGoogle Scholar
  54. Daniel McDuff, Rana El Kaliouby, Karim Kassam, and Rosalind Picard. 2010. Affect valence inference from facial action unit spectrograms. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops, IEEE, San Francisco, CA, USA, 17--24. DOI:https://doi.org/10.1109/CVPRW.2010.5543833Google ScholarGoogle ScholarCross RefCross Ref
  55. Elisa D. Mekler, Florian Brühlmann, Alexandre N. Tuch, and Klaus Opwis. 2017. Towards understanding the effects of individual gamification elements on intrinsic motivation and performance. Comput. Hum. Behav. 71, (June 2017), 525--534. DOI:https://doi.org/10.1016/j.chb.2015.08.048Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Marianne Miserandino. 1996. Children who do well in school: Individual differences in perceived competence and autonomy in above-average children. J. Educ. Psychol. 88, 2 (1996), 203--214. DOI:https://doi.org/10.1037/0022-0663.88.2.203Google ScholarGoogle ScholarCross RefCross Ref
  57. Steve Nebel and Manuel Ninaus. 2019. New Perspectives on Game-Based Assessment with Process Data and Physiological Signals. In Game-Based Assessment Revisited, Dirk Ifenthaler and Yoon Jeon Kim (eds.). Springer International Publishing, Cham, 141--161. DOI:https://doi.org/10.1007/978--3-030--15569--8_8Google ScholarGoogle Scholar
  58. Fred M. Newmann (Ed.). 1992. Student engagement and achievement in American secondary schools. Teachers College Press, New York.Google ScholarGoogle Scholar
  59. Manuel Ninaus, Simon Greipl, Kristian Kiili, Antero Lindstedt, Stefan Huber, Elise Klein, Hans-Otto Karnath, and Korbinian Moeller. 2019. Increased emotional engagement in game-based learning -- A machine learning approach on facial emotion detection data. Comput. Educ. 142, (December 2019), 103641. DOI:https://doi.org/10.1016/j.compedu.2019.103641Google ScholarGoogle Scholar
  60. Manuel Ninaus, K. Kiili, G. Wood, K. Moeller, and S. E. Kober. 2020. To Add or Not to Add Game Elements? Exploring the Effects of Different Cognitive Task Designs Using Eye Tracking. IEEE Trans. Learn. Technol. 13, 4 (October 2020), 847--860. DOI:https://doi.org/10.1109/TLT.2020.3031644Google ScholarGoogle ScholarCross RefCross Ref
  61. Manuel Ninaus, Gonçalo Pereira, René Stefitz, Rui Prada, Ana Paiva, Christa Neuper, and Guilherme Wood. 2015. Game elements improve performance in a working memory training task. Int. J. Serious Games 2, 1 (February 2015). DOI:https://doi.org/10.17083/ijsg.v2i1.60Google ScholarGoogle ScholarCross RefCross Ref
  62. Babette Park, Lisa Knörzer, Jan L. Plass, and Roland Brünken. 2015. Emotional design and positive emotions in multimedia learning: An eyetracking study on the use of anthropomorphisms. Comput. Educ. 86, (August 2015), 30--42. DOI:https://doi.org/10.1016/j.compedu.2015.02.016Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. Jonathan Peirce, Jeremy R. Gray, Sol Simpson, Michael MacAskill, Richard Höchenberger, Hiroyuki Sogo, Erik Kastman, and Jonas Kristoffer Lindeløv. 2019. PsychoPy2: Experiments in behavior made easy. Behav. Res. Methods 51, 1 (February 2019), 195--203. DOI:https://doi.org/10.3758/s13428-018-01193-yGoogle ScholarGoogle ScholarCross RefCross Ref
  64. Reinhard Pekrun and Lisa Linnenbrink-Garcia. 2012. Academic Emotions and Student Engagement. In Handbook of Research on Student Engagement, Sandra L. Christenson, Amy L. Reschly and Cathy Wylie (eds.). Springer US, Boston, MA, 259--282. DOI:https://doi.org/10.1007/978--1--4614--2018--7_12Google ScholarGoogle Scholar
  65. Jan L. Plass, Steffi Heidig, Elizabeth O. Hayward, Bruce D. Homer, and Enjoon Um. 2014. Emotional design in multimedia learning: Effects of shape and color on affect and learning. Learn. Instr. 29, (February 2014), 128--140. DOI:https://doi.org/10.1016/j.learninstruc.2013.02.006Google ScholarGoogle ScholarCross RefCross Ref
  66. R Core Team. 2020. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. Retrieved from https://www.R-project.org/Google ScholarGoogle Scholar
  67. M. Ramzan, H. U. Khan, S. M. Awan, A. Ismail, M. Ilyas, and A. Mahmood. 2019. A Survey on State-of-the-Art Drowsiness Detection Techniques. IEEE Access 7, (2019), 61904--61919. DOI:https://doi.org/10.1109/ACCESS.2019.2914373Google ScholarGoogle Scholar
  68. Valentin Riemer, Julian Frommel, Georg Layher, Heiko Neumann, and Claudia Schrader. 2017. Identifying Features of Bodily Expression As Indicators of Emotional Experience during Multimedia Learning. Front. Psychol. 8, (July 2017), 1303. DOI:https://doi.org/10.3389/fpsyg.2017.01303Google ScholarGoogle Scholar
  69. James A. Russell. 1980. A circumplex model of affect. J. Pers. Soc. Psychol. 39, 6 (1980), 1161--1178. DOI:https://doi.org/10.1037/h0077714Google ScholarGoogle Scholar
  70. Michael Sailer and Lisa Homner. 2020. The Gamification of Learning: a Meta-analysis. Educ. Psychol. Rev. 32, 1 (March 2020), 77--112. DOI:https://doi.org/10.1007/s10648-019-09498-wGoogle ScholarGoogle ScholarCross RefCross Ref
  71. Andrew Schall. 2014. New Methods for Measuring Emotional Engagement. In Design, User Experience, and Usability. User Experience Design Practice (Lecture Notes in Computer Science), Springer International Publishing, Cham, 347--357. DOI:https://doi.org/10.1007/978--3--319-07638--6_34Google ScholarGoogle Scholar
  72. Jesse Schell. 2015. The art of game design: a book of lenses (Second edition ed.). CRC Press, Boca Raton.Google ScholarGoogle ScholarDigital LibraryDigital Library
  73. Wolfgang Schnotz, Stefan Fries, and Holger Horz. 2009. Motivational aspects of cognitive load theory. In Contemporary motivation research: From global to local perspectives. Hogrefe & Huber Publishers, Ashland, OH, US, 69--96.Google ScholarGoogle Scholar
  74. Y. A. Sekhavat, S. Roohi, H. Sakian Mohammadi, and G. N. Yannakakis. 2020. Play with One's Feelings: A Study on Emotion Awareness for Player Experience. IEEE Trans. Games (2020), 1--1. DOI:https://doi.org/10.1109/TG.2020.3003324Google ScholarGoogle Scholar
  75. Stuart G. Shanker and Devin M. Casenhiser. 2013. Reducing the effort in effortful control. In A Wittgensteinian Perspective on the use of Conceptual Analysis in Psychology. Springer, 214--232.Google ScholarGoogle Scholar
  76. Nicolas Silvestrini and Guido H.E. Gendolla. 2019. Affect and cognitive control: Insights from research on effort mobilization. Int. J. Psychophysiol. 143, (September 2019), 116--125. DOI:https://doi.org/10.1016/j.ijpsycho.2019.07.003Google ScholarGoogle ScholarCross RefCross Ref
  77. James Steele. 2020. What is (perception of) effort? Objective and subjective effort during task performance. PsyArXiv (June 2020). Retrieved January 30, 2021 from https://pure.solent.ac.uk/en/publications/what-is-perception-of-effort-objective-and-subjective-effort-duriGoogle ScholarGoogle Scholar
  78. April Tyack and Elisa D. Mekler. 2021. Off-Peak: An Examination of Ordinary Player Experience. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, ACM, Yokohama Japan, 1--12. DOI:https://doi.org/10.1145/3411764.3445230Google ScholarGoogle Scholar
  79. Eunjoon "Rachel" Um, Jan L. Plass, Elizabeth O. Hayward, and Bruce D. Homer. 2012. Emotional design in multimedia learning. J. Educ. Psychol. 104, 2 (May 2012), 485--498. DOI:https://doi.org/10.1037/a0026609Google ScholarGoogle Scholar
  80. M.F. Valstar and M. Pantic. 2006. Biologically vs. Logic Inspired Encoding of Facial Actions and Emotions in Video. In 2006 IEEE International Conference on Multimedia and Expo, IEEE, Toronto, ON, Canada, 325--328. DOI:https://doi.org/10.1109/ICME.2006.262464Google ScholarGoogle Scholar
  81. S. Velusamy, H. Kannan, B. Anand, A. Sharma, and B. Navathe. 2011. A method to infer emotions from facial Action Units. In 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2028--2031. DOI:https://doi.org/10.1109/ICASSP.2011.5946910Google ScholarGoogle Scholar
  82. Harald G. Wallbott. 1998. Bodily expression of emotion. Eur. J. Soc. Psychol. 28, 6 (1998), 879--896.Google ScholarGoogle ScholarDigital LibraryDigital Library
  83. Ming-Te Wang and Jennifer Fredricks. 2014. The Reciprocal Links between School Engagement, Youth Problem Behaviors, and School Dropout during Adolescence. Child Dev. 85, 2 (March 2014), 722--737. DOI:https://doi.org/10.1111/cdev.12138Google ScholarGoogle ScholarCross RefCross Ref
  84. Ming-Te Wang and Rebecca Holcombe. 2010. Adolescents' Perceptions of School Environment, Engagement, and Academic Achievement in Middle School. Am. Educ. Res. J. 47, 3 (September 2010), 633--662. DOI:https://doi.org/10.3102/0002831209361209Google ScholarGoogle ScholarCross RefCross Ref
  85. David Watson, Lee Anna Clark, and Auke Tellegen. 1988. Development and validation of brief measures of positive and negative affect: The PANAS scales. J. Pers. Soc. Psychol. 54, 6 (1988), 1063--1070. DOI:https://doi.org/10.1037/0022--3514.54.6.1063Google ScholarGoogle Scholar
  86. Jacob Whitehill, Marian Bartlett, and Javier Movellan. 2008. Automatic facial expression recognition for intelligent tutoring systems. In 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, IEEE, Anchorage, AK, USA, 1--6. DOI:https://doi.org/10.1109/CVPRW.2008.4563182Google ScholarGoogle ScholarCross RefCross Ref
  87. Harry J. Witchel, Carlos P. Santos, James K. Ackah, Carina E. I. Westling, and Nachiappan Chockalingam. 2016. Non-Instrumental Movement Inhibition (NIMI) Differentially Suppresses Head and Thigh Movements during Screenic Engagement: Dependence on Interaction. Front. Psychol. 7, (February 2016). DOI:https://doi.org/10.3389/fpsyg.2016.00157Google ScholarGoogle ScholarCross RefCross Ref
  88. Matthias Witte, Manuel Ninaus, Silvia Erika Kober, Christa Neuper, and Guilherme Wood. 2015. Neuronal Correlates of Cognitive Control during Gaming Revealed by Near-Infrared Spectroscopy. PLOS ONE 10, 8 (August 2015), e0134816. DOI:https://doi.org/10.1371/journal.pone.0134816Google ScholarGoogle ScholarCross RefCross Ref
  89. Beverly Woolf, Winslow Burleson, Ivon Arroyo, Toby Dragon, David Cooper, and Rosalind Picard. 2009. Affect-aware tutors: recognising and responding to student affect. Int. J. Learn. Technol. 4, 3/4 (2009), 129. DOI:https://doi.org/10.1504/IJLT.2009.028804Google ScholarGoogle ScholarDigital LibraryDigital Library
  90. Georgios N. Yannakakis, Roddy Cowie, and Carlos Busso. 2021. The Ordinal Nature of Emotions: An Emerging Approach. IEEE Trans. Affect. Comput. 12, 1 (January 2021), 16--35. DOI:https://doi.org/10.1109/TAFFC.2018.2879512Google ScholarGoogle Scholar
  91. Yei-Yu Yen, Christopher D. Wickens, and Sandra G. Hart. 1985. The Effect of Varying Task Difficulty on Subjective Workload. Proc. Hum. Factors Soc. Annu. Meet. 29, 8 (October 1985), 765--769. DOI:https://doi.org/10.1177/154193128502900808Google ScholarGoogle Scholar

Index Terms

  1. Facial and Bodily Expressions of Emotional Engagement: How Dynamic Measures Reflect the Use of Game Elements and Subjective Experience of Emotions and Effort

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!