Abstract
Is it possible to detect toxicity in games just by observing in-game behavior? If so, what are the behavioral factors that will help machine learning to discover the unknown relationship between gameplay and toxic behavior? In this initial study, we examine whether it is possible to predict toxicity in the MOBA gameFor Honor by observing in-game behavior for players that have been labeled as toxic (i.e. players that have been sanctioned by Ubisoft community managers). We test our hypothesis of detecting toxicity through gameplay with a dataset of almost 1,800 sanctioned players, and comparing these sanctioned players with unsanctioned players. Sanctioned players are defined by their toxic action type (offensive behavior vs. unfair advantage) and degree of severity (warned vs. banned). Our findings, based on supervised learning with random forests, suggest that it is not only possible to behaviorally distinguish sanctioned from unsanctioned players based on selected features of gameplay; it is also possible to predict both the sanction severity (warned vs. banned) and the sanction type (offensive behavior vs. unfair advantage). In particular, all random forest models predict toxicity, its severity, and type, with an accuracy of at least 82%, on average, on unseen players. This research shows that observing in-game behavior can support the work of community managers in moderating and possibly containing the burden of toxic behavior.
- 2016. For Honor Code Of Conduct. https://forums.ubisoft.com/showthread.php/1490092-Code-of-Conduct. Accessed: 20-02--10.Google Scholar
- 2018. Dev Blog: Toxicity. ubisoft.com/en-us/game/rainbow-six/siege/news-updates/4mPLa195TWiXREafM4T4oC. Accessed: 20-02--10.Google Scholar
- Lucy A. Sparrow, Martin Gibbs, and Michael Arnold. 2021. The Ethics of Multiplayer Game Design and Community Management: Industry Perspectives and Challenges. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1--13.Google Scholar
Digital Library
- Sonam Adinolf and Selen Turkay. 2018. Toxic behaviors in Esports games: player perceptions and coping strategies. In Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts. 365--372.Google Scholar
Digital Library
- Anti Defamation League. 2020. Free to Play? Hate, Harassment and Positive Social Experience in Online Games 2020. https://www.adl.org/free-to-play-2020. Accessed: 20-02--10.Google Scholar
- Mary Elizabeth Ballard and Kelly MarieWelch. 2017. Virtual warfare: Cyberbullying and cyber-victimization in MMOG play. Games and culture 12, 5 (2017), 466--491.Google Scholar
Cross Ref
- Shaowen Bardzell. 2010. Feminist HCI: taking stock and outlining an agenda for design. In Proceedings of the SIGCHI conference on human factors in computing systems. 1301--1310.Google Scholar
Digital Library
- Christopher Barlett and Sarah M Coyne. 2014. A meta-analysis of sex differences in cyber-bullying behavior: The moderating role of age. Aggressive behavior 40, 5 (2014), 474--488.Google Scholar
- Nicole A Beres, Julian Frommel, Elizabeth Reid, Regan L Mandryk, and Madison Klarkowski. 2021. Don't You Know That You're Toxic: Normalization of Toxicity in Online Gaming. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1--15.Google Scholar
Digital Library
- Jeremy Blackburn and Haewoon Kwak. 2014. STFU NOOB! predicting crowdsourced decisions on toxic behavior in online games. In Proceedings of the 23rd international conference on World wide web. 877--888.Google Scholar
Digital Library
- Chris Bruzzo. 2020. EA and the war against toxicity. https://www.gamesindustry.biz/articles/2020-06--23-ea-and-thewar- against-toxicity. Accessed: 20-02--10.Google Scholar
- Lindsey A Cary, Jordan Axt, and Alison L Chasteen. 2020. The interplay of individual differences, norms, and group identification in predicting prejudiced behavior in online video game interactions. Journal of Applied Social Psychology 50, 11 (2020), 623--637.Google Scholar
Cross Ref
- Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer. 2002. SMOTE: synthetic minority over-sampling technique. Journal of artificial intelligence research 16 (2002), 321--357.Google Scholar
Digital Library
- Valve Corporation. 20013. League Of Legends.Google Scholar
- Amanda C Cote. 2017. "I Can Defend Myself" Women's Strategies for Coping With Harassment While Gaming Online. Games and Culture 12, 2 (2017), 136--155.Google Scholar
Cross Ref
- Pascal Debroek. 2020. The Impact of Toxicity on Retention and LTV. https://www.gamasutra.com/blogs/PascalDebroek/ 20201125/374272/The_Impact_of_Toxicity_on_Retention_and_LTV.php. Accessed: 20-02--10.Google Scholar
- Blizzard Entertainment. 2015. Overwatch.Google Scholar
- Niklas Ericsson and Hampus Bergström. 2020. How toxicity differ between male and female players in competitive Overwatch.Google Scholar
- David R Ewoldsen, Cassie A Eno, BradleyMOkdie, John A Velez, Rosanna E Guadagno, and Jamie DeCoster. 2012. Effect of playing violent video games cooperatively or competitively on subsequent cooperative behavior. Cyberpsychology, Behavior, and Social Networking 15, 5 (2012), 277--280.Google Scholar
Cross Ref
- Fair Play Alliance. 2021. https://fairplayalliance.org/Google Scholar
- Chek Yang Foo and Elina MI Koivisto. 2004. Defining grief play in MMORPGs: player and developer perceptions. In Proceedings of the 2004 ACM SIGCHI International Conference on Advances in computer entertainment technology. 245--250.Google Scholar
Digital Library
- Jesse Fox and Wai Yen Tang. 2017. Women's experiences with general and sexual harassment in online video games: Rumination, organizational responsiveness, withdrawal, and coping strategies. New Media & Society 19, 8 (2017), 1290--1307. https://doi.org/10.1177/1461444816635778 arXiv:https://doi.org/10.1177/1461444816635778Google Scholar
- Julian Frommel, Valentin Sagl, Ansgar E Depping, Colby Johanson, Matthew K Miller, and Regan L Mandryk. 2020. Recognizing Affiliation: Using Behavioural Traces to Predict the Quality of Social Interactions in Online Games. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1--16.Google Scholar
Digital Library
- Riot Games. 2009. League Of Legends.Google Scholar
- Spiros V Georgakopoulos, Sotiris K Tasoulis, Aristidis G Vrahatis, and Vassilis P Plagianakos. 2019. Convolutional neural networks for twitter text toxicity analysis. In INNS Big Data and Deep Learning conference. Springer, 370--379.Google Scholar
- Ayushi Ghosh. 2021. Analyzing Toxicity in Online Gaming Communities. Turkish Journal of Computer and Mathematics Education (TURCOMAT) 12, 10 (2021), 4448--4455.Google Scholar
- Kishonna L Gray. 2012. Deviant bodies, stigmatized identities, and racist acts: Examining the experiences of African- American gamers in Xbox Live. New Review of Hypermedia and Multimedia 18, 4 (2012), 261--276.Google Scholar
Digital Library
- Kishonna L Gray. 2012. Intersecting oppressions and online communities: Examining the experiences of women of color in Xbox Live. Information, Communication & Society 15, 3 (2012), 411--428.Google Scholar
Cross Ref
- Kishonna L Gray. 2014. Race, gender, and deviance in Xbox live: Theoretical perspectives from the virtual margins. Routledge.Google Scholar
- Emily Jane Hayday, Holly Collison, and Geoffery Z. Kohe. 2020. Landscapes of tension, tribalism and toxicity: configuring a spatial politics of esport communities. Leisure Studies 0, 0 (2020), 1--15. https://doi.org/10.1080/02614367. 2020.1808049 arXiv:https://doi.org/10.1080/02614367.2020.1808049Google Scholar
- Haibo He, Yang Bai, Edwardo A Garcia, and Shutao Li. 2008. ADASYN: Adaptive synthetic sampling approach for imbalanced learning. In 2008 IEEE international joint conference on neural networks (IEEE world congress on computational intelligence). IEEE, 1322--1328.Google Scholar
- Zorah Hilvert-Bruce and James T Neill. 2020. I'm just trolling: The role of normative beliefs in aggressive behaviour in online gaming. Computers in Human Behavior 102 (2020), 303--311.Google Scholar
Digital Library
- Jessica M Jerabeck and Christopher J Ferguson. 2013. The influence of solitary and cooperative violent video game play on aggressive and prosocial behavior. Computers in Human Behavior 29, 6 (2013), 2573--2578.Google Scholar
Digital Library
- Daniel Johnson, Lennart E. Nacke, and Peta Wyeth. 2015. All about That Base: Differing Player Experiences in Video Game Genres and the Unique Case of MOBA Games. Association for Computing Machinery, New York, NY, USA, 2265--2274. https://doi.org/10.1145/2702123.2702447Google Scholar
- Daniel Johnson, Xiang Zhao, Katherine M White, and Varuni Wickramasinghe. 2021. Need satisfaction, passion, empathy and helping behaviour in videogame play. Computers in Human Behavior 122 (2021), 106817.Google Scholar
Cross Ref
- Jeff Kaplan. [n.d.]. Our progress so far. https://us.forums.blizzard.com/en/overwatch/t/our-progress-so-far/159046. Accessed: 20-02--10.Google Scholar
- Bastian Kordyaka, Katharina Jahn, and Bjoern Niehaves. 2020. Towards a unified theory of toxic behavior in video games. Internet Research 30, 4 (2020), 1081--1102. https://doi.org/10.1108/INTR-08--2019-0343Google Scholar
Cross Ref
- Lindah Kotut and D Scott McCrickard. 2020. Amplifying the Griot: Design Fiction for Development as an Inclusivity Lens. Technical Report. EasyChair.Google Scholar
- Yubo Kou. 2020. Toxic Behaviors in Team-Based Competitive Gaming: The Case of League of Legends. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play. 81--92.Google Scholar
Digital Library
- Yubo Kou and Xinning Gui. 2021. Flag and Flaggability in Automated Moderation: The Case of Reporting Toxic Behavior in an Online Game Community. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1--12.Google Scholar
Digital Library
- Yubo Kou and Bonnie Nardi. 2013. Regulating anti-social behavior on the Internet: The example of League of Legends. In iConference 2013 Proceedings. iSchools, 616--622. https://doi.org/doi:10.9776/13289Google Scholar
- Rachel Kowert. 2020. Dark Participation in Games. Frontiers in Psychology 11 (2020), 2969. https://doi.org/10.3389/ fpsyg.2020.598947Google Scholar
Cross Ref
- Jeffrey H. Kuznekoff and Lindsey M. Rose. 2013. Communication in multiplayer gaming: Examining player responses to gender cues. New Media & Society 15, 4 (2013), 541--556. https://doi.org/10.1177/1461444812458271 arXiv:https://doi.org/10.1177/1461444812458271Google Scholar
- Haewoon Kwak, Jeremy Blackburn, and Seungyeop Han. 2015. Exploring Cyberbullying and Other Toxic Behavior in Team Competition Online Games. Association for Computing Machinery, New York, NY, USA, 3739--3748. https: //doi.org/10.1145/2702123.2702529Google Scholar
- Maud Lemercier-Dugarin, Lucia Romo, Charles Tijus, and Oulmann Zerhouni. 2021. "Who Are the Cyka Blyat?" How Empathy, Impulsivity, and Motivations to Play Predict Aggressive Behaviors in Multiplayer Online Games. Cyberpsychology, Behavior, and Social Networking 24, 1 (2021), 63--69.Google Scholar
Cross Ref
- Roger J Lewis. 2000. An introduction to classification and regression tree (CART) analysis. In Annual meeting of the society for academic emergency medicine in San Francisco, California, Vol. 14.Google Scholar
- Rachael Luck. 2003. Dialogue in participatory design. Design studies 24, 6 (2003), 523--535.Google Scholar
- Daniel Madden, Yuxuan Liu, Haowei Yu, Mustafa Feyyaz Sonbudak, Giovanni M Troiano, and Casper Harteveld. 2021. "Why Are You Playing Games? You Are a Girl!": Exploring Gender Biases in Esports. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1--15.Google Scholar
Digital Library
- Brendan Maher. 2016. Can a video game company tame toxic behaviour? Nature 531, 7596 (2016), 568--571.Google Scholar
- E Makuch. 2014. MOBAs are "notoriously toxic," says Strife developer. Retrieved July 6, 2015.Google Scholar
- Lavinia McLean and Mark D. Griffiths. 2019. Female Gamers' Experience of Online Harassment and Social Support in Online Gaming: A Qualitative Study. International Journal of Mental Health and Addiction 17, 4 (2019), 970--994. https://doi.org/10.1007/s11469-018--9962-0Google Scholar
Cross Ref
- "David Melhart, Antonios Liapis, and Georgios N. Yannakakis". "2021". "The Affect Game AnnotatIoN (AGAIN) Dataset". "IEEE Transactions on Affective Computing" ("2021"), "1--14". "in review".Google Scholar
- Marçal Mora-Cantallops and Miguel-Ángel Sicilia. 2018. MOBA games: A literature review. Entertainment computing 26 (2018), 128--138.Google Scholar
- Shane Murnion, William J Buchanan, Adrian Smales, and Gordon Russell. 2018. Machine learning and semantic analysis of in-game chat for cyberbullying. Computers & Security 76 (2018), 197--213.Google Scholar
Cross Ref
- M. Märtens, S. Shen, A. Iosup, and F. Kuipers. 2015. Toxicity detection in multiplayer online games. In 2015 International Workshop on Network and Systems Support for Games (NetGames). 1--6. https://doi.org/10.1109/NetGames.2015.7382991Google Scholar
Cross Ref
- Larissa Vivian Nägele, Merja Ryöppy, and Danielle Wilde. 2018. PDFI: Participatory design fiction with vulnerable users. In Proceedings of the 10th Nordic Conference on Human-Computer Interaction. 819--831.Google Scholar
Digital Library
- Joaquim AM Neto, Kazuki M Yokoyama, and Karin Becker. 2017. Studying toxic behavior influence and player chat in an online video game. In Proceedings of the International Conference on Web Intelligence. 26--33.Google Scholar
Digital Library
- Adewale Obadimu, Esther Mead, Muhammad Nihal Hussain, and Nitin Agarwal. 2019. Identifying toxicity within youtube video comment. In International Conference on Social Computing, Behavioral-Cultural Modeling and Prediction and Behavior Representation in Modeling and Simulation. Springer, 214--223.Google Scholar
Cross Ref
- Stephanie M Ortiz. 2019. "You can say I got desensitized to it": How men of color cope with everyday racism in online gaming. Sociological Perspectives 62, 4 (2019), 572--588.Google Scholar
Cross Ref
- Hunter L Paul, Nicholas David Bowman, and Jaime Banks. 2015. The enjoyment of griefing in online games. Journal of Gaming & Virtual Worlds 7, 3 (2015), 243--258.Google Scholar
Cross Ref
- Ari Schlesinger, Kenton P O'Hara, and Alex S Taylor. 2018. Let's talk about race: Identity, chatbots, and AI. In Proceedings of the 2018 chi conference on human factors in computing systems. 1--14.Google Scholar
Digital Library
- Konstantinos Sechidis, Grigorios Tsoumakas, and Ioannis Vlahavas. 2011. On the stratification of multi-label data. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 145--158.Google Scholar
Digital Library
- Sercan Sengün, Joni Salminen, Soon-gyo Jung, Peter Mawhorter, and Bernard J Jansen. 2019. Analyzing Hate Speech Toward Players from the MENA in League of Legends. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. 1--6.Google Scholar
Digital Library
- Sercan Sengün, Joni Salminen, Peter Mawhorter, Soon-gyo Jung, and Bernard Jansen. 2019. Exploring the relationship between game content and culture-based toxicity: a case study of league of legends and MENA players. In Proceedings of the 30th ACM Conference on Hypertext and Social Media. 87--95.Google Scholar
Digital Library
- Cuihua Shen, Qiusi Sun, Taeyoung Kim, Grace Wolff, Rabindra Ratan, and Dmitri Williams. 2020. Viral vitriol: Predictors and contagion of online toxicity in World of Tanks. Computers in Human Behavior 108 (2020), 106343. https://doi.org/10.1016/j.chb.2020.106343Google Scholar
Cross Ref
- Kenneth B Shores, Yilin He, Kristina L Swanenburg, Robert Kraut, and John Riedl. 2014. The identification of deviance and its impact on retention in a multiplayer game. In Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing. 1356--1365.Google Scholar
Digital Library
- Noah Smith. 2019. Racism, misogyny, death threats: Why can't the booming video-game industry curb toxicity? https://www.washingtonpost.com/technology/2019/02/26/racism-misogyny-death-threats-why-cant-boomingvideo- game-industry-curb-toxicity/. Accessed: 20-02--10.Google Scholar
- Lucy Sparrow, Martin Gibbs, and Michael Arnold. 2019. Apathetic villagers and the trolls who love them: Player amorality in online multiplayer games. In Proceedings of the 31st Australian Conference on Human-Computer-Interaction. 447--451.Google Scholar
Digital Library
- Wessel Stoop, Florian Kunneman, Antal van den Bosch, and Ben Miller. 2019. Detecting harassment in real-time as conversations develop. In Proceedings of the Third Workshop on Abusive Language Online. 19--24.Google Scholar
- Wai Yen Tang, Felix Reer, and Thorsten Quandt. 2020. Investigating sexual harassment in online video games: How personality and context factors are related to toxic sexual behaviors against fellow players. Aggressive Behavior 46, 1 (2020), 127--135.Google Scholar
Cross Ref
- Haydn Taylor. 2020. Valve introduces new measures to shut down team fortress 2 racist bot problem. https://www.gamesindustry.biz/articles/2020-06--18-valve-introduces-new-measures-to-shut-down-teamfortress- 2s-racist-bot-problem. Accessed: 20-02--10.Google Scholar
- Joseph J Thompson, Betty HM Leung, Mark R Blair, and Maite Taboada. 2017. Sentiment analysis of player chat messaging in the video game StarCraft 2: Extending a lexicon-based model. Knowledge-Based Systems 137 (2017), 149--162.Google Scholar
Digital Library
- Arjen Traas. 2017. The Impact of Toxic Behavior on Match Outcomes in DotA. Ph.D. Dissertation. Tilburg University.Google Scholar
- Selen Türkay, Jessica Formosa, Sonam Adinolf, Robert Cuthbert, and Roger Altizer. 2020. See No Evil, Hear No Evil, Speak No Evil: How Collegiate Players Define, Experience and Cope with Toxicity. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--13. https://doi.org/10.1145/3313831.3376191Google Scholar
Digital Library
- Ubisoft. 2016. For Honor.Google Scholar
- Rebekah Valentine. 2020. Infinity Ward pledges to do more to moderate racist content in its games. https://www.gamesindustry.biz/articles/2020-06-03-infinity-ward-pledges-to-do-more-to-moderate-racistcontent- in-its-games. Accessed: 20-02--10.Google Scholar
- Mark Verschoor. 2016. Eating seeds as a pastime activity: Predicting toxicity in online game chat using in-game events. Master's thesis. Tilburg University.Google Scholar
- Julia Crouse Waddell and Wei Peng. 2014. Does it matter with whom you slay? The effects of competition, cooperation and relationship type among video game players. Computers in Human Behavior 38 (2014), 331--338.Google Scholar
Digital Library
- Wargaming. 2010. World of Tanks.Google Scholar
- Tony Xiao. 2019. Confronting Toxicity in Gaming: Going Beyond ?Mute". https://www.nytimes.com/2019/06/06/ learning/confronting-toxicity-in-gaming-going-beyond-mute.html. Accessed: 20-02--10.Google Scholar
- Georgios N Yannakakis and Julian Togelius. 2018. Artificial intelligence and games. Vol. 2. Springer.Google Scholar
Digital Library
- James Zou and Londa Schiebinger. 2018. AI can be sexist and racist-it's time to make it fair. Nature (2018), 324--326. https://doi.org/10.1038/d41586-018-05707--8Google Scholar
Index Terms
For Honor, for Toxicity: Detecting Toxic Behavior through Gameplay
Recommendations
Don’t You Know That You’re Toxic: Normalization of Toxicity in Online Gaming
CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing SystemsVideo game toxicity, endemic to online play, represents a pervasive and complex problem. Antisocial behaviours in online play directly harm player wellbeing, enjoyment, and retention—but research has also revealed that some players normalize toxicity ...
Toxic Behaviors in Esports Games: Player Perceptions and Coping Strategies
CHI PLAY '18 Extended Abstracts: Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play Companion Extended AbstractsToxicity in online environments is a complex and a systemic issue. Esports communities seem to be particularly suffering from toxic behaviors. Especially in competitive esports games, negative behavior, such as harassment, can create barriers to players ...
The interplay between immersion and appeal in video games
A 2í 2 study on immersion and appeal of video games is presented.Immersion and appeal are found to be highly related.Immersion is found to be unaffected by playing experience.Appeal is found to be affected both by game played and playing experience. ...






Comments