Abstract
Video games, in addition to representing an extremely relevant field of entertainment and market, have been widely used as a case study in artificial intelligence for representing a problem with a high degree of complexity. In such studies, the investigation of approaches that endow player agents with the ability to retrieve relevant information from game scenes stands out, since such information can be very useful to improve their learning ability. This work proposes and analyses new deep learning-based models to identify game events occurring in Super Mario Bros gameplay footage. The architecture of each model is composed of a feature extractor convolutional neural network (CNN) and a classifier neural network (NN). The extracting CNN aims to produce a feature-based representation for game scenes and submit it to the classifier, so that the latter can identify the game event present in each scene. The models differ from each other according to the following elements: the type of the CNN; the type of the NN classifier; and the type of the game scene representation at the CNN input, being either single frames, or chunks, which are n-sequential frames (in this paper 6 frames were used per chunk) grouped into a single input. The main contribution of this article is to demonstrate the greater performance reached by the models which combines the chunk representation for the game scenes with the resources of the classifier recurrent neural networks (RNN).
- N. Aloysius and M. Geetha. 2017. A review on deep convolutional neural networks. In 2017 International Conference on Communication and Signal Processing (ICCSP). 0588--0592. https://doi.org/10.1109/ICCSP.2017.8286426Google Scholar
Cross Ref
- Leonard A. Annetta. 2008. Video Games in Education: Why They Should Be Used and How They Are Being Used. Theory Into Practice (2008). https://doi.org/10.1080/00405840802153940Google Scholar
- Elizabeth Boyle, Thomas M. Connolly, and Thomas Hainey. 2011. The role of psychology in understanding the impact of computer games. Entertainment Computing 2, 2 (2011), 69--74. https://doi.org/10.1016/j.entcom.2010.12.002 Serious Games Development and Applications.Google Scholar
Cross Ref
- Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. arXiv:1406.1078 [cs.CL]Google Scholar
- Jiyang Gao, Zhenheng Yang, and Ram Nevatia. 2017. RED: Reinforced Encoder-Decoder Networks for Action Anticipation. arXiv:1707.04818 http://arxiv.org/abs/1707.04818Google Scholar
- Roeland De Geest, Efstratios Gavves, Amir Ghodrati, Zhenyang Li, Cees Snoek, and Tinne Tuytelaars. 2016a. Online Action Detection. arXiv:1604.06506 [cs.CV]Google Scholar
- Roeland De Geest, Efstratios Gavves, Amir Ghodrati, Zhenyang Li, Cees Snoek, and Tinne Tuytelaars. 2016b. Online Action Detection. CoRR abs/1604.06506. arXiv:1604.06506 http://arxiv.org/abs/1604.06506Google Scholar
- Global Data. 2021. Video games market set to become a 300bn-plus industry by 2025. https://www.globaldata.com/video-games-market-set-to-become-a-300bn-plus-industry-by-2025.Google Scholar
- Matthew Guzdial, Boyang Li, and Mark O. Riedl. 2017. Game Engine Learning from Video. In Proceedings of the 26th International Joint Conference on Artificial Intelligence (Melbourne, Australia) (IJCAI'17). AAAI Press, 3707--3713.Google Scholar
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
Cross Ref
- Sepp Hochreiter. 1998. The Vanishing Gradient Problem During Learning Recurrent Neural Nets and Problem Solutions. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems 6 (04 1998), 107--116. https://doi.org/10.1142/S0218488598000094Google Scholar
Digital Library
- Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-Term Memory. Neural Computation 9, 8 (1997), 1735--1780. https://doi.org/10.1162/neco.1997.9.8.1735 arXiv:https://doi.org/10.1162/neco.1997.9.8.1735Google Scholar
Digital Library
- J J Hopfield. 1982. Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences 79 (1982), 2554--2558. https://doi.org/10.1073/pnas.79.8.2554Google Scholar
Cross Ref
- V. Janarthanan. 2012. Serious Video Games: Games for Education and Health. In 2012 Ninth International Conference on Information Technology - New Generations. https://doi.org/10.1109/ITNG.2012.79Google Scholar
- Michael I. Jordan. 1997. Chapter 25 - Serial Order: A Parallel Distributed Processing Approach. 121 (1997), 471--495. https://doi.org/10.1016/S0166-4115(97)80111-2Google Scholar
- S. Karakovskiy and J. Togelius. 2012. The Mario AI Benchmark and Competitions. IEEE Transactions on Computational Intelligence and AI in Games 4, 1 (2012), 55--67. https://doi.org/10.1109/TCIAIG.2012.2188528Google Scholar
Cross Ref
- Andrej Karpathy, George Toderici, Sanketh Shetty, Thomas Leung, Rahul Sukthankar, and Li Fei-Fei. 2014. Large-Scale Video Classification with Convolutional Neural Networks. In 2014 IEEE Conference on Computer Vision and Pattern Recognition. 1725--1732. https://doi.org/10.1109/CVPR.2014.223Google Scholar
- Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings.Google Scholar
- Michail D. Kozlov and Mark K. Johansen. 2010. Real Behavior in Virtual Environments: Psychology Experiments in a Simple Virtual-Reality Paradigm Using Video Games. Cyberpsychology, Behavior, and Social Networking (2010). https://doi.org/10.1089/cyber.2009.0310Google Scholar
- Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. ImageNet Classification with Deep Convolutional Neural Networks. In Advances in Neural Information Processing Systems 25, F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger (Eds.). Curran Associates, Inc., 1097--1105.Google Scholar
Digital Library
- Kangwook Lee, Hoon Kim, and Changho Suh. 2017. Crash To Not Crash: Playing Video Games To Predict Vehicle Collisions. In ICML 2017.Google Scholar
- Zijin Luo, Matthew Guzdial, Nicholas Liao, and Mark Riedl. 2018. Player Experience Extraction from Gameplay Video. CoRR abs/1809.06201 (2018). arXiv:1809.06201Google Scholar
- Zijin Luo, Matthew Guzdial, and Mark Riedl. 2019. Making CNNs for Video Parsing Accessible. CoRR abs/1906.11877 (2019). arXiv:1906.11877Google Scholar
- M. Ravanbakhsh, M. Nabi, E. Sangineto, L. Marcenaro, C. Regazzoni, and N. Sebe. 2017. Abnormal event detection in videos using generative adversarial nets. In 2017 IEEE International Conference on Image Processing (ICIP). 1577--1581. https://doi.org/10.1109/ICIP.2017.8296547Google Scholar
Digital Library
- Johanna Roettl and Ralf Terlutter. 2018. The same video game in 2D, 3D or virtual reality - How does technology impact game evaluation and brand placements? PLOS ONE 13, 7 (07 2018), 1--24. https://doi.org/10.1371/journal.pone.0200724Google Scholar
- Mark Sandler, Andrew G. Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. 2018. Inverted Residuals and Linear Bottlenecks: Mobile Networks for Classification, Detection and Segmentation. CoRR abs/1801.04381 (2018). arXiv:1801.04381Google Scholar
- David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap, Karen Simonyan, and Demis Hassabis. 2018. A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science 362, 6419 (2018), 1140--1144. https://doi.org/10.1126/science.aar6404Google Scholar
- Karen Simonyan and Andrew Zisserman. 2014. Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv 1409.1556 (09 2014).Google Scholar
- Khurram Soomro, Amir Zamir, and Mubarak Shah. 2012. UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild. CoRR (12 2012).Google Scholar
- Adam Summerville, Sam Snodgrass, Matthew Guzdial, Christoffer Holmgård, Amy K. Hoover, Aaron Isaksen, Andy Nealen, and Julian Togelius. 2017. Procedural Content Generation via Machine Learning (PCGML). CoRR abs/1702.00539 (2017). arXiv:1702.00539 http://arxiv.org/abs/1702.00539Google Scholar
- Jeremy Heng Meng Wong and Mark John Francis Gales. 2016. Sequence Student-Teacher Training of Deep Neural Networks. In INTERSPEECH. https://doi.org/10.21437/Interspeech.2016-911Google Scholar
- Mingze Xu, Mingfei Gao, Yi-Ting Chen, Larry S. Davis, and David J. Crandall. 2018. Temporal Recurrent Networks for Online Action Detection. arXiv:1811.07391 http://arxiv.org/abs/1811.07391Google Scholar
- Manzhu Yu, Myra Bambacus, Guido Cervone, Keith Clarke, Daniel Duffy, Qunying Huang, Jing Li, Wenwen Li, Zhenlong Li, Qian Liu, Bernd Resch, Jingchao Yang, and Chaowei Yang. 2020. Spatiotemporal event detection: a review. International Journal of Digital Earth 13, 12 (2020), 1339--1365. https://doi.org/10.1080/17538947.2020.1738569 arXiv:https://doi.org/10.1080/17538947.2020.1738569Google Scholar
Cross Ref
Index Terms
Investigating the Performance of Various Deep Neural Networks-based Approaches Designed to Identify Game Events in Gameplay Footage
Recommendations
Research on advertising content recognition based on convolutional neural network and recurrent neural network
The problem to be solved in this paper is to identify the text advertisement information published by users in a medium-sized social networking website. First, the text is segmented and then the text is transformed into sequence tensor by using a word ...
Synthesizing Game Levels for Collaborative Gameplay in a Shared Virtual Environment
We developed a method to synthesize game levels that accounts for the degree of collaboration required by two players to finish a given game level. We first asked a game level designer to create playable game level chunks. Then, two artificial ...
Chinese Text Feature Extraction and Classification Based on Deep Learning
CSAE '19: Proceedings of the 3rd International Conference on Computer Science and Application EngineeringWith the rapid development of deep learning, neural networks have been widely used in natural language processing tasks and achieved good results. Since convolutional neural networks can acquire high-level features that can better represent textual ...






Comments