Abstract
Problem-Based Learning (PBL) is a popular approach to instruction that supports students to get hands-on training by solving problems. Question Pool websites (QPs) such as LeetCode, Code Chef, and Math Playground help PBL by supplying authentic, diverse, and contextualized questions to students. Nonetheless, empirical findings suggest that 40% to 80% of students registered in QPs drop out in less than two months. This research is the first attempt to understand and predict student dropouts from QPs via exploiting students' engagement moods. Adopting a data-driven approach, we identify five different engagement moods for QP students, which are namely challenge-seeker, subject-seeker, interest-seeker, joy-seeker, and non-seeker. We find that students have collective preferences for answering questions in each engagement mood, and deviation from those preferences increases their probability of dropping out significantly. Last but not least, this paper contributes by introducing a new hybrid machine learning model (we call Dropout-Plus) for predicting student dropouts in QPs. The test results on a popular QP in China, with nearly 10K students, show that Dropout-Plus can exceed the rival algorithms' dropout prediction performance in terms of accuracy, F1-measure, and AUC. We wrap up our work by giving some design suggestions to QP managers and online learning professionals to reduce their student dropouts.
- Najwa Alghamdi, Nora Alrajebah, and Shiroq Al-Megren. 2019. Crowd Behavior Analysis Using Snap Map: A Preliminary Study on the Grand Holy Mosque in Mecca. In Conference Companion Publication of the 2019 on Computer Supported Cooperative Work and Social Computing (Austin, TX, USA) (CSCW '19). Association for Computing Machinery, New York, NY, USA, 137--141. https://doi.org/10.1145/3311957.3359473Google Scholar
- Nese Alyuz, Eda Okur, Utku Genc, Sinem Aslan, Cagri Tanriover, and Asli Arslan Esme. 2017. An Unobtrusive and Multimodal Approach for Behavioral Engagement Detection of Students. In Proceedings of the 1st ACM SIGCHI International Workshop on Multimodal Interaction for Education (Glasgow, UK) (MIE 2017). Association for Computing Machinery, New York, NY, USA, 26--32. https://doi.org/10.1145/3139513.3139521Google Scholar
Digital Library
- Ashton Anderson, Daniel Huttenlocher, Jon Kleinberg, and Jure Leskovec. 2013. Steering User Behavior with Badges. In Proceedings of the 22nd International Conference on World Wide Web (Rio de Janeiro, Brazil) (WWW '13). Association for Computing Machinery, New York, NY, USA, 95--106. https://doi.org/10.1145/2488388.2488398Google Scholar
Digital Library
- Sinem Aslan, Nese Alyuz, Cagri Tanriover, Sinem E. Mete, Eda Okur, Sidney K. D'Mello, and Asli Arslan Esme. 2019. Investigating the Impact of a Real-Time, Multimodal Student Engagement Analytics Technology in Authentic Classrooms. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19). Association for Computing Machinery, New York, NY, USA, 1--12. https://doi.org/10.1145/3290605.3300534Google Scholar
Digital Library
- R. Babbie. 2016. The Basics of Social Research. Cengage Learning. 2015936245 https://books.google.com/books?id=0CJTCwAAQBAJGoogle Scholar
- Ryan Baker, Jason Walonoski, Neil Heffernan, Ido Roll, Albert Corbett, and Kenneth Koedinger. 2008. Why students engage in "gaming the system" behavior in interactive learning environments. Journal of Interactive Learning Research, Vol. 19, 2 (2008), 185--224.Google Scholar
- Ryan Shaun Baker. 2005. Designing intelligent tutors that adapt to when students game the system. Ph.D. Dissertation. Carnegie Mellon University Pittsburgh.Google Scholar
- Ryan S.J.d. Baker. 2007. Modeling and Understanding Students' off-Task Behavior in Intelligent Tutoring Systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI '07). Association for Computing Machinery, New York, NY, USA, 1059--1068. https://doi.org/10.1145/1240624.1240785Google Scholar
Digital Library
- Ryan Shaun Baker, Albert T. Corbett, Kenneth R. Koedinger, and Angela Z. Wagner. 2004. Off-Task Behavior in the Cognitive Tutor Classroom: When Students "Game the System". In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vienna, Austria) (CHI '04). Association for Computing Machinery, New York, NY, USA, 383--390. https://doi.org/10.1145/985692.985741Google Scholar
Digital Library
- E.F. Barkley. 2009. Student Engagement Techniques: A Handbook for College Faculty. Wiley. 2009029117 https://books.google.com/books?id=muAStyrwyZgCGoogle Scholar
- Alexander Bartel, Paula Figas, and Georg Hagel. 2015. Towards a Competency-Based Education with Gamification Design Elements. In Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play (London, United Kingdom) (CHI PLAY '15). Association for Computing Machinery, New York, NY, USA, 457--462. https://doi.org/10.1145/2793107.2810325Google Scholar
Digital Library
- Jonathan Bassen, Bharathan Balaji, Michael Schaarschmidt, Candace Thille, Jay Painter, Dawn Zimmaro, Alex Games, Ethan Fast, and John C. Mitchell. 2020. Reinforcement Learning for the Adaptive Scheduling of Educational Activities. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--12. https://doi.org/10.1145/3313831.3376518Google Scholar
- Leonard E. Baum, Ted Petrie, George Soules, and Norman Weiss. 1970. A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains. The Annals of Mathematical Statistics, Vol. 41, 1 (1970), 164--171. http://www.jstor.org/stable/2239727Google Scholar
Cross Ref
- Johann Benerradi, Horia A. Maior, Adrian Marinescu, Jeremie Clos, and Max L. Wilson. 2019. Exploring Machine Learning Approaches for Classifying Mental Workload Using FNIRS Data from HCI Tasks. In Proceedings of the Halfway to the Future Symposium 2019 (Nottingham, United Kingdom) (HTTF 2019). Association for Computing Machinery, New York, NY, USA, Article 8, bibinfonumpages11 pages. https://doi.org/10.1145/3363384.3363392Google Scholar
Digital Library
- Muhammad Ilyas Bhutto. 2011. Effects of Social Reinforcers on Students'Learning Outcomes at Secondary School Level. International Journal of Academic Research in Business and Social Sciences, Vol. 1, 2 (2011), 71.Google Scholar
- Christopher M Bishop. 2006. Machine learning and pattern recognition. Information science and statistics. Springer, Heidelberg (2006).Google Scholar
Digital Library
- Nigel Bosch. 2016. Detecting Student Engagement: Human Versus Machine. In Proceedings of the 2016 Conference on User Modeling Adaptation and Personalization (Halifax, Nova Scotia, Canada) (UMAP '16). Association for Computing Machinery, New York, NY, USA, 317--320. https://doi.org/10.1145/2930238.2930371Google Scholar
Digital Library
- Leo Breiman. 2001. Random forests. Machine Learning, Vol. 45, 1 (2001), 5--32. https://doi.org/10.1023/a:1010933404324Google Scholar
Digital Library
- Elise Cappella, Ha Yeon Kim, Jennifer W. Neal, and Daisy R. Jackson. 2013. Classroom Peer Relationships and Behavioral Engagement in Elementary School: The Role of Social Network Equity. American Journal of Community Psychology, Vol. 52, 3--4 (Oct. 2013), 367--379. https://doi.org/10.1007/s10464-013-9603-5Google Scholar
Cross Ref
- Jonathan Carlton, Andy Brown, Caroline Jay, and John Keane. 2019. Inferring User Engagement from Interaction Data. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI EA '19). Association for Computing Machinery, New York, NY, USA, 1--6. https://doi.org/10.1145/3290607.3313009Google Scholar
Digital Library
- Xunru Che, Danaë Metaxa-Kakavouli, and Jeffrey T. Hancock. 2018. Fake News in the News: An Analysis of Partisan Coverage of the Fake News Phenomenon. In Companion of the 2018 ACM Conference on Computer Supported Cooperative Work and Social Computing (Jersey City, NJ, USA) (CSCW '18). Association for Computing Machinery, New York, NY, USA, 289--292. https://doi.org/10.1145/3272973.3274079Google Scholar
- Code Chef. 2019. Code Chef. https://www.codechef.com/ Retrieved January 24, 2019 fromGoogle Scholar
- Scott Chen, Ponani Gopalakrishnan, et almbox. 1998. Speaker, environment and channel change detection and clustering via the bayesian information criterion. In Proc. DARPA broadcast news transcription and understanding workshop, Vol. 8. Virginia, USA, 127--132.Google Scholar
- Tianqi Chen and Carlos Guestrin. 2016. XGBoost: A Scalable Tree Boosting System. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (San Francisco, California, USA) (KDD '16). Association for Computing Machinery, New York, NY, USA, 785--794. https://doi.org/10.1145/2939672.2939785Google Scholar
Digital Library
- Yuanzhe Chen, Qing Chen, Mingqian Zhao, Sebastien Boyer, Kalyan Veeramachaneni, and Huamin Qu. 2016. DropoutSeer: Visualizing learning patterns in Massive Open Online Courses for dropout reasoning and prediction. In 2016 IEEE Conference on Visual Analytics Science and Technology (VAST). IEEE, 111--120.Google Scholar
Cross Ref
- Yinghan Chen, Steven Andrew Culpepper, Shiyu Wang, and Jeffrey Douglas. 2017. A Hidden Markov Model for Learning Trajectories in Cognitive Diagnosis With Application to Spatial Rotation Skills. Applied Psychological Measurement, Vol. 42, 1 (Sept. 2017), 5--23. https://doi.org/10.1177/0146621617721250Google Scholar
- Justin Cheng, Michael Bernstein, Cristian Danescu-Niculescu-Mizil, and Jure Leskovec. 2017. Anyone Can Become a Troll: Causes of Trolling Behavior in Online Discussions. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (Portland, Oregon, USA) (CSCW '17). Association for Computing Machinery, New York, NY, USA, 1217--1230. https://doi.org/10.1145/2998181.2998213Google Scholar
Digital Library
- Chia-Fang Chung, Nanna Gorm, Irina A. Shklovski, and Sean Munson. 2017. Finding the Right Fit: Understanding Health Tracking in Workplace Wellness Programs. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI '17). Association for Computing Machinery, New York, NY, USA, 4875--4886. https://doi.org/10.1145/3025453.3025510Google Scholar
Digital Library
- Evandro B. Costa, Baldoino Fonseca, Marcelo Almeida Santana, Fabrísia Ferreira de Araújo, and Joilson Rego. 2017. Evaluating the effectiveness of educational data mining techniques for early prediction of students' academic failure in introductory programming courses. Computers in Human Behavior, Vol. 73 (2017), 247--256. https://doi.org/10.1016/j.chb.2017.01.047Google Scholar
Digital Library
- M Csikszentmihalyi. 1975. Beyond boredom and anxiety. San Francisco: JosseyBass. Well-being: Thefoundations of hedonic psychology (1975), 134--154.Google Scholar
- Ryan S. J. d. Baker, Albert T. Corbett, Ido Roll, and Kenneth R. Koedinger. 2008. Developing a generalizable detector of when students game the system. User Modeling and User-Adapted Interaction, Vol. 18, 3 (Jan. 2008), 287--314. https://doi.org/10.1007/s11257-007-9045-6Google Scholar
Cross Ref
- Matt Dixon, Nalin Asanka Gamagedara Arachchilage, and James Nicholson. 2019. Engaging Users with Educational Games: The Case of Phishing. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI EA '19). Association for Computing Machinery, New York, NY, USA, 1--6. https://doi.org/10.1145/3290607.3313026Google Scholar
- Edson B. dos Santos Junior, Carlos Sim oes, Ana Cristina Bicharra Garcia, and Adriana S. Vivacqua. 2018. What Does a Crowd Routing Behavior Change Reveal?. In Companion of the 2018 ACM Conference on Computer Supported Cooperative Work and Social Computing (Jersey City, NJ, USA) (CSCW '18). Association for Computing Machinery, New York, NY, USA, 297--300. https://doi.org/10.1145/3272973.3274081Google Scholar
- Gideon Dror, Dan Pelleg, Oleg Rokhlenko, and Idan Szpektor. 2012. Churn Prediction in New Users of Yahoo! Answers. In Proceedings of the 21st International Conference on World Wide Web (Lyon, France) (WWW '12 Companion). ACM, New York, NY, USA, 829--834. https://doi.org/10.1145/2187980.2188207Google Scholar
Digital Library
- Louis Faucon, Lukasz Kidzinski, and Pierre Dillenbourg. 2016. Semi-Markov Model for Simulating MOOC Students. International Educational Data Mining Society (2016).Google Scholar
- Sarah Foley, Nadia Pantidi, and John McCarthy. 2020. Student Engagement in Sensitive Design Contexts: A Case Study in Dementia Care. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--13. https://doi.org/10.1145/3313831.3376161Google Scholar
Digital Library
- G David Forney Jr. 2005. The viterbi algorithm: A personal history. arXiv preprint cs/0504020 (2005). http://arxiv.org/abs/cs/0504020Google Scholar
- Pascal E. Fortin, Elisabeth Sulmont, and Jeremy Cooperstock. 2019. Detecting Perception of Smartphone Notifications Using Skin Conductance Responses. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19). Association for Computing Machinery, New York, NY, USA, 1--9. https://doi.org/10.1145/3290605.3300420Google Scholar
Digital Library
- Jennifer A Fredricks, Phyllis C Blumenfeld, and Alison H Paris. 2004. School Engagement: Potential of the Concept, State of the Evidence. Review of Educational Research, Vol. 74, 1 (March 2004), 59--109. https://doi.org/10.3102/00346543074001059Google Scholar
Cross Ref
- Adabriand Furtado, Nazareno Andrade, Nigini Oliveira, and Francisco Brasileiro. 2013. Contributor Profiles, Their Dynamics, and Their Importance in Five Q&a Sites. In Proceedings of the 2013 Conference on Computer Supported Cooperative Work (San Antonio, Texas, USA) (CSCW '13). Association for Computing Machinery, New York, NY, USA, 1237--1252. https://doi.org/10.1145/2441776.2441916Google Scholar
Digital Library
- W.L. Gardiner. 1974. Psychology: a story of a search .Brooks/Cole Pub. Co. 73089595 https://books.google.fr/books?id=q50bAQAAMAAJGoogle Scholar
- Nate Gruver, Ali Malik, Brahm Capoor, Chris Piech, Mitchell L Stevens, and Andreas Paepcke. 2019. Using Latent Variable Models to Observe Academic Pathways. In Proceedings of The 12th International Conference on Educational Data Mining. EDM, 294--299. http://educationaldatamining.org/edm2019/proceedings/Google Scholar
- Juho Hamari, David J. Shernoff, Elizabeth Rowe, Brianno Coller, Jodi Asbell-Clarke, and Teon Edwards. 2016. Challenging games help students learn: An empirical study on engagement, flow and immersion in game-based learning. Computers in Human Behavior, Vol. 54 (2016), 170--179. https://doi.org/10.1016/j.chb.2015.07.045Google Scholar
Digital Library
- Siti Suhaila Abdul Hamid, Novia Admodisastro, Noridayu Manshor, Azrina Kamaruddin, and Abdul Azim Abd Ghani. 2018. Dyslexia adaptive learning model: student engagement prediction using machine learning approach. In International Conference on Soft Computing and Data Mining. Springer, 372--384.Google Scholar
Cross Ref
- Mitchell M Handelsman, William L Briggs, Nora Sullivan, and Annette Towler. 2005. A measure of college student course engagement. The Journal of Educational Research, Vol. 98, 3 (2005), 184--192.Google Scholar
Digital Library
- Kotaro Hara, Abigail Adams, Kristy Milland, Saiph Savage, Chris Callison-Burch, and Jeffrey P. Bigham. 2018. A Data-Driven Analysis of Workers' Earnings on Amazon Mechanical Turk. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI '18). ACM, New York, NY, USA, Article 449, bibinfonumpages14 pages. https://doi.org/10.1145/3173574.3174023Google Scholar
- Xiaofei He, Xinbo Gao, Yanning Zhang, Zhi-Hua Zhou, Zhi-Yong Liu, Baochuan Fu, Fuyuan Hu, and Zhancheng Zhang. 2015. Intelligence Science and Big Data Engineering. Big Data and Machine Learning Techniques: 5th International Conference, IScIDE 2015, Suzhou, China, June 14--16, 2015, Revised Selected Papers. Vol. 9243. Springer.Google Scholar
- Matthew Hertz. 2010. What Do "CS1" and "CS2" Mean? Investigating Differences in the Early Courses. In Proceedings of the 41st ACM Technical Symposium on Computer Science Education (Milwaukee, Wisconsin, USA) (SIGCSE '10). Association for Computing Machinery, New York, NY, USA, 199--203. https://doi.org/10.1145/1734263.1734335Google Scholar
Digital Library
- Hayati Hind, Mohammed Khalidi Idrissi, and Samir Bennani. 2017. Applying Text Mining to Predict Learners' Cognitive Engagement. In Proceedings of the Mediterranean Symposium on Smart City Application (Tangier, Morocco) (SCAMS '17). Association for Computing Machinery, New York, NY, USA, Article 2, bibinfonumpages6 pages. https://doi.org/10.1145/3175628.3175655Google Scholar
Digital Library
- Beryl Hoffman, Ralph Morelli, and Jennifer Rosato. 2019. Student Engagement is Key to Broadening Participation in CS. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education (Minneapolis, MN, USA) (SIGCSE '19). ACM, New York, NY, USA, 1123--1129. https://doi.org/10.1145/3287324.3287438Google Scholar
Digital Library
- Jeroen Janssen and Paul A Kirschner. 2020. Applying collaborative cognitive load theory to computer-supported collaborative learning: towards a research agenda. Educational Technology Research and Development (2020), 1--23.Google Scholar
- Kenneth Joseph, Wei Wei, and Kathleen M. Carley. 2017. Girls Rule, Boys Drool: Extracting Semantic and Affective Stereotypes from Twitter. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (Portland, Oregon, USA) (CSCW '17). Association for Computing Machinery, New York, NY, USA, 1362--1374. https://doi.org/10.1145/2998181.2998187Google Scholar
- Libor Juha?ák, Ji?í Zounek, and Lucie Rohlíková. 2019. Using process mining to analyze students' quiz-taking behavior patterns in a learning management system. Computers in Human Behavior, Vol. 92 (2019), 496--506. https://doi.org/10.1016/j.chb.2017.12.015Google Scholar
Digital Library
- Jutge. 2019. jutge. https://jutge.org/ Retrieved January 24, 2019 fromGoogle Scholar
- Saskia M. Kelders and Hanneke Kip. 2019. Development and Initial Validation of a Scale to Measure Engagement with EHealth Technologies. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI EA '19). Association for Computing Machinery, New York, NY, USA, 1--6. https://doi.org/10.1145/3290607.3312917Google Scholar
- Jina Kim, Kunwoo Bae, Eunil Park, and Angel P. del Pobil. 2019. Who Will Subscribe to My Streaming Channel? The Case of Twitch. In Conference Companion Publication of the 2019 on Computer Supported Cooperative Work and Social Computing (Austin, TX, USA) (CSCW '19). Association for Computing Machinery, New York, NY, USA, 247--251. https://doi.org/10.1145/3311957.3359470Google Scholar
- Juho Kim, Philip J. Guo, Daniel T. Seaton, Piotr Mitros, Krzysztof Z. Gajos, and Robert C. Miller. 2014. Understanding In-Video Dropouts and Interaction Peaks Inonline Lecture Videos. In Proceedings of the First ACM Conference on Learning @ Scale Conference (Atlanta, Georgia, USA) ([email protected] '14). Association for Computing Machinery, New York, NY, USA, 31--40. https://doi.org/10.1145/2556325.2566237Google Scholar
- Marios Kokkodis. 2019. Reputation Deflation Through Dynamic Expertise Assessment in Online Labor Markets. In The World Wide Web Conference (San Francisco, CA, USA) (WWW '19). ACM, New York, NY, USA, 896--905. https://doi.org/10.1145/3308558.3313479Google Scholar
- George D Kuh. 2001. Assessing what really matters to student learning inside the national survey of student engagement. Change: The Magazine of Higher Learning, Vol. 33, 3 (2001), 10--17.Google Scholar
Cross Ref
- Il-Youp Kwak, Jun Ho Huh, Seung Taek Han, Iljoo Kim, and Jiwon Yoon. 2019. Voice Presentation Attack Detection through Text-Converted Voice Command Analysis. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19). Association for Computing Machinery, New York, NY, USA, 1--12. https://doi.org/10.1145/3290605.3300828Google Scholar
Digital Library
- Young D. Kwon, Dimitris Chatzopoulos, Ehsan ul Haq, Raymond Chi-Wing Wong, and Pan Hui. 2019. GeoLifecycle: User Engagement of Geographical Exploration and Churn Prediction in LBSNs. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 3, 3, Article 92 (Sept. 2019), bibinfonumpages29 pages. https://doi.org/10.1145/3351250Google Scholar
Digital Library
- D Langley. 2006. The student engagement index: A proposed student rating system based on the national benchmarks of effective educational practice. University of Minnesota: Center for Teaching and Learning Services (2006).Google Scholar
- Hyunsoo Lee, Uichin Lee, and Hwajung Hong. 2019. Commitment Devices in Online Behavior Change Support Systems. In Proceedings of Asian CHI Symposium 2019: Emerging HCI Research Collection (Glasgow, Scotland, United Kingdom) (AsianHCI '19). Association for Computing Machinery, New York, NY, USA, 105--113. https://doi.org/10.1145/3309700.3338446Google Scholar
Digital Library
- LeetCode. 2019. LeetCode. https://leetcode.com/ Retrieved January 24, 2019 fromGoogle Scholar
- Pascal Lessel, Maximilian Altmeyer, Lea Verena Schmeer, and Antonio Krüger. 2019. "Enable or Disable Gamification?": Analyzing the Impact of Choice in a Gamified Image Tagging Task. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19). Association for Computing Machinery, New York, NY, USA, 1--12. https://doi.org/10.1145/3290605.3300380Google Scholar
Digital Library
- Zipeng Liu, Zhicheng Liu, and Tamara Munzner. 2020. Data-Driven Multi-Level Segmentation of Image Editing Logs. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--12. https://doi.org/10.1145/3313831.3376152Google Scholar
Digital Library
- Suman Kalyan Maity, Aishik Chakraborty, Pawan Goyal, and Animesh Mukherjee. 2017. Detection of Sockpuppets in Social Media. In Companion of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (Portland, Oregon, USA) (CSCW '17). Association for Computing Machinery, New York, NY, USA, 243--246. https://doi.org/10.1145/3022198.3026360Google Scholar
- Jorge Maldonado-Mahauad, Mar Pérez-Sanagustín, René F. Kizilcec, Nicolás Morales, and Jorge Munoz-Gama. 2018. Mining theory-based patterns from Big data: Identifying self-regulated learning strategies in Massive Open Online Courses. Computers in Human Behavior, Vol. 80 (2018), 179--196. https://doi.org/10.1016/j.chb.2017.11.011Google Scholar
Digital Library
- Hiroaki Masaki, Kengo Shibata, Shui Hoshino, Takahiro Ishihama, Nagayuki Saito, and Koji Yatani. 2020. Exploring Nudge Designs to Help Adolescent SNS Users Avoid Privacy and Safety Threats. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--11. https://doi.org/10.1145/3313831.3376666Google Scholar
Digital Library
- Miller McPherson, Lynn Smith-Lovin, and James M Cook. 2001. Birds of a feather: Homophily in social networks. Annual review of sociology, Vol. 27, 1 (2001), 415--444.Google Scholar
- Pardis Miri, Emily Jusuf, Andero Uusberg, Horia Margarit, Robert Flory, Katherine Isbister, Keith Marzullo, and James J. Gross. 2020. Evaluating a Personalizable, Inconspicuous Vibrotactile(PIV) Breathing Pacer for In-the-Moment Affect Regulation. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--12. https://doi.org/10.1145/3313831.3376757Google Scholar
- Reza Hadi Mogavi, Sujit Gujar, Xiaojuan Ma, and Pan Hui. 2019. HRCR: Hidden Markov-Based Reinforcement to Reduce Churn in Question Answering Forums. In PRICAI 2019: Trends in Artificial Intelligence. Springer International Publishing, 364--376. https://doi.org/10.1007/978-3-030-29908-8_29Google Scholar
Cross Ref
- Michael Morgan, Matthew Butler, Neena Thota, and Jane Sinclair. 2018. How CS Academics View Student Engagement. In Proceedings of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education (Larnaca, Cyprus) (ITiCSE 2018). ACM, New York, NY, USA, 284--289. https://doi.org/10.1145/3197091.3197092Google Scholar
Digital Library
- Abdallah Moubayed, Mohammadnoor Injadat, Abdallah Shami, and Hanan Lutfiyya. 2020. Student Engagement Level in an e-Learning Environment: Clustering Using K-means. American Journal of Distance Education, Vol. 34, 2 (March 2020), 137--156. https://doi.org/10.1080/08923647.2020.1696140Google Scholar
Cross Ref
- Benjamin Murauer and Günther Specht. 2018. Detecting Music Genre Using Extreme Gradient Boosting. In Companion Proceedings of the The Web Conference 2018 (Lyon, France) (WWW '18). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 1923--1927. https://doi.org/10.1145/3184558.3191822Google Scholar
- Saurabh Nagrecha, John Z. Dillon, and Nitesh V. Chawla. 2017. MOOC Dropout Prediction: Lessons Learned from Making Pipelines Interpretable. In Proceedings of the 26th International Conference on World Wide Web Companion (Perth, Australia) (WWW '17). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 351--359. https://doi.org/10.1145/3041021.3054162Google Scholar
- Junpei Naito, Yukino Baba, Hisashi Kashima, Takenori Takaki, and Takuya Funo. 2018. Predictive modeling of learning continuation in preschool education using temporal patterns of development tests. In Thirty-Second AAAI Conference on Artificial Intelligence.Google Scholar
- Tuan Dinh Nguyen, Marisa Cannata, and Jason Miller. 2016. Understanding student behavioral engagement: Importance of student interaction with peers and teachers. The Journal of Educational Research, Vol. 111, 2 (Oct. 2016), 163--174. https://doi.org/10.1080/00220671.2016.1220359Google Scholar
- Kenji Okamoto. 2017. Analyzing Single Molecule FRET Trajectories Using HMM. In Hidden Markov Models. Springer New York, 103--113. https://doi.org/10.1007/978-1-4939-6753-7_7Google Scholar
- Judith A Ouimet and Robert A Smallwood. 2005. Assessment Measures: CLASSE--The Class-Level Survey of Student Engagement. Assessment Update, Vol. 17, 6 (2005), 13--15.Google Scholar
- Steven Pace. 2004. A grounded theory of the flow experiences of Web users. International Journal of Human-Computer Studies, Vol. 60, 3 (2004), 327--363. https://doi.org/10.1016/j.ijhcs.2003.08.005Google Scholar
Digital Library
- Zilong Pan, Chenglu Li, and Min Liu. 2020. Learning Analytics Dashboard for Problem-Based Learning. In Proceedings of the Seventh ACM Conference on Learning @ Scale (Virtual Event, USA) ([email protected] '20). Association for Computing Machinery, New York, NY, USA, 393--396. https://doi.org/10.1145/3386527.3406751Google Scholar
Digital Library
- Sira Park, Susan D. Holloway, Amanda Arendtsz, Janine Bempechat, and Jin Li. 2011. What Makes Students Engaged in Learning? A Time-Use Study of Within- and Between-Individual Predictors of Emotional Engagement in Low-Performing High Schools. Journal of Youth and Adolescence, Vol. 41, 3 (Dec. 2011), 390--401. https://doi.org/10.1007/s10964-011-9738-3Google Scholar
- Sophie Parsons, Peter M. Atkinson, Elena Simperl, and Mark Weal. 2015. Thematically Analysing Social Network Content During Disasters Through the Lens of the Disaster Management Lifecycle. In Proceedings of the 24th International Conference on World Wide Web (Florence, Italy) (WWW '15 Companion). ACM, New York, NY, USA, 1221--1226. https://doi.org/10.1145/2740908.2741721Google Scholar
Digital Library
- Filipe D. Pereira, Elaine Oliveira, Alexandra Cristea, David Fernandes, Luciano Silva, Gene Aguiar, Ahmed Alamri, and Mohammad Alshehri. 2019. Early Dropout Prediction for Programming Courses Supported by Online Judges. In Lecture Notes in Computer Science. Springer International Publishing, 67--72. https://doi.org/10.1007/978-3-030-23207-8_13Google Scholar
- Jordi Petit, Omer Giménez, and Salvador Roura. 2012. Jutge.Org: An Educational Programming Judge. In Proceedings of the 43rd ACM Technical Symposium on Computer Science Education (Raleigh, North Carolina, USA) (SIGCSE '12). ACM, New York, NY, USA, 445--450. https://doi.org/10.1145/2157136.2157267Google Scholar
Digital Library
- Math Playground. 2019. Math Playground. https://www.mathplayground.com/ Retrieved January 24, 2019 fromGoogle Scholar
- Jagat Sastry Pudipeddi, Leman Akoglu, and Hanghang Tong. 2014. User Churn in Focused Question Answering Sites: Characterizations and Prediction. In Proceedings of the 23rd International Conference on World Wide Web (Seoul, Korea) (WWW '14 Companion). ACM, New York, NY, USA, 469--474. https://doi.org/10.1145/2567948.2576965Google Scholar
Digital Library
- Jiezhong Qiu, Jie Tang, Tracy Xiao Liu, Jie Gong, Chenhui Zhang, Qian Zhang, and Yufei Xue. 2016. Modeling and Predicting Learning Behavior in MOOCs. In Proceedings of the Ninth ACM International Conference on Web Search and Data Mining (San Francisco, California, USA) (WSDM '16). Association for Computing Machinery, New York, NY, USA, 93--102. https://doi.org/10.1145/2835776.2835842Google Scholar
Digital Library
- Rezvaneh Rezapour and Jana Diesner. 2017. Classification and Detection of Micro-Level Impact of Issue-Focused Documentary Films Based on Reviews. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (Portland, Oregon, USA) (CSCW '17). Association for Computing Machinery, New York, NY, USA, 1419--1431. https://doi.org/10.1145/2998181.2998201Google Scholar
Digital Library
- Juan D Rodriguez, Aritz Perez, and Jose A Lozano. 2009. Sensitivity analysis of k-fold cross validation in prediction error estimation. IEEE transactions on pattern analysis and machine intelligence, Vol. 32, 3 (2009), 569--575.Google Scholar
- Cristó bal Romero and Sebastiá n Ventura. 2016. Educational data science in massive open online courses. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, Vol. 7, 1 (Sept. 2016), e1187. https://doi.org/10.1002/widm.1187Google Scholar
- Matthew Rowe. 2013. Mining user lifecycles from online community platforms and their application to churn prediction. In 2013 IEEE 13th International Conference on Data Mining. IEEE, 637--646.Google Scholar
Cross Ref
- Victor B. Saenz, Deryl Hatch, Beth E. Bukoski, Suyun Kim, Kye hyoung Lee, and Patrick Valdez. 2011. Community College Student Engagement Patterns. Community College Review, Vol. 39, 3 (July 2011), 235--267. https://doi.org/10.1177/0091552111416643Google Scholar
Cross Ref
- Koya Sato, Mizuki Oka, and Kazuhiko Kato. 2019. Early Churn User Classification in Social Networking Service Using Attention-Based Long Short-Term Memory. In Lecture Notes in Computer Science. Springer International Publishing, 45--56. https://doi.org/10.1007/978-3-030-26142-9_5Google Scholar
- Junfeng Shang and Joseph E. Cavanaugh. 2008. Bootstrap variants of the Akaike information criterion for mixed model selection. Computational Statistics & Data Analysis, Vol. 52, 4 (2008), 2004--2021. https://doi.org/10.1016/j.csda.2007.06.019Google Scholar
Digital Library
- David J Shernoff and Mihaly Csikszentmihalyi. 2009. Cultivating engaged learners and optimal learning environments. Handbook of positive psychology in schools (2009), 131--145.Google Scholar
- Hongzhi Shi, Chao Zhang, Quanming Yao, Yong Li, Funing Sun, and Depeng Jin. 2019. State-Sharing Sparse Hidden Markov Models for Personalized Sequences. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (Anchorage, AK, USA) (KDD '19). Association for Computing Machinery, New York, NY, USA, 1549--1559. https://doi.org/10.1145/3292500.3330828Google Scholar
Digital Library
- Hyunjin Shin, Bugeun Kim, and Gahgene Gweon. 2020. Guessing or Solving? Exploring the Use of Motion Features from Educational Game Logs. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI EA '20). Association for Computing Machinery, New York, NY, USA, 1--8. https://doi.org/10.1145/3334480.3383005Google Scholar
- Manya Sleeper, Alessandro Acquisti, Lorrie Faith Cranor, Patrick Gage Kelley, Sean A. Munson, and Norman Sadeh. 2015. I Would Like To..., I Shouldn't..., I Wish I...: Exploring Behavior-Change Goals for Social Networking Sites. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (Vancouver, BC, Canada) (CSCW '15). Association for Computing Machinery, New York, NY, USA, 1058--1069. https://doi.org/10.1145/2675133.2675193Google Scholar
Digital Library
- A. Spanos. 2019. Probability Theory and Statistical Inference: Empirical Modeling with Observational Data. Cambridge University Press. https://books.google.com/books?id=nm_IDwAAQBAJGoogle Scholar
- Ivan Srba and Maria Bielikova. 2016. A Comprehensive Survey and Classification of Approaches for Community Question Answering. ACM Trans. Web, Vol. 10, 3, Article 18 (Aug. 2016), 63 pages. https://doi.org/10.1145/2934687Google Scholar
Digital Library
- Cor JM Suhre, Ellen PWA Jansen, and Egbert G Harskamp. 2007. Impact of degree program satisfaction on the persistence of college students. Higher Education, Vol. 54, 2 (2007), 207--226.Google Scholar
Cross Ref
- Stephanie D. Teasley. 2017. Student Facing Dashboards: One Size Fits All? Technology, Knowledge and Learning, Vol. 22, 3 (01 Oct 2017), 377--384. https://doi.org/10.1007/s10758-017-9314-3Google Scholar
Cross Ref
- Jacob Thebault-Spieker, Anbang Xu, Jilin Chen, Jalal Mahmud, and Jeffrey Nichols. 2016. Exploring Engagement in a 'Social Crowd' on Twitter. In Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work and Social Computing Companion (San Francisco, California, USA) (CSCW '16). Association for Computing Machinery, New York, NY, USA, 417--420. https://doi.org/10.1145/2818052.2869112Google Scholar
Digital Library
- Timus. 2019. Timus Online Judge. https://acm.timus.ru/ Retrieved January 24, 2019 fromGoogle Scholar
- Gustavo F. Tondello, Dennis L. Kappen, Marim Ganaba, and Lennart E. Nacke. 2019. Gameful Design Heuristics: A Gamification Inspection Tool. In Human-Computer Interaction. Perspectives on Design. Springer International Publishing, 224--244. https://doi.org/10.1007/978-3-030-22646-6_16Google Scholar
- Gustavo F. Tondello, Rina R. Wehbe, Lisa Diamond, Marc Busch, Andrzej Marczewski, and Lennart E. Nacke. 2016. The Gamification User Types Hexad Scale. In Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play (Austin, Texas, USA) (CHI PLAY '16). Association for Computing Machinery, New York, NY, USA, 229--243. https://doi.org/10.1145/2967934.2968082Google Scholar
Digital Library
- Richard Tucker. 2016. Collaboration and Student Engagement in Design Education .IGI Global.Google Scholar
- T. Vafeiadis, K.I. Diamantaras, G. Sarigiannidis, and K.Ch. Chatzisavvas. 2015. A comparison of machine learning techniques for customer churn prediction. Simulation Modelling Practice and Theory, Vol. 55 (2015), 1--9. https://doi.org/10.1016/j.simpat.2015.03.003Google Scholar
Cross Ref
- Wil Van Der Aalst. 2011. Process mining: discovery, conformance and enhancement of business processes. Vol. 2. Springer.Google Scholar
- Devin Waldrop, Amy L. Reschly, Kathleen Fraysier, and James J. Appleton. 2018. Measuring the Engagement of College Students: Administration Format, Structure, and Validity of the Student Engagement InstrumenttextendashCollege. Measurement and Evaluation in Counseling and Developmen , Vol. 52, 2 (Nov. 2018), 90--107. https://doi.org/10.1080/07481756.2018.1497429Google Scholar
- Danding Wang, Qian Yang, Ashraf Abdul, and Brian Y. Lim. 2019 b. Designing Theory-Driven User-Centric Explainable AI. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19). Association for Computing Machinery, New York, NY, USA, 1--15. https://doi.org/10.1145/3290605.3300831Google Scholar
- Qianwen Wang, Yao Ming, Zhihua Jin, Qiaomu Shen, Dongyu Liu, Micah J. Smith, Kalyan Veeramachaneni, and Huamin Qu. 2019 a. ATMSeer: Increasing Transparency and Controllability in Automated Machine Learning. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19). Association for Computing Machinery, New York, NY, USA, 1--12. https://doi.org/10.1145/3290605.3300911Google Scholar
Digital Library
- Wei Wang, Han Yu, and Chunyan Miao. 2017. Deep Model for Dropout Prediction in MOOCs. In Proceedings of the 2nd International Conference on Crowd Science and Engineering (Beijing, China) (ICCSE'17). Association for Computing Machinery, New York, NY, USA, 26--32. https://doi.org/10.1145/3126973.3126990Google Scholar
Digital Library
- Vicky Ward, Allan House, and Susan Hamer. 2009. Developing a framework for transferring knowledge into action: a thematic analysis of the literature. Journal of health services research & policy, Vol. 14, 3 (2009), 156--164.Google Scholar
Cross Ref
- Szymon Wasik, Maciej Antczak, Jan Badura, Artur Laskowski, and Tomasz Sternal. 2016. Optil.Io: Cloud Based Platform For Solving Optimization Problems Using Crowdsourcing Approach. In Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work and Social Computing Companion (San Francisco, California, USA) (CSCW '16). Association for Computing Machinery, New York, NY, USA, 433--436. https://doi.org/10.1145/2818052.2869098Google Scholar
Digital Library
- Szymon Wasik, Maciej Antczak, Jan Badura, Artur Laskowski, and Tomasz Sternal. 2018. A Survey on Online Judge Systems and Their Applications. ACM Comput. Surv., Vol. 51, 1, Article 3 (Jan. 2018), bibinfonumpages34 pages. https://doi.org/10.1145/3143560Google Scholar
Digital Library
- Jacob Whitehill, Kiran Mohan, Daniel Seaton, Yigal Rosen, and Dustin Tingley. 2017. MOOC Dropout Prediction: How to Measure Accuracy?. In Proceedings of the Fourth (2017) ACM Conference on Learning @ Scale (Cambridge, Massachusetts, USA) ([email protected] '17). Association for Computing Machinery, New York, NY, USA, 161--164. https://doi.org/10.1145/3051457.3053974Google Scholar
Digital Library
- Meng Xia, Mingfei Sun, Huan Wei, Qing Chen, Yong Wang, Lei Shi, Huamin Qu, and Xiaojuan Ma. 2019. PeerLens: Peer-inspired Interactive Learning Path Planning in Online Question Pool. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19). ACM, New York, NY, USA, Article 634, bibinfonumpages12 pages. https://doi.org/10.1145/3290605.3300864Google Scholar
Digital Library
- Wanli Xing and Dongping Du. 2018. Dropout Prediction in MOOCs: Using Deep Learning for Personalized Intervention. Journal of Educational Computing Research, Vol. 57, 3 (March 2018), 547--570. https://doi.org/10.1177/0735633118757015Google Scholar
- Yuan Xuan, Yang Chen, Huiying Li, Pan Hui, and Lei Shi. 2016. LBSNShield: Malicious Account Detection in Location-Based Social Networks. In Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work and Social Computing Companion (San Francisco, California, USA) (CSCW '16). Association for Computing Machinery, New York, NY, USA, 437--440. https://doi.org/10.1145/2818052.2869094Google Scholar
Digital Library
- Carl Yang, Xiaolin Shi, Luo Jie, and Jiawei Han. 2018b. I Know You'll Be Back: Interpretable New User Clustering and Churn Prediction on a Mobile Social Application. In Proceedings of the 24th ACM SIGKDD (London, United Kingdom) (KDD '18). ACM, New York, NY, USA, 914--922. https://doi.org/10.1145/3219819.3219821Google Scholar
Digital Library
- Yang Yang, Zongtao Liu, Chenhao Tan, Fei Wu, Yueting Zhuang, and Yafeng Li. 2018a. To Stay or to Leave: Churn Prediction for Urban Migrants in the Initial Period. In Proceedings of the 2018 World Wide Web Conference (Lyon, France) (WWW '18). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 967--976. https://doi.org/10.1145/3178876.3186144Google Scholar
Digital Library
- Zamzami Zainuddin, Samuel Kai Wah Chu, Muhammad Shujahat, and Corinne Jacqueline Perera. 2020. The impact of gamification on learning and instruction: A systematic review of empirical evidence. Educational Research Review (2020), 100326.Google Scholar
- Jiawei Zhang, Xiangnan Kong, and S Yu Philip. 2016. Social badge system analysis. In 2016 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM). IEEE, 453--460.Google Scholar
Digital Library
- Xi Zhang, Shan Jiang, and Yihang Cheng. 2017. Inferring the Student Social Loafing State in Collaborative Learning with a Hidden Markov Model: A Case on Slack. In Proceedings of the 26th International Conference on World Wide Web Companion (Perth, Australia) (WWW '17). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 149--152. https://doi.org/10.1145/3041021.3054145Google Scholar
- Sabrina Zirkel, Julie A Garcia, and Mary C Murphy. 2015. Experience-sampling research methods and their potential for education research. Educational Researcher, Vol. 44, 1 (2015), 7--16.Google Scholar
Cross Ref
Index Terms
Characterizing Student Engagement Moods for Dropout Prediction in Question Pool Websites
Recommendations
Predicting Student Dropout in a MOOC: An Evaluation of a Deep Neural Network Model
ICCAI '19: Proceedings of the 2019 5th International Conference on Computing and Artificial IntelligenceMassive Open Online Courses (MOOCs) have transformed the way educational institutions deliver high-quality educational material to the onsite and distance learners across the globe. As a result, a new paradigm shifts as to how learners acquire and ...
Student engagement in massive open online courses
Completion rates in massive open online courses MOOCs are disturbingly low. Existing analysis has focused on patterns of resource access and prediction of drop-out using learning analytics. In contrast, the effectiveness of teaching programs in ...
MOOC Dropout Prediction: How to Measure Accuracy?
[email protected] '17: Proceedings of the Fourth (2017) ACM Conference on Learning @ ScaleIn order to obtain reliable accuracy estimates for automatic MOOC dropout predictors, it is important to train and test them in a manner consistent with how they will be used in practice. Yet most prior research on MOOC dropout prediction has measured ...






Comments