Learning is Not a Spectator Sport: Doing is Better than Watching for Learning from a MOOC

ABSTRACT

The printing press long ago and the computer today have made widespread access to information possible. Learning theorists have suggested, however, that mere information is a poor way to learn. Instead, more effective learning comes through doing. While the most popularized element of today's MOOCs are the video lectures, many MOOCs also include interactive activities that can afford learning by doing. This paper explores the learning benefits of the use of informational assets (e.g., videos and text) in MOOCs, versus the learning by doing opportunities that interactive activities provide. We find that students doing more activities learn more than students watching more videos or reading more pages. We estimate the learning benefit from extra doing (1 SD increase) to be more than six times that of extra watching or reading. Our data, from a psychology MOOC, is correlational in character, however we employ causal inference mechanisms to lend support for the claim that the associations we find are causal.

References

1. Aleven, V., Beal, C. R., & Graesser, A. C. (2013). Introduction to the special issue on advanced learning technologies. Journal of Educational Psychology, 105(4), 929--931.
2. Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995) . Cognitive tutors:Lessons learned. The Journal of the Learning Sciences, 4, 167--207.
3. Anzai, Y., & Simon, H. A. (1979). The theory of learning by doing. Psychological Review, 86, 124--140.
4. Barab, S.A. & Squire, K.D. (2004). Design-based research: Putting a stake in the ground. Journal of the Learning Science, 13 (1) 1--14.
5. Bichsel, J. (2013). The State of E-Learning in Higher Education: An Eye toward Growth and Increased Access (Research Report), Louisville, CO: EDUCAUSE Center for Analysis and Research. Available online: http://www.educause.edu/ecar.
6. Bowen, W. G., Chingos, M. M., Lack, K. A., & Nygren, T. I. (2013). Interactive learning online at public universities: Evidence from a six-campus randomized trial. Journal of Policy Analysis and Management, 33(1), 94--111.
7. Champaign, J., Fredericks, C., Colvin, K., Seaton, D., Liu, A. & Pritchard, D. Correlating skill and improvement in 2 MOOCs with a student's time on task. In Proc. [email protected] 2014, Retrieved from: http://dx.doi.org/10.1145/2556325.2566250.
8. Clark, R.E., Feldon, D., van Merriënboer, J., Yates, K., & Early, S. (2007). Cognitive task analysis. In J.M. Spector, M.D. Merrill, J.J.G. van Merriënboer, & M.P. Driscoll (Eds.), Handbook of research on educational communications and technology (3rd ed., pp. 577--593). Mahwah, NJ: Lawrence Erlbaum Associates.
9. Collins, E.D. (2013) "SJSU Plus augmented online learning environment: Pilot project report." The Research and Planning Group for California Community Colleges. Available online:http://www.sjsu.edu/chemistry/People/Faculty/Collins_Research_Page/AOLE%20Report%20Final%20Version_Jan%201_2014.pdf
10. Collins, A., Brown, J. S., & Newman, S. E. (1989). Cognitive apprenticeship: Teaching the crafts of reading, writing, and mathematics. In L. B. Resnick. Knowing, Learning, and Instruction: Essays in Honor of Robert Glaser (pp. 453--494). Hillsdale, NJ: Erlbaum.
11. Dewey, J. (1916), (2007 edition). Democracy and Education, Teddington: Echo Library.
12. Hill, P. (2013). Emerging Student Patterns in MOOCs: A (Revised) Graphical View. e-literate. Available online: http://mfeldstein.com/emerging-student-patterns-in-moocs-a-revised-graphical-view/
13. Kaufman, J.; Ryan, R.; Thille, C. and Bier, N. (2013) Open Learning Initiative Courses in Community Colleges: Evidence on Use and Effectiveness. Mellon University, Pittsburgh, PA. Available online: http://www.hewlett.org/sites/default/files/CCOLI_Report_Final_1.pdf
14. Koedinger, K.R., & Aleven V. (2007). Exploring the assistance dilemma in experiments with Cognitive Tutors. Educational Psychology Review, 19(3), 239--264.
15. Koedinger, K.R., Baker, R.S.J.d., Cunningham, K., Skogsholm, A., Leber, B., Stamper, J. (2010). A Data Repository for the EDM community: The PSLC DataShop. In Romero, C., Ventura, S., Pechenizkiy, M., Baker, R.S.J.d. (Eds.) Handbook of Educational Data Mining. Boca Raton, FL: CRC Press.
16. Koedinger, K.R., Booth, J. L., & Klahr, D. (2013). Instructional Complexity and the Science to Constrain It. Science, 342(6161), 935--937.
17. Koedinger, K. R., Corbett, A. C., & Perfetti, C. (2012). The Knowledge-Learning-Instruction (KLI) framework: Bridging the science-practice chasm to enhance robust student learning. Cognitive Science, 36 (5), 757--798.
18. Koedinger, K. R., McLaughlin, E. A., & Stamper, J. C. (2014). Ubiquity symposium: MOOCs and technology to advance learning and learning research: Data-driven learner modeling to understand and improve online learning. Ubiquity, Volume 2014, Number May (2014), Pages 1--13. ACM New York, NY, USA. DOI:10.1145/2591682
19. Koedinger, K. R., & Sueker, E. L. F. (2014). Monitored design of an effective learning environment for algebraic problem solving. Technical report Carnegie Mellon University-HCII-14-102.
20. Lovett, M., Meyer, O., Thille, C. (2008). The Open Learning Initiative: Measuring the effectiveness of the OLI learning course in accelerating student learning. Journal of Interactive Media in Education. http://jime.open.ac.uk/2008/14/.
21. Meyer, O., & Lovett, M. C. (2002). Implementing a computerized tutor in a statistical reasoning Course: Getting the big picture. In B. Phillips (Ed.) Proc. of the Sixth International Conference on Teaching Statistics.
22. Morote, E. & Pritchard, D. E. (2002). What Course Elements Correlate with Improvement on Tests in Introductory Newtonian Mechanics? National Association for Research in Science Teaching -- NARST- 2002 Conference.
23. Ng, A. and Widom J. (2014). Origins of the Modern MOOC. http://www.cs.stanford.edu/people/ang/papers/mooc14-OriginsOfModernMOOC.pdf
24. Pane, J.F., Griffin, B., McCaffrey, D.F. & Karam, R. (2014). Effectiveness of Cognitive Tutor Algebra I at Scale. Educational Evaluation and Policy Analysis, 36 (2), 127--144.
25. Pappano, L. (2012, November 2). The year of the MOOC. The New York Times.
26. Pashler, H., Bain, P., Bottge, B., Graesser, A., Koedinger, K., McDaniel, M., & Metcalfe, J. (2007). Organizing Instruction and Study to Improve Student Learning (NCER 2007-2004). Washington, DC: National Center for Education Research, Institute of Education Sciences, U.S. Department of Education.
27. Ritter S., Anderson, J. R., Koedinger, K. R., & Corbett, A. (2007). Cognitive tutor: Applied research in mathematics education. Psychonomic Bulletin & Review, 14 (2):249--255.
28. Schank, R. C., Berman, T. R. & Macperson, K. A. (1999). Learning by doing. In C. M. Reigeluth (Ed.), Instructional Design Theories and Models: A New Paradigm of Instructional Theory (Vol. II) (pp.161--181). Mahwah, NJ: Lawrence Erlbaum Associates.
29. Singer, S.R. & Bonvillian, W.B. (2013). Two Revolutions in Learning. Science 22, Vol. 339 no. 6126, p.1359.
30. Sinha, T., Jermann, P., Li, N., Dillenbourg, P. (2014). Your click decides your fate: Inferring Information Processing and Attrition Behavior from MOOC Video Clickstream Interactions. Proc. of the 2014 Empirical Methods in Natural Language Processing Workshop on Modeling Large Scale Social Interaction in Massively Open Online Courses.
31. Strader, R. & Thille, C. (2012). The Open Learning Initiative: Enacting Instruction Online. In Oblinger, D.G. (Ed.) Game Changers: Education and Information Technologies (201--213). Educause.
32. Tetrad IV. http://www.phil.cmu.edu/tetrad.
33. Thille, C. (2014). Ubiquity symposium: MOOCs and technology to advance learning and learning research: opening statement. Ubiquity, Volume 2014, Number April (2014), Pages 1--7. ACM New York, NY, USA. DOI: 10.1145/2601337.
34. VanLehn, K. (2006). The behavior of tutoring systems. International Journal of Artificial Intelligence in Education, 16(3), 227--265.
35. VanLehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46(4), 197--221.
36. Zhu X., Lee Y., Simon H.A., & Zhu, D. (1996). Cue recognition and cue elaboration in learning from examples. In Proc. of the National Academy of Sciences 93, (pp. 1346--1351).

Index Terms

• Applied computing

  • Education