Oral Exams in Shift to Remote Learning

ABSTRACT

In this experience report paper we present our experience with the development of oral assessments as final examinations in three introductory computing courses. The choice of this type of summative assessment was prompted by the emergency remote instruction instituted in the middle of the Spring 2020 semester, across colleges and universities in the U.S., due to the coronavirus pandemic. The principles that guided our oral assessment design were: to develop a more comprehensive measure of student competence and mitigate exam cheating; to facilitate communication and workplace skills through student-teacher interaction; and to alleviate negative emotions associated with traditional summative assessments.

We report on the oral assessment features and logistics. To gain insights into the impact of this form of assessment, we conducted a student survey to learn about their emotional reactions and perceptions of assessment effectiveness. Mean scores of positive emotions (enjoyment, hope, pride, relief) were higher than negative emotions (anger, anxiety, hopelessness) across all three courses. Students found the personalized, interactive nature of the exam helpful in advancing their learning and communication skills. Many believed the oral exam to be a more accurate assessment of their knowledge than traditional methods. Additionally, feedback from the two instructors who implemented the oral assessments indicates that they enjoyed the experience and will use the lessons learned to improve the use of oral assessments in the future.

References

1. Lubna S. Alam. 2004. Is plagiarism more prevalent in some forms of assessment than others. In Proceedings of the 21st Australian Society for Computers in Tertiary Education Conference (ASCILITE'04). 48--57.
2. Jess Bidgood and Jeremy B. Merrill. 2017. As computer coding classes swell, so does cheating. New York Times. https://www.nytimes.com/2017/05/29/us/computerscience-cheating.html
3. Martin Dick. 2005. Student Interviews as a Tool for Assessment and Learning in a Systems Analysis and Design Course. In Proceedings of the 10th Annual SIGCSE Conference on Innovation and Technology in Computer Science Education (Caparica, Portugal) (ITiCSE '05). Association for Computing Machinery, New York, NY, USA, 24--28. https://doi.org/10.1145/1067445.1067456
4. Martin Dick, Judy Sheard, Cathy Bareiss, Janet Carter, Donald Joyce, Trevor Harding, and Cary Laxer. 2002. Addressing Student Cheating: Definitions and Solutions. In Working Group Reports from ITiCSE on Innovation and Technology in Computer Science Education (Aarhus, Denmark) (ITiCSE-WGR '02). Association for Computing Machinery, New York, NY, USA, 172--184. https://doi.org/10. 1145/960568.783000
5. Philip J. East and Ben J. Schafer. 2005. In-Person Grading: An Evaluative Experiment. In Proceedings of the 36th SIGCSE Technical Symposium on Computer Science Education (St. Louis, Missouri, USA) (SIGCSE '05). Association for Computing Machinery, New York, NY, USA, 378--382. https://doi.org/10.1145/1047344.1047472
6. Anna Eckerdal, Robert McCartney Jan Erik, Moström, Kate Sanders, Lynda Thomas, and Carol Zander. 2007. From Limen to Lumen: Computing Students in Liminal Spaces. In Proceedings of the Third International Workshop on Computing Education Research (Atlanta, Georgia, USA) (ICER '07). Association for Computing Machinery, New York, NY, USA, 123--132. https://doi.org/10.1145/1288580. 1288597
7. J. Helliwell, R. Layard, and J. Sachs. 2018. World Happiness Report. https: //worldhappiness.report/. Sustainable Development Solutions Network. New York.
8. Mark Huxham, Fiona Campbell, and Jenny Westwood. 2012. Oral versus written assessments: a test of student performance and attitudes. Assessment & Evaluation in Higher Education 37 (2012), 125--136.
9. P. Iannone and A. Simpson. 2012. Oral assessment in mathematics: implementation and outcomes. Teaching Mathematics and its Applications: An International Journal of the IMA 31, 4 (10 2012), 179--190. https:// doi.org/10.1093/teamat/hrs012 arXiv:https://academic.oup.com/teamat/articlepdf/31/4/179/4762864/hrs012.pdf
10. Päivi Kinnunen and Beth Simon. 2012. My program is ok -- am I? Computing freshmen?s experiences of doing programming assignments. Computer Science Education 22, 1 (March 2012), 1--28. https://doi.org/10.1080/08993408.2012.655091
11. Paul Laskowski, Sergey Karayev, and Marti A. Hearst. 2018. How Do Professors Format Exams? An Analysis of Question Variety at Scale. In Proceedings of the Fifth Annual ACM Conference on Learning at Scale (London, United Kingdom) (L@S '18). Association for Computing Machinery, New York, NY, USA, Article 54, 10 pages. https://doi.org/10.1145/3231644.3231667
12. Alex Lishinski, Aman Yadav, and Richard Enbody. 2017. Students? Emotional Reactions to Programming Projects in Introduction to Programming: Measurement Approach and Influence on Learning Outcomes. In Proceedings of the 2017 ACM Conference on International Computing Education Research (Tacoma, Washington, USA) (ICER '17). Association for Computing Machinery, New York, NY, USA, 30--38. https://doi.org/10.1145/3105726.3106187
13. David B. Lowe and Craig A. Scott. 1996. Reflections on a Novel Approach to Teaching Software Development. In Proceedings of the 7th Australasian Association for Engineering Education Annual Congress and Conference (Sydney, Australia). Australasian Association for Engineering Education, Barton, Australia.
14. David B. Lowe, Craig A. Scott, and R. Bagia. 1998. Engineering Students Throw Away Their Calculators. In Proceedings of the 9th Australasian Association for Engineering Education Annual Congress and Conference (Gladstone, Australia). Australasian Association for Engineering Education, Barton, Australia.
15. Peter Ohmann. 2019. An Assessment of Oral Exams in Introductory CS. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education (Minneapolis, MN, USA) (SIGCSE '19). Association for Computing Machinery, New York, NY, USA, 613--619. https://doi.org/10.1145/3287324.3287489
16. Reinhard Pekrun, Thomas Goetz, Anne C. Frenzel, Petra Barchfeld, and Raymond P. Perry. 2011. Measuring emotions in students? learning and performance: The Achievement Emotions Questionnaire (AEQ). Contemporary Educational Psychology 36, 1 (2011), 36--48. https://doi.org/10.1016/j.cedpsych.2010.10.002 Students? Emotions and Academic Engagement.
17. Eric Roberts. 2002. Strategies for promoting academic integrity in CS courses. In 32nd Annual Frontiers in Education, Vol. 2. F3G--F3G.
18. WHO Statistics. [n.d.]. World Health Data Platform. https://www.who.int/ healthinfo/statistics/en/.
19. Brenda Cantwell Wilson and Sharon Shrock. 2001. Contributing to Success in an Introductory Computer Science Course: A Study of Twelve Factors. SIGCSE Bull. 33, 1 (Feb. 2001), 184--188. https://doi.org/10.1145/366413.364581
20. Benjamin Yu, George Tsiknis, and Meghan Allen. 2010. Turning Exams into a Learning Experience (SIGCSE '10). Association for Computing Machinery, New York, NY, USA, 336--340. https://doi.org/10.1145/1734263.1734380
21. Yitong Zhao. 2018. Impact of Oral Exams on a Thermodynamics Course Performance. In 2018 ASEE Zone IV Conference. ASEE Conferences, Boulder, Colorado. https://peer.asee.org/29617.

Index Terms

• Social and professional topics

  • Professional topics

    • Computing education

      • Student assessment