One Solution to Addressing Assessment Logistical Problems: An Experience Setting Up and Operating an In-person Testing Center

To address the challenges of running exams in large enrollment CS courses, we set up and operated an in-person testing center at a minority serving institution. We have run the testing center for two quarters, proctoring over 6,000 exams for eight CS courses with approximately 1,800 students. In this experience report, we discuss the motivation for the testing center, its set-up and operation, and the lessons that we have learned from our first two quarters of operation. In addition, we present student and instructor feedback regarding use of the testing center, future steps, and improvements. By sharing, we hope that other schools can learn from our experience and improve upon our methods to help establish best practices for testing center configuration and operation.


INTRODUCTION
In the past decade, computer science departments nationwide have seen enrollment growth, both in terms of majors as well as non-majors taking CS classes [4,5].Our school is no different.Undergraduate enrollment campuswide has risen from 18,942 in 2013 to 23,161 in 2022 (a 22% increase), with engineering programs (including CS) growing from 2,362 to 3,797 (a 60% increase) in the same time period.This growth has increased the number of courses offered, increased the size of current courses, and limited the availability of classrooms.This growth has made summative assessment particularly challenging.For trustworthiness, our departmental norm is for exams to be performed in person, where they can be proctored, but enrollment growth has made finding suitable space difficult.The classrooms assigned to courses are typically only just large enough to seat the course, meaning that in-class exams would have students seated close enough that they could see each other's work.In general, larger rooms are not available during the day, and evening exams are logistically challenging for our student population.Our university is a minority serving institution and a large percentage of students have jobs or family obligations outside of school and a significant fraction of our students commute to campus.
In addition, during the pandemic many faculty shifted from paperbased exams to digital exams.Digital exams present many advantages, including saving time and money from not printing exams, saving time from not having to scan exams for online grading, facilitating auto-grading, and faster feedback to students.In contexts where digital exams are suitable, we find that they benefit both course staff and students, but the logistics of "bring your own device" (BYOD) exams are challenging.In addition to students needing reliable laptops, the rooms are not sufficiently provisioned for power that all students could plug in their laptops for a BYOD exam, resulting in additional complications when inevitably some students run out of power during the exam.
In a large course, there are invariably students that have conflicts with testing dates or times, due to illness, student sports/activities, and other personal circumstances.These conflicts cause a significant administrative burden to schedule and proctor conflict exams for these students.
Proctoring exams for a large class typically involves the participation of the course's teaching assistants (TAs) as proctors.This is suboptimal in two respects.First, proctoring to ensure that students don't cheat doesn't require the course-specific expertise of TAs, whose time is better spent teaching sections and helping struggling students.Second, we want our students to be comfortable approaching TAs to ask for help, and putting the TAs in charge of policing exams potentially jeopardizes the student-TA relationship.In addition, it is inefficient to train the TAs as proctors if they are only going to do it a few times during the semester.
Finally, when exams take place during scheduled lecture time, they take away from the contact hours that can be used to teach the material.If that time can be freed up, the material can be covered more slowly, with additional examples and activities, to ensure that even the least prepared students can be successful.
To address the above concerns, we implemented a dedicated inperson computer testing center [6,14].While most institutions have testing centers that focus on proctoring exams for a group of students with the instructor's consent (e.g., make-up and conflict exams, and students with special needs), our testing center is aimed to support entire classes (barring those with special accommodations that our center currently cannot accommodate).To do this, exams are run asynchronously, where students use a web-based scheduling tool to choose from many exam start times over several days so they can find a time that works with their schedule, therefore eliminating most scheduling conflicts.Using this framework, students from different classes may concurrently take exams, allowing the use of a relatively small room to handle large classes and accommodate students' availability.
Having a dedicated testing space addresses other exam issues.The center gives students privacy and space during an exam.Since the center has dedicated computers, no additional power outlets are required and all students are guaranteed to have a working computer.Computer-based exams decrease paper and ink usage and allow for more autograding.Exams are proctored by trained proctors whom the students do not have a pedagogical relationship with.Computer-based exams facilitate the use of question pools and parameterized items, enabling the creation of a unique test for each student.
Running auto-graded exams in the dedicated testing center greatly reduces the effort by the course to run an exam.This reduction in effort, coupled with the fact that exams no longer take up lecture time, facilitates a shift in assessment strategy from a few, high-stakes exams to smaller, more frequent low-stakes exams.More frequent testing benefits students [2,11], especially the ones in traditionally under-represented groups [1].Morphew et al. [9] found that students who completed frequent short assessments over the course of the semester scored higher on the final exam than students who completed a few longer mid-term examinations.Students also prefer shorter, more frequent exams [8] and find they reduce test anxiety [7].
In this paper, we describe our experience setting up (Section 2) and running (Section 3) the testing center.In Section 4, we share lessons learned.We then present feedback from students and instructors (Section 5) and our next steps (Section 6) before concluding (Section 7).

SETTING UP THE TESTING CENTER
Zilles et al. [15] suggest that any testing center must have some basic components: a dedicated space, a web-based scheduling tool, a tool for administering exams, and staff to operate and run the center.In this section, we discuss the choices we made while setting up these components.
To meet our secure assessment goals, we wanted a designated secure room, which led us to repurpose a space previously allocated for scheduled labs in CS courses.The acquisition of this room was made possible by a recent departmental decision to move CS lab courses from traditional computer labs to regular campus classrooms.This decision was motivated by three reasons.First, the growth of the course sizes exceeded the capacity of the computer labs and therefore the computer labs were no longer able to host all scheduled lab sections.Second, the college made a requirement for all CS major students to own a laptop meeting a minimum level of abilities.For non-major students, the campus offers a laptop checkout program where students can borrow a laptop for the day.Third, the department was able to make course specific Docker containers for each course so students were able to replicate necessary computer software and configurations on any computer.Since the acquired room for the testing center was assigned to the CS department, we were able to open it only during the testing hours and keep it locked and secured otherwise.
With the help of departmental IT staff, we were able to repurpose 29 computers from old computer labs to the new testing center.We assigned 28 computers for general use and 1 computer was saved for students who require additional testing time.Unlike prior work [16], we decided to have our center only offer accommodations for additional time.For other accommodations, students would be directed to have exams proctored by the Student Resource Center.
We decided upon a configuration of computers on tables placed against the four walls of the room with all monitors facing the center of the room.This configuration allows the proctors to easily see all screens from the middle of the room.Each computer station was separated by cardboard desk dividers.A desk with a proctor computer was placed in the middle facing the entry to the room.Figure 1 shows this room setup.
We wanted to make sure that students were unable to save files for other students or share information between testing sessions.Therefore, IT staff wrote a script that could be run either as a system process that runs every hour or manually when proctors need to reset computers.This script makes sure that all saved files and cached login information are removed from the machine.With the physical setup complete, we needed to decide what scheduling and testing tools we would use.To handle student exam scheduling, we chose the scheduling system PrairieTest [10], a choice driven by its user-friendly interface enabling students to make their own reservations, the ability to handle students with testing accommodations, and its existing proctorrun check-in/check-out process.PrairieTest, while serving as a great scheduling and proctoring tool, necessitated a complementary platform for hosting the computer-based exams.
One straightforward solution was to use PrairieLearn, an online platform for assessments designed to facilitate mastery-based learning [12], which has seamless integration with PrairieTest.PrairieLearn supports the auto-grading of randomized numerical, symbolic, drawing and programming questions [13].Using randomization and question pools has shown to mitigate cheating during asynchronous exams [3].However, a substantial number of instructors within our institution have developed an extensive repository of questions on our campus Learning Management System (LMS) -Instructure's Canvas -during the transition to online learning due to the Covid-19 pandemic.Canvas allows random question selection from question pools which would also create a unique exam for each student.Transferring this database of questions from Canvas to PrairieLearn would have required a considerable time investment, so we decided to accept both PrairieLearn and Canvas.To enhance the flexibility of the testing center, we also allowed instructors to use the space to offer traditional paper exams for their courses.
PrairieLearn exams become accessible to students only after they check-in at the testing center.Canvas allows for IP filtering, confining exam access solely to computers situated within the designated testing center.Paper exams are printed and stored in the testing center.Proctors distribute exams to students at the start of the exam session and collect them upon completion.These completed exams are stored securely, either inside lidded boxes or sealed envelopes, making sure other students do not have access to these exams.
To make the use of the testing center easier, the facilitator was tasked with creating "How To" documentation.The facilitator created documentation for future facilitators, course instructors, testing center proctors, and students.The different documents describe the steps necessary for each role to interact with the testing center and PrairieTest, the testing center scheduling tool.

LOGISTICS OF RUNNING THE CENTER
The job of the testing center facilitator is to make sure the center runs smoothly.This requires the facilitator to communicate with instructors, proctors, and students throughout the quarter.For these first quarters of operation, a member of the teaching faculty served as the center's facilitator.In this section, we will describe the tasks completed by the facilitator.
Before the start of the quarter, the facilitator reaches out to instructors to see who will be using the testing center and when they plan to have the exams open.The facilitator calculates how many exams will be administered each week and must verify that the number of exams does not exceed more than 70% of the available time slots.70% was used so that students would have plenty of choice of time slots.Sometimes meeting this goal requires working with course instructors to move exams earlier or later in the week.
Once the exam schedule is finalized, the facilitator adds the exam schedule to the scheduling tool.For each exam this consists of a beginning and ending of the range of days the exam is available and the day that students can start making reservations.We opted to have all reservations available at 8am on the Friday before the week of the exam.We did this to make sure that all students, independent of course, had an equal opportunity to make an exam reservation.When making reservations, students can choose any time slot that isn't full during the range of days the exam is run.Students can change their reservation at any time up until their scheduled exam time.
Before the first exam slot, the facilitator must hire and schedule proctors.Like previous work [15], we used two proctors in the testing center for each time slot.The idea behind this was, if an issue arises, one proctor can deal with that situation while the other proctor can continue to watch students.We chose to hire CS M.S. students who did not previously have a position as a TA or tutor at the university.As we were figuring out software and logistics, we thought graduate students would be able to learn the software and point out improvements.They also would have the maturity to create best practices.If they had not previously held a TA or tutor position, they would have no connection to students taking exams.
Proctors attended a training session so they could be introduced to the testing process.The testing process starts when students arrive at the testing center.Proctors check photo ID, check students in on PrairieTest, and tell students the computer they have been randomly assigned by PrairieTest.After the computers finish resetting, students can login.Our school requires two factor authentication in which students read a code from their phone.After logging in, students must put their phone in their backpack or pocket.No electronics are allowed on the desk and students are asked to remove earbuds, watches, hoodies, and hats.If students need a calculator, they are allowed to use the calculator on the computer.If students need scratch paper, they may take a blank piece of paper but must return it before leaving the testing center.During the exam, proctors continually walk the room checking that no additional tabs have been open and students are focused on their own exam.If proctors see something that is out of the norm, they can reach out to the facilitator via Slack and the facilitator can communicate information to the instructor.
During the quarter, the facilitator makes sure everything is set up correctly for instructors.The facilitator reaches out to an instructor a few days before an exam is open for reservations just to make sure nothing has changed and that they are okay with their students beginning to schedule their exam time slot.The facilitator also monitors a testing center email inbox in case any student or instructor has any question during the quarter.The majority of emails received on this email address have been from students who have missed their timeslot and need the facilitator to cancel their missed reservation so they can reschedule, or instructors who have allowed an exception and need the facilitator to enter the exception into PrairieTest so the student can schedule a time slot outside of the normal exam period.
The facilitator also makes sure that the testing center is functioning properly.Since the proctors are in the testing center, they are the ones who see any issues and need to tell the facilitator.We decided the best way of communication was via a Slack channel where proctors could contact the facilitator and IT staff quickly.Some of the common issues that we ran across were proctors becoming ill and not being able to attend their scheduled shift or a piece of hardware, like a keyboard, mouse, or monitor not working correctly.Once reported, the facilitator resolves the issues such as testing equipment and finding a replacement if necessary or finding another trained proctor to cover the missed shift.
The facilitator is responsible for distributing and updating the documentation.Based on feedback from various testing center roles, the facilitator is expected to improve the documentation and build best practices guidelines to help future users with onboarding.So far, the proctor and student sections in our documentation have seen the most improvement.For example, we have added new instructions for students who miss their scheduled slot.As we are adding more features and flexibility to the testing center, we want to make sure proctors are trained based on these new best practices.

LESSONS LEARNED
Operating the testing center for the first two quarters has been a very insightful experience.Although we based this center off previous work [15], that paper did not cover many day to day issues that arose.In this section, we will discuss six lessons that we learned.
The first lesson was regarding students' behavior scheduling exams.Prior work [15] noted that students often procrastinate in taking the exam by scheduling a slot on the last day.We observed the same behavior at our testing center.However, what was surprising is how late students would wait to make a reservation for a time slot.Students were allowed to make reservations starting the Friday before the week the exam was run.However, the facilitator and instructors would get many emails from students who waited until the night before or even the day of the last day of the exam window to make a reservation, only to find out that there were no available reservations.Course instructors were advised to remind students twice to make a reservation.The first reminder would be when the window to schedule exams opens with particular wording regarding scheduling early.The second reminder would be one or two days into the testing window with particular wording to schedule very soon.The second lesson was regarding scheduling exams so the testing center is utilized better.Figure 2 shows the utilization of time slots for the days active during one quarter.All exams were scheduled to end on Friday.Note, Monday of week 9 was a school holiday so exams for week 9 were due Monday of week 10.This figure shows that, because of the scheduling behavior of students to wait to take the exam until close to the due date, Friday time slots were typically full while Monday time slots were often widely available.This resulted in students not being able to make a new reservation if they missed a reservation earlier in the week or waited too late to make a reservation.The best practice would be to have courses stagger exam end periods, so the most desirable time slots for different courses would be staggered.This would bring the testing center closer to its goal of 70% usage.
The third lesson was preparing for unexpected technical issues.At first, we allowed up to 29 reservations for each time slot.We tried this as a way to maximize available reservations for the hours the testing center was open.This is only a problem if there is a technical issue with one of the machines.The best practice would be to move a student over to an available machine.However, when a time slot is completely booked, there is no available machine to move students to.We began to only allow a maximum of 26 reservations per time slot so there would be unused machines if necessary.The fourth lesson was the amount of time required by the facilitator to communicate with instructors new to the testing center.Each instructor has a unique testing philosophy and wants the testing center to be able to do different things.Because the testing center was brand new, we were more than willing to try to accommodate different wants from different instructors but it did take time and coordination by the facilitator.It also took a great deal of communication between the facilitator and instructors.The facilitator sends emails to instructors before the start of each quarter to gather their expectations.The facilitator also reaches out to instructors during the quarter to see if there are any changes to the testing plan.
The fifth lesson learned was the great importance of the proctors.The success of the testing center is in the hands of the proctors.If instructors and students are not confident in the results and integrity of exams administered by the testing center, the instructors will not use the center.However, it requires a unique blend of characteristics to be a good proctor.A good proctor must be a person who is personable, respectful of the different needs of students, punctual, able to solve problems quickly, and confident when speaking with students when they are not on task.Because they are vitally important to the success of the center, constant communication with the proctors is paramount.Slack was used for urgent communication whenever something unexpected happened.For less urgent communication, the facilitator would send out weekly messages informing proctors of what courses had exams that week, what media they would be using, any special allowances like a cheat sheet, and the total number of students expected.
The sixth lesson learned was how dirty the testing center got.Because of its high utilization and because exams are relatively short, a large number of students pass through the testing center every day.While in the test center, proctors should throw away any trash left behind.For more detailed cleaning, proctors should be assigned a cleaning task to complete each week.For example, a given proctor might be tasked by cleaning stations 1 through 5.
Cleaning should include dusting the computer including keyboard and mouse, wiping down the computer monitor, and wiping down the table.Some students also requested that a machine be sterilized before they begin their exam.Cleaning supplies including paper towels, towels, cleaner, anti-bacterial wipes, and hand sanitizer are stored in the testing center.Having them readily available means proctors and students can clean whenever they feel the room is not up to their standards or whenever they are scheduled to clean.

FEEDBACK FROM STUDENTS AND INSTRUCTORS
During the quarter, we surveyed students regarding their experience using the testing center.We asked students about the ease of scheduling an appointment, if they liked the flexibility of the testing center, and what they thought could be improved.Figure 3 shows the results of that survey.When asked "It was easy to make a reservation", over 90% of students agreed or strongly agreed with the statement.Some written comments we received were "It's nice and simple", "It's easy to schedule and reschedule if necessary", and "It's very accessible and the process is done with ease".There were some comments about not liking the additional step of having to schedule the exam, but we did not get any negative feedback about the scheduler or the ease of scheduling an exam.
We asked students if they liked the flexibility of scheduling their own time.84% of students said they agreed or strongly agreed with the statement "I liked the flexibility of using the test center".Some written comments that we received about the flexibility were "The system works great because I can work around my schedule" and "The test center is convenient because you are able to take it any time that you wish to do so".Negative feedback we got regarding the flexibility focused around having to do an extra step to schedule outside of classroom hours.We received a few comments like "I don't like having to schedule exams".
We asked students to tell us what they would improve about the testing center.There was not actually a lot of feedback regarding improvement of the center.Most students answered this question focused on the exam itself.We got many comments regarding how extra time would be beneficial or proctors should help explain questions being asked on the exam.The few comments we did get about improvement to the testing center were things like "the distance between people is too close", "the test center should be cleaned more often", and "some time slots should be different, starting at 12:30 or 1:30 instead of 12 and 1".
We have held discussions with instructors regarding their thoughts on using the testing center.We received very positive feedback.One instructor using paper exams liked the fact that, after the initial setup, he did not have to worry about getting a room for the exam and scheduling people to proctor it.He felt the testing center eased his overall workload.Another instructor using Canvas said that he liked the experience of using the testing center so well that he would be expanding to more of his courses.He also applauded the testing center staff for its quick response to emails and issues that arose.An instructor using PrairieLearn exams liked being able to test her students with smaller and more frequent assessments.This helped her identify students who might need additional help earlier in the term.

FUTURE STEPS AND IMPROVEMENTS
Setting up the testing center these past two quarters has been an educational experience.However, as the project scales up and lessons are learned, we do have a list of future steps and improvements.Some improvements will happen before the start of the next academic year while others are long term goals.
The first improvement will be to modify the current layout of the room.The current layout limits us to less than 29 reservations at a given time.As the usage of the testing center grows, the capacity of the testing center must grow too.Therefore, we will be moving the tables with computers into rows with all monitors facing the back of the room.The rows will have walkways between tables so proctors can walk around desks.With this configuration, we can add an additional 16 machines and 8 tables to bring the total number of student computers to 45.We will also be adding privacy screens to each monitor so taller students will not be able to view the monitor in front of them.
The next improvement is to add filters to limit web access during exams.Currently, proctors are able to see all the screens and can easily monitor the small number of students in the room.As the testing center size increases, monitoring all screens at the same time will become more challenging.We feel that this is a necessary step to maintain the integrity of the center.
Currently, all documentation is maintained in Google docs and distributed as PDF files.The files are quite long and require some time to navigate to the section that the user might need right then.Another problem is the documentation must be redistributed anytime a change is made.We have decided that creating a web page would be a better way to maintain this documentation.Users can navigate to the website and gather the information to answer their question with just a few clicks.The website will also link students to PrairieTest to make exam reservations.
We plan on shifting from hiring CS graduate students as proctors to undergraduate students.Our graduate students are interested in obtaining a TA position because then the school will cover their tuition and most of their fees on top of their salary, a much better financial position for them.However, this means that our proctors are unwilling to commit to a proctor position until after TA positions have been assigned, as much as a week into the quarter.Also, during the quarter, they are often looking for research or grader positions that align more with their career goals.This makes scheduling a challenge as the proctor's availability is changing during the quarter.Since hiring CS undergraduate students would be a challenge because they might be enrolled in a course using the testing center, we will be focusing on hiring undergraduate non-CS majors.
Eventually, the testing center will outgrow its current location.As more courses use the testing center and as current courses grow in numbers, it will not be possible to administer all the exams in one location.We have identified another lab room that we can convert to a second testing center.

CONCLUSIONS
We decided to set up an in-person computer-based testing facility based on prior work [16].This paper is a summary of our experience setting up and operating an in-person testing center.We hope that our discussion on how we used a testing center to help address some of the issues regarding assessments at our institution, setup of the center, the logistics of operating the center, lessons that we have learned from our first two quarters of operation, student and instructor feedback regarding using the testing center, and future steps and improvement will be helpful when other schools are facing similar assessment challenges.We hope that by documenting our experience these schools will have a defined process to follow.

Figure 1 :
Figure 1: First configuration of the testing center

Figure 2 :
Figure 2: Time Slot Utilization by Day of Week

Figure 3 :
Figure 3: Student's testing center experience survey results