Improving Knowledge of CS1 Terminology Through a Peer Reviewed Translation Activity: Results and Feedback

This paper describes a study in which a computer terminology translation activity was implemented in a CS1 class using a specially designed web application. Students translated an English technical term into their native language and wrote an accompanying definition of the term. These completed translations were then reviewed anonymously by their peers. Finally, students voted on the most highly rated translations to compose a glossary of translations. These activities were developed to prompt students to enhance their understanding of English computing terms, thereby aiding their academic pursuits. The activity, along with the web based application, was tested in a CS1 class comprising 90 non-native English speaking students. Additionally, a significant number of students voluntarily participated in a computer terminology test that had been administered to previous cohorts within the same class. Among the students who engaged in the translation activity, the average performance on this test was 8% higher. At the conclusion of the class, a survey was conducted to capture students' perceptions of the activity. Out of the total students, 67 completed the survey. The feedback received indicated that the majority of students recognized the advantageous impact of the translation activity on their programming learning. A considerable number of students expressed a keen interest in participating in another translation activity during their upcoming courses.


INTRODUCTION
There has been a significant expansion in the number of Englishmedium-instruction (EMI) programs taught in universities in nonnative English-speaking (NNES) countries [7] where in computer science, especially programming courses, the English language is particularly dominant.This is not simply an artefact of the courses' medium of instruction, but a reflection of the available materials, such as textbooks, language references and documentation, technical forums etc. [12].Much of these materials contain domainspecific terminology that can be ambiguous and difficult to comprehend, especially for NNES students [11,15].Additionally, all of the most popular and widely used programming languages contain keywords derived from English, which can present greater difficulty for NNES students [20].The meaning of some English terminology can change based on the domain and context in which they are used.This can cause confusion for NNES students especially when the available translations are poor and inaccurate [15].The difficulties students face in comprehending such terms have been found to impact their learning performance negatively [2,14].
The principle motivation of this study, which is part of a larger investigation on this topic, is to introduce and evaluate a peerreview activity along with an application supporting it that aims to improve students' knowledge of computer terminology in the CS1 course.In order to evaluate the efficacy of the tool as well as to assess avenues for improvement, a feedback survey that includes close and open-ended questions was created and responses were collected from the students.The main aim of the activity described in this paper was to achieve the following learning goals: • Improve student knowledge of computer terminology.
• Improve student critical thinking and problem-solving skills.
• Increase learning motivation and engagement in CS1 class.

TERMINOLOGY AND PEER-LEARNING 2.1 CS-Specialised Terminology Lists
A number of researchers have produced Computer Science (CS) word lists in an effort to help identify the relevant technical terminology for CS students.Minshall [22] developed the first specialised word list for the most frequently used terms in computer science which contained 433 single words and 23 multi-words.However, the sources used to generate this list were high-level CS materials including journals and conference papers.Another word list containing 356 words created by Bi [3] for CS students using textbooks covering different CS topics but excluding vocabularies that are expected to be known by Chinese undergraduate students.A more comprehensive CS academic word list that contains 904 words was provided by Roesler [26] along with a supplementary list containing 702 words.The sources used to build the two lists are journal articles and textbooks covering 10 different CS sub-disciplines.Alaofi and Russell [1] developed the first subject-specific word list in CS.This was the CS1 word list which included 110 single words and 13 multi-words compiled from introductory programming learning materials.This was designed to capture the technical terminology that would be essential within a CS1 class.
Students will often be encountering these terms for the first time and as such be learning them in two languages simultaneously [8].Specialised dictionaries might be considered an obvious resource for students to learn such terminology, however, those dictionaries are usually focused on formal terminology not subject field terms that undergraduates mostly need [9,25].In many studies, the lack of translated learning materials was one of the main obstacles faced by learners in the CS field [11,12].Additionally, the available translations for technical terms used in this type of material are usually of poor quality, inaccurate or ambiguous which cause more confusion for students [15].

Academic Performance and CS Terminology
Students learning complex concepts such as programming in a language other than their native language face heightened difficulties [4,11].Simultaneously learning a foreign language while also learning new terminologies, syntax, and developing the ability to code places a high cognitive load on students [28].Domain-specific terminology (such as computer terminology) are often not correctly understood by language educators and as a result are typically not learned in language classes [23].This adds extra pressure on NNES students, who are forced to learn terminology alongside their topic of study.
One of the suggested ways to support NNES students in CS is to offer a special course in CS terminology.The results of an empirical study by Chan [4] show a positive impact on the learning outcomes of students in an introductory information technology class when they were pre-taught technical vocabulary.However, applying this strategy involves a lot of preparation by teachers and consumes a lot of class time.
In a study by Keen and Etzkorn [14], the analysis showed that the average score in components of a class correlated with the density of "buzzwords" or technical terminology within the corresponding notes.This was measured within classes of native and non-native English speakers and shows that technical terms are not only a problem for non-native students.In two separate studies focusing on NNES students, knowledge of computer terminology was found to correlate with academic performance in introductory CS courses [2,21].
The computer science specialist vocabulary test (CSSVT) was created based on the first CS word list [22].The performance of 106 NNES students on the CSSVT was shown to correlate with academic performance in a computer science course delivered in English.The computer terminology test (CTT) is a validated instrument for assessing CS1 students' knowledge of computing terminology and was developed based on the CS1 word list [2].The performance of 150 NNES students on the CTT was shown to correlate with academic performance in a CS1 course delivered in English.

Peer-Review
Peer review or peer feedback is where students evaluate the work or performance of their peers and provide constructive feedback in relation to class-related activity [5].The feedback can be in the form of summative (e.g.score or grade) or formative (e.g.strengths and weakness without scoring) or both [31].For peer review to be more useful and effective, the students need to receive some form of training or guidance including examples of work and review criteria [31].
Besides its importance in enhancing the students' knowledge and skill-building in the specific-domain subject [30,31], peer review has many other benefits.From a wider perspective, peer review activities can help students develop critical thinking and problemsolving skills as well as positive impact on learning motivation and engagement [13,29].Peer review activities can also help students to develop many important skills that are required for future professional life such as following and developing evaluation criteria, the ability to fairly and objectively judge other people's work as well as the ability to self-assess themselves and accept and benefit from other people's reviews to improve their own work [24].
Peer review activities can be applied in a face-to-face form, which is the traditional method, or via different online means (e.g.learning management systems, mobile applications, blogs) [31].In recent years, online peer review has become the most attractive form for educators as it is believed it can bring better results regards learning achievements and thinking skills [24].The use of technology in peer feedback activities makes it more enjoyable and engaging for students [31].It also makes participation more flexible where students can complete the review at any time and anywhere [18,31].The most important aspect of using technology in peer review is the easy application of anonymity for both authors and reviewers.This has been proven to be more effective as the students will be more comfortable giving honest and critical feedback which increases the positive learning outcome of the activity [17].

Collaborative Translation Activities
One area of research that is particularly relevant to this study is English for Specific Purposes (ESP).Research in this area found that translation activity is beneficial in terminology learning including retention and accuracy of learned terms [6].While the area has a track record of peer learning and review translation activities [16], there is little research into tools that enable these practices.
The closest research in this area is a mobile application, named CoLecTer KIN, which can be used to collaboratively build a database of common terminology [25].This application was designed using the MIT App Inventor by researchers within the area of English language instruction.As a result, it is quite basic and functions by adding created entries to a shared Google spreadsheet.While students could access the sheet directly and see the entries made by their peers, there was no mechanism for the students to provide feedback on the translations.

THE ACTIVITY DESIGN
The terminology translation activity applied in this experiment includes three sub-activities, (1) translation, (2) review and (3) voting.In the first sub-activity, the teacher will assign one term to each student to translate and ask them to provide a short description or explanation for it in their native language.A number of terms are prepared in advance to be used in this activity.The English definitions for the terms are not provided within the tool to encourage students to create their own descriptions of the term.
The second sub-activity has two separate review activities.First, each translation is reviewed by two of the teaching assistants (TAs) from the class.This provides a baseline that can be used to evaluate the reviews of the students and allows the selection of the better translations for use in the later voting phase.Secondly, after all submitted translations are reviewed by TAs, each translation is assigned to a number of other students for review.Students are asked to select one of the options provided for review (inadequate, adequate, good or excellent).A short description for each of the reviews' options is provided for more accurate reviews.The same criteria are used for both TA and student reviews.The students are also encouraged to add constructive feedback in the form of a comment or explanation for their selected review option but this is not mandatory.These comments can be viewed by the student who submitted the translation after the review process ends to help improve their future submissions.
The third and last sub-activity is voting on the best translation for each term.At this stage, a list of all translations that have been nominated by teaching assistants is used.Each student will be asked to vote on the best translations for the terms that they reviewed.The result of the voting sub-activity will be used to build the course glossary that will be made available for all students in the class.
The activity is designed such that each student will interact with all of the terms.If there are 4 terms in the translation activity, a student will be randomly assigned one of them to translate and will be assigned a translation of each of the other three terms to review in the next sub-activity.In order to encourage the students to participate in the activity, a small portion of the course grade (4%) was contributed by all of the components in the translation activity.

THE TOOL DESIGN
The translation activity tool which enables this research is designed as a web application that can be used through a compatible learning management system (LMS).For an description of the implementation of the tool see [27].The design and implementation of the tool are based on the learning tools interoperability (LTI) specifications 1 .The LTI specifications allow LMS or platforms to integrate remote tools and content in a standard way [19].By implementing this application as a tool provider, it is possible for it to be integrated into any compatible LMS 2 .The primary objective of this tool is to simplify the process of using and managing peer-review translation activities in a CS1 course.
Using this tool the teacher can add a list of terms that will be used for the translation activity, randomly assign these terms to students, and then assign translated terms to TAs to review before distributing the translations to students for peer review.In this experiment, the instructor of the class does not speak the native 1 https://www.imsglobal.org/spec/lti/v1p3 2 The tool was only thoroughly tested with the Moodle LMS language of the students but the TAs do, which makes them play an essential role in reviewing submitted translations.
Access to all parts of the activity is enabled online through the LMS.When students open the translation activity, they see the term they have been assigned to translate as well as inputs to allow them to enter the translation and description.When students open the review activity, they see a list of translated terms that have been assigned for them to review.Selecting one of these translations enables the student to perform their review and provide feedback or comments about the translation.Similarly, when students open the review activity they can see a list of terms available for voting.Selecting one of these terms allows the students to rank all of the selected translations of the term.

SELECTED TERMS AND TIMING
The terms chosen for use in this activity were based on a lexical analysis of the course lecture notes in a manner similar to the generation of the various word lists discussed in Section 2.1.The topics taught in the course while the experiment was being conducted were the basic fundamentals of programming and the syntax of Python, choice (if statements), loops, lists, user defined functions, and modular design.As the class progressed, terms were chosen from the content that had most recently been covered.The activity was completed three times (A, B, & C), each with a total of 4 terms used.The lists of the terms used are (A) Literal Value, Source Code, Program, String; (B) Condition, Statement, Expression, Operator; (C) Sequence, Parameter, Index, Method.In this experiment, all terms that appeared in the CTT [2] were removed from consideration to prevent biasing the subsequent analysis.
For each use of the intervention, students were given one week to complete their translations and reviews were required to be completed in the following week.The voting sub-activity was combined for all three lists of terms and was completed after all reviews were complete.
The CTT was used as a post-test to explore the impact of the translation activity on the students' knowledge of computer terminology.The test has been made available to students to complete in the middle of the semester while the translation activity continued until the end of the semester.This decision was based on the timing of the same test in previous years for the control group.The goal was to make the CTT results more comparable for the two groups.

STUDENTS' FEEDBACK SURVEY
At the end of the semester, the students were invited to participate in an online feedback survey about the translation activity.The first question was a multiple-choice question with 8 statements.The goal of this question is to explore if the students agree with the expected benefits of the activity.For each statement, the students can choose one of the five options (strongly agree, agree, not sure, disagree, strongly disagree).The eight statements will be presented along with their results in the following section.
The multiple-choice question was followed by three open-ended questions to ask the students about what they liked or disliked about the activity as well as their suggestions to improve it.The qualitative data collected through open-ended questions were analysed using the inductive coding method.The first author generated themes and categorised all responses then the results were reviewed by the second author.No personal or demographic information was collected in this survey.

THE EXPERIMENT APPROACH
This research adopts a quasi-experimental approach to evaluate the effectiveness of the designed activity and tool 3 .In this case, it was not possible to have randomised experimental and control groups.As a consequence, the experimental group is defined as the entire class from the 22/23 academic year ( = 90), where the translation activity was performed, and the control group used for comparison was the students in the same class from the 20/21 and 21/22 academic years ( = 178).The only difference in the course across these two groups was the addition of the translation activity.The students in the class consist only of students in their first semester of a software engineering degree program and there were no changes to admission policies across these years.
The CTT does not have any randomisation and could not be applied both before and after the intervention.As such, a post-testonly design was used with the CTT completed voluntarily by the students after the intervention.

Participants
The experimental group included 90 NNES students enrolled in the CS1 course at a higher educational institution in China in the academic year 22/23.Out of these 90 students, 64 completed the CTT and 67 participated in the feedback survey.The control group is 178 students who attended the exact same CS1 course at the same institution in the previous two academic years (20/21 & 21/22).Out of these 178 students, only 83 completed the CTT.

RESULTS AND DISCUSSION 8.1 Results of the CTT
The result presented here is based on 64 participants from the experimental group and 83 from the control group who voluntarily completed the CTT.This test was administered at the same period in the course to all groups.The boxplot in Figure 1 compares the data collected for the two groups with regard to their performance in the CTT.The mean in the experimental group is clearly higher and the smaller size of the box shows that there was less variance in the results.Both groups also show some mild outliers in the data.
Moreover, by looking at the detailed descriptive summary in Table 1, all indicators showed higher numbers in favour of the experimental group.The average score in the experimental group was higher by approximately 8% with a lower standard deviation.The median was also showing very similar figures to the average, especially in the experimental group.The maximum score was also higher in the experimental group by 7% while the minimum shows a 10% difference.Based on the values of Q1 and Q3 it can be said that half of the students in the control group scored between 56.7 and 76.7 with an IQR of 20%.Whereas half of the students in the experimental group scored between 66.7 and 82.47 with an IQR of 15.7% indicating less variance among students' performance. 3The researchers received ethical exemptions for the study conducted in this paper with reference: LS-E-20-129-Alaofi-Russell from University College Dublin.A two-sample t-test was conducted to investigate the significance of the difference in the mean of the two groups.It is worth mentioning here that the sample size is sufficiently large to assume normality ( ≥ 15) and mild outliers are not expected to impact t-test results as well [10].The result of the two-sample t-test shows a significant difference in mean CTT score between the control and experimental groups ( (143.937)= -3.527, = 0.045).The negative  value indicates that the CTT scores of the experimental group were higher than the control group.This significant difference in the students' performance in CTT can be interpreted as a positive improvement in their knowledge of computer terminology as a result of participating in the peer-review translation activity.

Student Feedback
The first question in the feedback survey asked the students if they agree or disagree with a number of statements about participation in the translation activity.The options range from strongly agree to strongly disagree with a natural (not sure) option in the middle.For the sake of simplicity, the analysis combines the results of agree and strongly agree together as well as disagree and strongly disagree (Figure 2).In general, over 80% of the surveyed students agreed on the benefit of the translation activity on their learning process of programming.All parts of the activity including the translation, peer review and voting received highly positive feedback from the students.
Looking at the result of each statement it can be seen that 85% of the students agreed or strongly agreed that the translation activity improved their understanding of some programming concepts as well as lectures and learning materials.84% of the students agreed or strongly agreed that the translation activity had a positive impact on their learning of programming.Reviewing other students' translations was helpful in understanding programming concepts for 82% of the surveyed students.81% of the students said that this was also helpful in improving the next translation task they had to complete.The same percentage also agreed that the voting part of the translation activity encouraged them to do their best when they were completing the translation tasks.76% of the students enjoyed the voting task and they were excited to see the voting results.73% of the students who completed the survey stated that they would like to participate in another translation activity in their next programming course.
The second question in the survey asked the students about what they liked about the translation activity.This question was an open-ended question to allow the students to express their opinions freely.28 students answered this question and six themes emerged from those answers as shown in Table 2.The top theme was that the translation activity was generally fair, good, helpful and exciting which appeared in 46% of the answers.29% of the answers also expressed the benefit of this activity in improving and deepening their understanding of programming terms, concepts and programming language.14% of the students found it very interesting and helpful to see other students' translations which can improve their understanding of some terms and help them evaluate their own translations.Some students (11%) also see this activity as an opportunity to increase their knowledge in general.11% of the students believe that this translation activity was a good way to improve engagement and interactivity in programming class.4% think it also helped them in improving their understanding of the English language.These results are compatible with the improvement in the average score that has been noticed in the CTT results.Again, this supports the promising impact of adopting translation activity in the CS1 class.
The third question was also an open-ended question which asked the students what they disliked about the translation activity.27 students responded to this question and five themes appeared in the analysis of those responses ( (71%) showed that there was nothing to dislike or everything was good.11% of the respondents claim that some translations were directly copied from the internet and this can reduce the usefulness of the activity in their opinion.7% of the respondents said that they faced some technical or connection problems while completing the translation activity.4% said that the process of the translation activity is not easy enough to follow and the same percentage said that the translation activity is not helpful.The last question in the feedback survey was asking the students about their suggestions to improve the translation activity.10 students responded to this question and the main suggestions are listed here: • Add more terms to the translation activity • Publish the teacher's translation or a standard answer • Allow for more answers/ add a box for explanations • Create a forum for students to discuss the translation of technical terms • Link each translation activity with lessons in class • Conduct an interview/ oral test to prevent cheating Themes % of responses contributing to the theme Nothing to dislike 71% Some translations are copied from the internet 11% Technical/ connection issues 7% The process of the activity is not easy enough 4% The translation activity is not helpful.4% Table 3: What students disliked about the translation activity (open-ended question results) • Removing rating and voting and focus on participating in the translation • Add activities that include interpreting a line of code

LIMITATIONS AND RECOMMENDATIONS
While the aim of this experiment was not to conclude a generalisable result, the identification of limitations might be useful for future applications.The course in this study was provided online, it is not clear if there would be a similar effect if used in face-to-face classes.The terms that appear in the CTT were all excluded from the translation activity to prevent bias in the results.However, the impact of excluding those terms on the benefits of the translation activity can not be determined.
The main requirement in the peer review process applied in this activity involved asking the students to select one of the predetermined options for evaluating their peers' translations.A text box was provided to allow the students to write an optional explanation of their selected option or add more comments on the translation they are reviewing.This means that this activity is focused more on acquiring summative feedback rather than formative feedback.In previous studies in the area of peer review, it is been found that formative peer feedback can add more value to the learning outcomes of such activity.Accordingly, future applications for this activity might benefit from making formative feedback a requirement (not optional) in the peer review process.
It has been noticed that some students did not make enough effort, which appeared in the copied translations text.In fact, the researchers did not expect the students to translate the terms without looking for meaning or translations that are available on the internet.However, the students were encouraged to come up with their own translations but it seems that some of them did not follow this advice.This might have negatively influenced other students' learning and motivation.In future applications, the addition of some form of plagiarism checking or more punitive grading for copied answers could address this issue and add more value to this learning activity.
Finally, the positive impact of the translation activity on performance in the CTT has not been directly linked to improved academic performance in CS1 courses.Wider research on the impact of adopting translation activity in CS1 class is required.In order to aid this process, this tool is available as an open-source web application on GitHub4 .

DEPLOYING AND USING THE TOOL
Detailed instructions for deploying the tool are available on GitHub, however, a brief description of the process is given here for convenience 5 .
(1) Get SSL certificates for the server (2) Find configuration details from LMS (3) Update JSON configuration file with LMS details (4) Initiate server using docker-compose After these steps are completed, it should be possible to include the activity in your LMS (varies by LMS).All other configuration (addition of terms, TAs, etc.) is completed through the web interface by the coordinator of the course.

CONCLUSIONS
NNES students who are enrolled in English CS1 courses have to cope with not only the use of general English but also the specific terminology used only in CS.The teaching of such terminology mostly rests upon the teachers in CS, not the language teachers, as they have more knowledge of such terms.A previous study suggested pre-teaching vocabulary in the programming course where programming teachers are responsible for presenting and explaining new terms.This strategy could be very helpful for students especially since terms will be explained in class.However, this needs a lot of preparation which adds extra work and pressure on teachers.This research aims to help students in their learning of CS terminology but with a consideration of both the class time and effort required by teachers.The result was creating a lightweight activity that can help the students and does not overload teachers.
In this research, the process and impact of applying a managed and peer-reviewed translation activity on CS1 students' knowledge of computer terminology were presented.The results showed a significant increase in the average score of the students in a specialised computer terminology test compared to students in previous courses who have not been offered any translation-related activity.Thus, programming teachers can benefit from applying this activity in their classes after considering the suggested improvements.
These promising results might also help in designing supportive strategies for technical terminology acquisition and comprehension within the CS1 curriculum or any other subject in any field that requires students to learn domain-specific terminology.The tool designed in this study is open-source and available for any educators who would like to adopt it in their classes.
Finally, looking at the students' feedback on the terminology translation activity is very encouraging.Also, many students stated that they would like to have a similar activity in their next programming class highlighting their need for such activity not only in CS1 class but in more advanced programming courses.

Figure
Figure 1: Students' performance in CTT

Figure 2 :
Figure 2: Students' feedback on the translation activity

Table 1 :
1: Students' performance in CTT Descriptive Summary -Comparing the control and experimental groups' performance in CTT

Table 3 )
. The majority of the responses

Table 2 :
What students liked about the translation activity (open-ended question results)