Improving Software Engineering Teamwork with Structured Feedback

Teamwork is a key learning outcome for our course, Foundations of Software Engineering. However, conflicts are inevitable in teams, and if students cannot resolve conflicts, this can lead to decreased satisfaction for everyone on the team. In this experience report, we present our approach to help students deal with conflicts that can occur in team projects. Working with faculty from the School of Business, following best practices of organizational behaviour research, we instructed students on how to provide high quality peer feedback, and designed activities where students provided feedback to each other following these principles. After this intervention, we compared the results of our teamwork survey with the results from the previous semester. We saw a meaningful drop in teamwork problems directly after the intervention, and the effect persisted for the rest of the semester.


INTRODUCTION
Working together in a team is a major aspect of software engineering.Successful teamwork extends beyond technical processes and collaborative software tooling, it also depends on a strong internal dynamic.However, conflicts are inevitable in teams, and when unresolved, they can be detrimental to the success of software development projects, and lead to lower developer satisfaction in the team.[3,11,17].
In a third year course on software engineering, Foundations of Software Engineering , students collaborate in teams of 4 to 6 members, working on various projects through the semester.A major recurring challenge is that a portion of student teams struggle to work together.Despite trying various methods to help teams address teamwork issues over the years, the student teams continued to struggle.
"Conflicts with team members" was continuously reported as the largest concern that students had in the class.Moreover, members of such teams tended to report lower satisfaction with the course due to a poor team experience.Common student concerns included that team issues "caused a lot of stress." and "It was really tough this semester because I was stuck with a team that either just would not do any work on certain assignments and left the rest to me or wait until the very last second to work on assignments." In our reflection about why these problems persist, we hypothesized that our current methods did not teach the students how to address team issues themselves, and instead relied on the course staff and the course mechanics to resolve their team issues.
In this experience report, we describe a structured feedback intervention that we introduced, which addresses team conflicts within student teams in a new way.
The authors include a CS professor, the lead TA for the course, and two faculty from the Tepper School of Business.Team conflicts are well studied in the field of management and psychology [9], and it is known that feedback is important [6,12,16], as well as proper conflict management when conflict arises [4,7].We believe that strategies of feedback and conflict management would translate well into software teams, as the problem is inherently human in nature.The current course structure did not have peer-to-peer feedback built in, which provided a ripe opportunity to implement a feedback intervention.
To develop this intervention, we adapted structured feedback from a business context for the software engineering foundations class.We introduced this content at the midpoint of the semester.The intervention was intended to guide students in giving feedback that is effective and actionable, understand how to have a space to facilitate conflict resolution conversations, and work together as a team to address those issues in an action plan.
Despite having made various team related changes to the course over the years, we observed the most significant improvement in team dynamics after the structured feedback intervention, as observed through patterns observed in student surveys about their perception of their teams collaboration and the team cohesion in the final project.Furthermore we observed a positive improvement in impression towards feedback giving before and after the intervention.Thus, we believe our structured feedback intervention was particularly effective in helping student teams in a software engineering course, and our experience can be applied to similar courses in other institutions. 1

ORIGIN AND MOTIVATIONS
Previous to the intervention described in this paper, we covered lecture content on how to function well as a team, as well as an experiential exercise where students role-play different scenarios simulating dysfunctional team dynamics, which aimed to help them identify and address problematic behaviors within their own teams. 2 Despite this, in the semesters leading up to the Fall 2022 semester, we consistently received reports of troubled teams.The identification of such teams relied on student reports during office hours, assignment write-ups, end-of-semester teamwork reflections and peer evaluations.
When the course size doubled in Fall 2022, the absolute number of students grappling with team-related challenges also doubled.After reviewing final project submissions and peer evaluations, out of 31 teams, 13 teams expressed some level of conflict and 5 out of 13 teams had dysfunctions that warranted a grade deduction of at least one team member.This escalation prompted a reevaluation of the team experience in the course, as it highlighted a need for a proactive approach to this challenge.The underlying concern is that students' satisfaction with the course is closely linked to their experience with their teams.It has also negatively affected the learning of the students on the teams which experienced dysfunction.
In Spring 2023, we undertook various initiatives to enhance the team experience: (1) Team Contracts: 3 To ensure that students discuss their working styles, schedules, and come to a common understanding within their teams within the onset of their team formation.By discussing, aligning, and documenting expectations, it can help minimize misunderstandings that can cause team issues.(2) Software Engineering Team Retrospectives: 4 After completing their first major team project, student reflect their experience, and classify their experience into Keep Doing, Start Doing and Stop Doing.They were also asked to update their team contract accordingly.
Despite these initiatives, there was still a significant hurdle.We observed students often hesitated to provide feedback to teammates who were not cooperating effectively.When asked why, they would express that this reluctance stemmed from a desire to avoid conflict, and the potential of causing hurtful outcomes if the feedback is 1 All materials (referenced in footnotes) can be found at our report repository 2 Refer to Team Dysfunction 3 Refer to Team Contract 4 Refer to Team Retrospective done poorly.Thus the course staff recognized the need for a structured intervention that encouraged constructive feedback while circumventing potential pitfalls.
To address this challenge, the course staff engaged the business faculty to explore potential solutions.The result was the structured feedback intervention described in this paper that we adopted and implemented for students right after they completed their second major team project, which is right about mid-semester.

BACKGROUND AND RELATED WORK
Feedback is defined as information that is provided to recipients about their behavior, performance, or understanding [6,16,19].The purpose of feedback is to foster self-awareness and behavioral reinforcement (in the case of positive feedback) or change (in the case of critical feedback).Feedback enables recipients to compare their performance to a standard or goal with the aim of helping the recipient improve and grow.Feedback often includes appreciation, coaching, or evaluation of the recipient [31].
Feedback is a vital process for teams to learn and develop as it can foster better relationships between team members, better performance, and better outcomes overall (e.g., [12,19,25]).In a study comparing top-performing firms to middle-of-the-pack firms on the basis of financial performance, top-performing firms made more effective use of performance reviews and encouraged candid feedback [8].This suggests that effective feedback systems can help individuals and teams perform better.
Prior works on peer assessment systems in software engineering utilize peer ratings to assist instructors in evaluating student performance [14,32].These efforts are shown to ease instructor workload and provide a thorough perspective on students' performance within teams.Students are shown to be receptive to improving their behaviors based on feedback provided to them [29].
However, feedback conversations are often unsuccessful (see [19,30].There are many barriers to feedback conversations that may explain this phenomenon (see [13]).For example, people often have difficulties delivering feedback because they overestimate the emotional harm that honest feedback may cause [21,23], misjudge how much people desire feedback, and underestimate the benefits that recipients receive from feedback [2,21].Due to this, feedback providers often tell prosocial lies and deliver overly positive feedback (e.g., [22,28]) or avoid these conversations (e.g., [5,10].In a work that investigated quality of peer evaluations given in a software engineering course, students had difficulties giving concrete suggestions for improvement when identifying weaknesses.[26] People also struggle when receiving feedback.For example, recipients may discount and misinterpret feedback [15,18] or feel negatively toward the provider ( [24]).
Newer work points to effective strategies for successful feedback provision.Firstly, it is important to be honest in such conversations.Research shows that conversations in which people are honest are better than people anticipate them to be [21].For example, honest conversations are more enjoyable, lead to more social connections, lead to less relational harm, and elicit less negative reactions from recipients than people anticipate.Second, there are features of feedback provision that are more likely to lead to positive outcomes: Feedback should be goal-oriented, actionable, specific, and timely [6].Finally, how feedback is delivered matters.Feedback providers should provide honest information to the recipient in a way that promotes the recipient's well-being [13,23].That is, providers should try to provide support when delivering critical feedback and help recipients cope with the negative information [20].Building on this work, we devised a structured feedback intervention to help students provide effective feedback to their teammates.

COURSE ORGANIZATION
This section outlines the current course structure, providing context for the structured feedback sessions.We will first provide an overview of the course, how the teams are formed, and the relevant course support roles.We will then cover the structured feedback sessions that were added.Lastly, we will describe the team selfassessment surveys and pre-and post-structured feedback surveys.

Course Overview
Foundations of Software Engineering was a semester long course over 14 weeks.There were two weekly lectures of 80 min each by course instructors, and a weekly recitation of 50 minutes.The entire class attended the same lecture, and they were split into separate sessions for the recitation of 15-20 students each.Each recitation was run by two teaching assistants.
There were a total of 5 projects throughout the semester.The first project was an individual project, while the remaining 4 projects were team projects.
During the first project, we conducted a survey asking for students' preferred working hours, prior experience, and motivation in the class.We then formed teams based on the information, with greater importance given to matching the team members' preferred working hours.Teams were also formed only within each recitation group (i.e.every member of a team belongs to the same recitation).The teams persisted for the rest of the course.

Structured Feedback
After the second team project was completed, the students are introduced to the structured feedback sessions.We decided to conduct the structured feedback sessions after two projects to allow for students to have sufficient time working together to get to know each other, but also to have significant time to work afterwards so that team members would see the benefit of improving their teams' working relationship.This was done in three parts in the following order.
(1) In-lecture workshop on the objectives of giving feedback and structure of effective feedback The in-lecture workshop had several parts.First, it included a brief review of the benefits and difficulties of working in teams and strategies to deal with team conflict.Second, it defined feedback, explained that feedback conversations are difficult and why that is, and instructed students on how to effectively deliver and receive feedback (with an emphasis on critical feedback, as critical feedback is often less successful than positive feedback).Instructions included the importance of being honest when delivering feedback, examples of how to give feedback using the SBI, SBII, and SBIA models [1], and how to deliver feedback support.Finally, the workshop included instructions on how to make a team action plan in which teams would discuss their individual feedback and make a plan to overcome any issues or conflicts within the team.(2) A take-home anonymized peer feedback survey 5 was issued right after the in-lecture workshop, and it was due in 5 days after the workshop.Students rated the effectiveness of their teams and teammates and wrote positive and critical feedback for their teammates, with the knowledge that their feedback would be aggregated, anonymized, and presented to their teammates. 6 (3) Peer feedback reading, discussion and writing of the team action plan 7 .During their recitation section, students received instructions on how to constructively discuss the feedback they received, how to devise team goals, and how to make a team action plan.TA's were available to assist as needed, but the teams were responsible for leading their own internal discussion of their feedback.

Surveys
After the teams were formed, students were assigned team selfassessment surveys to complete weekly throughout the rest of the semester.The surveys included questions that asked students to self-assess their own performance in the team, as well as their perception of their team.Students were graded on participation, not on correctness.The survey questions come from previous work, [27] which used this instrument to help identify struggling teams in SE courses through weekly surveys.A similar self-assessment survey was also conducted in the prior semester (Fall 2022).
For the purpose of understanding students' perceptions and consequences of the structured feedback session, we also assigned a pre-survey and post-survey for students to complete prior to and after the structured feedback sessions.These surveys include questions about students' anticipated (pre-survey) and actual (postsurvey) experiences giving and receiving feedback.As described in Table 1, we present the demographics of students enrolled across the two different offerings that we are considering in the paper.There were 146 students in the Fall 2022, and 69 students in the Spring 2023.The class serves three main audiences, Information Systems (IS), Computer Science (CS), and Electrical and Computer Engineering (ECE).Most students take the course during their Junior year. 5Refer to Peer Feedback Survey 6 Feedback presented in the following format: Peer Feedback Report 7 Refer to Team Action Plan

OUTCOMES 5.1 Positive student perception of giving and receiving feedback
Students completed a pre-survey and post-survey before and after, respectively, the in-lecture workshop to assess their perceptions and experience of the workshop and team action plan recitation.We found that students underestimated the benefits and overestimated the harm caused by giving and receiving honest feedback.
In Figures 1 and 2, we compared the same question asked in presurvey and post-survey, with the statistically significant different ones identified with asterisks.We did not report every question that we asked of the students due to space constraint.

Giving honest feedback.
Students completed several items meant to measure their perceptions (pre-survey) and experiences (post-survey) of sharing their honest thoughts and evaluations when giving feedback to their teammates.Students answered questions about themselves, their perceptions of their teammates, and their perceptions of their team.We found that after giving honest feedback students reported that they had underestimated the benefits that their teammates experienced from that feedback (see Figure 1).That is, feedback providers felt that their teammates felt closer to them and trusted them more than providers had anticipated.Students also overestimated the emotional harm that their teammates would experience.Specifically, providers anticipated that their teammates would feel more distress and uncomfortable than they perceived their teammates as actually feeling.Additionally, students underestimated how well their teammates would perceive them: Providers reported that their teammates thought the provider was less unkind and less harsh than providers had anticipated.Students also underestimated the benefits that the team would experience after they provided honest feedback.Feedback providers reported that their team experienced less conflict ( = 1.77) than providers anticipated ( = 2.28;  (52) = 3.60,  < 0.001).Asterisks indicated a significant difference between pre and post-survey reports ( < 0.05).Scale ranges from 1 (strongly disagree) to 5 (strongly agree).Asterisks indicated a significant difference between pre and post-survey reports ( < 0.05).Scale ranges from 1 (strongly disagree) to 5 (strongly agree).

Receiving honest feedback.
Students also completed several items measuring their perceptions (pre-survey) and experiences (post-survey) of receiving their teammates' honest thoughts and evaluations.Paralleling our findings regarding giving honest feedback, we again find that students underestimated the benefits and overestimated the harm of receiving honest feedback (see Figure 2).Specifically, we found that recipients of honest feedback overestimated the emotional harm, such that providers reported believing that their teammates felt like they (the provider) were less unkind and felt less uncomfortable than providers anticipated.The results in the above two subsections (5.1.1 and 5.1.2) are in-line with prior work showing that people underestimate the benefits and overestimate the harm caused by providing honest feedback [21].

Reduction in team dysfunctions
We have also observed decreases in the number of teams with potential team dysfunctions.We observed that through the team self-assessment surveys conducted during the semester and the number of teams that visit office hours for team related issues.We also observed lower percentage of teams dysfunction in the final project through their project deliverable.

5.2.1
Lower amounts of recorded team dysfunctions.We used the team self-assessment surveys, which are assigned to all students weekly, to understand team issues and trends in Spring 2023.Response rates to the team self-assessment survey were relatively consistent, at around 80 percent across all weeks of the semester.There are three questions in our team self-assessment survey that reflect team cohesion to an appropriate degree, and students reply on a 5 point Likert scale.The team self assessment surveys were also assigned to students in Fall 2022.The exact projects changed slightly between the Fall 2022 semester and the Spring 2023 semester, however the content covered and the difficulties of each project are intended to be similar.
We plot the three questions from the two semesters beside each other in figure 3. We only plotted neutral to negative responses to observe negative sentiments better.In the graphs, the grey lines represents the due dates of the team self-assessment survey.In the graph representing the Spring 2023 semester, the two blue lines represent the start and end of the structured feedback intervention that we introduced.We observe that in Spring 2023, there was a decrease in individual's dissatisfaction with the team in the two surveys after the structured feedback session has been conducted.While for the Fall 2022 semester without the intervention, we observe that team dissatisfaction generally persisted throughout the semester without much change.
We further observe an increase in individual dissatisfaction with their team for both semesters near the end of semester.Near the end of the semester is when the final project is due.We also see that there is still less dissatisfaction at the end of semester in Spring 2023 as compared to Fall 2022 semester when comparing the percentages.
Additionally, in the Spring 2023's team self-assessment survey, we included an additional multi-checkbox question, "How do you feel about your team's collaboration process in the project" with a total of 6 possible responses.Students are allowed to check multiple boxes at once.We plot the counts across time over the semester in figure 4, removing the counts for "I feel like my team is collaborating well together" to see negative sentiments better.We see that similarly, we observe a drop in feelings that represented team dissatisfaction for the two surveys following the structured feedback intervention.

Figure 4: Feelings on collaboration process
The blue dash-dot line represents the period when the structured feedback intervention occurred.

5.2.2
Improvement in final project team cohesion.We observed better team cohesion in the teams through the final project deliverable, which included team reflections and peer evaluations.Out of the 15 teams, 2 teams (13%) expressed some level of conflict, and both teams have at least 1 member that warranted a grade deductions.This is lower than the prior semester, which out of 31 teams, there were 13 teams (42%) which expressed some level of conflict, and 5 of the 13 teams had dysfunctions that warranted a grade deduction of at least one member.

DISCUSSION
We hoped that providing students with an opportunity to provide feedback to their peers in the middle of the semester would likely be helpful.In previous semesters, proactive students would consult with the course staff about their problems without discussing those problems directly with their team beforehand.This was likely due to a mixture of not knowing how to give good feedback and a fear that the conversation would go worse than expected.Therefore through our collaboration with faculty from the School of Business, we provided a an intervention to help students with instructions and structure to address their teams issues themselves.
The results presented in the paper seem to suggest that students benefited from the activities more than just the information gained, as they also had more positive feelings towards giving feedback afterwards.In open-ended responses, students indicated that the feedback activity was beneficial in bringing issues to light and that they felt better about their teammates than prior to the activity.Moreover, the teamwork outcomes seemed to have improved over the previous semester, be it through self reported sentiments or through their final project deliverable.We believe that the strengths of our intervention are as follows: (1) Providing a framework for students to think about giving feedback and how to deliver feedback support.Students who aren't familiar with giving feedback are guided to provide quality feedback, as they are provided the framework.Quality feedback leads to better outcomes for the receivers of the feedback.(2) Having students document their feedback before discussion.
Students have the time to reflect and pen down their feedback, leading to better and clearer thoughts.(3) Providing a moderated class time for students to have feedback conversations.Some student teams will de-prioritize team conversations, and providing class time ensures that they have the feedback conversations.
We observed that this intervention shows significant promise.We were able to achieve improvements in the number of reported teamwork problems after this intervention.The fact that most students in the course want to work well with their teammates is probably an important contributor to the success of the intervention.This helps the students align their motivation between the students giving feedback and those students receiving feedback.

Possible areas for improvement
Firstly, from discussions with students, there are students who remarked that the anonymous form was not as anonymized as intended.Though the peer feedback submitted is shuffled before being presented to the teammates, students remarked they could identify the writer from their writing style.The negative effects of which would likely be minimal, as the students would ultimately discuss as a team about those feedback with each other.However, if students could be sure that their feedback was completely anonymized, it could provide more psychological safety.
Secondly, the teaching assistants observed that there were teams not fully engaging in discussions about their feedback.We believe that it is a mixture of the following reasons: (1) Some teams were already functioning and communicating their issues well and didn't know what else they needed to address apart from accepting the positive feedback from their peers.This is likely, since around 80 percent of students expressed sentiments that their team is functioning fine.In the future, we might have an alternative for teams that do not have major team issues to address during the feedback discussions.
(2) Teams went straight into working on the team action plan without discussion.Perhaps we could provide a discussion framework, and only let them have the action plan halfway through the session so they do not begin on the action plan without discussion.(3) Teams with members missing.This is still an open problem.
It also seemed as if the missing team member is likely the one who would have benefited most from the discussions.We, however, also believe this is a limitation of our approach, as it assumes that all members want to participate in improving their team.
Lastly, we found that, in the teamwork self-assessment surveys, the negative sentiments from the students did increase again near the end as they worked on their final project.This seems to suggest that while the intervention was beneficial, it does not eliminate all problems going forward.

CONCLUSION
This experience report covered the structured feedback intervention we implemented in Foundations of Software Engineering to improve student teams' outcomes.We believe that while the other course modifications, which included additions of team contracts and team retrospectives, definitely contributed to the lower overall team dissatisfaction throughout the semester, the structured feedback intervention was particularly effective.We note the positive impression that students reported about the experience of providing feedback to their peers.Moreover, after the intervention, we observed a meaningful decrease in teamwork issues, and the effect persisted for the remaining of the semester.This was observed through team self-assessment surveys, which did not reflect a similar decrease in the prior semester.Similarly, the percentage of teams with team cohesion issues in the final project also decreased when compared to the prior semester.We believe the outcomes of this experience can be utilized and built upon in advancing team collaboration education in software engineering education.

Figure 1 :
Figure 1: Perceptions of Giving Feedback to Teammates, Pre and Post Survey

Figure 2 :
Figure 2: Perceptions of Receiving Feedback from Teammates, Pre and Post Survey

Figure 3 :
Figure 3: Comparison of team self-assessment survey responses between two semestersThe blue dash-dot line in the 2023 semester represents the period when the structured feedback intervention occurred.The grey dotted lines in the graph represents the submission dates of the team self reflection survey, which is the real data point.

Table 1 :
Demographics of students enrolled in both Fall 22 and Spring 23 offerings.