The Development of Students' Professional Competencies on a Work-Based Software Engineering Program

Competencies may be defined as the knowledge, skills, and professional dispositions that an individual is required to demonstrate in order to be considered professionally competent. Competency-based education has long been a feature of professional degree programs, but the discipline of Computing Science has only recently begun to embrace competencies as a means of structuring or evaluating students' learning. Meanwhile, the practice of work-based learning - also well-established in other professional disciplines - has become more prevalent in Computing Science education, with increasing emphasis placed on work-based modes of learning, such as internships and apprenticeships. In this paper, we examine how students enrolled on a degree-level apprenticeship in Software Engineering have developed their professional competencies in the workplace. The paper is based on an analysis of 38 student assignments, wherein apprentices were asked to identify the competencies they have demonstrated, with reference to a portfolio of work. The UK Standard for Professional Engineering Competence and Commitment, which outlines the competencies required for certification as an Incorporated Engineer, provided the necessary framework. Competencies relating to communication and inter-personal skills were among those most often cited by apprentices, with competencies relating to knowledge and understanding and design and development systems also featuring prominently. Competencies relating to responsibility, management, or leadership were less prevalent, with professional commitment proving to be the least commonly cited category of competencies. We provide examples of how apprentices claim to have demonstrated each competency, and discuss the implications of these findings for competency-based learning in Computing Science education.


INTRODUCTION
In the past decade, computing disciplines have started to readjust or redesign their academic curricula in a move towards competencybased learning.A recent working group [18] reviewed such computing disciplines, with the IT2017 report [16] articulating for the first time the concept of competency for a computing curriculum and then the ACM/IEEE Computing Curricula 2020 (CC2020) [11] articulating competency as a combination of acquiring knowledge, practicing skills, and developing professional dispositions in the context of goal-oriented tasks in professional settings.
The UK Standard for Professional Engineering Competence and Commitment (UK-Spec) [8] governs the registration of individuals as Incorporated Engineers (IEng) in the UK.The applicants are assessed against five competence and commitment areas, each divided into several subareas: (A1-A2) Knowledge and understanding; (B1-B3) Design, development and solving engineering problems; (C1-C4) Responsibility, management and leadership; (D1-D3) Communication and interpersonal skills; (E1-E5) Personal and professional commitment.These areas can be mapped to the three components of computing competencies as follows [18]: A to knowledge; B, C, and D to skills; and, C and E to dispositions.
The work described here was carried out within the context of a work-based degree program -or apprenticeship -in software engineering (see [2] for more information on the design of this program).As is the case on most degree-level apprenticeships, the students spend a proportion of their time attending university classes, but the majority of their time is spent with their employer, here working as apprentice software engineers.These students are expected to be useful, contributing team members in the workplace.A specific Workplace Assessment module is designed to give the students an opportunity to demonstrate their ability to work in a team in their workplace as a valuable, contributing asset, as well as demonstrating their ability to manage their own time and justify the decisions they make along the way.This module is intended to create reflective life-long learners and bridge the gap between the education of computing professionals and software industry's expectation, by going beyond knowledge acquisition: the students are guided and supported in learning and/or improving their skills to apply their knowledge and developing engineering dispositions necessary in their workplace; hence to be competent professional software engineers.
Previous related work focused on understanding and perceiving [13,15], defining [5], assessing [3,4], and fostering [19] professional dispositions.These studies have focused generally on students enrolled in computing programs and competencies based on curricular recommendations.In our study, we focus on a workbased software engineering degree program and professional competencies based on the UK-Spec professional skills framework.Our goal is to investigate how do these students develop professional competencies in the workplace and in the university courses -as a combination of knowledge, practical skills, and professional dispositions, via an analysis of reflective reports submitted as university coursework.The research question may be expressed as: What evidence do students provide of the professional competencies developed on a work-based software engineering degree program?
This work contributes to our understanding of the professional competencies that students may develop on a work-based software engineering program.In particular, the study serves to highlight those competencies that are most apt to be developed by apprentice software engineers, and those that are less obviously exercised in the workplace.As such, the study offers insight into the use of a competency model based on the UK-Spec to assess work-based learning and development.

METHODOLOGY 2.1 Conceptual Framework
We have used the five dimensions of the UK-Spec [8] to design a framework for assessing professional computing competencies in a workplace-based module in a degree-level apprenticeship in software engineering program.Using such framework in assessing professional competency [18] is essential since the faculty members are not (expected to be) expert practitioners in the tasks students have completed in their respective workplace.

Participants
The participants here comprise a cohort of students (N = 38) enrolled on the third year of a four-year degree-level apprenticeship in software engineering.As apprentices, the students work in a range of workplace contexts, including multinational financial companies, public sector organisations, and small-to-medium enterprises.

Data collection
For one of the assessment components of the year-long work-based module, students are asked to maintain and submit a workplace journal comprising a reflective account of their achievements in the workplace.Students are asked to evidence how they have developed a preponderance of the competencies listed in the UK-Spec-based framework.Each student's reflective workplace journal is accompanied by a portfolio of evidence which is peer-reviewed (as formative assessment), validated by the student's workplace line manager in order to limit over-claiming against the competency framework, and assessed by one of the module instructors.As an assessed piece of work, the 'response rate' was 100%.Ethical approval for the study -including the analysis of submitted work -was granted by the host institution's College Ethics Committee.Details that might identify an apprentice or their employer have been removed from the data presented below.

Data analysis
The data were analysed using a deductive approach, with the UK-Spec competencies providing an a priori set of codes.In a sense, the first stage of analysis had been carried out by the students, who have -for the most part -labelled their workplace examples with an appropriate competency.What we are examining here is the competencies claimed by the students.To facilitate further analysis, all of the authors familiarised themselves with the data -typically the first step in thematic analysis [6].Then, the first author used qualitative data analysis software (NVivo) to code the relevant portions of the students' reports against each competency.In a very small number of cases, where students had claimed an example against a family of competencies (A, B, etc.), some additional analysis was required to determine the specific competency against which the example should be coded.The coded data were then checked independently by two other authors.

Limitations
Perhaps the most obvious threat to validity here is the self-report nature of the data: these are the competencies that apprentices report they have developed in the workplace.However, it should be noted that the apprentices are required to submit evidence alongside each competency claimed.Furthermore, the use of self-report measures -whether qualitative or quantitative -is well established in educational research [12,17].Another limitation is the scope of the study, which focuses on a single cohort.This limitation is addressed under future work below.

RESULTS & DISCUSSION
Table 1 lists all competency sub-areas identifiers with the number of participants and number of examples (quotes).Discussions on the analysis carried out on each competency sub-area are presented in the following subsections.

Knowledge and understanding
A1 Maintain and extend a sound theoretical approach to the application of technology in software engineering practice.In total, 22 apprentices claimed this competency, providing 30 distinct examples of how it had been achieved.As such, this was the third most prevalent competency overall, and the most prevalent competency concerned with the application of knowledge.Many apprentices saw this competency in terms of building upon the theoretical knowledge gained at university, by applying it in the workplace.For example: "Using the skills I learned in Algorithmics II this year, and Algorithmics I and Practical Algorithms last year, I was able to consistently make my code more performant by evaluating the time complexity and memory usage patterns of standard data structures" (P27).Another apprentice describes how knowledge of the Model-View-Controller pattern meant that "while I had never worked with Salesforce before, the knowledge I had gained about design patterns, was immediately valuable in understanding this new application of a familiar concept" (P18).A2 Use a sound evidence-based approach to problem solving and contribute to continuous improvement.19 apprentices claimed this competency, with many citing their work-based problem solving experience.For example: Having our own implementation allows us to have full control and tweak the algorithm to best fit our style of data.By method of cross-validation, specifically square mean loss comparison, we quantified how each tweak caused the prediction to have more or less loss accuracy -reflecting my proficiency in using an "evidence-based approach to problem-solving." (P21) Continuous improvement was linked to testing and debugging by several apprentices, for example: "One of the competencies I have gained implicitly is the skill of debugging code using a debugger in an IDE" (P20); "...creating automated test scripts and identifying areas for improvement in the application" (P13).The practice of code review was also identified here ("I had to review code written by a peer for the reports that are run daily for another area of our department to use" -P24), with one apprentice describing how they have helped improve their team's approach to reviews: I have learned a lot about the delivery pipeline and how it should be carried out.In the new team I am part of the delivery process had a few issues where the pull requests had no mandatory reviewers and people not making sure their release branches have been merged back into master.I have raised concerns with my team around this approach and after talking to our technical lead we have now implemented mandatory reviews by a tech lead and a developer for all pull requests.(P3) 3.2 Design and development of processes, systems, services and products B1 Identify, review and select techniques, procedures and methods to undertake software engineering tasks.19 apprentices claimed this competency, drawing on both their university and workplace experience in order to do so.For instance, one participant stated that "my understanding of software design patterns and architectural concepts from university has enabled me to effectively contribute to the design and development of technical solutions" (P4).The same participant went on to provide a specific example that involved developing a novel solution -a bot that "could pull data and compare over a multitude of databases" in order to identify inconsistencies in time-sheet data: This is a solution for a time-sheets issue that many within our area had not considered because of their day-to-day experiences in work which means they may automatically consider widely used existing solutions which in this case were not working.Yet my knowledge of databases which I have gained throughout my university experience enabled me to come at this problem from another angle.(P4) Others reported that they had developed this competency in the workplace by, for example, being "able to shadow several members of the team and identify the procedure and techniques required to carry out testing on the application" (P13).Another apprentice states that the skills of reading and writing "non-greenfield code" could only be developed in a workplace context: This skill required engagement with aspects of the competenc[y] B1 in a manner that would not be required to at university as any solution or technique considered for a task must be weighed against what has already been implemented within the programs being worked on in my workplace.(P5) B2 Contribute to the design and development of engineering solutions.The competencies listed under Design and development of processes, systems, services and products are closely related, with many apprentices using the same example to illustrate how they had met some combination of the B1, B2, and B3 competencies.One example unambiguously related to competency B2 follows: Instead of just considering the impact of a single ticket and its implementation, I was able to reason about why these tickets were required in the first place and how the overall project should be structured.I feel that this contributed to my development in competency B2 owing to my part in shaping the outline of the project.(P38) Competencies B2 and B3 were frequently claimed together, where apprentices are involved in both the design and the implementation of solutions -for example, "due to my role as the main driver for my group, where I contributed to the design of our solutions and implemented most of the changes" (P28).
B3 Implement design solutions and contribute to their evaluation.While this competency -also claimed by 19 apprentices -is frequently linked with the competency above, several apprentices were able to provide examples of how they had contributed to the evaluation of software.For example: "in terms of the link between backend and frontend code it is important it not only works but is efficient which involved myself creating appropriate functions then evaluating them by the speed of loading in the browser" (P21).Another apprentice noted that this competency can also involve corrective action: Another key role for me in the development work was gathering requirements, not only at the beginning of work on a ticket but during it, too, as sometimes gaps in their specification were present.Identifying missed detail in requirements led to liaising with concerned teams, and action being taken to remediate subsequent design issues.(P22)

Responsibility, management or leadership
C1 Plan for effective project implementation.The ability to plan for effective project implementation was just as frequently claimed by apprentices as the more development-focused competencies above.With these apprentices in their third year, it is clear that many have begun taking on leadership roles in the workplace, highlighting the opportunities that an apprenticeship can offerespecially when compared with a traditional degree program: The skills that I greatly improved upon were time management, estimation of projects and giving feedback to the rest of the team.I do not think I would have had a chance to improve these skills if it was a formal assessment given out at university.I knew that I had achieved competency C1 given that I accurately identified any blockers affecting the implementation, carried out experiments to minimise risks of the project going awry and agreed on concrete implementation plans with the rest of the team.(P37) As with most competencies, there is significant overlap where these are claimed.For example, it is not surprising that planning for a project involves managing people and resources: As one of two remaining internal staff on the team, I assumed accountability [for] the relational database from working on a long term project.Project implementation in this context includes design, development, risk management, planning and manual deployment of database code changes.By working alone in the project at first then acquiring and training additional resources, I demonstrated the responsibility and management competencies highlighted in section C1 and C2 of the competencies list.(P10) C2 Manage tasks, people and resources to plan and budget.While there is certainly overlap between these competencies, slightly fewer apprentices (N = 13) claimed this particular competency.With the exception of a few examples, such as that of P10 above, apprentices mostly claimed this competency in relation to their own time management: "I had to prioritise the important tasks, ranking the tasks by importance and finishing the most important or urgent tasks first" (P7); "Time management allowed me to meet the deadline effectively regardless of the kind of task that was set as I could properly estimate each aspect of the task" (P36).
It is notable, perhaps, that most of the examples of apprentices managing budgets and other people came from those in less technical roles.For example, as one product owner notes, "my involvement in the data and analytics [project] has enhanced my understanding of capacity and budgeting challenges, contributing to managing projects more efficiently" (P4); and a business analyst: "[C2] has been obtained by carrying out workplace tasks such as monitoring ongoing [reporting platform] requests and allocating them to members of the BA team" (P26).Indeed, some of the apprentices claim this competency against workplace activities that are only tangentially related to software engineering: "I send out communications and promote all the [well-being] services available within the bank to colleagues who need it or come to me for advice" (P24).
C3 Manage teams and develop staff to meet changing technical and managerial need.Just four apprentices claimed this competency, perhaps reflecting their relatively junior status within their respective organisations.Examples include streamlining the management of fellow apprentices: "Setting this [Microsoft] Team up helped remove the numerous emails being sent constantly to the apprentices and had separate channels within the team to add appropriate content" (P24); and, stepping up to manage a team where more senior team members had moved on: "...with people leaving my team at work, a vacuum opened that I found it quite easy to fill.Filling this hole in the team meant accepting much more responsibility by leading my own large projects" (P27).Another apprentice, on reflection, notes having assisted with on-boarding new staff: Since the end of the first year of [the apprenticeship], I've been working to help onboard new joiners into my team, and, thinking back, I've been there to support my line manager in welcoming the newcomers and getting them setup with the tech and knowledge they need to start working as quickly as possible.(P20)

Communication and inter-personal skills
D1 Communicate in English with others at all levels.This was the most frequently claimed competency, with 29 apprentices providing 42 distinct examples of their ability to communicate.While there are a small number of instances where this competency has been developed through observation (for example, "observing how other team leaders communicate with their members during meetings" -P33), most apprentices refer to learning how to communicate by being required to do so.For example, many refer to the need to deliver presentations at work: "This was achieved when I presented [the project] in front of my entire 300+ person team ranging from remote viewers in India and England, to an in-person audience of around 50 people in our [local office]" (19); "time constraints placed on presentations have taught me to be able to fit technical and complex conversations into a limited time frame" (P11).
Being required to collaborate with others is another common theme: "working directly with stakeholders and end users within the bank to better understand what's required of the task at hand" (P30); "I've grown more versatile, resilient, and empathetic, allowing me to collaborate effectively with diverse team members from across the world" (P4).Indeed, apprentices frequently cited the forms of collaboration associated with modern software development -particularly agile ceremonies -as opportunities for developing this competency.For example: During stand-up meetings [...] I shortened my daily updates and made them more specific.I gave them according to the following format: The Jira ticket number, a brief description of what work I was doing in relation to it, whether the ticket was complete and, if it was not, an estimate of when I would complete it or a description of any blockers I was experiencing.Sticking to this format helped me to avoid unnecessary rambling, which according to feedback I received from colleagues made me sound more professional, and the information I was trying to communicate easier to understand and respond to.(P22) However, this was not necessarily a competency that apprentices expected to exercise: "Perhaps unusually considering the usual role of a software engineer, giving presentations is a very regular part of my job" (P20).The same apprentice goes on to suggest that the opportunities to develop their presentation skills while in the workplace are more significant than those afforded by university: "with a work presentation, often I've found myself presenting to those who are much higher-up within the organisation than me, and often the audiences are larger in size" (P20).
This competency was frequently discussed in conjunction with competency D3 (Demonstrate personal and social skills -discussed below).For example, as one apprentice describes: Taking this new role again also helped me achieve the D1 and D3 competencies as I was working with a team based in India to learn these new skills.I feel like my knowledge with both competencies was vastly improved during this as there was a lot of documentation and meetings set up to transfer knowledge over to me.(P9) D2 Present and discuss proposals.This was another frequently claimed competency, with 20 apprentices providing relevant examples.Again, the meetings associated with agile software development provided opportunities for apprentices in this regard: "I was able to present and discuss proposals, this was done as part of sprint planning and retros where I would assist in estimating user story complexities and times" (P14); "Scrum meetings involved presenting and discussing proposals for projects or for developments and solutions for projects.I occasionally had to present my own findings or work" (P7).While most of the examples provided by apprentices relate to the workplace, some refer to how the experience of presenting their work at university has helped prepare them for doing so professionally: I have been provided with industry transferable skills such as ability to create documentation, gained through academic assignments, which has proved useful in documenting procedures and successfully presenting ideas.Furthermore, the development of presentation abilities during university presentations has aided in the presentation of proposals and the presentation of solutions to stakeholders.(P4) D3 Demonstrate personal and social skills.This was the second most prevalent competency overall, claimed by 24 apprentices.The examples cited by apprentices often connect this competency with confidence, for example: "Engaging with colleagues and fellow apprentices has allowed me to further strengthen my interpersonal and intrapersonal skills enabling me the confidence to get involved in events that I would have previously avoided" (P26); "I have noticed that I am now more confident in asking for help [...] I think that I've learned skills like D3 without even realising it through interactions like these" (P28).
As with some of the competencies associated with Responsibility, management or leadership, many of the examples cited here relate to workplace activities that are not directly concerned with software engineering.For example: Over the past year, I have volunteered for several networking and citizenship events, where I have communicated with wider teams in [the organisation] and I have attended school careers fairs where I have spoken to teachers, parents and students about the apprenticeship.On reflection [...] I can see how my social confidence has developed and enabled me to achieve competency D3. (P13)

Professional commitment
E1 Comply with relevant codes of conduct.As was the case with the Responsibility, management or leadership competencies discussed above, those relating to Professional commitment were amongst the least frequently claimed.However, ten apprentices did provide evidence of complying with relevant codes of conduct, often as a result of workplace training, which connects directly with competency E4 below: "completing mandatory training to ensure I am up-to-date with the procedures required to maintain security whilst working" (P13).As might be expected, many of the examples of meeting this competency came from the heavily regulated financial sector: "I have been able to learn different laws and regulations which banks must conform to.As such, [the bank] provides training to all employees every quarter to ensure myself and everyone else conforms to all policies and procedures" (P30).
E2 Manage and apply safe systems of work.While six apprentices claimed this competency, they generally did so in conjunction with other competencies, such as E1 above.For example: "We have to complete mandatory training to keep us up to date with relevant codes of conduct (E1), and work safely (E2).These modules can be banking related, software engineering related, or health and safety related depending on what has changed or needs refreshed" (P24).
E3 Undertake engineering activities in a way that contributes to sustainable development.Just one apprentice provided an example relating to sustainable development, drawing on an awareness of sustainability gained through a university module: We also have the knowledge and awareness to consider the impact that our continuous development and deployment can have on our environment.My time within [the professional issues course] at university has allowed me to raise environmental concerns confidentially and knowledgeably which furthermore allowed me to ensure that this solution was not only viable but environmentally friendly with the promise we can host on cloud.(P4) Sustainability-related learning outcomes were added to a number of modules on the apprenticeship program in 2022, including the professional issues course to which P4 refers.However, it is striking that just one apprentice made reference to sustainable development in relation to their experience of working as a software engineer in industry.
E4 Carry out and record CPD necessary to maintain and enhance competence in own area of practice.The other most commonly claimed competency under Professional commitment was that related to CPD, which, as noted, was frequently cited in conjunction with complying with relevant codes of conduct.However, several apprentices referred to on-the-job training that was used to enhance their competence in relation to the specific technologies used in the workplace.For example: Part of improving myself has required gathering specialized knowledge of the Salesforce platform.This has been done through Salesforce's training platform and certification program [...] This helped greatly in cementing the knowledge gained from several university courses and could be applied to gaining competency E4. (P12) Platforms such as Amazon Web Services -not explicitly taught at university -also featured in the apprentices' responses, as did undertaking training in languages and frameworks not covered in class, for example: "Further to the pursuit of performance, I've been learning about Angular's internals too" (P27); "I will be moving team soon and plan to take courses in React to help me with the work I will be undertaking in my new role" (P24).
E5 Exercise responsibilities in an ethical manner.Six apprentices claimed this competency, referring to how they conducted themselves in the workplace, as one apprentice describes: I have gained competency E5 by treating my colleagues with respect and conducting myself ethically.As an example, there were times when my university schedule clashed with taking charge of bigger projects.In those situations, I made sure to communicate effectively and responsibly to avoid any expectations that I could handle those particular tasks.(P28) Other examples of working in an ethical manner included ensuring that "[customer accounts] data be transferred in a safe and secure manner" (P7), or "that any sensitive information has been redacted and communicated with the necessary colleagues to identify areas of concern regarding this" (P13).

CONCLUSION & FUTURE WORK
The data presented here indicate which professional competenciesbased on the UK Standard for Professional Engineering Competence and Commitment (UK-Spec) -are reported to have been developed by students enrolled on a work-based degree program in software engineering.Of the five areas (A-E) into which the UK-Spec is organised, those competencies related to D. Communication and inter-personal skills, A. Knowledge and understanding, and B. Design and development of processes, systems, services and products proved to be the most widely claimed.Meanwhile, those competencies that fall under C. Responsibility, management or leadership were less prevalent, and those related to E. Professional commitment were much less so.
It is expected that work-based programs should develop professional skills and competencies [9,21] and, indeed, this was part of the rationale for creating computing-related apprenticeships in the first place [10,20].As such, the findings here shed light on how (and which) competencies are developed on a work-based degree program.The references to confidence -particularly in relation to Communication and inter-personal skills -also align with previous work suggesting that confidence underpins the development of a range of apprentices' transferable skills in the workplace [1].
The less frequently reported E2, E3, and E5 competencies address professional commitment to work environment safety, sustainability and social/ethical responsibility.Participants' limited self-reports in these areas may indicate a gap between the education of computing professionals and the industry's expectations of software engineers, as was also identified in the German computing education system [14].Acting responsibly towards society needs to be embedded in current computing curricula and pedagogical methods, with very recent work [7,14] providing recommendations and ideas for designing education in responsible computing.However, the data presented here relate primarily to what apprentices have learned in the workplace.Therefore, the relatively sparse evidence for Professional commitment may reflect current industry norms, rather than highlighting a gap in the taught curriculum.Indeed, it was notable that the only mention of sustainable development originated from a university module.
As noted, this study was limited to a single cohort of apprentices.In the future, we aim to replicate the work with successive cohorts in order to gather a greater quantity and variety of data.Ideally, the work would be expanded to include software engineering apprentices on other programs.Also, since the dispositions component of competencies maps to area C and E [18], the relative lack of examples here may indicate a lack of clarity in the UK-Spec-based framework as to what dispositions are, and how students can identify them in the workplace.Therefore, future work should investigate how CC2020 dispositions are understood [13], defined [5], and assessed throughout students' studies.Future work may also incorporate opportunities for more objective measures of the competencies claimed.

Table 1 :
Competencies ordered by the number of participants who claimed each competency.The examples column shows the number of distinct examples provided by participants in relation to each competency.N=38 participants overall.