Modeling Women's Elective Choices in Computing

Evidence-based strategies suggest ways to reduce the gender gap in computing. For example, elective classes are valuable in enabling students to choose in which directions to expand their computing knowledge in areas aligned with their interests. The availability of electives of interest may also make computing programs of study more meaningful to women. However, research on which elective computing topics are more appealing to women is often class or institution specific. In this study, we investigate differences in enrollment within undergraduate-level elective classes in computing to study differences between women and men. The study combined data from nine institutions from both Western Europe and North America and included 272 different classes with 49,710 student enrollments. These classes were encoded using ACM curriculum guidelines and combined with the enrollment data to build a hierarchical statistical model of factors affecting student choice. Our model shows which elective topics are less popular with all students (including fundamentals of programming languages and parallel and distributed computing), and which elective topics are more popular with women students (including mathematical and statistical foundations, human computer interaction and society, ethics, and professionalism). Understanding which classes appeal to different students can help departments gain insight of student choices and develop programs accordingly. Additionally, these choices can also help departments explore whether some students are less likely to choose certain classes than others, indicating potential barriers to participation in computing.

(including fundamentals of programming languages and parallel and distributed computing), and which elective topics are more popular with women students (including mathematical and statistical foundations, human computer interaction and society, ethics, and professionalism).Understanding which classes appeal to different students can help departments gain insight of student choices and develop programs accordingly.Additionally, these choices can also help departments explore whether some students are less likely to choose certain classes than others, indicating potential barriers to participation in computing.

INTRODUCTION
In most parts of the world, women are less likely to study computing and less likely to work in tech-related industries than are men [67].Women's participation in technical fields of study varies both by cultural beliefs about appropriate gender roles and by nations' educational policies [19].According to Organization for Economic Co-operation and Development data, women are underrepresented among computer science (CS) degree recipients in nearly all countries in the industrialized world [19,28].For example, in 2018, European women earned only about 20 percent of information and communication technology bachelor's degrees [33], while in the United States, 22 percent of bachelor's degrees in computing were awarded to women in 2021 [34].Australia's figures are consistent with these, as women earned about one-fifth of all bachelor's degrees in CS in 2021 [87].In 2022, 26 percent of CS bachelor's degrees in New Zealand were awarded to women [69].Less data is available for Latin American countries and other parts of the world.A small group of nine universities presented data at the Microsoft Latin American Research Summit in 2011.The percentage of women in these computing departments ranged from single digits to low 20s.
The continued underrepresentation of women in computing is deeply concerning both from the viewpoint of social justice and from the need for a highly qualified technical workforce.Computing professionals work on the cutting edge of technology, are placed into influential positions, work creatively, enjoy freedom relative to other fields, and earn some of the highest salaries among professionals [36].Too many women are missing out on these benefits.Economic well-being is more than a financial or material goal; it has a well-established, measurable effect on women's health.Five decades of research in Western and Eastern Europe, North America, the Middle East, and other locations demonstrate that economic factors such as higher occupational class and income are positively associated with women's better physical and mental health [4,5,23,46,52,54,56,64,77,93].Women's low participation deprives organizations of the well-documented contributions of gender-diverse teams for problem solving, decision making, product development, and avoidance of algorithmic bias [6,29,41,44,72,88,92].
While we do not intend to be gender essentialist -a belief that women and men have fundamental differences that are consistent across cultures -it is important to point out that women are less likely than men to participate in technical fields of study across most, but not all, countries of the world.Patterns of behavior can be explained not only by innate differences but also by differences in socialization and the stereotypes people are exposed to, and there are many social cues pushing women away from computing.These include social stereotypes about who studies computing and the judgments the people around women and girls make as to whether computing is suitable for them.Stereotypes also shape the social environment women face in computing, which can be hostile and discriminatory.It is plausible that the differences in average men's and women's behavior influenced by these social pressures lead to different choices being made within computing degrees, thus potentially creating the possibility of designing computing degrees that are more appealing to women.Whilst eradicating such social pressures would be ideal, this is infeasible in anything but the long term, so appealing to women within these constraints is important.
Although CS has become an increasingly popular field of study, and the most rapidly changing field for the past 30 years, this change may not be reflected in current CS curricula [48,53,79,94].Irrelevant curriculum might be one of the reasons why women are underrepresented in CS and technology-related fields.However, the ability to select from elective classes provides an opportunity to allow students to make informed decisions as to their education and career pathways.Further, it is plausible that offering more elective courses that counteract CS stereotypes could make a CS degree more interesting to women [20].Additionally, integrating a range of perspectives into the CS curriculum has the potential to promote diverse thinking, a crucial element in cultivating critical thinking skills that can motivate women to stay in CS [80,90].
Research into why students choose CS [59], why they leave classes [78], and what support they find during classes [50] has been carried out over the years.However, research into these issues is limited and are often single-institution studies that are difficult to generalize.When looking beyond CS at the STEM fields, the research is similarly minimal; the work that exists includes a study that explored the motivations of women in choosing STEM subjects across many colleges in Turkey [31].Furthermore, for women students enrolled in a CS undergraduate program, there is little research into how their choices differ from those of men in selecting the specific electives that make up their program.Elective classes allow students to choose the classes that make up their degree and may encourage students to immerse themselves in new areas [35,49,79].Students take electives not only because they are interesting but because these classes help them to improve skills necessary for their careers [55].The ability to choose empowers students to take ownership of their educational pathway.
Our work builds upon a systematic review of the evidence of effectiveness in broadening participation that was carried out by Working Group 3 at ITiCSE 2021 [66].Among their recommendations was the following advice: "Make connections from computing to your students' lives and interests (Make it Matter), but don't assume you know what those interests are; find out![66, p. 80]".This statement prompted the creation of our working group.
In this working group, we aimed to find out whether women and men tend to take classes with different content when they have a choice.We specifically relate features of the curriculum content (which we refer to as content features) to enrollment numberswe do not collate or analyze data relating to non-content features of electives, such as the instructor for the class.By carrying out a multi-national multi-institution study, we aimed to identify how much of the variability in elective choice is due to specific content features -which we assume is constant between institutions and nations -and how much is due to non-content features, which we assume varies between electives, institutions, and nations.
In particular, we compiled data on elective classes from multiple institutions, coded for class content, and related them to the numbers of women and men who elect those classes.By so doing, we can address whether women and men tend to have different interests in studying CS.The study uses Bayesian networks to determine the impact of curriculum content on elective choice.We theorize and connect with related studies to discuss what drives the choices and discuss the study's implications for improving the gender balance of computing.
We address the following research questions: RQ1: Do women and men enroll in elective classes differently?RQ2: How does elective class enrollment vary by class content?How is this different for women and men?
In this study, we only explore elective classes because the popularity of these classes gives us data about the popularity of the content that they cover.Institutions can use our results not only to redefine their elective offerings but also to re-evaluate the content coverage of the core/compulsory classes that students must take as part of their broader program of study.
As a result of answering our research questions, our contributions to the research community include: • A multi-institutional, multi-national dataset on undergraduate computing elective class enrollment, including class topic content and student numbers by gender -272 classes involving 49,710 enrollments across 9 institutions; • A Bayesian model built from this data to identify how and to what extent class enrollment is related to content and gender We start by explaining the context and terminology of the research in Section 2 before outlining how the existing literature relates to our work in Section 3. Section 4 describes our data and data processing steps, the codes we used to categorize the data, the process of coding we used, and the approach to analysis and modeling.Section 5 explores how the results answer the research questions from various perspectives, and the implications of these are discussed in Section 6.We discuss the scope and limitations of the work in Section 7 and outline intentions for future work in Section 8.The paper is concluded in Section 9.
Given the international context of this work, a common terminology was established for a program of study, class, compulsory class, elective class, and class cohort, which can be found in Table 1.In our study, students in a university setting are enrolled in a program of study for CS.The term class is used in this paper to refer to the individual element of study that is taught to a group of students.In some countries, this may be referred to as a module or a course.Within any given program of study, a class taken by all students is referred to as a compulsory class.Where students have a choice or optional element, this is referred to as an elective class.The number and type of electives may be large, running across multiple fields of study.Elective classes may be selected by CS students and non-CS students.
An example program of study with three compulsory classes is illustrated in Figure 1.In this example, the third year of the Bachelor of Computer Science program consists of compulsory classes and elective classes.The CS student must take all three compulsory classes and any three of the elective classes.A non-CS student may select a CS elective as part of their own, separate program of study.We have also included in this diagram the tags that we would apply to them within the context of this study, which are described in Section 4.2.3 and Tables 3 and 4.

RELATED LITERATURE
Research on enhancing diversity in post-secondary CS programs, particularly concerning women's participation, has explored a multitude of factors within cultural, social, and educational systems.These factors encompass a spectrum of influences, including culturally specific expectations for individuals, socialization processes that may guide individuals of different genders toward distinct occupations, the culture within the computing discipline, secondary educational policies, and prior experiences (e.g., the availability and compulsory or elective nature of computer science in secondary schools).Additionally, these factors can also include post-secondary efforts to promote diverse enrollment in computing degree programs, policies (e.g., admission barriers for students with or without prior computing experience), support systems (e.g., mentoring, role models, advising, extracurricular groups), pedagogy, classroom experiences (e.g., collaborative learning opportunities), studentstudent and student-faculty interactions, and the relevance and meaningfulness of the curriculum for women [1,25,73].The focus of the study presented here is within the latter: the content of the curriculum, and specifically, topic choice within CS degrees.
Below, we briefly discuss how the outputs of the prior ITiCSE working group that inspired this research relate to our goals in Section 3.1.We then discuss existing work on students' elective choices in Section 3.2.Exploring existing scholarship allows us to understand better what aspects of computing are appealing to students in general and introduces frameworks and theories that look at how people make choices.We then analyze literature that looks either specifically at women's choices or contrasts women's and men's choices in Section 3.3.
We aim to maintain a focus on the external and societal factors that influence choices in CS while avoiding any language that might imply that these choices are inherently gendered.Our discussion here also emphasizes the need for cultural sensitivity and the limitations of generalizing findings across different global contexts.The literature we discuss in this section is largely situated in the United States as there is very little research from elsewhere, and the data we are using in this paper is from institutions from North America and Western Europe.This limits the global applicability of our conclusions, as we are aware that trends around women in computing vary across the globe, as discussed in Section 1.

Project Motivation Background
One of the earliest studies of women's interests related to CS was an ethnography conducted at an elite, private university in the United States [62].In their study, Margolis and Fisher found that while women and men shared many interests, men were more likely to express interest in hacking and in studying the computer itself, while women were more likely to express interest in the context in which computing was applied, the link between computing and other disciplines, and computing's contributions to society.Other researchers supported the generalizability of these findings through quasi-experimental and interventional studies [8,9,40].The National Center for Women & Information Technology's (NCWIT) Engagement Practices Framework [30] is based on these and other studies related to connecting the content of the curriculum (degree

Class Text Mining
The title of the individual element of study whether core or elective.

Compulsory class Bachelor Thesis
This is a required class that students in the Bachelor in Computer Science must take in year three of the program of study.

Elective class Computational Intelligence
This is a class that students in the Bachelors in Computer Science may choose to take as one of their 3 optional classes in year three of the program of study.

Class Cohort
Fall 2023 Text Mining Students The group of students that attend a particular iteration of a class in a given term.

Cohort
Third year honors students A group of students choosing from the same list of elective classes.Using the NCWIT framework as a guide, a working group from ITiCSE 2021 carried out a systematic literature review to identify empirical support for the recommendations of teaching practices that support engagement by women in computing programs [66].That literature review focused on the computing topics and application areas that influenced the program choices that women made in addition to interactive approaches to pedagogy, collaborative teaching, and a focus on socially relevant issues.The 2021 ITiCSE working group noted that "most of the work [. . .] evaluated interventions implemented at only one institution" and identified the need to "disentangle effects".The study presented here focuses only on topics and application areas and intends to address this criticism.

Students' Choices in Computing
A great deal of scholarship discusses what leads students to make the choices they do.In this section, we first discuss theory around, then empirical studies on, student choice.Expectancy-value theory [32] postulates that student motivation comprises two factors: students' expectations for success and their perceptions of the value of doing something.Both are required to motivate a student.The premise states that a student will be more motivated to take a class if they are expecting to do well and perceive there is value to taking the class.By contrast, if students think they will do well, but see no value in the class, the theory postulates that they will not be motivated to take it.The reverse is also true: if students see value, but do not think they will be successful, they are less likely to take the class.
What counts as success goes beyond academic performance, which is explained by other theories, such as social learning theory [7] and field theory [63].Social learning theory emphasizes that people interact with and learn from learning environments.In this view, students are psychologically embedded in a classroom social space and the experience of this learning environment shapes their expectations, values, and then, their motivation.This is different from thinking about motivation as though it's an inherent and unchangeable feature of a person and different from a deficit model, which implies something is missing in the student herself [65].In addition to physical space, educational learning environments include affiliations with other students and faculty and a sense of belonging.Affiliation is the sense of friendship with students and teachers, getting to know them, learning collaboratively, and enjoying working together.Belonging is a student's feeling of being accepted, valued, included, and encouraged by other students and teachers.[38].In classrooms where students feel they belong and that they are supported interpersonally, they are more motivated to engage in important academic tasks and events [82].
Field theory is similarly concerned with the experience of an individual in a social and physical space.A field, or social space, consists of actors who interact, cooperate, and compete with one another [13].Drawing from the physical sciences, the notion of field focuses attention on how elements within a space or setting are influenced by other elements in the setting and the setting's characteristics [63].A football field illustrates the relationship well: the positions in the field are occupied by agents, who then follow rules and operate within borders and on turf with specific features (e.g., artificial or natural).The game is competitive, with agents using strategies to maintain or improve their positioning vis a vis other players and features of the playing field.Educational settings can be viewed as socially reproductive fields.When the setting is competitive, those who benefit the most often hail from socially, economically, or educationally advantageous positions [14].Bourdieu refers to these people as the dominant class, people who are deeply embedded in the system with their practices and discourses.These possess higher levels of social capital, which is then used for leverage in the field.When a field (such as CS) is dominated by a subgroup (such as men), then the rules of the game are set by this subgroup, who then define what is "proper" computer science in a way that is arbitrary and to their advantage.
In CS research, educational advantage has been referred to as preparatory privilege [61].Those with preparatory privilege are students with prior programming experience, knowledge, and/or skills not held by their less-prepared counterparts.They often have positive views and dispositions towards the discipline that aligns with their future goals.These students may also share a common language and discourse related to CS, and a specific style of communication and interaction with their peers of the same position and with their teachers [47].Those who do not share that common language and communication style are likely to have greater difficulty feeling they belong.
Beyond issues related to the social setting, there are very practical aspects of student choices.In addition to completing a set of required classes within a program of study, students often need to complete a set of elective classes to fulfill graduation requirements.Electives allow students to explore or to focus on areas of interest, though students do not always desire to take electives nor do they always get to take the electives they wish to take.Recent research on elective choices has examined students' reasons, which often reflect interests.For example, a study of why students choose to enroll in Massive Open Online Courses (MOOCs) highlighted four clusters of students: those concerned with good grades, those who wanted to learn, those who wanted to be with or make friends, and those who both wanted to learn and get good grades [58].Another study of students in the sciences found that students were most concerned with career usefulness, good grades, and ease of getting credit [71].Interestingly, that study suggested that men may be more likely than women to take classes for social reasons, which is consistent with a recent study of CS majors [49,71].The study also found that social science classes were taken more for interest than STEM classes, that humanities classes were taken for self-development, and that CS, engineering, or physics class-taking stemmed from a desire for career skills [55].
Some research has been done on students' elective choices within CS.Elective classes are available from the CS department itself as well as other departments for multidisciplinary learning [79].Electives give students the flexibility of focus, providing students the opportunity to choose different paths in the program of study [60], and which may ultimately support different career paths.Electives may also give a different perspective on the current career with integration or collaboration of other fields.In CS, offering students more elective choices can promote autonomy, which can encourage lifelong interest and may encourage broader participation [35].When students initially start choosing classes, they may not have an interest but are trying to find out if they have an interest or will enjoy the class for other reasons [43].A recent survey-based study asked students the importance of seven concerns when choosing classes, though the paper was non-specific as to whether the classes were elective or compulsory [49].Consistent with the NCWIT Engagement Practices framework, personal interest in the class content was considered most important, with no differences found between women and men.Interest was followed closely by a non-contentrelated factor, class credits that contribute to one's degree program, and a factor that could be both content-and non-content-related, preparing the student for future work.Other typical non-content related factors included fitting into students' schedules and knowing another student in the class.The study presented in this paper addresses only content-related factors.

Women's Choices in Computing
Interests are not necessarily inherent to gender.Instead, stereotypes and social expectations play a role in forming values and beliefs, which in turn influence students' educational and career choices and success [18].This understanding is crucial when considering the Things-People dimension, which has been used to explain differences in men's and women's preferences [22,83].While it has been observed that "men may show a preference for working with things and women for working with people" with women expressing stronger 'Social' and 'Artistic' interests than men, these trends should not be viewed as essential differences.These are more likely reflective of societal pressures, where men are often expected to prioritize income as breadwinners, and women may have broader leeway to pursue interests that are less directly tied to financial outcomes.
In the context of computing, these findings are mirrored.Largescale studies have indicated that women are more likely than men to prioritize helping others and maintaining a work-family balance, while men place more value on earning potential [11,12].These differing values can lead to different elective choices.Research by Kilkenny et al. [49] and Payton et al. [74] supports this, showing that women CS majors often seek to understand and address community challenges.Cohoon [24] further found that women's participation in CS is bolstered when they perceive their involvement as meaningful and socially contributive.The presence of electives that appeal to women's interests may also attract women to the major.
However, it is essential to recognize that the social climate within CS and related fields can be unwelcoming to women and that these negative forces can deter women from pursuing and remaining in computing fields, which is a critical aspect that requires more immediate and prominent attention in discussions about gender and CS.
A few studies have delved into specific elective choices of students within CS departments.For instance, a study in Greece found that over six years, women tended to select more theoretical computing classes, while men appeared to choose systems classes, although the study did not statistically test these observations [51].They also noted that women enrolled in more humanities and social sciences classes.Another study found that while women may not be attracted to traditional gaming projects, they are more drawn to serious game projects, which avoid endorsement of the geek gaming culture [84].
When it comes to publication trends, women have been found to contribute more to fields such as human-centered computing and less to areas like computer vision and algorithms [2,16,26,89].In summary, while research has examined why women pursue CS majors and what retains women, little is known about what topics they pursue when they have a choice.The study presented below provides insights into the elective classes of undergraduate women, choices that may influence later CS subfield choices.

METHODS
Our study focused on bringing together data from nine institutions to answer our research questions about women students and elective classes.Details of the nine institutions, the computing programs they offer, and the data they contributed are given in Appendix A. We used convenience sampling techniques for data collectioncollecting data only from the authors' institutions -to collect data regarding undergraduate student enrollment in elective classes and class descriptions that are presented to students.We then engaged in two stages of coding the class descriptions, followed by using our coding and enrollment data to model women's elective class enrollment using Bayesian techniques.An overview of our data collection and analysis methods can be seen in Figure 2.
All authors that retrieved data from their institution approved this research with their respective ethics review boards.In most cases, the ethics boards declared that this research was not human subjects research due to the level of anonymization within the data being requested.After receiving approval from their respective institutions, the authors requisitioned data for all CS elective classes at their institution.The data was expected to include overall enrollment numbers, as well as a separation of the number of women and men enrolled in that class in a given academic year.The data on gender were extracted directly from local information systems, which are based on student self-reports.A summary of this data can be seen in Table 2.
Our data represented class enrollment across three academic years (2020-2021, 2021-2022, 2022-2023) whenever possible.In one case (Kiel University), only one year was provided.Collecting this class data over multiple institutions and years allowed us to treat non-content-related features, e.g.instructor characteristics, as "noise" in the system -we did not collect data on these noncontent-related factors.
Any data regarding students who did not identify as a woman or a man (e.g., non-binary) were not included due to low sample sizes increasing the risk of identification of the student (and thus de-anonymization).Students were included in the data regardless of their program of study.All the data was historical, meaning the data represents the enrollment in the class at the end of the class; the data does not reflect students who initially enrolled and subsequently withdrew from the elective class.We did not collect data on whether a student may have been repeating a class, either due to being able to take electives multiple times or failing and needing to take the elective again.One exception to this was Uppsala University, where the data consisted of only first-time registered students in the class.
Cohort sizes were also gathered to capture the proportion of students enrolled in an elective compared to all the students who had the potential to enroll in that elective.As defined in Table 1, a cohort is a group of students choosing from the same list of elective classes.
In some cases, it was challenging to determine the cohort total.Open universities, such as UOC Spain, often have a different, flexible approach to structuring classes and student progression compared to traditional universities.This flexibility means that students can enroll in classes at different times, take classes in any order, and complete them at their own pace.As a result, there is no fixed starting point or progression timeline for a specific group of students, making it challenging to form traditional cohorts.Further details of the institutional contexts for each of the contributing departments are given in Appendix A.
4.1.2Data Processing, Cleaning, and Validation.Once data were received from the institution, the data were standardized in terms of format.Any enrollment data were rounded to the nearest multiple of three to minimize the risk of de-anonymization.
For each institution, a set of validation checks was carried out by a working group leader to identify if any of the following situations were true: • Missing enrollment data for cohorts at an institution.
In some cases it was difficult to identify exactly what the cohort was, i.e. the group of students that had access to the same set of CS electives, particularly where there were students from other programs taking classes within CS departments.• Classes that had enrollment higher than the specified quota.Few classes had reported quotas, or the maximum number of students allowed to enroll in that class (this may also be referred to as a "cap" on enrollment in some contexts).For the few that did, none of the raw enrollment data exceeded the stated quota.• Classes that had enrollment higher than the specified cohort.The only time this happened was when the cohort of CS majors was smaller than the number of students choosing the elective because the full cohort was difficult to identify.This was because access to classes was very open -even to the public -so adjustments to the cohort size were made to reflect this.See Appendix A.8 for details • Uneven distribution of class enrollment across academic years.Having decided that our sample would be taken from three academic years, enrollment data outside this time frame were removed.Where there was uneven distribution, this highlighted some missing data that was followed up for inclusion in the final dataset.
Questions on the completeness and accuracy of the data were followed up with the author based at the institution.
In cases where researchers received identifiable data from their institution, that researcher worked to anonymize the data for inclusion in this study.These data were then shared privately amongst the group via a shared Google Drive while these validation checks were carried out.Once the data were cleaned and validated, they were moved into a publicly accessible GitHub repository 1 , where they will stay, along with the scripts used to process the data.This allows for clarification, verification, reproduction, and use in future studies by us or others -see Section 8.

Class Descriptions
4.2.1 Data Collection.The authors gathered textual descriptions of the elective class content as provided to students, based on the electives for which they submitted enrollment data.Class descriptions are often publicly available documents or websites.In some cases, class descriptions were scraped from these public-facing websites.In other cases, class descriptions were gathered by hand (i.e., copied and pasted from a site into a document).
Three of the institutions included in this study provide class descriptions in a language other than English.Each of these cases, and how the descriptions were translated, are described below.
UOC. Class descriptions were only available in Spanish and Catalan.They were translated using Google Translate and then reviewed to ensure they were intelligible.The adequacy of the aforementioned approach was deemed sufficient for coding.
Uppsala University.Class descriptions were available in Swedish and English.For this study, the English class descriptions were chosen, as they provided sufficient information for coding purposes.
Kiel University.Some class descriptions were in German.These were translated into English by two scholars bilingual in German and English and subsequently reviewed by three additional bilingual scholars to ensure the translation had been done properly.

Data
Processing, Cleaning, and Validation.Any duplicate entries were removed, and the results were compared with total enrollment data as a validity check.

Coding.
Our coding process aimed to provide quantitative elective characteristic data based on topic areas from the ACM curriculum guidelines from 2023 [53] and non-computing application areas taken from the top level of the Common Aggregation Hierarchy (CAH), developed by the UK Higher Education Standards Agency (HESA) [42].The codes used for computing topics and application areas are summarized in Table 3 and Table 4 respectively.
When looking for appropriate software for coding, the goal was to find something that was accessible for all authors, collaborative in nature, and appropriate for the coding goals.Ultimately, the decision was made to use DiscoverText [81], which fulfilled the requirements mentioned above.
As researchers coding qualitatively, we recognize that our backgrounds and experiences can influence our interpretations of both our qualitative and quantitative data.In our group, eight authors identified as women and four authors identified as men.The group consisted of graduate students, researchers, and faculty to capture a range of perspectives, with some researchers being members of groups supporting women's empowerment in computing (NCWIT and ACM-W).
There were multiple stages of coding involved with the elective descriptions.Each of these stages is described in detail below.Stage Zero -Trial coding.The goal of the trial coding stage was for the authors to familiarize themselves with the codebook, dataset, and the coding software, DiscoverText.In addition, the authors wanted to ensure that the ACM topic areas and non-computing application areas, as can be found in Tables 3 and 4, were usable for coding.Each of the authors was assigned 50 class descriptions for the trial.During this process, the authors found that some codes were difficult to distinguish, while other codes were found to be too broad.Take, for example, the codes Software Engineering and Software Development Fundamentals.Both of these topics seem to apply to certain classes, even though the codes themselves are distinguishable in the ACM guidelines.Other classes were difficult to code because the topics covered, such as Information Systems, were not mentioned in the ACM guidelines.And, some application areas were found to apply to many classes, such as Engineering and Technology (CAH10).While some discussions were had between the authors during this process, the goal was not yet to reach a high level of agreement between coders.Instead, authors were encouraged to familiarize themselves with the codes and class descriptions and to code as they see fit.
Stage One -Main Coding.After trial coding, all authors engaged with the main coding stage.This entailed each author coding 142 elective classes using the ACM and CAH codes.Each elective class was coded by six authors.After this stage, some elective classes were excluded because it was decided the class was not within our scope, either because it wasn't an undergraduate class or because it wasn't a computing elective.Additionally, we coded some classes that did not have corresponding enrollment data which were subsequently excluded from the analysis.After these exclusions, we had 272 elective classes in our analysis.
A Krippendorff alpha value was also calculated for the main round of coding (=0.50).This value is not high according to some measures of significance.However, it is worth noting the context: there were several degrees of freedom across multiple dimensions of the process (6 participants making judgments on 272 classes using 37 possible codes), and the ratification process addressed the most controversial classes.We consider the initial level of agreement and the steps taken to improve consensus to be reasonable in this context.
Stage Two -Ratification & Consensus.To address classes that had low agreement in the codes assigned, the authors collaborated to ratify codes to resolve disagreements.This stage involved authors working in groups of two to three to discuss a specific content code or group of codes, from ACM or CAH.All classes associated with a certain ACM code were investigated and discussed by the group, which ultimately assigned each class a '1' if the class did contain that content or a '0' if it did not.All ACM codes with low reliability were discussed in turn in this manner; HCI, NC, PDC, and SF were the only ACM areas that were not reviewed again in this stage because they had high reliability from the Stage One coding.
Policies regarding coding the ACM areas were codified during this process (see Appendix B).
Following this, 17 out of 272 of the elective classes were identified as not having at least one code assigned by the majority of coders in Stage One, or a group of coders in Stage Two, leading to high inter-coder disagreement for these classes.Examples include Quantum Computing, Game Design, and Information Systems.For such cases, the decision was made to discuss the issues and reach a consensus among the authors.Game Design, for example, was coded with Graphics and Interactive Techniques (GIT) and Specialized Platform development (SPD), with an application area of Creative Arts and Design (CAH21).Similar coding rules were designed for other classes with high inter-coder disagreement.For the complete list of coding policies that we developed and used, see Appendix B.

Interdisciplinary Content.
Classes that focused strongly on interdisciplinary content proved difficult to code.These electives tended to assume a prior level of competency in a wide range of disciplines such as mathematics, architecture, and biology.For example, in classes focusing on quantum computing, linear algebra, algorithms, circuit models, graph theory, calculus, modeling, and cloud computing all feature in the content section of one class.Given the description provided, there is no single code that stands out clearly as a viable representation of the class content.This applies to several frequently encountered classes, including game design, big data, computer vision, bioinformatics, and information systems.
In some cases, policies were established to code such classes in specific ways to ensure that the coding was consistent.However, in many cases it was not clear exactly which codes should be used consistently and which classes were interdisciplinary, leading to discussion and a lack of consensus on some of these codes and classes.
A possible solution to this issue was to create additional codes to use with the dataset.However, this would also introduce several confounds.If new codes were introduced, the entire dataset would have to be re-coded by all the participants.We decided early in the process to use an existing code book, and attempting to change this later would have caused substantial threats to the validity of the coding process.
Nevertheless, it is worth noting that, in our experience, the ACM curriculum and CAH areas lacked the coverage and nuance to effectively code every class.Despite this, many classes did fall neatly across one or two codes, making this a problem that was not encountered with concerning frequency.

Data Analysis and Modeling
Our data were analyzed using a hierarchical Bayesian network, in which content-related factors (i.e., ACM topics and CAH application areas) and non-content-related factors are represented as latent variables.The content-related factors have the same values across all electives and institutions, and the model identifies the best fit.The non-content-related factors are independent for each elective class, but they are identically distributed (i.e., same standard deviation).The model identifies the best-fit value for the standard deviation of the non-content-related factors.

Bayesian modeling.
The Bayesian approach is rooted in the idea that probabilities reflect the degree of belief or evidence in support of an outcome.A Bayesian model expresses a prior set of beliefs, encoded as a probability distribution.When new evidence is available, the probabilities representing beliefs are updated in light of the new evidence, but also taking into account previous evidence or beliefs.This is captured in Bayes' theorem, which relates the new (posterior) probability to the old (prior) distribution through the newly available evidence.
In a Bayesian network, a set of different variables is defined and are related to one another.For example, the standard deviation of one variable may itself be another variable in the network.This means that Bayesian networks can be used to represent situations where observed data are random variables whose distributions have parameters that depend on a range of different features.A classic example is to consider the effect of an educational intervention by carrying it out in multiple schools.Each school by itself is not large enough to provide statistically significant results, but the characteristics of the schools vary so we can't simply aggregate all the results.The Bayesian network approach is to have the effect of the intervention modeled in one variable, parameterized by an unknown mean and standard deviation, and the effect of each individual school modeled by another variable.Sometimes we have an unknown parameter that is used in more than one place in a model.For example, our model may assume that the same standard deviation of test results applies in all schools, but with different means.This is referred to as a 'pooled' parameter because it is shared by more than one of the distributions in the model.
The prior distributions of these unknown parameters are set up to be appropriately 'vague', i.e. a wide distribution of values is likely.Each observation, i.e. a real data point, provides more evidence about the likelihood of certain parameter values.Once all of the observations have been taken into account, through repeated applications of Bayes' theorem, we have new (posterior) distributions for the parameter values.This is all complicated by the fact that we usually can't solve the equations that arise from the multiple applications of Bayes' theorem analytically, so we have to use numerical estimation, often using a Monte-Carlo Markov Chain simulation.Once this process is complete we will have new estimates for the distribution of our unknown parameters, for which we can find a mean and standard deviation, in the usual way.We can also find what is referred to as a 'credible interval' for each parameter, i.e. upper and lower limits between which a certain proportion of the distribution lies.This is the Bayesian analog of the 'confidence interval' used in classical frequentist approaches.For a more complete introduction to Bayesian modeling see Nicenboim and Vasishth [68].

Scoring and ranking electives.
Central to our model is the evidence we have relating students' enrollment in certain elective classes.We assume that the way students choose electives is to rank the options available to them and then choose from the top of the list downwards.In our model, we aim to construct a scoring function for electives, which will vary from student to student, so that the ranking that students carry out agrees with the scoring function: higher-scoring electives are picked in preference to lower-scoring ones.This is an oversimplification because the choice of electives may not be independent of each other: there may be combinations of electives that are not allowed or there may be restrictions on the number of students that can take particular electives.We don't know what this scoring function is directly, and we can only see its effects when choices are made between electives, but can try to infer how it operates.
Theories such as Expectancy-Value Theory (EVT) [91] and Theory of Planned Behavior (TPB) [3] identify a range of factors that affect people's decisions.The likelihood of any particular action being followed depends on the characteristics of the action and how important the different characteristics are to the person deciding.The importance of these factors is represented by weights, which vary from person to person and are socially and culturally influenced.These theories are used to predict decisions about whether or not a person will do something, but we follow the same idea to construct elective scores and hence ranked lists of electives and hence sets of chosen electives.Each elective has its own fixed characteristics, but individuals weigh these characteristics in different ways and so score the electives differently and make different choices.

4.3.3
Content-related and non-content-related scoring.We separate the characteristics affecting elective choice into features based on the content -computing topics and application areas -and non-content-related features, e.g.instructor.We start with the assumption that the score () given by a learner () to an elective class () will be distributed as follows: This says that the score is formed from the sum of two normally distributed variables, corresponding to a content-related score and a non-content-related score.
() is the mean of the content-related term, which depends on the learner and the content of , and    () is the standard deviation of the the content-related term.There are similar terms,    () and    (), for the non-content () related factors.The content-related factors are based on the curriculum coding we carried out.Although different students choose different electives, we assume there will be some trend in the choices made.In coding electives, we ended up with a binary association of content features with electives, so each topic and application area was labeled as either present or absent within the elective.A further simplifying assumption is that the distribution of the content-related score for an elective depends only on the presence or absence of these content features so that the distribution of the content-related score is the same for electives with the same content at different institutions.Assuming that the scores are normally distributed and that students have the same standard deviation in their scoring of electives, this simplifies down to saying the distribution of the content-related score depends only on the contribution that individual content features make to the mean of the distribution.More popular topics will have a higher mean than less popular topics.The contributions from different topics are added together, so if an elective includes a popular topic and an unpopular topic these features will tend to cancel each other out.Figure 3 shows how this works: each of the classes has a binary pattern of features that describe its content.Each content feature has a parameter in the model which is included in the score summation for the elective if We didn't collect any data on the non-content-related characteristics (e.g.instructor characteristics) so all of the non-content-related characteristics are combined into a single value per elective within the model.We assume that the non-content-related score is also normally distributed.
When ranking based on scores, the result depends only on the difference between the scores, so if we add a constant to all of the scores it does not affect the ranking, and hence on elective choices.Therefore we can fix the mean of the distribution of the noncontent-related score without affecting the outcome, as it is added to all elective scores.As we are characterizing the non-contentrelated score as noise, it makes sense to fix its mean to be zero.The non-content score was then parameterized only by its standard deviation and appeared essentially as noise since we did not collect data about the possible non-content features.The content score is slightly more complex because the mean of the distribution depends on the content area itself -we are expecting some topics and applications to score higher than others.We can fix the mean and standard deviation of the content factor distributions arbitrarily -the absolute value of the mean is unimportant because we are only considering orderings.The size of the standard deviation for the distribution of content weightings is also arbitrary: all that is important is its size relative to the standard deviation of the noncontent-related features.If the variance of the non-content score is noticeably larger than that of the content score, the non-content noise will hide any noticeable structure from the content features.We arbitrarily fix the mean and standard deviation of the content weighting distribution to be 1000 and 200 respectively.
We have described a lot of assumptions that we have made in building the model.If these assumptions are incorrect, then there will be no discernible pattern in the content-related scores, and students will choose electives independently of the content-related score that we calculate.This would show up in the model as small values for the individual content scores and/or large variance for the non-content-related score.

Gender effects.
With no information to distinguish the students, their prior class scoring distribution would be identical.However, we can include different versions of each parameter for women and men.For content-related features, this was done by providing a parameter for the overall popularity score and another parameter for the difference between the popularity with women and men.These we combined to define the mean of the content-related portion of the score as follows: We separately modeled the non-content variance for women and men, to see whether content-related features had different importance according to gender.

Modeling in STAN.
We removed from the model any content features (ACM topics or CAH application areas) that were not identified as tags by a majority of coders for at least two electives.If there are no electives tagged, then there is no data for the model to base its estimation of content feature weights.Because of this, we excluded Systems Fundamentals (SF) from the ACM topic list which, as the name suggests, is foundational content more likely to be taught early on as a core part of a program.The other ACM topic that was identified as foundational is Software Development Fundamentals (SDF), but this is included because there are elective classes in SDF offered mainly to non-CS students.
We determine the weightings of the content features by finding the optimal value of the parameters to fit the data through Monte Carlo Markov Chain (MCMC) simulation.The whole model was implemented in Stan probabilistic programming language [86] and the full model is listed in Appendix D. Bayes modeling and Stan in particular have been used for a wide range of applications, including previous research in computing education [10].Various Python libraries were used to prepare the data for the Stan model and to analyze and visualize the results.
MCMC estimates the posterior by repeated random sampling.In general, the more samples that are taken, the better the approximation to the posterior distributions, but the more time the computation takes.Once a model seems to converge, the intent is to increase the number of samples until the estimate is 'good enough', which can be measured by a variable referred to as R. The usual guidance is that a value of less than 1.1 for R is desirable, but the closer to 1 the better.

Elective Class Enrollment
Our first research question (Do women and men enroll in elective classes differently?)focuses not on the content of the classes and whether that contributes to choices, but on whether or not women and men make different choices.We address this by looking at the electives individually and comparing the proportion of all women who can choose to take the elective, rather than the proportion of all people in the class who are women, to the proportion of all men who choose to take the elective.We do this by calculating the odds ratio, dividing the proportion of women choosing an elective by the proportion of men taking an elective.If this ratio is one, then the class is equally popular with women and men; if it is greater than one then it is more popular with women than with men.
To calculate the odds ratio, we need to know the size of the cohort from which the students taking the elective are drawn.Figure 4 shows the distribution of the natural log of these odds ratios, where the log transformation was taken to allow for symmetry about the x-axis to enhance interpretability and also to relate to effect size.Here, the zero on the y-axis corresponds with an odds ratio of one, i.e. equal popularity.Small, medium, and large effect sizes correspond with log odds of 0.3628, 0.9069, and 1.4510 respectively [21], and these are indicated as horizontal lines in Figure 4.All of the odds ratio calculations aggregate over all available offerings of the elective-typically the same class offered across multiple academic years.
We calculate the confidence interval for each elective and also plot them on Figure 4 where they indicate statistical significance at the 95% significance level.When considered independently, 59 of the 272 electives show statistically significant differences between enrollment for women and men.Combining these by using the binomial distribution, the probability of these occurring by chance is 0, to as many decimal places as we can calculate.For comparison, if the number of significantly different individual electives was only 22, then we would have  < 0.05, and 26 classes or more would give  < 0.005.Because we have 59 modules with significantly different enrollments, we see that there is a highly significant difference between the elective choices of women and men, positively answering RQ1: women and men choose electives differently.
As seen in Figure 4, it is notable that substantially fewer of the electives are more popular with women (bar above the x-axis) than with men (bar below the x-axis).We can also see in Tables 5a and 5b that a substantial number of electives have a notable effect size: all classes with an odds ratio indicating a moderate effect size (equivalent to Cohen's  > 0.4) are included in the tables, where the total rounded class size is 15 or more.
In Table 5, we see the most relatively popular and least relatively popular elective classes in our data set.These are specific classes at a specific institution in our data.Some of these align with reports in the literature.For example, HCI was introduced as an intervention to encourage women's participation [62] and there are two HCIrelated classes listed in Table 5a.Other themes within the topics and application areas reflect prior work, including a focus on society and community [24,49,74], such as with IT, Ethics, and Organization or Public Engagement in Computer Science.There are also different electives with the same title (Natural Language Processing) that are very different in terms of popularity: in one case all of the women in the cohort chose it, but in the other case no women chose it.This demonstrates that, while there are significant differences between the electives the women and men choose, it looks like these differences are not solely based on content or non-content factors, but on a combination of the two.Teasing out the contributions of these different factors is the subject of our next set of results.The coding scheme we use does not allow us to go into the specifics of every sub-topic (e.g.Natural Language Processing is part of the Artificial Intelligence topic with the ACM classification) but the broad patterns identified within the ACM topics are covered next.

Elective Content: Gender-based Differences
Our second research question (How does elective class enrollment vary by class content?How is this different for women and men?) brings in our computing topics and non-computing application area coding analysis.Using the results of coding the class descriptions (as detailed in Section 4.2.3) in a hierarchical Bayesian network (described in Section 4.3), we could determine how popular different elective class topics were with students, both overall and comparing women and men.6 shows the number of classes tagged with each ACM computing topic and CAH non-CS application area.

Overall Popularity of Topics. Table
Software Development Fundamentals (SDF) proves to be the most popular of all topics, as can be seen in Table 6 and also in Figures 5a, 6a and 6b.SDF is the topic of Introduction to Computing classes offered at Kennesaw State University and Universitat Oberta de Catalunya.Traditionally, SDF classes have been included in the mandatory/compulsory curriculum of computing programs.However, these two universities have taken a different approach by utilizing this class as a more gradual introduction to programming, intending to develop more pathways for students to pursue a computing academic career.
The other most popular ACM topics overall are SEC (Security), SE (Software Engineering), and HCI.The interpretation of the posterior mean values has to be done with care, because of the height of the credible interval.While Figures 5 and 6 order the bars by the mean of the posterior distribution (marked with a dot), where the line joining the dots is close to horizontal there is little certainty about the ordering because the credible intervals have significant overlap.FPL (Foundations of Programming Languages) and PDC (Parallel and Distributed Computing) show a markedly unpopular overall opinion and a reasonably tight credible interval.OS (Operating Systems) and AR (Architecture) are also unpopular overall, but with a broader credible interval-largely because they are only associated with a small number of electives (five each).SDF only labels two electives, which is why it has such a broad credible interval, but it is so markedly popular that the bottom end of its credible range is still higher than the top end of any other topic.
Within the CAH application areas shown in Figure 5c, the credible intervals are generally much broader than for the ACM topics, again because of the small number of associated electives.Only CAH17 (Business and Management) has more than ten electives, and so has a tighter interval.We have good evidence that CAH03 (Biological and sport sciences), CAH21 (Creative arts and design), Horizontal dashed lines indicate effect sizes as calculated from [21].Vertical error bars show 95% confidence intervals where they do not include 0, i.e.where the difference is statistically significant.CAH16 (Law) and CAH17 (Business and management) are overall more popular than CAH22 (Education), but otherwise, the outcomes for application areas are inconclusive -the credible intervals overlap so there is not strong evidence of difference.

Differences in content popularity between women and men.
Figures 5b and 5d show the popularity difference for women and men in computing topics and non-computing areas.The magnitudes of these differences are generally smaller than the differences between topics overall, which is to be expected.Also, as we might expect, the credible intervals are relatively large because they rely on data about women's choices.There is less of this data because the model is working on subsets of the whole dataset, and imbalanced subsets at that.There is good evidence that FPL (Foundations of Programming Languages) is more popular with women than with men.Yet, this is not to say that FPL is often chosen with women, given that the overall popularity of FPL is low.However, we can say that FPL is less unpopular with women than with men.GIT (Graphical and Interactive Techniques, which would include games electives) and AL (Algorithms and Complexity) are also more popular with men than women, according to our model results.
Figure 5d shows the differences between the popularity of different application areas for women and men.This graph is characterized by very wide credible intervals compared with the magnitude of the differences.This reflects the small amount of data, due both  Table 5: Elective classes at individual institutions with large magnitude log odds ratio.  and  are the number of women and men taking the class respectively, and   and  are the total number of women and number of men in the cohort respectively.() is the natural log of the computed odds ratio, which acts as an effect size calculation in our model.Only electives with a rounded class size of at least 15 and at least a medium effect size ( > 0.5) are included.
to the small number of elective classes involved and also the relatively small number of women within the whole dataset.There is a slight uptick at the positive end of the graph, with CAH02 (Subjects allied to medicine), CAH22 (Education and teaching), and CAH15 (Social Sciences) perhaps showing a tendency of popularity with women.This would be in line with national-level HESA statistics in the UK, where these three areas are all in the top six subjects chosen by women at the degree level.
In the model, the contribution of a particular topic to average elective popularity for a woman is equal to the overall popularity of the topic plus half the popularity difference between women and men.We can easily calculate the means of these distributions, but there is not a simple way to find the credible interval.Thus, Figure 6 plots these mean values without error bars.We should take care though, because the credible interval will generally be wider than the originals.There is a discussion of how these findings relate to the literature in Section 6.

Importance of non-content features.
All non-content features for an elective (e.g.instructor characteristics) are combined into a single parameter.The non-content features are assumed to be independent and normally-distributed variables, with a mean of zero and an unknown, but shared (pooled), standard deviation.Based on the data provided, the model updates the distribution of this standard deviation.The larger the non-content-related standard deviation, the less impact the content-related factors will have on student choices.
Separate standard deviation () parameters are included in the model and the summary of their posterior distributions is given in Table 7.What is most noticeable is that the mean of the standard deviation for non-content (women) is noticeably lower than that for the non-content (men).Further, the upper bound of the credible interval for the standard deviation for non-content (women) is less than the lower bound of the credible interval for non-content (men).This means that we have credible evidence that women are less influenced than men by non-content features in their elective choices.This, in turn, means that offering popular elective content should disproportionately increase the popularity of electives with women.We also report the values for R achieved after 100,000 samples in our model.Because these are all less than 1.1, we have confidence that they are a good estimate of the true posterior distribution.

DISCUSSION
Whilst it is clear that women have a wide range of interests and that patterns of preferred classes mirror those of men to a fair extent, there are also interesting differences.To return to our research questions, we can conclude the following: RQ1: Do women and men enroll in elective classes differently?The data indicates that the enrollment patterns in elective classes do differ between women and men, with more available classes that appeal to men than to women.If degree curricula favor subdisciplines that do not appeal to women or are not inclusive of women, then this creates an issue in equity of access to computing degrees as a whole.This can be interpreted from a Bourdieusian perspective, as the rules of the field -which control which subdisciplines are selected -are being defined by men.
The results presented in Section 5.1 illustrate that there are far more classes that appeal to men than to women, with classes that are proportionally more appealing to women being clearly in the minority.When the available elective classes are unappealing to women, it may reduce the chance that they pursue the degree program or may result in greater attrition of women.
RQ2: How does elective class enrollment vary by class content?How is this different for women and men?Class enrollment varies substantially between women and men across different topic areas, with System Development Fundamentals, Security, Software Engineering, HCI and Society, Ethics, and Professionalism as the most popular topics.The least popular topics for the population as a whole were Foundations of Programming Languages, Parallel and Distributed Computation, Operating Systems, Algorithms and Complexity, and Architecture.
The finding that HCI (Human-Computer Interaction) is the most popular non-foundational elective with women is consistent with research on the Things-Humans dimension, suggesting that women are more interested in humans than are men and that men are more interested in things [22,83].However, at fourth place in men's top five, men were not far behind women in terms of the appeal of HCI classes.The HCI appeal to women at the undergraduate level no doubt contributes to research findings that women in research careers publish heavily in HCI journals [2].
As the number of jobs in AI continues to increase [57], we expected AI classes to be a popular elective choice.In terms of gender, we also noticed that while AI appears to be in the top five choices   for men, it did not appear to be one of the top five choices for women.This might indicate that women students are hesitant to take classes in AI despite its growing popularity.A recent interviewbased study suggests that women and men perceive AI to require mathematical skill [70], which may put off women because of lower self-efficacy in math [37].Interestingly, MSF (Mathematical and Statistical Foundations), which is also mathematics-heavy, was more popular than AI for women.In the United States, women typically make up about 40 percent of mathematics and statistics majors, and high school girls make up more than half of Advanced Placement statistics and calculus class-takers.Thus, we speculate that a factor other than mathematics itself, such as the belief that one might belong in the social setting or possibly a competing goal, such as the desire to take a class more focused on humans than on things, The finding may also reflect the fact that our data goes back to 2020 and so some of it is from a period before the recent AI explosion, when students may be less likely to view AI as exciting and leading to remunerative careers.A factor in the lower levels of popularity for women may be that AI careers are now viewed as having high earning potential, which the literature suggests would motivate men more than women [45].The relatively low popularity of AI electives among women motivates further investigation into why students are not taking AI classes and how we might support them, in case there are barriers.Given that AI is now so influential in shaping modern life, and also increasingly provides exciting, dynamic, and socially meaningful careers, we must encourage women to study AI.The social and ethical implications of AI are becoming more and more relevant as its impact on society increases, so the higher interest of women in Society, Ethics and Professionalism (SEP) and, as evidenced in the literature, in social good and altruistic activities in general, mean they are an essential part of the future workforce.
Prior literature suggests that women have greater interest than men in classes that integrate social issues [24,49,51].Thus, we expected to see women showing high enrollment in classes with a SEP component and that their enrollment in SEP classes would be much greater than men's.As Lee et al. [55] mention, men might choose classes for career aspirations and take fewer humanities classes.However, these findings suggest that men do not understand the degree to which SEP topics might help them with their career goals.Considering that SEP covers topics about ethics, interacting with teams, and professionalism that can be important in the workplace, it is important to investigate why men are less inclined to take these classes [75].That the prior literature is not supported in terms of the appeal of these classes to women is perhaps because SEP is a broad topic, which may reflect prosocial values, but also may include a range of other issues.Additionally, very few elective classes are predominantly SEP; the tag was most often used for classes that focused primarily on another area but contained a significant SEP element.Exploring the other tags of SEP-tagged classes may provide more insight into this question.
Non-content factors being less influential for women than men was also an interesting result for us.It provides further support for studies in which men noted choosing classes for social reasons more than women [49,71].While our results cannot confirm if the men in our data chose classes for social reasons, they do indicate that they might have more non-content reasons to choose classes than women.These results also suggest strong possibilities for improving women's uptake: if we can get the content right such that they will appeal to women, the related uptake is likely to be high.Non-content factors are harder to control and influence.
Prior work also suggested differences in enrollment in information security classes based on gender [70].While we did notice that men took these classes more than women, the gap was not as big as some of the other gaps we observed.This might be because security did not appear to be a very popular topic overall.However, since we still did see a slight gap, it is important to continue investigating if women might face challenges such as stereotypes, as suggested by [70] to take classes in information security.

SCOPE AND LIMITATIONS
The scope of our study concentrated on how undergraduate elective classes with different contents are chosen by women and men students.The bounded study does not consider data regarding students who did not identify either as a woman or a man (e.g., non-binary) due to low sample sizes.It was felt that this could increase the risk of de-anonymization, thereby identifying individual students.
There are some limitations that we faced during our study.The limitations are grouped here into three categories: construct validity, internal validity, and external validity.
Where classes are listed as electives, we have assumed that it is open to all students in the relevant cohort to take that class, and hence not taking that class is a deliberate choice.However, this is not necessarily the case.We are aware that a small number of the classes have quotas and so are not available to all students.Quotas are particularly relevant for education classes, where only two classes were tagged, one of which has a low quota; hence the apparent unpopularity of this topic may be because students are prevented from taking the class.The results obtained would be more informative if quotas were factored in.It is also possible that some classes had quotas that were not reported -particularly given the recent trend of rising student numbers in computing.
Construct validity as a limitation started with operational definitions.The operational definition of electives may vary across institutions.We mitigated this by classifying each class using the ACM 2023 curriculum guidelines, ensuring consistency.However, it's important to note that the actual content covered in each class may differ from what was described and coded.The second limitation we faced under construct validity is Unmeasured/Confounding variables.Our results are based solely on the coding of classes using class descriptions and learning outcomes.This approach does not account for other factors in detail, such as social influences or time constraints, which could influence the participation and outcomes of students in elective classes.All non-content factors are grouped together and only the overall size of the impact is estimated, although it is differentiated between women and men.We also assume that elective choices are independent of each other and that topics independently affect topic scores.We did not collect data on the choices of individual students -this would have made de-anonymization a distinct possibility -so we can't assess the independence of choice with our data set.Incomplete Assessment is the third dimension of construct validity in our limitations.Our study focused on coding classes offered by the participating institutions, and we did not capture if students were learning computing material outside of these classes (e.g., through MOOCs or textbooks).The potential influence of alternative learning sources on students' computing knowledge and skills was not considered.The last item that appeared on our construct validity limitation is Code Applicability, which is to use a standardized approach to coding.To mitigate bias, we used the 2023 ACM curriculum guidelines to develop our initial coding scheme.While these guidelines provide a useful framework, they may overlook important nuances and variations in the offerings of different universities.
Selection bias is the first item of internal validity limitations that we considered.The universities and classes included in our study were not randomly selected.Researchers who applied to be part of the working group may have had specific interests or observations in mind, introducing a potential bias.The classes studied were based on convenience sampling from these universities.Attrition/-Mortality is another element of internal validity.Since we were interested in looking at the choices students made when they enrolled in elective classes, we did not include students who dropped the class.Thus we cannot make assumptions about why students might drop a class.
External validity is one of the final limitations considered here.Generalizability was challenging because, while we aimed for diversity in the data, our data set consists of universities from Western Europe and North America.As a result, the findings may not fully represent the choices and patterns observed in other contexts and geographical regions.Time-frame effects also can be considered as a limitation as we included data from the most recent years (2020,2021,2022).Notably, this included years that were affected by COVID-19, which could have influenced student choices if classes were online.In some cases, this made classes available that might otherwise have been restricted due to time or space restrictions.

FUTURE WORK
We would like to explore these results on a regional basis -that is, are women in a certain region making different choices to women in other regions?There was insufficient data to carry out that research given the relatively low number of institutions from which we have data and all of our institutions are from similar regions (the US, Canada, and Western Europe).Individual differences between institutions such as size, intake, and online/face-to-face teaching are likely to be at least as influential as any overarching regional differences.However, the worldwide differences in access to CS degrees for women described in Section 1 suggest that this could be a fruitful area for research.To carry out such research, we would need to choose specific regions to target and recruit a sufficient number of institutions from each region, ideally attempting to include similar institutions in each region.Readers interested in carrying out such research for particular countries or types of institutions are invited to read through the descriptions of our institutions in Appendix A and make use of the data from any institution that meets their criteria.
The data we currently have is only on aggregate numbers taking elective classes.It would be interesting to look further into how students choose combinations of classes to explore whether topic areas tend to be chosen together.As commented earlier, this would require a much more detailed data set on elective choices made by individual students and would make de-anonymization a distinct possibility.
We currently have only quantitative data, which helps highlight what choices women are making, but does not give us any insight into why they are making these choices.For this, we would like to conduct qualitative research within our institutions.This could take multiple forms, for example: • Interviews and/or focus groups with students who have recently chosen elective classes (i.e., registered for classes) or are about to choose elective classes to understand how and why they made those choices.• Think-aloud methods for students going through the registration process or a simulated registration process, to elucidate how they are making their choices.This can give more accurate information than interviews or focus groups as people are not always good at explaining or understanding how they made choices, and post-hoc rationalizations can unconsciously influence feedback.• Large-scale surveys asking students to choose reasons for their elective classes.• Presenting elective descriptions to high school students to get their views on what looks appealing.This could help craft degrees that are more relevant and meaningful for women.
We only considered how class titles and descriptions influence class choice from one point of view: how they map into the ACM and CAH codes.However, there are many other ways in which these could be influencing students.One of the key findings of the literature is that women are strongly influenced by factors such as social good and humans over things, and we did not see this reflected as strongly as expected in our results.However, this may well be because we are using SEP (Society, Ethics, and Professionalism) as a proxy for these factors.SEP includes IDEA (Inclusion, diversity, equity and accessibility) in its definition, which makes SEP broader than just social issues, such as social well-being and human connections.A more thorough approach would be to identify themes that the literature suggests we may expect to be influencing women and perform thematic analysis on the class titles and descriptions, including allowing for emerging codes.Correlating these with class choice would be much more informative in understanding the extent to which these factors are affecting women's choices.However, performing this for a significant number of classes would be extremely time consuming.
There are characteristics of classes that merit further investigation, either from a gender perspective or simply an analysis of the CS curriculum structure in different institutions.We have no account of the stage of study, the number of credits, or the notional learning hours associated with the electives.Although this would require more detailed work, mapping the different levels of study across institutions and national boundaries could highlight where there are differences in what content is covered, in how much depth, and at what level.
This research is exclusively concerned with inequality from a gender point of view, but there are other important axes of inequality in computing, such as race and ethnicity, socio-conomic status, and neurodiversity.If class registration data disaggregated by such factors is available, a similar analysis could be done for these characteristics.
We also recognize that we have reduced this conversation around gender in computing to merely one of women and men.There are many theoretical perspectives on how gender is defined and experienced, and although there is no space in this paper to fully explore them, the reader is referred to [15,27] as starting points for further reading.These perspectives can, and should, be integrated into future work to deepen the understanding of gender dynamics in CS.

CONCLUSION
The problem of gender inequality in computing is complex and deep.Making meaningful steps towards resolving this gap requires not only action on multiple fronts but also a deeper understanding of the causes of the problem and the different ways in which it could be alleviated.This paper focused on the differences between women and men in the enrollment of undergraduate-level computer science elective classes.Our data and results showed that there is a difference between women and men in the choice of elective classes, and this difference appears across a variety of subjects.The research in this paper explores a specific aspect of this problem that has so far been largely neglected in the literature: namely, once women are in computing degrees, what topics in computing are most and least appealing to them, judged by the elective classes they choose to take.We used multi-institutional, multi-national data to analyze this question, with up to three years of data from nine institutions: six in Western Europe and three in North America.We hypothesize that a better understanding of the answer to this question can allow computing departments to develop degree programs (both core and elective content) that are more popular with women, hence taking steps towards improved equality.The limitations of this study, as we stated above, will light future work by helping to develop a body of research on what topics within computing appeal most to women, with the ultimate goal of improving gender equality in computing.and integrative experience for students in their final year of study, providing a platform for showcasing their mastery of the subject matter and the application of their skills and knowledge acquired throughout their academic journey.

A.4 University of Edinburgh
The University of Edinburgh, founded in 1583, is one of Scotland's four ancient universities and one of the world's top universities, ranked 15th overall in the QS World University Ranking [85], and 23rd in the world for Computer Science.
The The student intake is highly selective and highly international, with around 1200 undergraduate students in the school and a large number of both taught and research postgraduate students.The majority of undergraduates take four-year degrees resulting in a BSc or BEng qualification, with a minority continuing for a fifth year to achieve a masters-level MInf.The four years of an undergraduate degree in a Scottish university -unlike 3-year English degrees -correspond directly with the four years of a US degree.At the University of Edinburgh, students have a choice of electives outwith the school in first and second year, with compulsory classes in Informatics and Mathematics.In third, fourth and fifth year (for those progressing to MInf degrees), all classes are elective, with the exception of a compulsory dissertation.In practice, there is significant overlap in class choice between the different degrees and students have flexibility to change their degree title if they want to make a choice of electives that fits another degree better.Cognitive Science students, as well as a wide choice of electives in Informatics, have compulsory and elective classes in other schools.Hence we consider all degrees as a single cohort for the purpose of this research, as any student is able to take almost any of the available electives.Third-year students take 70 credits of compulsory classes and choose 50 credits from a range of electives.Fourth-and fifth-year students must do a 40-credit dissertation and then choose 80 credits from any of the 3rd-year electives (20 credits maximum) or a larger range of electives at a higher level.

A.5 University of Glasgow
The University of Glasgow is a Russell Group university in the UK and one of the four ancient universities in Scotland.The University of Glasgow ranks within the top 20 universities for Computing Science in the UK and within the top 150 universities worldwide.The university enrolls roughly 26,000 students across the institution.
The University of Glasgow offers four-year BSc programmes in Computing Science and Software Engineering.The taught content of the Software Engineering programme is very similar to the Computing Science programme with one additional compulsory class in the fourth year of study and a compulsory summer internship between the third and fourth year, so the programmes are comparable for the purpose of this work.Students do not have any elective choices in the first and second year of study, so only data from the third and fourth year are considered here.In their third and fourth years of study, students must make up 120 credits worth of computing classes.In the third year of study, students have 50 credits of compulsory classes and a 30 credit compulsory team project, so must take four 10 credit elective classes.In the fourth year of study, students have a 40 credit individual project, so are able to select up to eight 10-credit elective classes.The elective classes are selected from the same pool across both years, which means that some students may take particular electives in either their third or fourth year of study.

A.6 Universitat Oberta de Catalunya
The Universitat Oberta de Catalunya (UOC, Open University of Catalonia in English) is a pioneering institution in online education based in Catalonia, Spain.Established in 1994, UOC offers a wide range of undergraduate and postgraduate programmes entirely delivered through their innovative online learning platform.UOC students span a wide range of ages, from recent high school graduates to mid-career professionals and older adults [76, p. 22].Many UOC students have family responsibilities and work commitments, making the flexibility of online education a crucial aspect of their academic journey.
Students taking the four-year Computer Science Engineering programme have to enroll into 72 ECTS from elective classes (all classes are 6 ECTS but one which is 12 ECTS, so they need to take 11 or 12 elective classes).There are 29 different elective classes that can be taken by students at any moment.Elective classes are organized in learning itineraries, and it is imperative for students to undertake a minimum enrollment of all elective classes offered within their selected elective itinerary.

A.7 University of Toronto
The University of Toronto is a large research-intensive university located in Toronto, Ontario, Canada.It is recognized as one of the top universities globally.It is Canada's oldest and most esteemed university, ranking first in Canada and 21st worldwide with a QS ranking of 21 as of 2023.The university has a QS ranking of 12 for computer science.Across its three campuses, approximately 97,066 students attend the university in 2023.
The university offers a four-year HBSc program in computer science.Students in the campus where our data comes from are admitted into a general entry program as they enter the university.After taking some required classes in the first year, students can declare to specialize in computing or major in it, with the former being more restrictive on the number of classes taken by students.For simplicity, we only include electives that both majors and specialist students can optionally take.Students are not offered elective choices in their first and second years of the program, however, in the third and fourth years, they can choose a number of elective classes (from either third or fourth year).Specialist students often need to take more electives (2 in 2023) than students majoring in computer science (2.5 credits in 2023).Students can also take some independent classes (research opportunities or projects), but they are not required to do so.These independent classes are often taken by 1-2 students, so we include these from our dataset as not everyone who wants to take them is able to take them (lack of supervision opportunities).

A.8 Uppsala University
Uppsala University is the oldest university in Sweden and the Nordic countries still in operation, founded in 1477.Uppsala University belongs to the Coimbra Group of European universities and to the Guild of European Research-Intensive Universities.It is a prestigious university, with a global QS ranking of 124 in 2022, and places within the 151-200 of top universities for Computer Science and Information Systems [85].
The data from Uppsala University is from the three-years Bachelors' Programme in Computer Science.Students usually take elective classes in their third year of studies, for a minimum of 25 credits.These elective classes, while offered by the Information Technology department, are open for students from other departments and programmes as well.
Uppsala University has a unique accreditation system in the sense that elective classes can be 5, 7.5, or 10 credits.This means that while some students might take three courses worth 7.5 credits, others will do a combination of 5 and 10 credit courses.In addition, while computer science students are expected to take 25 credits in elective classes, they can also take more, especially if they are prolonging their education.This is not uncommon among students from Sweden and the European Union, as education in Sweden for them is free.
To make the data workable for the statistical model, some parameters had to be estimated.For example, the minimum amount of elective classes had been set to five, and the maximum to seven, a reasonable estimation given that elective classes are usually 5 credits.This number was then multiplied by the amount of women and men students to calculate the total number of elective classes taken by these students.We then had to estimate cohort size based on this number and total number of enrolment for the elective classes, in order to also include non-computer science students in the cohort.

B.14 Common Aggregation Hierarchy (CAH)
Twenty-four general codes and 231 specific codes make up the Common Aggregation Hierarchy [42].These range from creative art and design to physical sciences.We used these categories, less the categories for computer science.
In general, we used the following policy in applying CAH codes.When classes were about computation applied to or integrated into specific topics, we wanted to capture the nature of the topic by coding for it.For example, a class with the title "quantum computing" and described as centrally concerned with physics was coded with the CAH code for physics.Similarly, a class titled "natural computing" and described as centrally concerned with biological algorithms was coded with the CAH code for "biology."Likewise, three classes focused on health informatics, but which were not about biology or medicine, were coded with the CAH code for "subjects allied to medicine." In contrast, many classes require knowledge of mathematics or science in order to be successful.However, just because a student must use some mathematics or science (or another competency), the class is not necessarily about that topic.In these cases, no CAH code was applied to the class.

C AUTHORS' OPINION SURVEY RESULT
We surveyed the authors prior to the model results being revealed.The two Working Group leaders abstained from voting.Our predictions of topic and non-CS application areas, compared with the actual results of the model, for overall enrollment are presented in Figure 7.The comparison of the predicted with the actual model in terms of the differential enrollment by women and men are presented in Figure 8.

Figure 1 :
Figure 1: Elective classes in a Program of Study.Electives are coded with ACM-defined curricular areas (e.g.DM, AI) and/or CAH-defined application areas (e.g.17) as described in Section 4.2.3

Figure 2 :
Figure 2: An overview of the data collection and coding process

Figure 3 :
Figure 3: Topic-related features of classes are combined with parameters for topic weights, that can be positive or negative.

Figure 4 :
Figure 4: A graph of the natural log of the odds ratio for the proportion of women to men taking an elective class for every elective class in our dataset.The log odds ratio in our distribution is proportional to effect size in normally distributed data.Bars on the left indicate higher odds of men taking a class; bars on the right indicate higher odds of women taking a class.Horizontal dashed lines indicate effect sizes as calculated from[21].Vertical error bars show 95% confidence intervals where they do not include 0, i.e.where the difference is statistically significant.
Elective classes with the lowest proportion of women to men enrolled.
(a) Popularity of ACM computing curriculum topics across all students (b) Difference of popularity in ACM computing curriculum topics (c) Popularity of CAH non-CS application areas across all students (d) Difference of popularity in CAH non-CS application areas

Figure 5 :
Figure 5: Mean posterior popularity of content features as generated by our model.Credible intervals of 96% are marked.

( a )Figure 6 :
Figure 6: Estimates of overall mean popularity of content features as generated by our model School of Informatics was formed in 1998 from the merger of the Department of Artificial Intelligence, the Department of Computer Science and the Department of Cognitive Science, and is a major international research institution.It is one of the oldest Informatics institutions in the world, with the formation of both the first AI research group and the Computer Unit (foundation of Computer Science) in 1963 -the 60th anniversaries of AI and CS research are being celebrated this year -and the foundation of the School of Epistemics (later Cognitive Science) in 1969.

Figure 7 :Figure 8 :
Figure 7: The authors' opinion regarding the popularity of computing curriculum topics and non-CS application areas versus model outcomes

Table 1 :
Terminology used in this paper with examples and explanations Keyword Example Explanation Program of Study Bachelors in Computer Science The title of the program of study.

Table 2 :
Number of elective classes and enrollments rounded to the nearest multiple of three

Table 4 :
[42]CS application areas defined by HESA in the CAH, as used for coding[42].CAH code 8 was a duplicate and was removed.CAH11 covers computing, and so was also removed.

Table 6 :
Content codes with the number of applicable electives and means of posterior popularity estimates

Table 7 :
Summary of posterior distributions for standard deviation parameters within the model.c3 and c97 refer to the lower and upper bounds on the credible interval.
BScs are awarded in Computer Science, Artificial Intelligence, Artificial Intelligence and Computer Science, and Cognitive Science, and BEngs are awarded in Computer Science, and Software Engineering.Joint UG degrees are offered in Computer Science and Maths, Physics or Management Science.Five-year MInf degrees are offered in Informatics.