The Matchmaker Inclusive Design Curriculum: A Faculty-Enabling Curriculum to Teach Inclusive Design Throughout Undergraduate CS

Despite efforts to raise awareness of societal and ethical issues in CS education, research shows students often do not act upon their new awareness (Problem 1). One such issue, well-established by HCI research, is that much of technology contains barriers impacting numerous populations—such as minoritized genders, races, ethnicities, and more. HCI has inclusive design methods that help—but these skills are rarely taught, even in HCI classes (Problem 2). To address Problems 1 and 2, we created the Matchmaker Curriculum to pair CS faculty—including non-HCI faculty—with inclusive design elements to allow for inclusive design skill-building throughout their CS program. We present the curriculum and a field study, in which we followed 18 faculty along their journey. The results show how the Matchmaker Curriculum equipped 88% of these faculty with enough inclusive design teaching knowledge to successfully embed actionable inclusive design skill-building into 13 CS courses.


INTRODUCTION
Recently, several universities and faculty members have begun increasing course coverage of societal issues that arise in computing professions.Examples include inserting critical thinking activities or ethics lectures into certain courses, and adding stand-alone ethics courses (e.g., [9,17,21,24,33,59], and others discussed in Section 2).Despite these efforts, however, researchers have recently reported that CS students are not acting in accordance with their new awareness (e.g., [23]).
Although this problem is not likely to be solved with a single "silver bullet" solution, this paper addresses one factor that is impeding progress-a shortfall in students' ethical training.As one recent work showed, HCI faculty have reported that students in their classes struggle to break free from "conventional" design patterns, thereby preventing more inclusive software solutions [41].In the same work, HCI students reported that while they understood bias may influence their work, they did not know how to identify and fix it [41].
We believe addressing this requires students to get hands-on experiences in which they act upon societal issues continuously across their CS degree program.To address this problem for the target societal issue of inclusivity, our approach inserts a subarea of HCI-namely inclusive design-into CS education at every level and in almost every course.The approach's end goal is to gradually build and reinforce students' hands-on inclusive design skills across all years in the major, as per Figure 1.Ultimately, we hope students will use this knowledge to build more inclusive software in their schoolwork and future careers.In this paper, we describe the first Figure 1: (Left): A pattern common in approaches that cover societal issues in CS (Right): The approach we were training faculty to carry out across a 4-year undergraduate CS degree program.The differences (blue) from many others' approaches are: (1) faculty collaborate on creating (2) gradual, coordinated content across the CS curriculum to (3) engage students throughout their CS program to create more inclusive software, ultimately directly impacting users of products these budding CS professionals will build.
step toward this approach: equipping HCI and non-HCI faculty alike to carry out this HCI-centered approach.
This step requires buy-in and change from a sizeable number of faculty.Some might initially object because most CS courses are not HCI courses.Even so, inclusive design is relevant to anything CS developers create for others to use-e.g., APIs, libraries, designs, documentation, databases, and of course user interfaces-because all such products need to be usable and maintainable by diverse people.Feasibility could also be an issue, such as faculty taking on too much extra work, the new HCI material occupying too much time in already-packed courses, or non-HCI faculty's ability/willingness to teach HCI material.
These points raise numerous challenges.Will faculty want to embark on such a project?Will those without HCI backgrounds be able to embed inclusive design elements into their "core CS" courses?Can we equip them with the motivation, background, and specific teaching skills they need to succeed at such an endeavor?
These points boil down to two key challenges.The first, Challenge-M, is motivating the faculty enough to embark on this endeavor at all.The second, Challenge-S, is equipping them well enough so their efforts are successful in their classrooms): • Challenge-M (motivating): Will CS faculty-even non-HCI faculty-want to embed inclusive design into their course(s)?
Will they find it feasible to do so?• Challenge-S (succeeding in the classroom): Even given motivation, can CS faculty succeed at this approach?Can we equip even non-HCI faculty with the knowledge and teaching skills needed to embed inclusive design elements into their courses and teach them successfully?
Addressing these challenges requires enabling faculty to make the right matches: the right inclusive design elements for their courses, their strengths, and their comfort levels.To help them make the right matches, in this paper we present and evaluate the Matchmaker Curriculum to enable CS faculty-including non-HCI faculty-to pair up with appropriate inclusive design elements while also addressing Challenge-M and Challenge-S.
The Matchmaker Curriculum is analogous to a dating app's matchmaking: it introduces faculty to a buffet of inclusive design elements and offers suitable potential matches for their course.Also like a dating app, the faculty are in full control; they can "swipe left" to skip any offered matches, and can even reject them all, finding the right matches some other way.
The Matchmaker aims to not only help faculty pair up with and learn inclusive design elements, but also motivate why they might want to teach those concepts to their students and how to do so effectively given their own contexts.In support of these goals, the Matchmaker Curriculum aims to make embedding and teaching inclusive design low-cost and minimally invasive, while also minimizing repetitive content across courses.The Matchmaker Curriculum consists of: • Curriculum Element #1: Getting faculty motivated: Because the success of the approach depends on a sustained and coordinated effort by faculty, this element uses multiple mechanisms to motivate the faculty to engage and stay engaged.• Curriculum Element #2: Teaching faculty inclusive design content: For faculty to embed bits of inclusive design into CS courses, they first need to understand an inclusive design method-in this curriculum, the GenderMag method for inclusive design [10].This element teaches faculty the GenderMag method and its components.• Curriculum Element #3: Guiding faculty through embedding inclusive design concepts into their courses: To enable faculty to embed the GenderMag components they want into their own courses, this element includes multiple mechanisms and scaffolding to support their efforts, with feedback iterations all along the way.• Curriculum Element #4: Developing faculty's PCK: Knowing content and making changes are not enough for effective teaching; faculty also need pedagogical content knowledge (PCK) to know how to teach this content.This element blends hands-on practice teaching with actionable PCK research findings on how to teach inclusive design effectively.
The first contribution of this paper is the above Matchmaker Curriculum, which contributes to HCI education a new pathway for introducing HCI across a computing major.But does the Matchmaker Curriculum overcome Challenges M and S? To answer this question, we conducted a field study in which faculty in the Computer Science department at University X (a primarily undergraduate, Hispanic-serving institution) participated in the Matchmaker Curriculum.Thus, our second contribution is the investigation of whether and how the Matchmaker Curriculum overcame Challenge-M and Challenge-S, starting from faculty's early interest in the vision shown in Figure 1 through their carrying out of that vision in their own classes in the ensuing fall term.
Positionality statement: We are of multiple races (Asian, Latinx, White), with national/ethnic backgrounds from Asian, South American, and North American nations.Several of us also have the intersectional identity of women of color.As such, a number of us have experienced lack of representation in computing courses firsthand.However, we recognize that, as academic researchers and people with access to higher education, we are in positions of privilege.Two of us have inclusivity leadership positions, which brought us credibility with the participating faculty.We are committed to using these privileges to use HCI methods to contribute to CS education's inclusivity to not only broaden participation in CS, but also to make CS education a better experience for everyone.

BACKGROUND AND RELATED WORK
Our Matchmaker Curriculum can be considered a form of Responsible Computer Science [36] or Critical Computer Science [33].Both Responsible and Critical CS are efforts to teach students mindfulness of societal and ethical implications of their work and respect for stakeholders [17,21,33,36,59].Examples of approaches include one university that had students discuss and reflect about targeted advertising, bias, etc. [17] and another university aiming to teach students awareness, reasoning, and communication about ethical problems [24].
Additionally, some Responsible CS and ethics approaches keep faculty workload viable by minimizing faculty involvement.For example, some approaches have brought in philosophy graduate students and postdocs [24] or undergraduate teaching assistants [17] to develop and teach ethics content, or have leveraged premade ethics modules [9].Similar examples are in [4,13,18,[47][48][49] as well as approaches with separate ethics courses (e.g., [12,20]).
Our approach instead emphasizes taking action and increasing faculty control.Specifically, it emphasizes equipping faculty with the knowledge needed to take ownership of inclusive-design Responsible CS elements suitable for their own courses, and to teach students inclusive design skills for solving societal issues.It uses the GenderMag inclusive design method to fuel both of these emphases, as we explain next.

Inclusive Design with GenderMag
Our Matchmaker Curriculum leverages, as its inclusive design method, the GenderMag method's components and foundations.GenderMag [10] is an existing method for avoiding, finding, and fixing inclusivity "bugs" in software.We chose this method because it is evidence-based [9] and used in software development by technologists around the world (e.g., [3,11,19,28,30,32,40,43,50,56]).Additionally, GenderMag uses an analytical method that does not require user involvement and has a very high accuracy rate (e.g.[10,43,56]).
At the core of GenderMag are five facets that categorize different areas of cognition individuals bring to their use of technology: motivations for using technology; information processing style; computer self-efficacy; learning style1 (by process or by tinkering); and attitude toward risk.Each facet has multiple facet values representing a spectrum of cognitive styles (e.g., from selective to comprehensive information processing).GenderMag defines an inclusivity bug as technology failing to support a cognitive style.Such barriers are cognitive inclusivity bugs because they disproportionately impact people with that cognitive style.The barriers are also gender inclusivity bugs because the facets capture (statistical) gender differences in how people problem-solve [2,10,15,16,53,56].
GenderMag uses three personas to bring the facets to life: Abi (Abigail/Abishek), Pat (Patricia/Patrick), and Tim (Timara/Timothy).For each facet, Abi's and Tim's facet values are at opposite ends of the spectrum and Pat has values within.Abi's facet values are disproportionately displayed by women, Tim's by men, and Pat provides a third set of values [10].The principle behind GenderMag is that, when technology simultaneously supports all three personas, every combination of facet values is also supported.Cognitive styles of the three personas are shown in Figure 2.
The GenderMag method integrates these personas and their facets into a specialized cognitive walkthrough [10,35].As with other cognitive walkthroughs [35], a GenderMag walkthrough involves walking through every step of a use-case/scenario and answering questions about each subgoal/action a user "should take" to succeed at the use-case.The user in a GenderMag walkthrough is a persona and there is a facet question at each step: • Before taking any actions: Will <persona> have this subgoal/take this action?Why/what facets?• After taking the "should take" action: If <persona> does the right thing, will they know that they did the right thing and are making progress toward their goal?Why/what facets?
Figure 2: A representation of the GenderMag personas' cognitive styles (facets) [34], which we leveraged in our approach.
In the realm of CS education, the only works relating to teaching GenderMag are Oleson et al. 's Action Research investigation into how HCI-oriented faculty teach GenderMag in face-to-face university CS classes [40], Letaw et al. 's use of GenderMag in two online courses [34], and our prior investigation on the impact of embedded inclusive design on students [22].Oleson et al.'s work produced Pedagogical Content Knowledge to enable effective teaching of GenderMag content, which we leveraged for our approach (described later in Sections 2.3 and 5.4).In Letaw et al.'s work, GenderMag concepts in 2 online courses provided early evidence for their proposed "embedded inclusive design" approach.Our approach can be seen as an example of that concept.In our prior investigation, we found that students in courses with GenderMag had improved grades and reported improved climate outcomes.However, no prior work has investigated how to embed elements of GenderMag into courses across a four-year CS degree program, including in non-HCI courses taught by non-HCI faculty.

Inclusive Design Education
Prior work suggests that the time is right for CS courses to teach students to design for diverse users.For example, over 6,000 individuals registered for access to a high-school level web design curriculum featuring accessibility concepts [1].Also, Oleson et al. found HCI students were lacking ability to design for diverse populations [41] and subsequently introduced the new CIDER (Critique, Imagine, Design, Expand, Repeat) technique to both help students design inclusively and encourage students to value inclusivity [42].Blaser et al. proposed that including universal design principles in engineering courses could increase feelings of inclusion for both women students and students with disabilities [6], as other work suggested women might be drawn to inclusive design (e.g., [25]).Similarly, Izzo and Bauer found teaching universal design to include people with disabilities helps both students and instructors to improve accessibility, awareness, and instructional flexibility [29].
Closest to our approach is work from Waller et al. and Putnam et al.Waller et al. experimented with integrating accessibility across a curriculum [57], although their approach is less minimally invasive than needed for our goals.Putnam et al. suggest guidelines for achieving this goal in the accessibility domain [45]: (1) multiple learning experiences (and pointers to key open problems); (2) resources to enable CS faculty who are not inclusion specialists to integrate teaching inclusion (for Putnam, accessibility); and (3) helping CS faculty evaluate the effectiveness of curriculum that include these aspects of diversity [45].Our Matchmaker Curriculum harnesses all three of these recommendations.
In prior years, individual faculty at various institutions have also included elements of GenderMag (e.g., the personas or the method itself) in their courses, especially HCI and Software Engineering courses (e.g., [34,40]).We have also previously reported on student outcomes of a multi-year GenderMag approach [22].However, prior work has not investigated how to match a broad spectrum of faculty, including those not trained in HCI, to inclusive design elements they can teach to continually add to students' ability to build inclusive software across a coordinated, 4-year CS curriculum.

Educating Faculty: Three Foundations
Our Matchmaker Curriculum also draws upon three foundations to support faculty's teaching: Communities of Practice (CoP), the Training of Trainers (ToT) model, and Pedagogical Content Knowledge (PCK).
A CoP is a learning community that often engages in informal learning and professional networking [58].Scholars have applied the concept to different types of learning communities such as professional learning communities/networks [31,54] and faculty inquiry groups [7].CoP approaches recognize professional development among faculty as a social activity, where communication among participants is key.A CoP has three main components: a shared area of interest (the domain), a group of people who engage and share knowledge (the community), and a shared collection of resources (the practices) [58].In our setting, the domain was embedding suitable inclusive design elements into existing courses, the community was university faculty (mostly CS), and the practices were shared inclusive design teaching and learning resources.Our Matchmaker Curriculum used active learning exercises and small-group hands-on work to draw the faculty into a CoP.
These active, hands-on aspects also helped align our Matchmaker Curriculum with the Training of Trainers (ToT) model [8,14,44], which is widely used in medical settings [14] and in some educational settings such as for teaching students to facilitate public deliberations [44].Our Matchmaker Curriculum draws upon multiple ToT properties, such as modeling and skill practice, active learning activities, opportunities for feedback, follow-up support, and action planning.This also included two properties that were essential to our approach: teaching relevant content and using evidence-based approaches to teach the content.The relevant content was GenderMag [10].The evidence-based teaching approach was PCK for teaching inclusive design.
PCK [51] is the integration of pedagogical knowledge (background in effective teaching) and content knowledge (background in a topic) that enables faculty to teach particular content.PCK is topic-specific and audience-specific [55] so we supported faculty's curricular changes by building upon Oleson et al. 's investigation into PCK enabling faculty to teach inclusive design skills using GenderMag [40], a point we will expand upon in Section 5.4.

CURRICULUM OVERVIEW
We built upon the above foundations to implement our Matchmaker Curriculum.It consists of the four elements in Table 1.
These four elements help faculty pair up with the "right" element(s) of inclusive design for their courses.Toward this goal, the Matchmaker Curriculum is non-prescriptive; with no fixed way to include an inclusive design element in any given course.Instead, it uses Curriculum Elements #1 and #2 to introduce faculty to the GenderMag method and its by-products and to offer suitable potential matches for their courses.However, as with any matchmaker or dating app, full control lies entirely with the faculty as to which elements to choose and how to proceed with them.
To evaluate the extent to which the Matchmaker Curriculum met its Learning Outcomes (LOs) and overcame the challenges identified in Section 1, we conducted a field study.For clarity of results, we present the field study methodology before detailing the Matchmaker Curriculum, so as to present the study results in the context of each Curriculum Element's details.

FIELD STUDY METHODOLOGY
In our field study, 18 faculty participated in the Matchmaker Curriculum and then acted upon it in their own classes.The field study investigated the on-the-job endeavors of these faculty to embed inclusive design into their own courses with the overall goal of evaluating the extent to which the Matchmaker Curriculum met the challenges enumerated in the introduction via the following RQs: RQ-M (motivating, before-the-fact): Will CS faculty-even non-HCI faculty-want to embed suitable inclusive design elements into their course(s), and see it to be feasible?
RQ-S (succeeding in the classroom): Can we equip even non-HCI faculty with the knowledge and teaching skills needed to embed suitable inclusive design elements into their courses and teach them successfully?
To investigate these RQs, we turn to each Curriculum Element's LOs for measurements by which to answer these RQs.Thus, we investigated RQ-M by measuring how many faculty achieved Curriculum Element #1's LOs, and investigated RQ-S by measuring how many faculty achieved Curriculum Element #2-4's LOs.

Education Contexts and Field Study Span
During the field study, faculty participated in the Matchmaker Curriculum from May through the end of fall term (December) at University X, a U.S. regional Hispanic-serving Institution.Their learning context for the summer was virtual due to COVID lockdowns so interactions consisted of shared/exchanged documents, emails, and Zoom virtual meetings.The lockdowns ended by September, so faculty taught their updated courses fall term in-person.Finally, we interviewed them over Zoom after they taught (November/December).
Faculty participated in the Matchmaker Curriculum through: (1) a 12-hour workshop series, (2) a set of resources including a wiki of shared teaching resources, (3) feedback and group work, and (4) emails with questions/answers and updates.The workshop series was offered twice-on two consecutive 6-hour Saturdays in May, and on three consecutive 4-hour weekdays in June (roughly same number of participants in each).Part of this workshop series (Curriculum Element #2, detailed in Section 5.2) was derived from two longstanding CS courses at another university.We then piloted and refined the workshop at University N and University U.

Participating Faculty
We recruited University X faculty through the CS department chair, who canvassed the department faculty.The faculty's response was positive, and when we initiated the study by offering a modest $500 summer stipend for participation, 15 opted in. 3 more faculty heard In total, 16 of the 18 participating faculty were CS faculty at University X engaged in the across-the-degree-program effort described earlier.As shown in Table 2, there were a total of 3 HCI faculty participants (two who currently teach HCI and one with previous experience teaching HCI).The non-HCI CS faculty participants taught a variety of courses ranging from CS0 through capstone and covering programs in Computer Science and Information Technology.The two non-CS (also non-HCI) faculty were an education faculty member at University X and an electrical/computer engineering faculty member at a different public university; these two were each changing one course, without coordinating with other courses.

Procedures and Data
Before beginning the activities, we reviewed the IRB-approved informed consent document with participants to gather their consent for our data collection.During the field study, the Matchmaker Curriculum was carried out via 21 activities, detailed chronologically in Table 3.
We collected faculty data throughout the field study (Table 3) via questionnaire responses, workshop recordings, faculty-created artifacts, faculty emails to facilitators, and interviews.The questionnaire data were qualitative with text-entry, Likert scale, and multiple-choice questions.Faculty-created artifacts included products of workshop activities and faculty's updated course materials.The full questionnaires and workshop activities are in the Supplemental Documents.
We used these data as measures to evaluate the Matchmaker Curriculum LOs through how many individual faculty members achieved the LO.For LOs that were not binary (LO-2, LO-3, LO-4), if a faculty member succeeded in at least 60% of the LO measures, we say they met the LO (because 60% is the passing grade threshold in 90/80/70/60 grading).
To triangulate faculty's pre-classroom expectations with their actual classroom experiences, at the end of fall term, we conducted and recorded a 30-minute semi-structured interview with the 10 faculty who taught at least one updated class during fall term.These faculty taught 16 sections of 7 courses covering CS0, CS1, CS2, WWW, Mobile, SE, and Ed (non-CS).Interview questions are given in the Supplemental Documents.
We qualitatively coded the faculty interview data using codes corresponding to the LO measures, which will be shown in Tables 4, 5, 7, and 8 (e.g., "Burden/prep light?" from Table 4).To code the data, we first segmented each interview by question.Then, two researchers independently coded 21% of the data, with 80% agreement (Jaccard method) [52].Given this level of agreement, the same researchers divided up coding the remaining data.The detailed code list can be found in the Supplemental Documents.
A final source of triangulation was our prior study which measured outcomes of these faculty's work from the student perspective [22].We discuss this further in Section 5.4.

THE MATCHMAKER CURRICULUM AND FIELD STUDY RESULTS
Throughout the Matchmaker Curriculum's design, we applied several HCI principles.To facilitate that perspective, below we motivate attributes of each Curriculum Element using Nielsen's wellknown 10 Usability Heuristics [39].Also, a cross-cutting example is that the entire approach is an application of Minimalism (as per Nielsen's Heuristic 8) in its aim to be "minimally invasive, " requiring very few additions to any one course.Picture" and Costs vs. Benefits.Motivating the faculty was critical to the project's success because, if a faculty member was not motivated, they might not effectively contribute to the coordinated effort.In fact, research indicates that faculty excitement about a curricular innovation is key to their adoption of the innovation [37].Further, we wanted their expectations to be realistic so they would not be disappointed.Curriculum Element #1 was also strongly influenced by Blackwell's model of Attention Investment [5], which emphasizes how users (here, faculty) weigh costs, benefits, and risks in deciding whether to take a cognitively demanding action.Thus, this element's hoped-for Learning Outcomes were that the faculty would be able to (LO-1a): analyze the costs and benefits of embedding inclusive design into their own course(s), and (LO-1b): be motivated to embed inclusive design into their course(s).Toward these ends, we implemented Curriculum Element #1 using the following five mechanisms: (1) Appeal to costs/benefits/rewards as per the faculty reward system (2) Explain the coordinated "big picture" (3) Emphasize each faculty member's control over their own courses (4) Explain the equity and inclusion benefits (5) Provide data on prior student outcomes and experiences In our case, mechanism (1) was easy because participating in our Matchmaker Curriculum aligned with University X's faculty reward system and retention criteria: (University's faculty retention criteria): "List any new teaching materials, teaching techniques, etc., . .." (CS Dept Chair, interview): "Professional development is encouraged and must be documented." We also mentioned that the approach is intended to be "minimally invasive, " requiring very few changes to any one course.We presented the "big picture" (Figure 3) to demonstrate this aspect concretely and to provide some cost/benefit information.
For mechanism (2), we used Figure 3 to appeal to some faculty's enjoyment of collaboration.We also drew upon a Community of Practice approach (Section 2.3) by emphasizing the importance of each faculty member's role and that effective collaboration was necessary for success.
For mechanism (3), we emphasized that, despite the coordination needed, faculty were not being asked to hand over control of their course content.To support this, we offered suggestions on which elements of inclusive design might be right and how to embed them, not requirements, as per Nielsen's Heuristics 3 (User control) and 7 (Flexibility).We then reinforced this point with Curriculum Elements #3 and #4 in which faculty designed their own course embeddings.
For mechanisms (4) and ( 5), we presented data on students' responses, successes, and diversity/equity/inclusion results from other studies on teaching with GenderMag [34,40].This served two purposes: to show initial success of teaching with GenderMag and to demonstrate potential benefits to students, as per Nielsen's Heuristic 2 (Match to <faculty>'s real world.(Prior success and student benefits have been found to be key motivators for faculty adoption [37,38].)We evaluated LO-1a and LO-1b to determine the success of Curriculum Element #1 using the data shown in Table 4.The evaluation of these LOs also answers RQ-M: whether faculty will want to embed suitable inclusive design into their courses, and believe it to be feasible to do so.
We measured LO-1a, whether faculty could analyze costs/benefits of embedding suitable inclusive design into their courses, using the "Benefits & drawbacks" row in Table 4.This row aggregates 6 optional questionnaire questions about benefits/drawbacks faculty members foresaw at this stage for their course(s), workload, or students.In total, 88% (14/16) responding faculty members, including 86% (12/14) non-HCI faculty, produced at least one benefit/drawback to their courses.In fact, the data from other GenderMag studies were compelling to some faculty: P17 (teaches CS1, Day 2 Feedback): "Great to hear that students felt more inclusive and learned about their own processing style." Still, some participants anticipated spending significant time on embedding efforts: P15 (teaches CS0+WWW, Day 3 Feedback): "I think I need to update a lot of assignments. .."However, by Day 3 all reporting participants had converged on anticipating a light or medium burden.Some participants even said the work was so important it was not a burden at all: P06 (teaches EE, Day 3 Feedback): "<It's> an important part of teaching, it is not a burden to include <GenderMag in courses> and learn about how to be more inclusive." Thus, we conclude that at this point, faculty believed the approach to be feasible.
LO-1b is the success of Curriculum Element #1 to motivate faculty to embed inclusive design.Our interest here was faculty's motivation before teaching their updated courses.We measured their motivation at this point using all three data rows in the top portion of Table 4.If a faculty member's responses to all three rows indicated a more positive than negative set of expectations, we considered LO-1b achieved for that faculty member.By this measure, all 16 of the responding faculty members were motivated.The fact that all except P08, who withdrew due to illness, continued their effort past this point confirms this LO-1b result.
But were their cost/benefit analyses and motivations realistic?To find out, we coded the faculty's reflections from the end-of-fall interviews as explained in Section 4.3; these interviews occurred after completing the workshop and teaching a term of classes.As Table 4's bottom section shows, these faculty members' fall classroom experiences met or exceeded their expectations in all comparisons except two.P01 and P11 reported their burden to be somewhat heavier than they had expected, and P18 also reported a non-trivial burden; however, these three also reported that the benefits outweighed the costs.As several faculty put it: P01 (CS2, Interview-CS2): "I don't know what the cost is. . .students have extra reading for their final project but also motivated more. . .I don't see very much cost and benefit is large" P11 (CS0+Mobile, Interview-CS0): "the cost is minimal and the effect. . .much outweighs [it]. . .[but] it's very hard to squeeze…in" P15 (CS0+WWW, Interview-CS0+WWW): "The topic is a little bit unusual for computer science but definitely important" P17 (CS1, Interview-CS1): "So there is cost involved. . .only 10 minutes, so you're talking about negligible. .."

Curriculum Element #2: Teaching Faculty
Inclusive Design Content 5.2.1 Curriculum Element #2's Implementation: Mostly Hands-On Activities.For faculty to teach inclusive design, they would first need to learn it, so Curriculum Element #2 taught inclusive design in the form of the GenderMag method (Section 2.1).The associated Learning Outcome, LO-2, was that faculty would be able to evaluate software using the GenderMag method and recognize its use to identify meaningful issues in software.
To accomplish this goal, we aligned this element with steps from Training of Trainers (ToT) research (Section 2.3) by using Gender-Mag to show how people learn new technologies, and through active learning to practice new skills [14].Specifically, our mechanisms for Curriculum Element #2 were as follows:   Why teach GenderMag?And why the full GenderMag method, if most faculty would be teaching only a portion of it?Two reasons were to illustrate the proficiency students should gain by the end of their degree, and to model teaching all of the method's components (setting the stage for Curriculum Element #4).Another reason was that the full GenderMag method provided a concrete, handson way to introduce faculty to inclusive design.Also note that ToT emphasizes evidence-based methods; empirical studies have produced evidence of GenderMag's efficacy [10,11,19,43,50,56] and of practices for teaching it [34,40].
Mechanism (1) was a cognitive styles sharing activity where faculty shared their facet values via the scale shown in Figure 4.These facet values were defined earlier in Figure 2, which we also presented to the faculty.
The activity served several functions.Educationally, it raised awareness on how users use technology in different ways, which also introduced central concepts of GenderMag.As a team-builder, it showed faculty different ways their peers problem solve.The activity also served as an example suitable for any CS/IT course requiring groupwork.The activity resulted in a rich whole-group discussion, and faculty were able to connect the activity's implications to their own teaching styles and students.P07 (Ed, Day 1 transcript): ". ..I think I am more like Tim. ..I need to feel that I am free to make errors.This is also a part of my teaching style."  Triangulating evidence (end-of-fall Interviews, faculty-created GenderMag Artifacts.) P14 (Capstone, Day 1 transcript): ". ..in the context of teaching, students would be all over the spectrum. .."P12 (ProjMgt, Day 1 transcript): "It also helps you spot the conflicts in the groups. ..the ones who want to complete tasks versus the ones who are tech interested. ..you get a boiling point at some point …" Building upon faculty's understanding of cognitive styles, mechanism (2) presented the full GenderMag method as a 30-minute lecture.This lecture (see Supplemental Documents) modeled how to teach the GenderMag method and was reusable to whatever extent faculty wanted.
The lecture was followed by mechanism (3), a two-hour active learning session in which faculty worked in small groups to apply GenderMag.The activity (see Supplemental Documents) involved each small group walking through a use-case for Canvas, a popular education platform.They answered the questions listed in Section 2.1 to evaluate the experience the Abi persona might have.We coached each faculty group along the way (as per Nielsen's Heuristic 1 about the importance of feedback) and periodically gathered them for sharing-out.This mechanism modeled an active-learning in-class activity they might use in their own classes.By the end of the activity, they had gained skills in locating "inclusivity bugs" in the software and were suggesting fixes.P05+P06+P08 (during the GenderMag activity, Day 1): "There's no indication of progress/process.(inclusivitybug for Abi, relating to Computer Self-Efficacy and Learning Style) P01+P02 (during the GenderMag activity, Day 1): ". ..the association between the account and the actual video is not clear and <Abi is not> a risk taker. ..Maybe show the video list.Then show the account. .."

Curriculum
Element #2's Learning Outcome Results.We measured the LO-2 results using faculty's Day 1 self-assessments, shown in Table 5's top two data rows, and then triangulated those data against their artifacts and post-teaching interviews.At the end of Day 1, 82% (14/17) achieved LO-2.Specifically, 82% (14/17) faculty members reported being able to perform a GenderMag evaluation and 94% (16/17) said their GenderMag evaluation of Canvas during the workshop had revealed meaningful issues (Table 5).These totals included 67% (13/15) of the non-HCI faculty, suggesting Curriculum Element #2 was appropriate for both HCI and non-HCI faculty.
We triangulated their self-reports against the artifacts the faculty created in their hands-on GenderMag work, and their interview reflections.This triangulation produced strong results; as the bottom of Table 5 shows, all faculty for whom artifacts or pertinent interview data were available performed GenderMag evaluations at least as well, and occasionally better, than their LO-2 measures indicated.
The ability of faculty members to learn and apply this type of inclusive design is the first part of answering RQ-S: it shows that the Matchmaker Curriculum was able to equip most faculty with the content knowledge they would need to succeed at this approach.

Curriculum Element #3: Guiding Faculty
Through Embedding GenderMag Concepts into Courses  As the above list suggests, the mechanisms provided extensive scaffolding.They also leveraged Training of Trainers principles by emphasizing hands-on practice and feedback, action planning with backward design, and multiple support opportunities [8,14,44], as per Nielsen Heuristics 1 about feedback and 10 about help and documentation.Finally, the mechanisms continued foster a Community of Practice through an ongoing emphasis on peer collaboration and support [58].
Mechanism (1) focused on the "how".It provided faculty with a well-known process they could follow to make changes to their courses: backward design [46].Backward design starts by considering desired student outcomes, then considers assignments through which students could demonstrate these outcomes, and finally designs course elements that would enable students to succeed at such an assignment.For example, Figure 5 shows P06's use of this process.
To further support faculty's action planning, in mechanism (2) we provided "starter packs": templates for each year of a 4-year CS degree, with suggested inclusive design element matches, fillin stages for backward design, and reusable materials housed in an online community.The online community content included lecture slides, homeworks, readings, in-class activities, and exam questions that participants could reuse or adapt.For example, nine faculty used the GenderMag personas graphic shown in Figure 2. In the Day 2 feedback, 88% (14/16) of reporting faculty responded that the starter packs were useful and 50% (8/16) of post-workshop respondents said they used the online community frequently.
Mechanism (3) encouraged collaboration as the faculty began their course changes.Faculty joined small Zoom breakout rooms with groups teaching similar courses.Facilitators visited each room to offer additional coaching if needed.Though some of their collaborations had rocky starts, by the end of the workshop all reported that peer work had eventually gone well (Table 7): P06 (EE, Day 1 feedback; peer work went somewhat poorly): ". ..I ended up trying to get folks involved and then stepped back because I do not like being in that role continuously." P16 (WWW, Day 3 feedback): "The breakout rooms were very collaborative and lots of interesting ideas and insights were exchanged." Mechanism (4) added a feedback loop in which participants turned in their course materials and received up to two rounds of feedback from facilitators (who were GenderMag experts).With one exception, this feedback respected faculty control over their courses, but sometimes brought cross-course matters to faculty's attention.Specifically, feedback was about improving wording or presentation with occasional suggestions for adding or removing content to keep consistency between courses.For example, some feedback to faculty explained there was too much or too little GenderMag material embedded, which might cause overlap/gaps between courses: Facilitator to P02(OOD) and P10(CS2+OOD): "The GenderMag Survey is not needed. ..students have already assessed themselves and their facets." Other faculty received feedback suggesting improvements in their wording of the assignments: Facilitator to P17(CS1): "Why these changes: it's important not to make Abi seem 'deficient'. . .Abi's problem-solving approaches are different from Tim's, but. . .<not> inferior. .." Unfortunately, the facilitators strayed outside these bounds in one case.In this case, P03(DB) and P04(DB) had added a GenderMag element to one of their non-GUI course assignments in a reasonable way, but the facilitators envisioned more extensive uses of GenderMag and tried to convince them to try that vision.At this point, P03 and P04 withdrew, commenting that GenderMag was not a good fit for their course.We will revisit this point in Section 6.

What the Faculty Created for their
Courses.The faculty who remained produced embeddings for their courses at three levels: early-level, mid-level, and upper-level.Recall the overall goal was to gradually build students' inclusive design knowledge over the 4-year CS/IT curriculum and avoid overlap between courses.Thus, embeddings they created for earlier courses would, one-by-one, introduce a few elements of inclusive design; later courses would gradually add elements with more breadth and depth, with students applying the concepts they were learning (Table 6).
Early-level courses began by introducing students to "Not Like Me" [27]-the idea that software's users are (mostly) not like the developer (here, the CS student).To do this, faculty introduced one or more personas and their facets by making small modifications to their existing assignments (both UI-oriented and non-UI-oriented).As Figure 6 shows, some faculty had students reflect on the personas/facets whereas others added simple questions requesting students consider the personas' usage of technology.Faculty of mid-level courses then built upon this familiarity with the personas by asking students to apply inclusive design in more depth.For example, Figure 7 shows how P10, P15, and P16 built on early-level concepts to have students evaluate use-cases for web sites using one or more personas.P16 went about this by adding an active learning activity (Figure 7a) whereas P10 and P15 leveraged GenderMag to evaluate class projects that were already part of the course (Figure 7b, 7c).
Finally, by the upper-level courses, faculty taught students to apply the full GenderMag method to software they were creating.The HCI, Mobile, and SE courses explicitly taught how to follow this method (typically in one lecture that replaced previous material covering a different usability process).As shown in Figure 8, these  courses included full use of customized personas and cognitive having students apply it to the projects they were working on in walkthroughs on the students' own designs.The subsequent Cap-those courses on their own.CS, Cap-IT and ProjMgt courses reinforced use of the method by Table 7: Curriculum Element #3's Learning Outcome results.(Top): From faculty's questionnaire responses and materials.17 faculty provided the data, including 14 non-HCI faculty.The first four rows are attitudes, the next two are accomplishments.We use "approved" to mean faculty's materials received feedback that did not suggest further changes.P03 and P04 dropped after the first round of feedback and P18 did not submit materials for feedback.Blue, X,X,-,n/a: same as Table 4. Multiple marks indicate multiple courses.(Bottom): Triangulating evidence: All faculty who were not ill/retired followed through by teaching their updated course(s) except P03 and P04.Black, dark gray, light gray, no color: same as Table 4. n/a: course not scheduled that term, or faculty member ill/retired Triangulating evidence: these faculty followed through by teaching their updated courses with suitable elements of inclusive design.

Curriculum Element #3's Learning Outcome Results.
To see how many faculty achieved LO-3 (being able to embed suitable inclusive design concepts into their existing courses), we measured participant responses using the six data rows in the top portion of Table 7.By the end of this Curriculum Element, 94% (16/17) of faculty and 100% (14/14) of non-HCI faculty achieved LO-3.Corroborating evidence (Table 7's bottom portion) strongly confirmed these outcomes; 14 of the 16 faculty who could have taught their updated courses during the upcoming academic year did so.
This result provides the second part of the answer to RQ-S.It says that the Matchmaker Curriculum was able equip faculty with the knowledge needed to embed suitable inclusive design elements they wanted into the courses they would be teaching.We consider this result somewhat remarkable, because this LO required these faculty members to embed inclusive design (HCI) concepts into their courses-14 of whom had no HCI background.

Curriculum Element #4: Developing
Faculty's Pedagogical Content Knowledge 5.4.1 Curriculum Element Implementation: Teaching how to teach it.As discussed in Section 2.3, the Training of Trainers (ToT) model notes the importance of not only providing content knowledge, but also introducing evidence-based approaches to teaching the content [8].Thus, Curriculum Element #4 introduced faculty to Pedagogical Content Knowledge (PCK), knowledge of how to effectively teach this kind of content.The associated learning outcome, LO-4, aimed for faculty to be able to engage and guide students on learning inclusive design concepts.
To help faculty harness and/or develop appropriate PCK, we used three mechanisms that aligned with ToT principles-modeling skills, skill practice, and feedback [8,14,44]-as well as aspects of Community of Practice including peer collaboration and support.Thus, this Curriculum Element's mechanisms were: (1) See teaching of GenderMag concepts modeled (2) Practice teaching their new materials with peers 14/16   2) faculty formed small groups to collaborate on each other's materials and take turns teaching each other.This mechanism was intended to help faculty practice teaching, find potential problems with their materials before unleashing them on the students (consistent with Nielsen's Heuristic 10 on error prevention), collaborate, and iteratively improve their plans.At first, their plans were so rough that problems were easily spotted.However, as they iterated, the peer playing the "student" role sometimes needed to deliberately act as an uninterested, resistant, or obtuse student to bring out different problems.(Some of the faculty were quite inventive in playing these roles.) Once the faculty had unearthed problems and attempted to resolve them, mechanism (3) introduced four of the 11 PCK elements from Oleson et al. 's field study of faculty members teaching inclusive design (Figure 9) [40].

Curriculum
Element #4's Learning Outcome Results.To evaluate the success of LO-4, we measured faculty's pre-teaching assessments of their abilities to teach inclusive design concepts and then triangulated with postteaching interviews and with students' post-teaching ratings and retention results.16/17 (94%) of reporting faculty successfully achieved LO-4, including 14/15 of the non-HCI (Table 8).
Corroborating evidence from the end-of-fall interviews, shown in the bottom of Table 8, mostly confirmed faculty's early assessments: all but two faculty remarks aligned with their previous self-assessments from the questionnaires.These remarks were from faculty who encountered student questions they were unable to answer.This highlighted the need for mechanism (3) and possibly the addition of a frequently-asked questions document, as suggested by P05: P05 (Interview-WWW): "I would have felt more prepared if we had like a. ..document with like commonly asked questions. .." We have also investigated the effects of these faculty's efforts from the students' perspectives [22].That investigation spanned periods of time in which COVID, George Floyd, escalated anti-Asian hate, and the January 6 th insurrection arose in the U.S. Because of these complexities, we compared the post-intervention students' experiences (in which all the traumatic events were still affecting people's lives) with baseline (1): the pre-intervention period that was the most similar to the post-intervention period, and provided additional comparisons when possible with baseline ( 2): the pre-intervention period that was before any of these events.The post-intervention period outperformed baseline ( 1) and most baseline ( 2) comparisons [22].
For example, the students' attrition rate improved (Figure 10 Left), as measured by number of Incompletes, Failures, and Withdrawals (IFWs), and the gap between targeted and non-targeted courses closed.The grades most students received on their fully graded inclusive design assignments (not pictured) were at least as high as their grades on their fully graded other assignments, suggesting that most students did actually learn some inclusive design.The students also reported several improvements in their educational climate.For example, students' ratings of their instructors' ability to create an inclusive environment for students was higher than it had been in the baseline period (Figure 10 Right).These results answer the third and last part of RQ-S: they show that the Matchmaker Curriculum was able to equip most faculty to teach these inclusive design elements successfully.

DISCUSSION 6.1 Triangulation
Central to our methodology's validity is triangulation: whether the same results manifest themselves multiple times from multiple sources of evidence.Triangulation not only guards against construct validity flaws, but also adds confidence in the reliability of the results.
Table 9 shows how we triangulated the results of our study.As the table shows, the results for all the LOs were evidenced across multiple sources-2 to 8 for each LO.Of the 30 sources, 28 produced results above acceptable thresholds, and only 2 below.

Costs, and Lessons Learned
Throughout this effort, we were aware that faculty would take a final accounting of the costs of embedding suitable inclusive design into their courses.Faculty communicated their conclusions during post-teaching interviews and, in two cases, by their decision to withdraw from the effort.The faculty's remarks such as those in Table 10 pointed out three orthogonal dimensions of costs (1) who or what caused it; (2) the type (course  (Left): All students' IFW (Incomplete/Fail/Withdraw) rates before and after GenderMag was embedded into the CS/IT curriculum, out of >5000course grades total.IFWs in targeted (dark green), non-targeted (gold), and all (gray) courses, in the before-intervention periods ("Baseline") vs. the postintervention terms ("Post").Brackets show gaps between targeted and non-targeted; trend lines show targeted-baseline vs. targeted-post.Low = good.[22].(Right):>400 students' average rating of instructors' ability to create an inclusive classroom environment ratings before (gold) vs. after embedding GenderMag (dark green).[22].
P18 pointed to a different cost issue: during-the-term grading costs from a "poor assignment."They pointed out this faculty-time cost was self-inflicted by their up-front preparation.Fortunately, P18 noted that addressing it was within their control: P18 (SE+HCI, Interview-SE): ". ..theHCI <course> is taught in spring so I'll be doing it there. . .and then I would certainly do it again. . .next fall in <course> software engineering and again make sure I'm getting the information I want without <making grading and data collection hard>." The dimension of who or what caused a cost and who can fix it comes back to faculty control over their own courses.Curriculum Element #1 introduced this point in its third mechanism ("Emphasize each faculty member's control over their own courses"), and the remaining Curriculum Elements reinforced it with faculty embedding suitable inclusive design content that they chose into their own courses however they saw fit.Giving faculty so much control brought the advantage that if problems arose, faculty members tended not to blame the content, but rather to feel that their upfront work needed tweaking-and that they could fix it themselves.We suspect that this power may be critical to the sustainability of the approach.
However, giving faculty so much control also led to costs, as sometimes faculty turned out to cover essentially the same material: The faculty is trying to find ways to resolve this issue through more collaboration among faculty who teach adjacent courses; another possibility would be to embed some of the coverage into courses' official learning objectives, which might help to stabilize which material gets introduced in which courses.
We also observed what happened when we strayed too far into faculty's control of their courses.We believe the core reason P03 and P04 withdrew from the effort over summer was because we inadvertently took too much control.These two participants had been collaborating on materials for backend-focused courses and had drafted a non-GUI related homework question.But our feedback asked them to additionally apply inclusive design content to a different, GUI-related part of an assignment: Facilitator to P03 (DB) and P04 (DB): "We see a great opportunity for incorporating GenderMag into the UI portion. .." It was at this point that they withdrew: P04 (DB, Email after first iteration of feedback): "We think. ..good candidate courses for GenderMag should be GUI-related. .." In retrospect, we think that our feedback pressured them to relinquish too much control and autonomy.A lesson learned.

Embedding Inclusive Design into Non-GUI Courses
A different hypothesis about P03's and P04's withdrawal could be that, just as P04 surmised, the approach is suitable for GUI-related activities only, not for backend-focused courses like databases, programming languages, compilers, computer architecture, etc.However, evidence from University X runs counter to this hypothesis.At least seven of the faculty embedded inclusive design into non-GUI assignments/activities.Common strategies in these embeddings were (1) asking students to consider how diverse people might use their artifacts (e.g., understanding code or UML diagrams written by others), and (2) team-building exercises, which we had modeled during the workshop for Curriculum Element #2.
Examples of strategy (1) were P15(CS1)'s assignment, P10(OOD)'s assignment, P03(DB)'s and P04(DB)'s ill-fated database assignment (Figure 11a, 11b, 11c, respectively) as well as P17(CS1)'s assignment (recall Figure 6).Examples of strategy (2) included P06(EE)'s use of the cognitive styles for team-building (not shown), P10(OOD)'s assignment (Figure 11b top) and P07(Ed)'s use of the personas to discuss education strategies (not shown).We have also seen examples of non-GUI uses outside of this study: researchers have reported use of strategy (1) to find and/or fix inclusivity  2).It began by asking team members to share how their facets might impact the assignment (which was about use case diagramming) and then asked the team to consider how to make their assignment work for different personas.(c) P03 and P04 planned to include a class-wide discussion as part of one of the DB assignments.
issues in documentation and issue tracking sites [26,43] and strategy (2) for team-building in online classrooms [34].As with other field studies, our study investigated the real-life outcomes of faculty who participated in our Matchmaker Curriculum.As a result, our results are specific to only that context: the particular university, the particular faculty, the particular courses they taught, and their particular ways of teaching.Generalization beyond this context is not possible without further follow-up studies.

Threats to Validity
For example, faculty who chose to participate might have had different characteristics from those who chose not to.Faculty also had autonomy over how to embed inclusive design into which subset of their courses, so different faculty might make different decisions about the same courses.
Another threat is construct validity: whether the data accurately measured the LOs we intended to measure.To mitigate that threat, we measured each LO using a variety of measures to guard against overreliance on any one.We triangulated faculty's pre-teaching reports/achievements against their post-teaching reports/achievements, as discussed in Section 6.1.Even so, a different choice of measures might yield different results.
One "big picture" threat comes back to an important motivation for this work, which was to equip tomorrow's computing professionals to build more inclusive technology (recall Figure 1).Because our data did not contain enough samples to compare the students' work products, we have not yet been able to evaluate whether their products actually became more inclusive.In a future study we will collect data to make this comparison possible.

Generality
Threats and Boundary Conditions.The main limitation is the generality of Curriculum Element #1 (Motivating the faculty).Our study was conducted in an undergraduate institution with a focus on quality teaching.In other contexts, such as Ph.D.-granting universities that prioritize research, it is unclear how to tie into their reward system; one possibility might be linking to their existing efforts to broaden participation in computing.For four-year colleges that emphasize quality teaching to faculty, we hypothesize that those colleges could implement the Matchmaker Curriculum with minor adjustments, and two-year colleges such as community colleges might be able do so also, but with a less populated "big picture" (recall Figure 3).
Another generality threat is that in some cases it may not be possible to apply our approach with GenderMag if gender inclusion efforts are not possible or not accepted.In this case, we believe that it is possible to use the Matchmaker Curriculum by substituting another inclusive design method for GenderMag.
Another possibility is that there may individual faculty members interested in our approach who are not able to or do not want to lead a curriculum-wide effort through the full Matchmaker Curriculum.We understand that a curriculum-wide effort may not be an option for everyone and believe that our curricular materials can still be used to create a standalone GenderMag embedding.Even without the curriculum-wide implementation, prior research has found that including GenderMag into individual classes can benefit students [34].

Sustainability Threats.
A final threat is sustainability without the original facilitators, specifically (Sustainability1) whether existing University X faculty could continue; (Sustainability2) whether incoming University X faculty members could successfully engage; and (Sustainability3) whether another university could succeed without the original facilitators.
Promising evidence for (Sustainability1) is already available: University X faculty are now in their third year, the last two of which have been without external facilitators.For (Sustainability2), University X is using two mechanisms: an online course version of the Matchmaker Curriculum which we created and are currently beta-testing, and a pass-it-on model, where incoming faculty are coached as needed by faculty with experience embedding GenderMag elements.However, there is no answer yet to (Sustainability3).Still, the online course version provides all elements of the Matchmaker Curriculum except those dependent on local context and those involving human facilitators/peers, and our hope is that the University X pass-it-on model might fill the human roles with faculty who already teach elements of GenderMag at their own universities.(Faculty at universities in 45 countries have used or taught GenderMag, which provides a starting point.)That said, we do not yet know how well the Matchmaker Curriculum will perform with different facilitators or what the full criteria for future facilitators might be.Thus, only future studies can fully answer the Matchmaker Curriculum's sustainability.
To support such future studies, we have made the Matchmaker Curriculum materials freely available.These materials are available in the Supplemental Documents and on Open Educational Resources (OER) Commons.The online course mentioned above is freely available via Canvas's Free-for-Teacher platform.Descriptions and links to all these online resources can be found on the GenderMag for Educators webpage 2 .

CONCLUDING REMARKS
Is it possible to enable a wide swath of a department's CS faculty-mostly non-HCI faculty-to embed bits of inclusive design across almost all their undergraduate CS courses?If so, how?Our field study showed the answer to our "is it possible" question was "yes" and that the Matchmaker Curriculum's four elements provided the "how." Two specific challenges the Matchmaker Curriculum aimed for were: (M): Motivating faculty, most of whom were not HCI faculty, to undertake the endeavor (measured in RQ-M); and (S): enabling even the non-HCI faculty to succeed in the classroom (measured in RQ-S).Curriculum Element #1 tackles the Motivating challenge and Curriculum Elements #2-#4 tackle the success-in-the-classroom challenge.
Motivating: Our field study results show that 14 of the 16 faculty members (88%) who could have gone forward with the approach did so (Table 7).We attribute this success at motivating the faculty and keeping them motivated to three key attributes, which are emphasized starting with Curriculum Element #1 and reinforced throughout: • Minimally invasive: The Matchmaker Curriculum emphasizes that no course should change much and includes an expandable buffet of ideas for how faculty can make minimally-invasive changes while still achieving the goal.• Student data: The Matchmaker Curriculum shows data from related work showing strong benefits to student retention and education climate.Recall from Section 5.1 that faculty found these data compelling.• Non-prescriptive (Faculty control everything): Most critically, the Matchmaker Curriculum only makes suggestions; individual faculty choose, create, and/or adapt the inclusive design material(s) that seem appropriate to their courses.If things were not perfect the first time, this control motivated several faculty to iteratively finetune.
Success in the Classroom: In our field study, the mostly non-HCI faculty at University X succeeded well enough to see improvements such as increased instructor ratings and student grades that indicated students were learning inclusive design [22].We attribute these successes to four key elements: • Evidence-based: Our Matchmaker Curriculum brought in evidence extensively.We already discussed bringing student data into Curriculum Element #1 (motivating).We also brought evidence and foundations behind the GenderMag method into Curriculum Element #2 (content), so faculty would be equipped to answer student questions and into Curriculum Element #4 (PCK) in providing the faculty with evidence-based teaching practices for this kind of content.• Scaffolded: Curriculum Element #3 (teaching) was heavily scaffolded, with process ideas (e.g., backward design, starter packs), content ideas (examples for faculty to reuse as desired), and iterative feedback.• Hands-on: Curriculum Elements #2 (content) and #3 (teaching) used mostly active learning activities.In Curriculum Element #2, these activities demonstrated engaging ways to teach inclusive design content; and in Curriculum Element #3, they enabled faculty to make 2 https://gendermag.org/educators.phpprogress during the workshop on developing their own embedding ideas.• Collaborative: Curriculum Elements #3 (teaching) and #4 (PCK) were highly collaborative.Faculty not only shared materials but also developed them and practiced together, which they found engaging and helpful (Section 5.4).
Perhaps most important, the technological world in which we spend increasing portions of our lives needs to become more inclusive.In the CHI literature alone, reports abound of underserved groups of users.Inclusive design skills can help address such problems, but not enough computing professionals possess these skills-yet.We hope the Matchmaker Curriculum can accelerate HCI's rate of changing the world.

Table 1 :
Matchmaker Curriculum overview with the associated Learning Outcomes.Section 5 details the mechanisms we used to carry out each Curriculum Element.faculty's PCK LO-1a: Analyze costs/benefits of embedding into their own course(s).LO-1b: Be motivated to embed inclusive design into their course(s).LO-2: Evaluate software using the GenderMag method and recognize its use to identify meaningful issues in software LO-3: Embed suitable inclusive design into their existing course materials with provided resources and collaboration.LO-4: Engage and guide students on learning inclusive design concepts (1) Relate costs/benefits to faculty reward system (2) Explain the coordinated "big picture" (3) Explain their control over their own courses (4) Explain equity and inclusion benefits (5) Provide data on prior student outcomes (1) Activity: Cognitive styles sharing (2) Lecture: GenderMag (3) Active learning: GenderMag hands-on (1) Explain process ideas and include resources: backward design, starter packs, and example embedding (2) Intro to content ideas: in the online community (3) Coaching/collaboration in creating materials (4) Feedback on materials submitted (1) See teaching of GenderMag concepts modeled (2) Practice teaching their new materials with peers (3) Known PCKs for teaching inclusive design about it through word-of-mouth and joined without the stipend, bringing the total to 18.

Figure 3 :
Figure 3: The "big picture," shown to faculty to guide their efforts and emphasize the need for coordination.(Full version in Supp Doc.) (Left): Excerpt from 1st two years' course list, suggesting how the pieces might fit together and the minimal classtime needed.(Right): The 2nd two years.HCI1 and SE1 were the only courses for which we suggested significant lecture/classtime additions.

Figure 4 :( 1 )
Figure 4: Part of the cognitive style sharing activity, filled out by P13 and P14, who learned they process information differently.

5. 3 . 1
Curriculum Element #3's Implementation, Successes, and Tribulations along the Way.In Curriculum Element #3, faculty needed to act upon what they had learned-i.e., decide what inclusive design content was suitable to use in their own courses and how.The associated Learning Outcome, LO-3, was that faculty would be able to embed suitable inclusive design concepts into their existing courses, in ways that did not introduce undue workload to the faculty or detract from the course's existing learning goals.The four mechanisms we used were:

Figure 5 :( 1 )
Figure 5: Excerpt from P06's three-week teaching plan created using the backward design template.

Figure 6 :
Figure 6: Early-level course assignments created by faculty: (a) Snippet from P01's CS2 class explaining the personas (Figure 2).(b) P10 and P11's finalized CS0 assignment.(We underlined their updates.)(c) Changes made by P10 (purple) and (green) as they collaborated on curricular materials for CS0 during the workshop.(d) Snippet from P17's CS1 (non-UI) lab asking students to compare their information processing and learning styles during coding to the three personas.(e) Another part of P17's (non-UI) CS1 lab asking students to determine which persona they most identified with as they wrote their code.

Figure 7 :
Figure 7: Mid-level course examples where students complete larger assignments to learn how to apply inclusive design.(a) P16's in-class think-pair-share activity for WWW, in which students shared their facet values and looked for inclusivity bugs.(b) Snippet from P15's first of two WWW assignments, which guided students through using the Abi persona to evaluate course projects presented to the class.(c) Snippet from P10's OOD assignment which adds usage of multiple personas to an existing course project milestone (green text is per instructor's formatting).

Figure 8 :
Figure 8: Upper-level course examples: (a) P05's additional Mobile assignment.(b) In P18's SE course, students conducted GenderMag evaluations on their own projects (excerpt).(c) P13 and P14's Cap-CS and Cap-IT weekly project status report included a new section asking about inclusivity issues each team discovered.(d) P12's ProjMgt final report included an additional notes section where groups could comment on their use of GenderMag.

Figure 9 :
Figure 9: (Left): Excerpt from a handout showing the four PCKs highlighted from Oleson et al.'s study [40], used on workshop Day 3. (Right): Examples: Excerpts from workshop slides on PCK11.

( 3 )
Known PCKs for teaching inclusive design Mechanism (1) took place on Day 1 as part of Curriculum Element #2's teaching faculty GenderMag concepts (Section 5.2).Having already seen us model strategies for teaching, in mechanism (
P05 (Mobile+HCI, Interview-Mobile):"I know a couple. ..students <who are> doing <inclusive design content> for all of their other classes and they are just kind of burnt out over it.I think it's. ..because. ..new stuff coming at them from several different professors, several different ways of doing it."

Figure 11 :
Figure 11: (a) P15 used strategy (1) in their CS1 lab assignment where students are asked to consider how different personas might read and understand the code they wrote.(b) P10's OOD assignment used strategies (1) and (2).It began by asking team members to share how their facets might impact the assignment (which was about use case diagramming) and then asked the team to consider how to make their assignment work for different personas.(c) P03 and P04 planned to include a class-wide discussion as part of one of the DB assignments.

Table 2 :
(Left): Faculty participants.In total there were 18 participants (9 men and 9 women) teaching a combined 14 courses.Blue: non-HCI faculty.(Right): Courses the faculty participants identified for adding embedded inclusive design elements.

Table 3 :
Faculty engaged in the 21 activities shown, producing the data shown at the end of each group.Activity handouts and materials in the table are in the Supplemental Documents.(Time lengths approximate.)

Table 4 :
Curriculum Element #1's Learning Outcomes.(Top): Relevant questionnaire responses from each participant.Participants are shown with their course(s).16 faculty provided data, including 14 non-HCI faculty.X: yes/agree, ×: no/disagree, -: neither agree nor disagree, X>×: benefits outweighed costs, blank: did not respond to this question, n/a: could not respond (e.g., did not do the questionnaire).(P08 withdrew due to illness after Day 1.) Multiple marks indicate multiple questions in that category.Blue: non-HCI faculty.(Bottom): Triangulation with post-teaching interviews.X: positive reflection, ×: negative reflection, blank: not mentioned during interview.Comparisons with Top: black: better than faculty member initially reported, dark gray: same as initially reported, light gray: worse.No color: no comparison possible.

Table 6 :
(Left) Levels of inclusive design learning, and the courses teaching that level.(Right) Year-by-year summary of inclusive design levels taught across the 4-year undergraduate CS/IT curriculum.

Table 10 :
Examples of each of the three dimensions of costs.