Equity-Forward Learning Analytics: Designing a Dashboard to Support Marginalized Student Success

Student outcomes in US higher education exhibit deep and persistent inequities. The continued underperformance of historically marginalized students remains a serious concern across higher education, reflected in increasing efforts among institutions to infuse diversity, equity, and inclusion into their academic and social communities. Yet despite widespread recognition of these inequities, few studies in the learning analytics literature engage in practical ways with issues of educational equity or DEI considerations. In this paper, we share our work supporting a large college's strategic DEI goals through the creation of a Course Diversity Dashboard informed by research into how students’ study behaviors and performance interact with their gender and ethnic identities to impact course outcomes. The dashboard enables users to explore inequalities in course outcomes and take concrete actions to improve student study strategies, time management, and prior knowledge. Results from our research revealed the existence of previously hidden learner inequities in all courses included in our study as well as critical differences in underrepresented minority students’ prior knowledge. And while we did not find evidence of meaningful differences in the study behaviors of student subgroups, our findings further validate the effectiveness of evidence-informed study strategies in an authentic educational setting.


INTRODUCTION
As educators we know that no two learners are the same, but as researchers and data scientists we often treat them as though they were.We estimate the average effect of interventions while ignoring ever-present variation in outcomes and model the behaviors of interchangeable anonymous ids that obfuscate differential impacts on important student subgroups.The consequence is a literature full of potentially misleading and incomplete findings, and the recent push in education research away from identifying "What works" to focusing on "What works, for whom, and in what circumstances" reflects this discomforting conclusion [62].
And while the shift in education research from analyzing average effects to understanding the complex interactions that exist in any educational setting is laudable, equally pressing is the need to appreciate students as individuals to reveal hidden injustices and redress inequities lurking beneath the educational status quo.In their recent book, learning design experts Mirjam Neelen and Paul Kirschner argue that good learning experiences should exhibit three characteristics: efficiency, effectiveness, and enjoyment [37].To these three features, we argue a fourth is equally important: equity.Even a learning experience that is maximally effective, enjoyable, and efficient for the "average learner", may consistently produce inferior outcomes for students with specific intersectional identities and belonging to particular social categorizations [8].This includes the many marginalized students who face unrecognized microaggressions in the classroom, experience a lack of representation in course materials, encounter subtle and overt sexism, and carry stigmas about their inherent ability to be successful in certain degrees or subjects [35].As educators and learning analysts, we have an ethical obligation to identify and root out these inequities to ensure all students have equal opportunity to succeed.
With the rapid adoption of digital courseware and growth of online learning, there is increasing access to data sources that can meaningfully support the goal of greater educational equity.And the field of learning analytics, which is defined as "the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs" [26], must play an integral role in supporting diversity, equity, and inclusion (DEI) efforts across education.In particular, the increasing use and acceptance of learning analytics dashboards (LADs) in higher education, creates an opportune moment for learning analysts to incorporate DEI-relevant data points and targeted analyses that enable educators, learning designers, and administrators to explore important questions about the equity of their courses and support them in taking remedial action.
Yet recent summaries of LAD research, as well as our own review of the literature, suggest that issues of diversity, equity, and inclusion are neglected areas of attention within the learning analytics community.And despite the growing emphasis in higher education on ameliorating disparities in the outcomes of women and historically underrepresented minorities (URM)-a term that includes students identifying as African American (Black), Hispanic (or Latino), American Indian, and/or Alaskan or Hawaiian Nativestudies investigating the impact of learning interventions on these groups or describing learning analytic tools intended to support educational DEI goals are rare.
In this paper, we respond to this gap in the learning analytics literature by describing our work to research and build a learning analytics dashboard intended to support institutional DEI efforts at a large community college.The design of the dashboard was informed by exploratory research into differences in student course outcomes, study strategies, and prerequisite knowledge associated with students' varying intersectional identities.This research, in turn, was guided by a review of key findings in the learning and educational sciences that directed us to investigate important factors known to impact student success in college.Our hope is this work contributes to an increasingly equity-forward approach to learning analytics and reflects a growing commitment in the learning analytics and educational research communities toward developing evidence-informed tools intentionally designed to expose and dismantle educational injustices.

RELATED WORK 2.1 Prior LAD Research
Numerous researchers have described the potential benefits of LADs in higher education, highlighting their ability to communicate important patterns in student behavior and performance to guide educators in making data-informed changes that improve course quality and learner outcomes [16,34,41,55].Yet despite these positive depictions, a growing number of studies exploring the practical impact of LADs report users often struggle to make effective use of dashboards and face significant hurdles incorporating them into existing processes.Commonly cited reasons include the steep learning curve associated with data-informed decision-making, lack of instructor familiarity with key findings in the learning sciences, challenges integrating LADs into existing instructor workflows, and failure to include educator perspectives and feedback during the dashboard development process [57,60].Failure to anticipate and address these obstacles often results in low faculty adoption and poor decision-making [2,49].Many researchers, for example, point out that both during LAD development and implementation, there is often a notable lack of any reference to evidence-informed learning theory [47,48].The selection of visualizations and data surfaced in LADs often lack clear ties to research-supported actions known to demonstrably improve learning or connect meaningfully to a learning experience's educational goals, two factors that make it unsurprising that faculty struggle to find them useful [50].As Jivet and colleagues note, LADs that merely raise awareness of available educational data without a clear pedagogical focus and in the absence of tools to take remediating actions are unlikely to support and improve learning [22].And while researchers cite the potential for improved student outcomes as faculty take actions based on dashboard-derived insights, the reality is that few instructors possess strong enough backgrounds in the learning sciences to know what effective pedagogical actions to take in response to revealed data patterns [32].
LAD developers also often underestimate the steep learning curve associated with taking data-informed action.As Li and colleagues note [25], it is a non-trivial process to translate the data patterns surfaced by LADs into real-world, educationally effective, actions.Simply providing analytics to teachers does not entail agency and teachers are rarely exposed to instruction related to issues of data literacy [27].Furthermore, surveys suggest many faculty are understandably resistant to taking on the additional responsibility of analyzing course data, with the increased workload and role expansion it entails [18].Successfully integrating LADs into existing instructor workflows, therefore, requires developing a sustainable practice of analytics use and adequate support during the analytics sense-making process.

DEI & Learning Analytics
The last decade has witnessed a growing public commitment among U.S. higher education institutions regarding the importance of DEI efforts on their campuses.Decades of research show that historically marginalized minorities consistently earn lower college grades than their peers, are more likely to withdraw from enrolled courses, and have much lower six-year graduation rates [29].And both marginalized minorities and women continue to be severely underrepresented in most STEM majors, such as math and science [24].Explanations for these disparate outcomes are manifold, and efforts to ameliorate these inequities have taken a variety of approaches.These include infusing social justice topics into course curriculums, adopting universal design principles to guide course design, reconsidering course prerequisite and admission requirements, and establishing institutional-level committees entrusted with regularly monitoring and reporting on the outcomes of URM students [17,45].
Additionally, educational researchers have increasingly recognized the need to adopt DEI-conscious practices when conducting and reporting their work.Noting the historical neglect of DEI concerns in educational research and the persistent lack of diversity in many college disciplines (particularly STEM), recent papers emphasize the importance of undertaking research specifically designed to understand how factors like systematic bias, marginalization, sexism, and microaggressions interact with students' intersectional identities to impact their success in school [39,59].These factors are further exacerbated by the fact that many women and URM students come to college less prepared due to legacies of systemic racism and sexism in our country, which has resulted in fewer educational opportunities and resources during their pre-college years [12,30,52].
Yet despite the increasing importance of DEI issues in education, learning analytics research has only begun to scratch the surface of this important topic [54].While several DEI-related topics, such as issues of power and representation in the design of learning analytics tools [50,61] and the topic of algorithm fairness in predictive models [15], have figured prominently in learning analytics papers, there are few examples of learning analytics research undertaken explicitly to understand and redress educational inequities or support institutional DEI efforts.Our review of the learning analytics literature suggests that engagement with DEI issues is typically limited to including minority and/or gender status as subgroups in traditional research analyses.Examples of this approach include investigating whether students' use of different learning strategies is associated with URM status [44] and assessing whether gameful learning experiences impact minority and female students differently than their non-marginalized peers [21].A notable exception is the paper by Reinholz and Shah [42], where the authors propose a quantitative approach to measuring student participation-which they term 'equity analytics'-with the goal of identifying inequities in classroom discourse.
With respect to LAD research in particular, a recent review of the literature found little engagement with DEI-related issues [58].In fact, the authors were at such great pains to find examples of DEI themes in LAD research that the only examples cited included one where demographic information was mentioned as a worthwhile inclusion, but ultimately excluded [14], and another where there was only a brief mention of the importance of including the ability to explore subgroups in the dashboard interface without explicitly mentioning any marginalized or vulnerable groups [25].Ultimately, the authors of the review conclude with a call for greater focus on DEI issues in LAD research, a call amplified by recent papers emphasizing the importance of developing analytic tools to help identify educational inequities and inform curricular changes that advance institutional DEI goals [7,19].

DESIGN CONTEXT & METHODS
In the following sections we share the motivations and context surrounding the development of the Course Diversity Dashboard, describe the data sources used, and walk readers through the research behind the chosen visualizations.

Institutional Context
Ivy Tech Community College (ITCC) is a large community college in the state of Indiana.It is a critical player in the higher education and workforce training areas across the state.As part of ITCC's Strategic Plan, the promotion of diversity, equity, and belonging is an area of emphasis for the college, with a stated goal to eliminate systematic inequities for students.Metrics for achieving this goal include reducing the female and URM equity gap in applicant conversion, retention, and completion rates.A first step in carrying out this strategy required understanding existing student equity gaps, which was the initial impetus for creating the Course Diversity Dashboard.Subsequent discussions about how to empower stakeholders to act in response to any revealed equity gaps led to an expansion of the initial dashboard concept to capture additional student behavioral and performance data to guide course-based interventions.
Contemporaneous with ITCC's focus on educational equity, was increased scrutiny on pass rates in key gateway courses and student struggles in early math courses.At ITCC there are two typical pathways for students to fulfill their math requirements.For non-STEM programs, the math requirement is Quantitative Reasoning (MATH123) and for STEM programs the pre-requisite is College Algebra (MATH136).Historical challenges with student achievement in these courses led to the team to select these courses for the initial CDD pilot.Eventually, two additional math courses-Finite Math (MATH135) and Brief Calculus (MATH201)-were also included in the pilot to provide further insight into female and URM student math performance and behavior.

Data
The Course Diversity Dashboard was created using data collected from multiple sources and spanning several years.Data on student course outcomes-in the form of final grades and withdrawals-was collected from ITCC's Student Information System (SIS).This outcome data was combined with student admission data, which provided demographic information about each student's self-identified ethnicity and gender.Students are given the option not to answer these demographic questions on their application, with roughly 4% of students electing not to provide information about their gender and 9% choosing not to identify their ethnicity.After removing students who declined to provide the information necessary to determine their gender and/or URM status, the final data set consisted of 46,073 individual students who were enrolled in courses between Spring 2020 and Fall 2022.Among students in the dataset, 30,915 identified as female and 15,158 identified as male.With respect to ethnicity, 33,595 students identified as White, 1,338 identified as Asian or Pacific Islander, and 11,140 students identified as belonging to a URM.
In addition to student outcomes and demographics, the Course Diversity Dashboard also incorporates data collected by students' digital courseware.This courseware includes homework and assessment tools, interactive course-specific content, and adaptive skillbuilding features to support student learning.As students interact with the courseware, the platform captures data on when students access and submit assignments, what tools and hints are used while working on homework questions, and how students perform on individual assignment learning objectives.This data was critical for the Course Diversity Dashboard because a primary goal of the project was to go beyond investigating general course outcomes (e.g., pass rates and grades) to surface behavioral and performance insights that could be used to inform potential interventions.Although ITCC sets institution-wide curricular standards for every math course, it affords campuses and faculty the freedom to choose among a variety courseware options to use in their courses.Given limits on data access, only math courses adopting MyLab Math contributed to the study strategy and objective performance data discussed later in this paper, a subset of 11,621 students.

Course
Outcomes.The first step in redressing inequities in course outcomes is awareness that disparities exist.At ITCC, passing students are defined as those receiving an A, B, or C in their course.Although overall pass rates were available for each course included in our study, these rates had not previously been broken down by student gender and ethnicity.As a result, analyses investigating how differences in student pass rates were associated with students' gender and minority status had not been previously available to college stakeholders, including the faculty and instructional designers who teach and design the courses.While it is not uncommon for institutions to report on course pass rates at the level of gender and minority status, the team sought to go beyond this level of analysis to explore deeper potential patterns of inequality that might be revealed by analyzing the relationship between course pass rates and students' varying intersectional gender and ethnic identities.
Another potentially important metric for assessing inequalities in course outcomes is student withdrawal and retake rates.Research shows that course withdrawals and retakes negatively impact 2ndyear retention rates, derail student degree plans, and often have serious financial and time costs for students [1].Thus, the team investigated the percentage of students withdrawing and retaking each course, again breaking these results down by students' gender and ethnic identities to uncover any existing inequalities.Because higher withdrawal and retake rates among certain groups may indicate the need to provide additional supports or revise course content to better reflect the preparedness of all incoming students, the team wanted to understand the value of including these data points on the dashboard.

Study Strategies, Time Management, and Prior Knowledge.
While awareness is a critical first step in addressing educational disparities, too often institutional DEI analyses are devoid of practical guidance for practitioners-i.e., the faculty and instructional designers responsible for teaching and designing courses-regarding how to tackle inequities revealed in their courses [58].Similarly, most learning analytics dashboards are limited to raising awareness of easily accessible educational data with little consideration of concepts from the learning sciences or the incorporation of pedagogical tools intended to aid dashboard users in taking subsequent action [23].These observations led the team to consider the inclusion of dashboard visualizations intended to shed light on student study strategies, time management, and prior knowledge-analyses that were selected based on a review of key findings in education research.If associations between students' prior knowledge, study strategies, and intersectional identities were found, presenting this information on the CDD could guide ameliorative stakeholder actions.
Extensive research in the learning sciences has identified several study strategies that consistently produce superior learning outcomes.These strategies include spacing studying rather than cramming, engaging in retrieval practice rather than rereading or highlighting, and viewing worked examples in place of problemsolving when learning new material [10,53].Efforts to understand what strategies students use in practice, however, reveal that students frequently adopt less-effective strategies when studying, resulting in lower academic performance and less learning [4,20].Fortunately, studies have found that simple study skill interventions and course design changes, which nudge students toward better learning strategies, can be helpful in improving student studying habits [28].
Given these observations, the team utilized courseware interaction data to explore student use of two effective learning strategies: spaced practice and worked examples.In the case of spaced practice, data collected on the average number of days students worked on weekly homework was investigated to determine how distributed students' studying efforts were across their assignments.Student use of worked examples was analyzed by calculating the percentage of times a student viewed a worked example-an option available within their courseware-after getting a homework question incorrect on their first try.Previous research into whether URM and female students employ effective study strategies at different rates than their non-URM and male peers is nascent, with recent work reporting conflicting findings [43,45].
Poor time management is also frequently identified in the research literature as a major contributing factor to URM student underachievement [13], and interventions designed to improve URM time management skills are common [36].Although there are many facets to effective time management, and student interactions within their digital courseware constitute only a small slice of all the actions students take to plan and manage their studying activities, the team undertook an analysis of the percentage of homework turned in prior to the due date and how this was associated with a student's gender/ethnicity and course success.The hypothesis being that students who consistently wait until an assignment's due date before turning in their homework may be struggling with time management and potentially underachieving due to rushing to complete homework at the last moment.
Finally, studies have found that URM students often come to college less academically prepared than their peers and may not have mastered prerequisite concepts presupposed by course designers and instructors [46].This knowledge difference is often explained by majority minority high schools having less resources, poorer quality curriculums, and/or teachers who present lowerlevel instruction in what are nominally the same courses offered at more affluent schools [9,56].Critically, learning scientists note that learner prior knowledge is perhaps the most important factor influencing students' ability to learn successfully.Prior knowledge is the foundation upon which new information is built and integrated, and without an understanding of the relevant background concepts it is difficult for learners to make sense of new information [37].Consequently, the team investigated possible associations between students' intersectional identities and prior knowledge by exploring students' performance on learning objectives assessed in their first two quizzes in each course-content intended primarily as review material.

RESULTS
In this section, we discuss the results of our exploratory research undertaken to inform the design and analyses included on the Course Diversity Dashboard.We then share how our findings were translated into the dashboard's initial design and describe several dashboard features intended to address previously identified challenges with instructor use and adoption of LADs.

Course Outcomes
Examining average course pass rates during the last three years, our analyses revealed that URM students pass at much lower rates than their non-URM peers across all math courses (see Figure 1).Although college administration was aware of low pass rates in these courses, this figure reveals how inequitably these pass rates are distributed across different intersectional identities.White and Asian students, for instance, historically pass courses at rates that, in some cases, approach 20% to 30% greater than their Hispanic/Latino and Black peers.The gaps are largest in MATH123 and MATH135, a finding that is particularly concerning given that MATH123 is the course non-STEM majors typically take to meet their math credit and progress in their degree program.
Figure 1 also highlights the importance of visualizing students' intersectional identities and looking beyond general categorizations like URM status to explore deeper patterns in student outcomes.Specifically, this plot reveals that Black students, and Black male students in particular, are at or near the bottom in course pass rates for every course.This insight would have been missed if students were lumped together solely according to their URM status.The figure also shows that female URM students are overrepresented in the lower half of course pass rates in most courses.
Looking at course withdrawal and retake data shown in Figure 2, we find that male and female URM students withdrew and retook courses at notably higher percentages than their non-URM peers.Like course pass rates, we see that Black and Hispanic students of both genders are more likely to withdraw and retake these courses compared to other intersectional identities.In many cases, the withdrawal and retake percentages of these groups is close to twice the rate of their Asian and White peers.Regrettably, these higher withdrawal and retake rates among URM students likely have a serious negative impact on their progress toward a degree and result in lost time and greater tuition expenses.Furthermore, students who are forced to withdraw from a course may become discouraged and decide to leave the school altogether, which can have long-term consequences for their career prospects and earning potential.

Study Behaviors & Prior Knowledge
Figure 3 shows our findings regarding passing and non-passing student study behaviors, broken down by URM status and gender.It should be noted that this and the following figure do not break down student identities by ethnicity given supplementary analyses (not shown) did not show meaningful differences between specific ethnic groups.Additionally, while our research is exploratory in nature and does not align with the traditional framework of null hypothesis testing, it is worth noting that all reported differences highlighted below achieved statistical significance at the conventional p < 0.05 level.
Overall, students had a median average worked example usage rate of 30.5%, indicating that students viewed a worked example after incorrectly answering a homework question almost a third of the time.As expected, passing students had an average worked example usage rate higher than non-passing students (+5.9%), but there was little evidence of a meaningful difference between URM and non-URM students.Interestingly, female students had a worked example usage rate 5.1% higher than their male peers.
The students in our study averaged just under two days (1.8) working on their weekly homework assignments.Students were given credit for one day for each distinct day they recorded any activity in their homework, regardless of duration.Passing students had a slightly higher average numbers of days working on homework than non-passing students (2.0), but there was again no evidence of a meaningful difference between URM and non-URM students in the number of days spent working on weekly homework.Female students, however, also averaged slightly more days working on homework than male students (+0.18).
Finally, analyses of submission times revealed that passing students, on average, turned in over half of their homework assignments (52%) prior to the due date, whereas non-passing students turned in less than a third of their homework assignments prior to the due date (30%)-a considerable 22% difference.Consistent with earlier findings, there was no discernable difference between URM and non-URM students' submission times, but once again female students had a higher average early submission percentage compared to their male peers (+5.4%).These results further validate, in an authentic educational setting, the positive association between academic success and students' use of evidence-informed study strategies and effective time management.Across all three behaviors, passing students were more likely than non-passing students to view worked examples, interact with homework multiple times during the week, and turn in assignments prior to the due date.Interestingly, we also found consistent evidence that female students employed these strategies more frequently than their male peers.Finally, we did not find strong evidence of differences between URM and non-URM students in their propensity for using any of the strategies examined.
Analyses of early quiz results in the four math courses revealed several sizeable learning objective knowledge gaps between URM and non-URM students.Figure 4 shows a subset of four quiz learning objectives, highlighting the differences in student performance across gender and URM status.Concerningly, we see that URM students averaged 14%-19% lower scores than their non-URM peers on these learning objectives, equating to a performance gap of almost two letter grades.It is likely that these and other knowledge differences among URM students on early course learning objectives help explain their lower course pass rates and higher withdrawal/retake rates.Finally, we did not find any consistent patterns in learning objective performance associated with student gender.5.This first screen includes a visualization allowing users to explore changes in course pass rates across each semester during the last three years, while also allowing users to breakdown these rates by URM status, gender, and ethnicity.Crucially, by showing pass rates longitudinally over multiple terms, the intention is to enable users to identify historical trends and monitor changes in response to potential interventions.This view also includes a visualization enabling users to explore differences in course withdrawal and retake rates among students with different intersectional identities.The inclusion of these two visualizations, as well as the ability to slice learner outcomes at the level of students' ethnic and gender identities, were directly informed by our research showing concerning inequities in these outcomes and the importance of being able to explore underlying patterns beneath broad categorizations such as 'underrepresented minority'.

From Research to Dashboard Design
The second screen of the dashboard is shown in Figure 6 and includes analyses of student in-course data and provides users with insights into student study strategies and prior knowledge.Although our findings did not show evidence that URM students would uniquely benefit from being targeted with study skill interventions, they did reaffirm the importance of helping all students adopt more effective studying behaviors.Using the Course Diversity Dashboard to reinforce the efficacy of these strategies, while also highlighting their possible underutilization, may help convey to instructors and learning designers their educational value-an important outcome given research showing that many instructors  are not familiar with the most effective study strategies to recommend to learners [33].For example, an instructor might consider reviewing effective study behaviors at the beginning of her course if the dashboard reveals that student use of these strategies is low.And a course learning designer might consider replacing a single weekly homework with smaller sub-assignments due throughout the week to encourage leaners to space their practice and avoid cramming on the due date.
Finally, given our findings revealing significant differences in early URM learning objective performance, the second dashboard screen also includes a visualization highlighting the largest gaps in URM learning objective performance on early course quizzes.Prior to the Course Diversity Dashboard, these gaps were hidden from faculty and instructional designers, but the dashboard now enables an instructor to identify learning objectives that may need additional reinforcement early in her course before diving into novel material.Learning designers can also review these findings to decide whether existing course content coverage accurately reflects the spectrum of prerequisite knowledge possessed by incoming students.

Additional Dashboard Considerations.
As noted in section 2.1, the team was keenly aware of previously reported challenges with stakeholder use of LADs and took several actions to ensure the presentation and communication of the dashboard maximized the likelihood that it would be used and lead to positive action.For example, prior research has found learning analytics dashboards rarely include supporting information to aid user interpretation [5, 40], severely limiting their impact and usefulness.Thus, every visualization included on the Course Diversity Dashboard was paired with an accompanying textual explanation of the analysis providing important context and interpretive guidance.Several reflective questions were also included with each visualization, supporting faculty and instructional designers during their sensemaking process.
It was also important that the Course Diversity Dashboard include a separate section listing various evidence-informed suggestions for how faculty and learning designers might respond to any inequities revealed in their courses.As previous studies have noted [23,31], rarely do LADs support users in transitioning from awareness to action, so the team worked with the ITCC diversity office to include relevant school resources and identified promising intervention ideas in the learning science literature.These resources and intervention ideas were collected and provided on a linked dashboard resource screen.
Finally, the team recognized that although LADs are often useful for communicating data insights, they can be challenging to integrate into existing stakeholder workflows and in many cases other modalities may be more effective [38].Consequently, although the Course Diversity Dashboard is natively built using the data visualization software Tableau, the team created a script using the statistical language R to output a PDF report for each course.Although this output provides less interactivity, we've found it is often better suited for casual and non-specialist audiences while also being easier to disseminate to faculty and other stakeholders.
The team has also proposed changes to the current course revision process whereby instructional designers and faculty work together to review and interpret CDD visualizations during their regular course revision conversations to seamlessly integrate the dashboard into existing workflows and increase uptake [23].

DISCUSSION
The Society for Learning Analytics Research's (SoLAR) 2020 Statement of Support and Call for Action included the following appeal: "We encourage members of our Society to mobilise our expertise and connections with communities to actively contribute to the hard work of promoting social justice and dismantling injustices in education" [51].Yet as previously discussed, recent reviews of the literature show DEI-related topics are still a neglected area of focus in the learning analytics community.
This paper describes our process researching and designing a Course Diversity Dashboard to support ITCC's strategic DEI goals.This dashboard not only raises awareness of existing course inequities, but also incorporates learning science informed visualizations and resources to guide concrete remedial actions while also providing longitudinal views that empower users to assess the impact of their actions.Although the Course Diversity Dashboard is currently in pilot phase, it has already raised the awareness of instructors, learning designers, and administrators about the existence of inequities in early math courses and prompted actions to redress them.For example, a math department leader at ITCC, after viewing early CDD findings, immediately pushed to start a peer tutoring program on her campus after the dashboard showed clear evidence of prerequisite knowledge disparities among incoming URM students.She also articulated plans to use the dashboard to monitor the impact of this intervention on subsequent student success rates at her campus.And after viewing the CDD visualizations, the instructional design team initiated efforts to adapt the content coverage in current math courses to ensure all students have equal opportunity to succeed and further nudge students in adopting more effective learning strategies.The CDD will also aid this team in monitoring the impact of their changes on student study strategies and course performance.
While these are exciting developments, the process of creating the dashboard was not without its challenges.We briefly discuss three challenges encountered during our project and share some thoughts on how the learning analytics community can begin to address these obstacles to make equity-forward analytics the norm rather than the exception.
The Course Diversity Dashboard required collecting and aggregating data stored in a variety of formats and controlled by multiple teams across ITCC and external partners.This necessitated a coordinated effort across several different groups to acquire the necessary data and then substantial data cleaning and wrangling to match students across these disparate datasets.In our experience, this data fragmentation, both within an institution and across external partners, is typical in higher education and creates a logistical and technical challenge to obtaining the data needed to support equityforward analytic projects.Consequently, an increased focus on issues of equity within the learning analytics community will require more attention to making DEI-relevant data accessible and useful to researchers.This might include developing better processes for researchers to request necessary data, providing anonymized, synthetic, or simplified datasets that lower the technical bar for investigating DEI-related questions, and establishing better data standards to ensure there are common identifiers across systems.
A second challenge the team faced was resistance from multiple groups to sharing student demographic data.Concerns with data sensitivity and student privacy were common, despite the team going to great lengths to ensure data was sufficiently protected and the purpose of the Course Diversity Dashboard was fully explained along with its alignment to ITCC's strategic goals.In fact, the team was unable to obtain access to several important student characteristics originally intended to be part of the dashboard's launch-including students' socio-economic, disability, and firstgen statuses.While concerns with student privacy and worries of potential misuse are important, our obligation to understand whether the educational experiences we provide students are just necessitates moving beyond disembodied conceptions of data and requires interrogating student outcomes ". ..with a keen eye to gender, race, sexuality, class, disability, nationality, and other forms of embodied difference" [6].The omission of DEI-relevant factors in educational research, as Ferguson notes, often serves merely to entrench current systems that "disempower and disenfranchise vulnerable groups" [11].As a community we need to do a better job articulating that the inclusion of DEI-relevant data is not merely a nice addendum to an educational research project, but a critical requirement.
Finally, although the analyses included in the Course Diversity Dashboard provide valuable insights into student outcomes broken down by ethnicity and gender, these features capture only a small subset of the social categorizations that make up students' complex intersectional identities, and more work is needed to collect richer DEI-relevant data.For example, use of the 'Black' ethnic category indistinguishably combines the descendants of African slaves with African immigrants, two groups with very different backgrounds and experiences [3].And admission forms limiting students to selecting either a male or female gender impose an artificial binary categorization that does not reflect the realities of non-binary, intersex, and non-gender conforming students.

LIMITATIONS AND FUTURE WORK
One limitation of our work is the necessity of having to rely on student interaction data in courseware as a proxy for actual student studying behaviors.Although this data is suggestive of student strategies, learning management systems and digital courseware data alone can never capture all the offline activities students undertake, and the same behaviors may be consistent with different interpretations.For instance, although initial exploratory analyses revealed that students printing homework to work offline is rare, some students may have chosen this option.And although consistently turning in homework on or after the due date may indicate poor time management skills, in some cases it may simply reflect conscientious students availing themselves of all the time afforded to them.Another limitation is that our initial dashboard analyses focused on the binary outcome of passing/not passing a course and thus are unable to reveal important inequities in the underlying distribution of letter grades within these categories.More recent ad hoc analyses undertaken by the team indicate that even among passing URM students, these students are disproportionately more likely to earn grades of Cs and Bs rather than As when compared to their peers.This outcome contributes to URM students achieving lower GPAs and experiencing related adverse effects that could hinder their academic progress.
In terms of next steps, we've initiated conversations with the learning design team to adopt additional standardized course elements across all courses to enable further investigations into student learning strategies.For example, only one of the math courses investigated included practice tests for students to use in preparation for their exams, a feature that would provide an excellent opportunity to explore student use of retrieval practice, another research-supported learning strategy.We also plan on working with the learning design team and course faculty to incorporate surveys to better understand students' perceptions of their course climate and sense of belonging.Much of the research on ameliorating the inequitable outcomes experienced by women and URM students points to the importance of student perceptions and attitudes, highlighting the importance of instrumenting courses to collect data about how learners perceive their course climate, their feelings of self-efficacy, and capturing broader metrics of student mental and social wellbeing.

Figure 1 :
Figure 1: Average course pass rates broken down by ethnicity and gender.Error bars indicate +/-one standard error of the mean.

Figure 2 :
Figure 2: Average course withdrawal and retake rates broken down by ethnicity and gender.

4. 3 . 1
Selection of Dashboard Visuals.Applying the insights gained from our exploratory analyses, the initial screen of an anonymized version of the initial Course Diversity Dashboard design is displayed in Figure

Figure 3 :
Figure 3: Boxplots showing differences in student use of worked examples, spaced practice, and early submission of homework.Plots are broken down by whether students passed the course, URM status, and gender.

Figure 4 :
Figure 4: Average student scores on a selection of learning objectives evaluated on early course quizzes.Scores are broken down by gender and URM status.

Figure 5 :
Figure 5: First anonymized screen of the Course Diversity Dashboard enabling users to identify inequities in course outcomes and the second view provides insight into URM students' study strategies and prior knowledge.

Figure 6 :
Figure 6: Second screen of Course Diversity Dashboard providing insight into URM students' study strategies and prior knowledge.