Watching the Election Sausage Get Made: How Data Journalists Visualize the Vote Counting Process in U.S. Elections

Election results in the United States are visualized online in real time by news outlets as vote counting persists over days or weeks. They are a massive public-facing exercise in managing audience understanding of uncertainty in partial data, breaking news web traffic records as the public seeks information about winners. We categorize designs of real-time election results from 19 U.S. news outlets and election results providers for the 2020 and 2022 general elections to create a visual vocabulary of live results. We then use this vocabulary to guide interviews with data journalists who worked on these designs to understand their design goals and challenges. Tying these conversations back to our visual vocabulary, we map out how communication goals like balancing certainty and uncertainty in the journey towards finding out winners, alongside challenges like determining thresholds at which information is shown, manifest in the designs displayed.


INTRODUCTION
In the United States, for as long as competitive elections have existed, people have satisfed their curiosity about election outcomes with numbers [21].News media has delivered on this front, publishing numbers-flled, data-driven coverage of elections that includes political polling 1 , probabilistic election forecasts, and real-time updates of vote tallies on election night.As technology evolvedthink telegraph lines [45, p. 47], to the television [45, p. 131], to the Internet -election results were broadcast and updated to a hungry audience at increasingly fast speeds.Today, with the help of the Internet, most major news outlets visualize results in real time through fickering online dashboards, aided by third party results providers that aggregate results across the country.These dashboards are incredibly successful and break web trafc records for digital news media [28].
Though technology has made real-time updating possible, it still takes time to count and report all votes cast in the U.S. because of its decentralized electoral system [8,39].Between the time when polls close and when winners are called for races, partial results can create mirages suggesting one party (and candidates for that party) will emerge victorious across races [13].Publishing partial data comes with the responsibility of managing the public's understanding of uncertainty in the electoral outcomes as they view incomplete data.It also comes with the responsibility of explaining the processes underlying vote counting and reporting -after all, some have interpreted spikes in visualizations of vote counts as evidence of election fraud [51,88].In the realm of quantifed news coverage of elections, which Littlewood [45] calls "public opinion on a conveyor belt", election results dashboards and the data they display are signifcant because they ofer the public a unique opportunity to watch electoral processes unfold while dealing with the aforementioned responsibilities.This prompts the question: why and how are election results communicated and visualized in real time?
In pursuit of this question, we conducted a study comprised of two parts: a visual analysis of election results dashboards during the 2020 and 2022 general elections to produce a visual vocabulary of real-time election results, and a qualitative study interviewing 13 U.S.-based data journalists about their design decisions for these dashboards.Considering the sizeable majority of adults that obtain news "often or sometimes" "from a smartphone, computer or tablet" [34], we focus only on online presentations of real-time election results in this study.
The visual vocabulary illustrates the status quo of how real-time election results are displayed online, while fndings from the qualitative study shed light on the priorities and challenges infuencing designswhat were the journalists trying to achieve, and what difculties did they run into?We also got existential with it, asking interviewees about incentives for publishing election results in real time and whether they would stop doing so if they could, instead waiting until all votes are counted to tell readers. 2 To illustrate a bird's-eye view of the design process, we map participants' goals and challenges to design choices.Our mapping shows how goals, like not implying something too early about election outcomes and contextualizing election results for diferent readers, materialized into designs while in tension with diferent challenges, like determining at what point of vote counting to show information and the possibility of losing reader confdence.We also catalog, with input from our interviewees, how several data sets "fow" over the course of vote tallying into results dashboards and some design decisions made throughout to display this data, providing a "timeline" of reporting results.

BACKGROUND 2.1 Idiosyncrasies of U.S. elections
There is no central election authority in the U.S., which means that local election ofcials count ballots.After counting, ballots are reviewed, or "canvassed", at the local and state levels to make sure every vote is accounted for and to correct any errors [20].Finally, results are certifed and made ofcial after review [8,39].The decentralization of elections means that votes tallied are not aggregated and published in one place by the federal government -instead, they are published by local and state election authorities across the country, scattered across government websites.Fortunately, third party organizations have taken up the mantle of aggregating and disseminating results.The Associated Press, a wire service, stations stringers locally to collect results from county clerks in addition to scraping results from websites [62].Other providers of election results include Decision Desk HQ, founded in 2012 [36], and Edison Research [68], which provides results to ABC News, CBS News, CNN, and NBC News.These three providers also provide exit polling, project winners for races, or "call races", and calculate estimated expected vote (EEV) -an estimate of the total vote in an election -as votes are reported.News outlets buy access to these products and publish them online.
Results published in real time on election night and the days following are unofcial, meaning they have not been reviewed and certifed yet.Race calls (also referred to as winner projections) made by results providers and sometimes new outlets are also unofcial.Throughout election night, lead margins are calculated, which is typically the percentage point diference between share of votes reported thus far for leading candidates.Percentage of estimated expected vote (EEV) counted thus far communicates progress in counting.The Associated Press estimates this metric using "turnout in recent elections, details on votes cast in advance and -after polls close -early returns" [63].Some outlets opt to show percentage of precincts reporting instead, which is the percentage of total precincts that have begun reporting votes. 3owever, this metric can be deceiving as a measure of counting progress because a precinct may have not yet released all of its results [62] when it is shown as reporting.

How quantitative data became part of journalism
To understand why quantitative data is prevalent in political news, or news at all, we turn to historical recounts of how data increasingly permeated journalism practices.Precision journalism, a vision of journalism relying on social science methods of collecting empirical data to assess the validity of claims and to contextualize social events, was advanced by Phillip Meyer in his 1973 book [1,48,56].Precision journalism came to be known as the incorporation of surveys and opinion polling in news coverage

How news media narrates elections using numbers
The media's data-driven coverage of elections includes political polling [58,84], probabilistic forecasts published before Election Day and on Election Day after polls close [17,32,33,60,79,81,92], election results published and analyzed in real time, exit polling [24,77], and post-election analyses assessing how much of and how the electorate voted [16,18,73,90].
It is clear that media coverage of elections in the U.S. impacts public perception: it directs the public's awareness to certain issues [47] as well as candidates and traits of candidates [91, p. 28], all of which can infuence how voters evaluate candidates over time [90]. 4How does coverage that specifcally centers numbers and statistics impact the public's perception of elections and democracy?Political polling, which systematically samples participants to relay their candidate preferences and issue stances, and media coverage of polling can infuence future political coverage of candidates [94] and change voter preferences to align with the majority [3,87].Polls often serve as input data to pre-election probabilistic forecasts [35] built by news organizations that predict winners of races with some statistical uncertainty.An overarching critique of news coverage of polls and probabilistic forecasts relegates it to "horserace" journalism [58,92].Horserace journalism furthers the collective perception of elections as a sports game by focusing on "who is ahead in the race and whether a candidate's position is improving or deteriorating throughout the race" [12].This critique notes that it is convenient for the U.S. news media to cover elections as a numbers game rather than assessing which candidates promise better policies in order to adhere to partisan neutrality [58].Journalists also frequently leave out statistical uncertainty in polls when writing about them [9,55,84] or portray survey error as actual change in voter preferences [58].
Real-time election results can be understood as the fnale to pre-election political polling and forecasts, as the citizenry fnally sees votes cast and winners called after months of coverage about what might happen.Election results were not always broadcast live by news outlets [45, pp. 15-16].As more people, specifcally adult white males, became eligible to vote, people turned to newspapers to stay informed about candidates, how popular they were among the public, and election outcomes [45, pp. 20-24].Later on in the late nineteenth century, there would be considerable audience demand for live election results; masses of people would gather outside of newsroom buildings after polls closed, ready to watch freworks and bulletin boards communicating results [45, pp. 54-57].
Today, online election results dashboards are where examples of data-driven elections coverage come together -election results, information from past political polls, live election forecasts activated early on in the vote counting process to predict winners -making them an important news artifact to understand.

Visualizing data in real time
One of the primary goals of election results dashboards is to show ongoing information about the state of races, which aligns with Few's [30] defnition of dashboards as "a visual display of the most important information needed to achieve one or more objectives; consolidated and arranged on a single screen so the information can be monitored at a glance."Sarikaya et al. [72] and Bach et al. [7] assess large samples of dashboards and code the dashboards by purpose, audience, visual and interactive features, data semantics, structure, and components.Though their impressive, expansive design spaces helped us categorize and situate election results dashboards, neither drill into how the visual encodings of data change over time or the specifc sociotechnical challenges of creating dashboards.A close parallel to our study is research on COVID-19 dashboards and visualizations [25,95,96].Similarly to election results dashboards, COVID-19 dashboards were published by news outlets, share the unique responsibility of engaging with a wide audience, and display and contextualize dynamic data with editorial content.Our study contributes a domain-specifc, public-facing 5example of how dashboards can inform the public and contextualize data with uncertainty, building on studies of public-facing dynamic dashboards like COVID-19 dashboards.
To show election results as soon as votes are counted and reported is also an example of communicating data progressively.Though progressive visual analytics (PVA) and progressive visualizations are targeted more for complex data exploration and analysis [2,50,76], 6 research in this area reveals relevant benefts, challenges, and recommendations for visualizing data progressively before the requested computation fnishes.One beneft of PVA applicable to viewing live election results is monitoring the computation to understand the process -monitoring vote reporting may enhance understanding of the process, depending on how news outlets frame election results.Applicable challenges that users encounter with PVA include the perils of judging partial data, monitoring the succession of results to determine when results are stable and trustworthy enough, and handling fuctuations [2].For progressive visualizations to be useful, they should cue users when new results are available, communicate uncertainty in partial results and the computation process, and keep visual complexity at a minimum [2,76].
Some studies tested how well participants judged early trends in progressive visualizations showing converging data, reducing the sample's error bounds over time [57,64].However, as we discuss in our interview fndings, creators of election results dashboards explicitly caution readers against drawing conclusions as quickly as possible from early vote counts.In addition, the tallying and reporting of election results is not done at random -there is a time bias, as rural, Republican-leaning counties tend to report their ballots frst [13] and mail-in ballots that require more time to count have leaned Democratic in recent elections [65].Procopio et al. [64] did fnd that participants declined in performance when false patterns were introduced early in the data progression, falling prey to illusion bias, making PVA a relevant domain to study people's cognitive biases when viewing partial data.Viewers also reference outside information in their interpretation of partial data; people who believed Donald Trump won the 2020 presidential election cited a constellation of factors contributing to their belief in election fraud, like the furry of changes to voting rules because of COVID-19, a belief that Democrats and the media wanted Trump out of ofce, and spikes in visualizations of vote counts they perceived as suspicious [51].

Communicating uncertainty
Because it takes time to count votes and election results are communicated piecemeal, there is a window during which news outlets manage the public's understanding of uncertainty around race winners.Who won an election is considered information with epistemic uncertainty because winners are set in stone the moment polls close and we can reduce the amount of uncertainty as we gain more information.This difers from aleatoric uncertainty that is introduced by the "fundamental indeterminacy or randomness of the world" [86].The window of uncertainty after polls close before races are called can be mere minutes, hours, or months in the case of a runof election. 7Any expressions of uncertainty on dashboards change in real time as we know more about races, presenting visual and editorial challenges.
van der Bles et al. [86] provides a helpful framework for communicating epistemic uncertainty that includes who is doing the communication, what uncertainty is expressed, how the uncertainty is expressed, and who the audience is and how they are impacted by the information. 8Additionally, Spiegelhalter [75] ofers advice for how to communicate and visualize uncertainty, like expressing uncertainty as a proportion, frequency, or percentage with a clear denominator and visualizing uncertainty several ways to reach diferent audiences.Newer visual encodings for uncertainty and probability, especially those that express the probability distribution in discretized formats, have been shown to improve participants' task performances [23,38,41].But despite all of this, a survey by Hullman [37] showed that the perceived norm among participants, some of whom are journalists, is to not communicate uncertainty in visualizations.They rationalized this decision with several reasons, e.g.not wanting to confuse readers, not having uncertainty information, viewing uncertainty as tangential information.We discuss expressions of uncertainty in our sample of election results dashboards when we present our visual vocabulary, as well as reasons for and against highlighting uncertainty in our interview fndings.

A VISUAL VOCABULARY OF REAL-TIME ELECTION RESULTS
To understand the status quo of how election results are communicated and visualized in real time, we conducted a visual analysis of results dashboards from the last two general elections in the U.S -the 2020 presidential and 2022 midterm elections -to create a visual vocabulary from the designs.Our goals for the vocabulary were to tease out the key data components of election results what data comprise election results?-and how they are communicated and visualized.We focus more on the content of dashboards, though there are some examples of composition.

Sampling
We focus on the 2020 and 2022 general elections because they are the most recent elections, had high turnout, 9 showcase diferent seats up for election, and are opportunities to observe how data journalists handled large increases in mail-in ballots due to the COVID-19 pandemic [19].To determine which news outlets to include in our analysis, we frst sourced a list of broadcast and print news outlets with a digital presence from Pew Research Center's methodology for examining news coverage of Joe Biden in 2021 [6].Their methodology selected outlets using Comscore Media Metrix® data, only including an outlet if its digital site received a large enough viewership of "at least 30 million average unique monthly visitors in October-December 2020." In addition, an outlet had to report extensively on "national afairs or political issues."Their list provided a helpful starting point, as we wanted to analyze well-trafcked digital news outlets that report on political matters in some capacity.
After reviewing this list, we excluded outlets without election results dashboards for our targeted elections (Breitbart, Business Insider, New York Post, Newsweek, and Vox).We added some local news outlets that created bespoke election results dashboards for our targeted elections, intentionally including outlets in states (California, Vermont, Washington) with universal mail-in voting.We also included Reuters because of our interest in how an outlet approached communicating U.S. election results to an international audience.Some outlets in our list sourced their results visualizations from The Associated Press or Decision Desk HQ (The Hill, The Washington Examiner, The Seattle Times, BuzzFeed), so we include The Associated Press and Decision Desk HQ in our list and remove outlets that used their visualizations for both the 2020 and 2022 elections.Note that Decision Desk HQ is purely a results provider, not a news outlet.Our fnal list is shown in Table 1.

Capturing designs
We captured dashboard designs as votes were actively reporting and once election results were no longer updated.For the more recent 2022 midterm elections, we searched for links to each outlet's election results dashboard using search terms like "live election results 2022 midterms" coupled with the outlet name.After polls closed on Nov. 8 (Election Day) and the days following, we took screenshots of the pages as votes were reported.We also took screenshots of the fnal, archival versions of the dashboards after elections fnished.For the 2020 presidential elections, we searched for dashboards using similar terms but substituted in "2020 presidential" and took screenshots of the fnal, archival versions.
Wayback Machine was used to screenshot any pages we missed as votes were reported during the 2022 elections and 2020 elections.For the 2020 elections, we sampled dashboard designs on Nov. 3 (Election Day) and the days following.Screenshots from Wayback Machine make up our entire sample of 2020 live designs.Some components on pages sampled from Wayback Machine did not load properly, and we note the fdelity of screenshots in Table 1.Later, during our interviews, we asked participants to send screenshots of their dashboards as results were reported but did not receive any.All screenshots and links to dashboards are included in supplementary materials.

Analyzing designs and creating a vocabulary
To analyze our sample of designs, the frst author frst broke down each page into distinct, visually discrete components that communicated election results or electoral processes.White space was usually a good indicator of a discrete component.Text was considered if it concerned the status of the race, like an announcement about a race call, or meta information about election results like a disclaimer or data source.Scanning all of the screenshots, the frst author frst grouped components across dashboards by the data communicated, then further categorized components by the visual encoding of the data.Components that communicated new data or used a new visual encoding were sorted into existing groups or began a new group.Groupings were iterated on and refned over several sessions in conversation with the second author to ensure that groupings made sense and aligned with goals.The fnal groupings are shown in Fig. 3 and Fig. 4, with the headers stating the data or information communicated and examples of how they are visually encoded listed underneath.Details on groupings and a table of information and visual encodings present in each dashboard are included in supplementary materials.

Results
The components in Fig. 3 show the meat of election results -vote counts and vote shares, which are used to fnd out which candidates are leading at any given moment and by how much.They also show the outcomes of races, which are comprised of winners called, control of a chamber for the party that won enough seats to claim a majority, and whether a race advances to a runof election.
Text labels and iconography were often used together to communicate the status of races, like how much candidates were leading by or who won a race.Dashboards also made liberal use of tables and maps to show vote counts and vote shares, shading them with varying hues, luminance, and patterns to visualize the progress of vote counting and who or what was leading and by how much.1: Fidelity of screenshots of election results dashboards from each organization, election, and phase of vote counting.We also note which organizations we interviewed from.Wayback Machine was used to capture all dashboards during vote counting for the 2020 general election and some dashboard pages during counting for the 2022 general election.Most of our 2022 sample during vote counting was captured on election night and the days following.Partial fdelity means some components on the page did not load, e.g.geographic maps.Dashboard that were not captured or did not load on Wayback Machine were not included.

Organization
The use of simpler chart types aligns with other studies of dashboards that recommend minimizing visual clutter and supporting non-experts [7,72] -individuals of varying data and visualization literacy levels should be able to comprehend the information.Components in Fig. 4 show the progression and process of vote counting, as well as live election forecasts.Some of these could be considered strategies employed to manage uncertainty while viewing partial results, which we explore and confrm by asking participants during our interviews.The progress bar visualizing percentage of estimated expected vote reporting communicates to readers that we are early in the process of tabulating votes, so trends may be misleading.Examples of qualitative expressions of uncertainty state that there are few votes in and encourage readers to be patient.Shading visualizations with a neutral palette that communicates counting is in progress instead of shading by leading candidates is an example of waiting to show information and deterring readers from drawing conclusions early.Forecasts that predict winners and party control of congressional chambers with qualitative and quantitative uncertainty may correct for audience misinterpretations of early false patterns in election results.There were more expressions of qualitative uncertainty in forecasts, e.g."leans", "likely", and "favoured to win", across dashboards than quantitative uncertainty.These uncertainty expressions mirror expressions of uncertainty included with COVID-19 visualizations -COVID-19 visualizations added disclaimers and explained the data collection process to show uncertainty in data quality [96].
However, we are only scratching the surface of real-time election results with our visual vocabulary, hypothesizing strategies that data journalists used to communicate uncertainty around this ephemeral data.It is worth uncovering what journalists intended to communicate with these designs because that is refection of what they think readers want to see and fnd valuable. 10We had questions about what we observed in the dashboards -what determines the colors and visual salience of components on pages?How do you choose what information to show at diferent points of the vote counting process?How do your designs respond to claims that visualizations of election results look suspicious?Might it be confusing for readers to see live election forecasts as votes are reported because it may suggest that the election outcome is in fux when it is not?(Pre-election forecasts are published before Election Day when the outcome is indeed still up in the air.)Guided by our vocabulary of designs, we interview the individuals who created these displays to ask them about their design decisions for real-time election results, which we discuss in the next section.

QUALITATIVE STUDY OF DATA JOURNALISTS
After summarizing designs of real-time election results in our vocabulary, we craved richer explanations for why election results were communicated in these ways.In pursuit of answers, we interviewed the data journalists and designers who designed these dashboards.Interviews revolved around the following research questions: • Why are election results published in real time?
• What are the resources and organizational structures for designing real-time election results?• What broad communication goals guide the designs of realtime election results?• What communication challenges are faced when designing in service of these goals?Elections are covered regularly by newsrooms, with several of our participants working on live election results across multiple elections and newsrooms.Given that elections are not one-of events and some newsrooms are constantly preparing for elections, we were optimistic that journalists would have strong recollections of their decisions.We also discuss potential follow-up studies in Sec.5.2 to further understand newsroom processes for live results.

Positionality statement
The frst author previously covered elections as a data journalist at The Texas Tribune.This context was included in recruitment messages to potential participants.The frst author's experience also guided the questions asked and fow of conversations during interviews.We lean into this expertise and see it as an advantage, because the frst author could probe further beyond the surface of design processes for election results given a baseline knowledge of the process.

Recruitment
To recruit participants, we emailed at least one data journalist or designer who had likely worked on an election results dashboard sampled in our visual analysis, making sure to cover every organization in our list.Dashboards often included a list of contributors' names on the page, and newsroom employees will typically have public professional emails.Some individuals had since left the new outlet where they designed the dashboard for the 2020 and 2022 general elections, complicating recruitment.We also snowball sampled by asking participants to refer our study to others, aiming for participants who were known to be enthusiastic or skeptical about the value of live election results.In addition, we attended a community call hosted by Open News 11 to discuss the study and recruit participants and posted a recruitment message in the Slack workspace News Nerdery.
When prompted, we sent our research agenda and questions to share with news organizations' standards departments in order to 11 Open News is an organization dedicated to fostering collaboration and open practices in journalism-technology networks.
clear the individual for participation.However, some outlets (The New York Times, CNN) ultimately chose not to participate due to internal policies for research participation, so we do not include their perspectives.

Participants
We interviewed 13 data journalists and designers who built, designed, or managed the design of election results dashboards at The Associated Press, Decision Desk HQ, The Los Angeles Times, NPR, Politico, Reuters, The Seattle Times, Star Tribune, The Texas Tribune, VTDigger, The Wall Street Journal, and The Washington Post.
The organizations we interviewed individuals from are marked in Table 1.
During the time they worked on designs for real-time election results, our participants were formally titled as a data journalist or graphics reporter, and ranged in seniority from reporter to editor or director, except for our participant from Decision Desk HQ.As a proxy measure of experience, we asked participants to report the news outlets where they had previously covered real-time election results.Most (8 out of 13) had worked on live results at one or two news outlets, and four had worked at three or more.Our participant from Decision Desk HQ had only worked on live results there.Eight participants had worked on election results at local news outlets, and nine had worked on election results at national news outlets.

Procedures
Interviews were conducted by the frst author in a semi-structured format over Zoom and recorded using its recording function. 12Each interview lasted around an hour and participants were ofered $20 compensation. 13The frst author came prepared with a blueprint of interview questions touching on each of the four research questions for the qualitative study.Interview questions are included in supplementary materials.If the participant worked on a dashboard for the 2022 midterm elections, we began the interview discussing the 2022 dashboard because it was designed more recently than the 2020 dashboard.We screen shared screenshots of the dashboards designed by the participant during the interview, asking the participant to refer to specifc components on the page that were designed in service of goals and in tension with challenges.
In addition to our interview questions, we sought input from participants on how data fows into election results dashboards on election night and the days following.This fow is catalogued in Fig. 11, and it evolved signifcantly as we incorporated participant feedback -much of their feedback complicated the process of designing real-time election results, introducing complexity to a seemingly straightforward process.

Thematic analysis
After transcribing interviews, we sent transcriptions back to participants and gave them the option to correct or clarify anything said.Five responded and requested no changes.The frst author then conducted a hybrid thematic analysis [10,11] by breaking down the text, coding each text snippet with themes and insights, and grouping snippets into emerging categories while positioning the categories as answers to our research questions. 14We focused codes on the why and how: reasons for publishing election results live, resources and organizational structures for designing and building results dashboards, and goals and challenges around communicating live results.If a text snippet referred to a specifc component in the dashboard design, we preserved the relationship so we could refer back to how goals and challenges manifested in designs.The frst author iterated and refned the initial groupings while consulting the second author, eventually clustering them into higher level themes and drawing connections between themes.The higher level themes were then further refned over several sessions before settling into coherent themes that we present in the next section.A list of higher level themes and sub-themes are included in supplementary materials.

Results: Reasons for publishing election results in real time
We frst explore the reasons and incentives for publishing election results in real time.It is not universally agreed among data journalists that election results should be communicated the way they are, or communicated in real time at all [5,93].Yet all major news outlets publish results live.Why?For one, they are a reader service that make election processes more transparent in a complex, decentralized system, hold elections ofcials accountable, and are opportunities to set the record straight when misinformation spreads about electoral procedures and the meaning of results.When there are errors in vote tallies, like clerical entry errors, journalists and results providers can fag these errors for immediate correction and seek clarifcation from elections ofcials [46,62,69].It is the public's right to "see into the mechanisms of this kind of Byzantine system of the government just counting all these votes" (P5).However, partial election results are typically not useful for any decisions in the moment (everyone has presumably already voted), existing more for a reader to satisfy their information-seeking needs (P1, P2, P4, P5, P7, P13).This reasoning was used to rationalize publishing and not publishing results in real time -it is harmless because it is not impacting decision-making; but if it is not impacting decision-making, why spend all this efort publishing results live?Some (P4, P8, P13) entertained the idea of not publishing partial results at all, but added the caveat that in the current world and news landscape we inhabit, it would not be possible given the potential for conspiracy theories to spread in the absence of live results (P7, P8).Regardless, almost all participants acknowledged that real-time results do serve some beneft and will continue to be published, but that communication should be improved (P1, P5, P6, P7, P9, P10, P11, P13).
From a news business standpoint, real-time election results generate reader engagement and revenue in a competitive news landscape. 15Competition is a part of journalism, in which "there's a little bit of double counting and maybe a little bit of wastefulness as you're trying to pursue scoops and news" (P6).The felt duty to provide live election results as a public service coupled with record-high reader engagement and revenue gains means that it would be difcult to stop publishing live election results due to expectations from readers, political reporters, editors, and customers (of organizations like The Associated Press, Reuters, and Decision Desk HQ) for this coverage (P1, P4, P5, P8, P9, P10, P11).
From an internal organizational standpoint, the massive collaborative efort of building and designing real-time election results forces teams to clean up their team structure and lines of communication, which is useful for other cross-newsroom projects (P6, P11).Some teams also would not exist without the project of publishing real-time election results (P6, P11).Other benefts include increased data literacy and elections literacy across newsroom members working on election results (P11, P13).Major technical advancements also occur (P2, P3, P7, P11), like using a new web framework for election results dashboards that is adopted by the rest of the news website.

Results: Resources and organizational structures for designing real-time election results
Design decisions do not occur in a vacuum, so we asked participants about the process of creating results dashboards, focusing on factors like resources available and organizational structure.It is generally agreed upon that election results dashboards are massive projects involving considerable time and resources, even while news outlets, especially smaller local outlets (P4), have few resources to begin with.Designs for the upcoming general election typically build of of designs from the last general election (P1, P3, P4, P5, P9, P12).At the organizations represented by our participants, the number of people who contribute to real-time election results for the general election ranges from one person to more than 20 people, and the duration of the calendar year devoted to the project ranges from weeks before the election to 100% of the year as a full-time job.Real-time election results often involve data, graphics, engineering, product, and political reporting teams (P1, P2, P6, P7, P12) -a feat of project management (P6).Although teams can simplify their designs or rely on paid graphics from results providers (P3, P9), the project burns people out.For those in charge of real-time election results, the project can end up being "what got them to to walk out of the building" (P2).One solution to reducing eforts is streamlining all historical election results dashboards into one standard application with a design template set up to easily ingest data in the future (P7).We point out the amount of work involved in designing, building, editorializing, and maintaining rigs for live election results, and the organizational and resource constraints that news outlets face, because it is a direct factor in how ambitious the designs are.When a team is small and a developer is hired months before the election (P1), or when resources are limited as a local news outlet (P9), the design of live results can hinge on two questions: "What do we know we have the capability to do because we've done it before?"and "What do we think people would want to see?" (P9).With limited resources, there is a "measure twice cut once" (P9) attitude that fosters hesitancy in trying new things when it comes to displaying election results that trumps suggestions for new features.This calculus may vary by newsroom and by election.But no matter how limited the resources, election results will likely continue to be published in real time for all the reasons mentioned in the previous section.Incentives for broadcasting real-time election results and organizational resources available infuence the goals and challenges of designs that we discuss in the following sections.

Results: Goals and challenges of designing real-time election results
4.8.1 Mapping goals and challenges to designs.At the risk of presenting an overwhelming amount of information up front, we map the goals and challenges discussed by participants in the next few sections to specifc design choices, illustrated by components of our visual vocabulary, in Fig. 5.We encourage you, the reader, to print out this fgure and use it as a visual guide as you read the next sections.
4.8.2Goals.We asked participants to talk about the goals and priorities they set for their designs of real-time election results, helping us understand what they set out to achieve, what they set out to inform readers about.
Provide and interpret election results.The frst and foremost goal is to 1A provide basic information on who and what the electorate could vote for in the current election and the status of races (P1, P4, P5, P6, P7, P8, P9, P12, P13).As mentioned in the previous section, this goal is rooted in a sense of duty to readers (P6, P12), to provide complete information in a easy format that aligns with what readers saw on their ballots (P1, P4, P8).Dashboards often placed balance of power bars (Fig. 6) and geographic maps at the tops of pages to communicate a simple, holistic overview of the status of races.A related goal of providing election results is 1B organizing them on the pages in service of reader needs given the large amount of data and boundaries of screen sizes.Races were organized to encourage audiences to critically think about how race outcomes aligned with expectations (P2) -for example, races could be organized by their Cook Political Report ratings (solid Democrat, likely Democrat, lean Democrat, toss up, solid Republican, likely Republican, or lean Republican) [66] which would clue readers in to which races defed expectations once winners were called.Races could be surfaced earlier or highlighted on the page as key races, isolated onto separate pages, and fltered using drop down menus to draw readers to the most important or relevant races.Selectively emphasizing races was usually an editorial decision.For example, Star Tribune chose to elevate statewide races over national congressional races because statewide races impact the greatest amount of people in the state (P4).
There is also a general recognition that it is not enough to just show results -news outlets have a responsibility to 1B 1C explain and contextualize election results for diferent readers.They can do so by weaving in historical voting patterns, trends in how the electorate voted, and implications of results for Americans.There are several levels to contextualizing results: "One is basic information display.Yeah, this person's winning the state.Then the next level is some sort of analysis or cross comparison, like, oh, you know, the margin shifted this much from 2020, right?[...] I think the third and the Holy Grail is like the margins have shifted from 2020 and here is the state, and we're circling it, that like decided the election or had the most change" (P6).Results were layered with other data sets, like campaign fnance or demographics, and context from reporting to show who voted for and supported candidates.Dashboards also sought to provide race-specifc context (Fig. 7), issue-specifc context, context about U.S. electoral processes, and generally the downstream impacts of election outcomes on people's lives.Each race has its own idiosyncrasies worth reminding readers about, e.g. by adding text above races "to fag certain edge cases, like California primaries have two winners, for instance, for House, which is like a weird thing that they just do" (P10).Issue-specifc context can include explanations around issues represented by ballot initiatives, issue stances of certain candidates, and the ways people's lives will change with these issues in play.Reuters created simple visual explanations of U.S. elections and linked them on dashboards for its international audience (P3), an example of adding context about the electoral system.Some outlets, like NPR and The Texas Tribune, were more conservative, only showing leads when percent EEV reached 50% (P2, P8).During the 2020 presidential election, The New York Times did not show leads on its results map, only coloring states when races were ofcially called [4].The Washington Post showed leads when 35% of EEV and ideally 5% of two diferent types of votes (mail-in, in person, etc.) were reported (P11).On the Los Angeles Times' dashboard, it states "Map will not show a leading candidate in a county until 25% of the vote has been reported here" [78].Star Tribune showed leads at 10% of precincts reporting (P4), and Reuters showed leads as soon as any votes were reported (P3).Other outlets were more cryptic about their thresholds, as exhibited by ABC News' dashboard that states "Counties are colored red or blue when the % expected vote reporting reaches a set threshold.This threshold varies by state and is based on patterns of past vote reporting and expectations about how the vote will report this year" [53].Once information was shown, designs should 2B emphasize the most meaningful changes in results (P2, P4, P5, P6, P8, P10) -emphasizing every change in results could mislead people into thinking the outcome is in fux, making "people get upset or get nervous with the whole system [...], feel like it's not working for them because they see it as these are coming in and these can change at any time" (P8).It is important to intentionally prioritize "not having stuf moving on the page, not calling things until they're actually called like not turning it into a story about movement and change that has to be observed and making it more of like, if you want to know, here it is" (P2).Pulling back on colors used to shade races in tables and geographic maps was one strategy to subtly show leads and changes in leads when percent EEV was low (P2) (Fig. 8).Race calls take precedence over any other information, even if no votes are shown to be tallied (P5, P8), and are usually designed with the highest visual salience.Not emphasizing every change was also done to avoid inspiring anxiety in readers.Elections are often "very emotional and fraught events for a lot of readers", and if we keep treating them like sporting events, "it's really easy to end up with a visual display or a vis or whatever that is too active and too overwhelming and assailing to the reader" (P5).Sources of potential anxiety discussed include overwhelming amounts of animation and visual clutter (P2, P5, P6).
One strategy to sidestep implying something too early was to just 2C provide a best guess of election outcomes with quantifed statistical uncertainty.The Washington Post (Fig. 9) and The New York Times included live election forecasts alongside real-time results to predict winners after polls closed, encoding prediction uncertainty using opacity, luminance, and range of motion.These forecasts formalized a "gut level assessment" about who the winner of an election might be into a more rigorous statement of certainty (P11).However, the strategy of a forecast was not used or even considered appropriate by many of our participants.Several (P2, P3, P4, P6, P10) said they did not have in-house expertise to build an elections forecast.Several also said they were "not in the business of forecasting" (P3, P4, P7), mentioning the hesitancy to venture into forecasting because it's a "very political space to enter into" (P7) and the possibility of engendering mistrust in the media (P4).
1A GOAL 3 Show vote counting as an incremental process.The goal of emphasizing the most meaningful changes is complicated by another goal: 3A showing readers that vote counting is progressing.Results dashboards should give readers "an idea of not only what was called or not, but what was still a battle in progress, and with those battles in progress, being able to see how each race was going" (P9).It is still necessary to subtly communicate that results are changing without emphasizing changes too dramatically.Vote counting can also span days, so "everything on this page should be screaming at you that this is a process that takes time" (P2).To show progression, percent EEV was visualized through small bar charts (Fig. 10) and encoded as dot density shading on geographic maps of election results (P7).Dashboards included text about percent EEV above vote counts and text disclaimers explicitly stating that the process of vote counting takes time (P2).Before lead margins appeared at some threshold of percent EEV, dashboards also shaded visualizations with shades of gray and black to communicate that votes are still being counted and there is no leader yet.A related goal of showing the progress of vote counting is communicating the live-ness of results, because otherwise "all the work that you put into making it live is useless" (P3).A pulsing red dot infuenced by red live indicators in broadcast journalism shows that results are actively updated on the dashboard, although this animation could potentially create anxiety for readers (P2, P6).

3A 3B
4.8.3Challenges.After describing goals, we asked participants to describe why these goals were hard to achieve and the challenges they experienced.We recognize that these are a subset of challenges faced, because results dashboards are "challenges all the way down" (P13) when you're designing for data that is a "pile of edge cases" (P1).
Determining the threshold at which information is shown.In order to avoid implying something too early, news outlets had to decide the exact threshold (of EEVP or percentage of precincts reporting) at which margins of lead would be shown (P4, P8, P11).However, there is no one, magical threshold across races at which, past that point, meaningful leads are shown.
It depends on where the votes reported thus far are from, what types of votes have been reported, and how many and where votes are left.(Remember that mail-in ballots in recent elections leaned Democratic [65].)P11 illustrates this point with a previous election experience: "I remember in Florida, for example, when I was early in my election career, they would count all the Election Day ballots last, right?And then all the early ballots frst.And that was back when early ballots were Republican retirees.So what happened in Florida is that 50% of the vote would drop the minute the polls closed because it was all the mail-in votes and absentee votes and early votes.And it would be all devastating Republican wins up and down the whatever right.[...] And in Florida, like in the 2000's, like early 2010's, the Election Day vote was all Democratic and the early vote was all Republican." Avoiding the dopamine lever efect.It was challenging to avoid dramatizing partial results, which may entice readers to continuously refresh the page like a dopamine lever (P2, P5), while simultaneously alerting readers that results are changing.The entire efort seems to contradict itself, telling readers "don't count your chickens before they've hatched, but also tune in so you can see how many chickens have hatched" (P9).An additional tension is introduced here by news outlets' incentives to frame updates as newsworthy, which complicates eforts to temper the importance of small changes in vote counting (P8).Readers also crave certainty and up-to-date information about election results, and though news outlets try to provide some certainty and make clear that election outcomes are uncertain early in the vote counting process (P8, P9, P10, P11), readers may refresh repeatedly anyway.Updating less frequently so that each new batch of data carries more meaningful information is an option, but it is difcult to do this because milestones -the meaningful changes -happen unexpectedly and at various times across races (P2, P4).
Losing reader confdence.News outlets want to appear authoritative and knowledgeable on election night.They want readers to have confdence in their results dashboards (P3, P4, P6), and they risk losing reader confdence when waiting to release information about who is leading and avoiding speculation about election outcomes.This could ultimately lead to a loss of readers in a competitive news landscape, that if a news outlet "suddenly stopped or really pared it back, that there is a danger that we lose readers' confdence in our ability to tell them what they want or need to know about their communities and the place they live" (P4).Newsrooms are also designing against misinformation (P2, P6, P8), which means displaying and interpreting election results as soon as possible to explain what is going on during vote counting.On the fip side, if news outlets do not accurately communicate the state of races or imply something too early in their fght for readers, their coverage could become "fodder for misinformation" (P5) or "part of a misinformation campaign" (P8), which would also impact reader trust.
Explaining earlier race calls with no votes reporting.A repeatedly discussed challenge was communicating a race call disseminated by a results provider for a race without any votes reporting (P2, P3, P8, P13).P12 explains the approximate logic that Decision Desk HQ uses for race calls: "It's an algebra problem.News outlets then publish this information on dashboards alongside the other information.
Okay.We know how many votes are in.We know how many and we know where they're from and we know who's gotten them.The question becomes how many are left?And based on what we know about the geography of where those votes could come from and the partisan historical leans and how other things are going [...] that's how we do it, is how many are in, how many are expected back, and what would the second place candidate need of those votes to come back and overtake the frst place candidate." It is statistically possible to be certain enough about the outcome of a race before votes are reported when the race is not contested or a party or candidate has a strong history of winning [80].However, this presents an awkward challenge of communicating the logic to readers who may not understand the nuances of race calls (P13).
Explaining and signalling election protocols.Part of explaining elections as a process that takes time is communicating the protocols involved, like ranked choice voting (P4) and runof elections (P6).It is assumed that ranked choice voting is less well-known because it is less common across U.S. elections compared to the frst-past-the-post system (P4).Runof elections are triggered with diferent criteria across states and types of elections [85], which can be confusing.Election results dashboards employed various ways of visually signalling these processes -in the case of a runof election (Fig. 12), it is considered an outcome of the current election, so it is therefore reasonable to choose a party-neutral color with similar luminance and salience as race call colors to represent races advancing to runofs (P3).Breaking out of traditions.Pushing back against institutional traditions around how real-time election results should be visualized was challenging (P2, P5, P6, P7, P8, P9, P11, P12).Because real-time election results are an immensely successful product, outlets may be hesitant to try something new that readers are not accustomed to.This is especially true for smaller news outlets, as discussed in Sec.4.7.This is also true for the Associated Press, who has customers to consider when designing results graphics and as a result, is "a lot more creatively constrained" (P5).
One visualization subject to much fruitful debate is the geographic map."Land doesn't vote" (P8), but geographic maps visually over represent states with larger surface areas and do not consider high population density in urban areas.However, it was difcult to overturn the geographic map in favor of a cartogram (Fig. 13) because readers still wanted geographic maps out of familiarity, and "familiarity in many cases is more important to us than any of these other, more trivial concerns" (P11).Editors also felt the cartogram was "too weird" (P6).
Contextualizing results to not feel removed from peoples' lives.The presentation of results on dashboards can feel too abstract, where citizens' interests are fattened into numbers and colors (P1, P2, P5, P9).It can also feel removed from the downstream impact of election outcomes on people's lives (P1), which some (P5, P12) regarded as the "scorecard" element of election results.Adding nuance and meaning to election results is more challenging when the design of real-time election results feels immutable (P1), given everyone's expectations of what they should look like.We point out that opinions difered on the scorecard element of election results and its presentation -adhering to a scorecard format was viewed as adhering to the procedural reality of our elections system, a "straightforward" presentation that isn't making any kind of commentary (P7), while also viewed as framing elections as a sports game that could create cynicism (P1, P2, P5).The presentation of the numbers could be a distraction to the nuanced interests of citizens, while simultaneously a representation of those interests.
Designing useful, intuitive navigation for races.Given that organizing races to best service readers is a goal and there is a vast amount of results data to display, it is no surprise that designing intuitive navigation presents as a difculty (P3, P4, P10).Good navigation should "get them to all of the things we have to ofer" and make it "easy to get back to where you were, to get to the next place you want to go" (P3).Dashboards often double encoded navigation from the national level to state level, providing readers with the option to click on states in geographic maps, tables, and drop down menus to view state results.Local news outlets showing a long list of races for various counties and cities in their dashboards encountered additional navigation challenges.

Open design questions
If you are a data journalist, you may be wondering at this point of the paper: how can I better communicate election results for the upcoming 2024 presidential elections?If you are a researcher, you may be wondering: what areas are worth researching to understand how to improve the communication of live election results and progressive data generally?We hesitate to make concrete design recommendations, because we did not investigate perspectives from readers of real-time election results to confrm or deny our interviewees' assumptions of what they fnd useful.Instead, we list several open design and research questions requiring further studies to validate, blending questions that concern uncertainty communication, progressive visualization of data, and areas that our interviewees suggested needing improvement.We also emphasize that not every news outlet needs to create a bespoke display for real-time election results.Perhaps it is enough to recirculate results from other organizations, and spend more time reporting on the health of electoral processes, downstream consequences of election outcomes, and nuances in citizens' interests and voting participation.
• How can dashboards better emphasize uncertainty in partial, progressive data?The uncertainty expressed with real-time election results usually references the incompleteness of partial results, cautioning readers against drawing premature conclusions about winners.We are curious if readers internalized and understood qualitative expressions of uncertainty in partial results, given that past research has shown there is variability in the probabilities that individuals attach to verbal expressions of uncertainty [14].Considering the challenge of losing reader confdence, we also wonder if expressions of uncertainty in real-time election results afect trust in results, electoral processes, and news organizations.The efect of disclosing epistemic uncertainty on trust and credibility has shown mixed results, suggesting the efect may be context dependent [86].This is also relevant to how progressive visual analytics display uncertainty in order to build trust with analysts [2].However, uncertainty shown during the progression of vote counting should be understood by laypeople, not just elections analysts.Election results are also a relevant area to study data hunches which is defned as "an analyst's knowledge about how and why the data is an imperfect and partial representation of

Limitations
the phenomena of interest."While Lin et al. [44] focused on use cases of recording data hunches with analysts and experts, we wonder how data journalists can externalize data hunches about election results, described as a "pile of edge cases" (P1, P2, P11, P12), throughout the vote counting process to the general public, and how the public receives this information.• What strategies are successful in correcting for illusion bias [50,64], when readers fall prey to false patterns during the progression of data?Live election forecasts were one strategy implemented to correct for this bias, but we need to empirically evaluate whether they do so.Forecast predictions were shown with verbal qualitative uncertainty, e.g."Candidate A is slightly favored to win, but Candidate B still has a chance", and visualized quantitative uncertainty.It is worth studying whether they introduce confusion with this uncertainty by promulgating the idea that election outcomes can change after polls have closed.• How can displays of real-time election results better inform readers of the vote counting, reporting, review, and certifcation processes?Election winners will eventually be known, but watching the process unfold is unique to real-time election results until votes are certifed.We mentioned earlier that one beneft of progressive visual analytics is the opportunity to understand the computation process as intermediate results are shown [2].How can this beneft extend to viewing real-time election results?This goes hand in hand with the research questions of why readers view real-time election results and what they want to learn.If readers simply want to know who might be the winner when viewing live election results, adding more context on electoral processes may not increase understanding.Relatedly, if readers have already solidifed their opinions on electoral integrity, then learning more about electoral processes may not increase their trust in the system.It would be fascinating to systematically explore the types of misinterpretations readers have of real-time election results and the vote counting process, and examine the reasoning that individuals employ to rationalize their misinterpretations.• What elements of real-time election results cause anxiety and why?Our participants repeatedly emphasized that elections are anxious afairs and that they did not want displays of live election results to exacerbate readers' anxieties.This question can be investigated against the landscape of afective visualization design, which concerns data visualizations designed to communicate and infuence emotion [42].Morini et al. [49] examined emotions in response to visualizations of climate-related issues, another anxiety-inducing topic.Emotion can be considered a precondition to and a result of viewing live election results, though it is unclear how much visual design choices impact the anxiety that readers already feel around elections.It may be the case that dashboard designs only serve to not exacerbate anxiety, rather than alleviate anxiety.Furthermore, we question what is the appropriate amount of anxiety that readers should have when viewing real-time election results. [44], One limitation of our visual vocabulary is that we relied on Wayback Machine to capture dashboards from the 2020 and 2022 general elections as votes were reporting, though for the 2022 election we captured most of the pages on election night and the days following.
Because some components on dashboards did not load via Wayback Machine, we missed some designs that appeared temporarily during the vote counting process.In the future, we plan to automate the collection process by deploying a script to screen capture dashboards during vote counting and reporting.We also did not examine all editorial text published on dashboards, focusing only on text concerning the status of races and metadata about election results.Additionally, we did not include visual displays of real-time election results from broadcast television or print news, which each have diferent design constraints and extent to which they control viewers' pace of receiving information [26].
Regarding our qualitative study, we did not speak to a data journalist or designer from every news outlet on our targeted list, missing perspectives from ABC News, CBS News, CNN, Fox News, NBC News, The New York Times, and USA Today.We also recognize that this study would be strengthened by ethnographic studies within newsrooms as data journalism teams are planning election results dashboards before election day, on election night after polls have closed, and in the period following until all votes are tallied.Ethnographic feldwork would corroborate participants' descriptions of their design processes and more closely trace the organic, messy, and cross-newsroom evolution of the designs.
Overall, our study did focus on live election results in the U.S., so it is unclear how these fndings may translate to the publishing of results in other countries.Vote counting tends to go faster in other countries with centralized election authorities and shorter ballots, decreasing the window of uncertainty in election outcomes [22].

Conclusion
This study was initially inspired by the frst author's curiosity about how displays of live elections results were modifed amid unfounded allegations that the 2020 general election was stolen [27].Some allegations hinged on "suspicious" spikes in visualizations of vote counts, exploiting the procedural complexities of the U.S. electoral system to sow doubts in the legitimacy of election outcomes [51,88].Our study then evolved into a broader investigation of how to communicate results in real time to a curious audience.We frst created a visual vocabulary to understand the landscape of current designs, and then interviewed data journalists and designers to gain rich insight into their design decisions.We hope that the vocabulary serves as a helpful guide to visualizing progressive data to a wide audience.Our qualitative fndings uncover numerous future research opportunities around real-time election results, from a journalism, visualization, and political communication perspective, as it is a piece of the larger puzzle around how numbers are used and visualized in the American political media ecosystem.
We again return to this question of why new organizations publish election results in real time.From our study, we learned that it is because of a belief that the citizenry has a right to see and engage with the most up-to-date information, and that there are organizational, reputational, and fnancial benefts to publishing results live.The next step is understanding why and how readers engage with this information.
[1, p.  125].The founding of the National Institute for Computer-Assisted Reporting (NICAR) and Investigative Reporters and Editors (IRE) in the late 1900's also advanced computer-assisted reporting and the use of data and databases, particularly for investigative journalism[67].Both institutions still provide training to data journalists today[31].As computers gave journalists easier and faster access to data [1, p. 134]), data and databases became associated with the journalistic value of accountability[56]; data could reveal problems or inconsistencies in society that became central to reporting[1,  pp.123-124].

Figure 3 :
Figure 3: First part of our visual vocabulary of real-time election results.It is organized by the information communicated with examples of the visual encodings underneath.

Figure 4 :
Figure 4: Second part of our visual vocabulary.Information communicated includes the process and progress of vote counting, as well as predictions made by live election forecasts and qualitative and quantitative uncertainty of predictions.

Figure 5 :
Figure 5: Mapping of participants' design goals to design choices that incorporate components of our visual vocabulary.Challenges faced during the design process are listed under the corresponding design and goal.

Figure 7 :
Figure 7: Context for a race in 2020 by The Texas Tribune [83].

GOAL 1 GOAL 2
Balance certainty and uncertainty in the journey towards fnding out winners.It is important to focus on the current state of the race and 2A avoid stating or implying something too early when trends in results are meaningless (P1, P2, P4).Again, this difers from the goal of progressive visualizations to encourage experts to analyze and draw conclusions earlier from dynamic data[2].This careful balance was refected in the thresholds of percentage of estimated expected vote (EEV) or percentage of precincts reporting at which vote margins between leading candidates were shown.

Figure 11 :
Figure11: How various data sets "fow" into election results rigs, to be displayed on dashboards for the public to view.Before polls close, news outlets have access to information like historical voting data, political polling, and demographic data.After polls close, results providers collect votes reported at the local level and disseminate them to news outlets who buy access.News outlets then publish this information on dashboards alongside the other information.