Lessons Learned: Citizen Forecasting, Candidate Resignations, and the 2024 US Presidential Election

Abstract

Every four years, numerous election-forecasting models attempt to predict the results of the US presidential election. Regardless of the stability of any election system, such as the bipartisan system in the United States, conditions can arise (e.g., candidate resignations) that negatively impact forecasters’ ability to predict electoral outcomes. Citizen forecasting—that is, directly asking respondents who they think will win an election—has a long track record of successfully predicting presidential elections. This study proposes adapting a citizen forecasting measure originally intended for use in multiparty systems to predict the US presidential election in 2024. Using this measure, we created a forecast of the national-level popular vote and vote-share forecasts for seven swing states.

The results of future elections are a significant concern to different groups of stakeholders, including voters, policy makers, and elected representatives. Researchers have developed several types of models to forecast elections. Currently, they generally use three types to forecast US presidential elections (Murr and Lewis-Beck 2020, 91). The first type, econometric models, use aggregate-level data and regression techniques to estimate incumbents’ vote or seat shares (Bélanger and Trotter 2017, 821). These models assume that the electorate rewards or punishes incumbents based on economic performance. In the second type, researchers use prediction markets where traders can buy and sell contracts that correspond to real-life election outcomes (Luckner 2012, 7). The third type of election-forecasting model analyzes individual-level responses to vote-intention and vote-expectation items on survey instruments. Vote-intention items ask respondents which candidate or party they intend to vote for in an upcoming election. Vote-expectation items ask respondents which candidate or party they think will win an upcoming election (Lewis-Beck and Tien 1999, 175–76). Although researchers have used citizen forecasting extensively in US presidential elections, there is no study that has used a likelihood citizen forecasting measure to predict election outcomes within that context.

The results of future elections are a significant concern to different groups of stakeholders, including voters, policy makers, and elected representatives.

This article makes three contributions to the literature. First, it represents the first use of a likelihood citizen forecasting measure in US presidential elections, a generally bipartisan system. Murr (2011) used this type of measure to aggregate citizen forecasts in the British multiparty system. Although most US presidential elections pit candidates from two major parties against one another, sometimes a strong third-party candidate appears in bipartisan systems. Before ending his presidential campaign on August 23, 2024, Robert F. Kennedy, Jr., appeared on the ballot in at least 23 states and sought ballot access in 25 others (Slisco 2024). Although Kennedy ended his campaign and endorsed Donald Trump, his campaign’s rise demonstrates the need to use citizen forecasting measures that can capture levels of support for more than two candidates.

Collecting citizen forecasts using a likelihood measure has several advantages over a categorical measure. For example, the level of (un)certainty that citizens have in their forecasts at the national and state levels can be observed directly. In addition, vote shares can be estimated more easily. Previous research generally used historical data to predict a party’s vote share in US presidential elections (Murr 2015, 922). Asking citizens to rate a candidate’s chances using a likelihood measure represents an improvement over this process. It allows us to estimate a vote-share forecast for each electoral candidate without resorting to historical data.

The second contribution of this article includes a forecast of the share of the national popular vote that citizens expected Joe Biden and Donald Trump to receive. We caveated this forecast and our swing-state forecast by emphasizing that both apply to the state of the race only before July 2024. On July 21, 2024, Joe Biden announced that he would end his campaign for reelection and endorsed Vice President Kamala Harris to replace him on the Democratic ticket (Baker and McNamee 2024). Biden ending his reelection campaign, Harris entering the race, and RFK Jr. ending his campaign a month later changed the race to the extent that forecasts asking about these candidates’ chances of victory should not be assessed retrospectively. However, this article shows how a likelihood measure can be applied successfully at the national and state levels in future US presidential elections and the lessons that future election forecasters can learn from this highly volatile election.

[T]his article shows how a likelihood measure can be applied successfully at the national and state levels in future US presidential elections and the lessons that future election forecasters can learn from this highly volatile election.

The article’s third contribution includes vote-share forecasts for seven states that we identified as swing states: Florida, Georgia, Michigan, North Carolina, Ohio, Pennsylvania, and Virginia. We chose to focus on these states as swing states for three reasons. First, the Electoral College renders most states uncompetitive in US presidential elections. Previous citizen-forecasting studies recommend focusing on competitive versus noncompetitive electoral districts within a country because they represent a more stringent test of citizen forecasting (Thompson-Collart, Brie, and Dufresne 2024, 8). Second, the states in our sample demonstrated competitiveness in recent elections. The winning candidate carried five of seven states in our sample by five points or less in the 2016 and 2020 elections (Wolf and Rigdon 2024). Third, polling aggregators identified four of the states in our sample—Georgia, Michigan, North Carolina, and Pennsylvania—as particularly consequential states that could “tip” the election; that is, push a candidate over the 270 electoral votes needed to win the presidency (FiveThirtyEight 2024). As of September 2024, polling aggregator FiveThirtyEight had given Pennsylvania a 17.5% probability of tipping the election, North Carolina a 12% probability, Georgia an 11.5% probability, and Michigan an 11.4% probability. For these reasons, we chose to limit our sample to these four states.

CITIZEN-FORECASTING METHODOLOGY

Citizen forecasting aggregates individual predictions to provide a forecast of which candidate or party will win an upcoming election. This technique relies on Condorcet’s jury theorem (Murr 2011, 771; Murr 2015, 917; Temporão et al. 2019, 3). Under Condorcet’s original formulation, each citizen had to have a greater than 50% probability of making a correct prediction, their votes had to be uncorrelated, and the predicted outcome had to be binary (Murr 2011, 772). Subsequent research relaxed these assumptions, allowing for both the competence levels of citizens and the correlation of votes to vary as well as for predictions with multiple outcomes (Murr 2015, 918). Citizens’ competence levels represent the key to a successful citizen forecast. If a group of citizens has a greater than even chance of predicting the correct outcome, the probability of making a correct election forecast approaches 100% as citizens are added to the group (Murr 2015, 917). Whether unrepresentative samples of citizens can predict election results remains an open research question. Previous studies show that unrepresentative but highly competent samples of citizens within US states usually can predict the presidential election in their state (Murr 2015, 919). However, other studies find that unrepresentative samples do not outperform a representative sample (Ganser and Riordan 2015, 124).

Citizen-forecasting studies use two methods for aggregating citizens’ predictions about upcoming elections. The first, plurality voting, tallies the percentage of respondents who believe a specific party will win an election (Murr 2011, 774). The party with the highest proportion of individuals expecting that party to win is forecast as the election winner. Although plurality voting is relatively straightforward, this method discards a considerable amount of information, such as which party came in second or third and the level of certainty that each respondent had in their forecast. Murr (2011) proposed range voting as an alternative method to plurality voting in multiparty elections. Range voting sums and normalizes expectation scores from a likelihood measure (Murr 2011, 774). The range-voting method provides two advantages over plurality voting: (1) information on which parties will come in second and third; and (2) analysts can observe individual respondent’s level of certainty in their election prediction.

This study aggregated citizens’ forecasts about the 2024 US presidential election using a range-voting procedure (Thompson-Collart et al. 2025). We asked citizens which candidate they thought would win the election at the national level and in their state. The national-level question asked respondents, “How likely do you think it is that Donald Trump, Joe Biden, or RFK Jr. will be elected president in November? Please assign a probability to each candidate.” The response options included Donald Trump, Joe Biden, and Robert F. Kennedy, Jr. The state-level question asked respondents, “How likely do you think it is that Donald Trump, Joe Biden, or RFK Jr. will win your state in the presidential election? Please assign a probability to each candidate.” These response options also included Donald Trump, Joe Biden, and Robert F. Kennedy, Jr. Respondents could assign a probability of victory to each candidate ranging from 0% to 100%. These measures allow analysts to predict not only the election winner but also the closeness of an election (Temporão et al. 2019, 4). These items produced two pieces of information needed to predict the outcome of a presidential election. First, this measure provided an average likelihood figure for each candidate. We predicted that the candidate with the highest average likelihood would win the election. Second, we could use data collected using this measure to estimate vote shares for each candidate in the election. We obtained vote shares for all candidates in two ways: (1) we divided the average likelihood for each candidate by the sum of all likelihoods for all of the candidates; and (2) we repeated this process using the median likelihood. We estimated vote shares using both mean and median likelihoods because the likelihood distributions for all three candidates were not symmetrical. As a result, using only the mean might have led us to overestimate support for a minor candidate while underestimating support for the major candidates (Penn State Eberley College of Science n.d.). The use of median likelihoods addressed this concern.

A practical example illustrates the vote-share estimation process. To normalize the likelihood scores for each candidate, we first took the average likelihood score for each. For example, in the state of Michigan, the average likelihood score for Biden was 3.6, for Trump was 5.5, and for Kennedy was 1.8. To estimate the vote share for Biden, we divided his average likelihood of 3.6 by 10.9, or the sum of the likelihood scores for all of the candidates (i.e., 3.6+5.5+1.8=10.9). This procedure resulted in a vote-share forecast of 33% for Biden in Michigan. We estimated vote shares in this way at the national level as well as within each state. We repeated the process also using the median likelihood.

This technique for estimating vote shares from citizen forecasts represents an improvement compared to previous methods. To obtain vote shares from citizen forecasts, researchers typically regress the percentage of citizens who believe that a party will win on the vote share obtained by that party in an election (Lewis-Beck and Tien 1999, 181; Murr 2011, 777). Although this method provides accurate results, it also faces three pragmatic limitations. First, we require historical data to estimate a regression equation. Although historical data exist for the two major parties, we do not have historical data for new, third-party candidates that may emerge. Second, estimating vote shares in this way is less time-consuming than collecting historical data and estimating a regression. Third, this method for estimating vote shares appears to provide accurate predictions. The technique demonstrated a mean absolute error of 2 percentage points for the first round of the 2017 French presidential election and 1.5 points for the second round (Dufresne et al. 2022, 732). The method provides an error rate that is similar to using the final Gallup poll in an election campaign (Lewis-Beck and Tien 1999, 183).

CITIZEN FORECASTING IN US PRESIDENTIAL ELECTIONS

Both vote-intention polling and citizen forecasting vary in their accuracy for predicting election results. Lewis-Beck and Tien (1999) first compared the accuracy of citizen forecasting to vote-intention polling. They found that both vote-intention polling and citizen forecasting correctly predicted the winner in nine of 11 elections between 1956 and 1996. Moreover, when comparing vote-share estimates obtained from citizen forecasts to those from vote-intention polls, they found a similar mean average prediction error across the two forecasting methods. Graefe (2014) extended this line of research to other election forecasting methods. He compared the accuracy of citizen forecasting to vote-intention polling, prediction markets, and quantitative models for US presidential elections. He found that citizen forecasting predicted vote shares similarly to quantitative models and better than vote-intention polling and prediction markets. From these results, we conclude that citizen forecasting represents an accurate election-forecasting method that complements vote-intention polling.

Lewis-Beck and Skalaban (1989) first demonstrated that US citizens could accurately predict presidential elections. Across eight presidential election years between 1956 and 1988, 69% of citizens correctly predicted the winning party. Lewis-Beck and Tien (1999) subsequently found that contextual factors explained why some citizens could better predict elections than others, including the respondent’s level of education, date of the interview, and whether the respondent expected a close election. However, partisan affiliation also exerts a considerable effect on citizen forecasts (Lewis-Beck and Skalaban 1989, 419; Lewis-Beck and Tien 1999, 179; Mongrain 2021a, 11). Furthermore, Dolan and Holbrook (2001) found that political knowledge improves citizen forecasts and attenuates the effects of wishful thinking at the state level, but they uncovered no such attenuation at the national level. Taken together, citizen forecasting provides researchers with a valuable method for accurately predicting US presidential elections.

DATA AND METHODOLOGY

We elicited citizen forecasts from a probabilistically selected sample of adults residing in the United States. Léger, a private polling firm, collected the survey data between May 13 and July 2, 2024. The sample contained 1,607 respondents at the national level. Figure A1 in online appendix A lists the number of respondents answering the survey per day during the data-collection period. The largest number of respondents in a single state (i.e., 164) were from California. By contrast, the state with the smallest number of respondents was Vermont, with only three. At the national level, the sample mostly reflected the US population on key demographic characteristics. For example, 38% of the sample had a college degree and 44% of respondents were female. At the state level, only 16 states had at least 30 respondents, seven of which we identified as swing states (i.e., Florida, Georgia, Michigan, North Carolina, Ohio, Pennsylvania, and Virginia). Florida and Virginia had the most and the fewest respondents (i.e., 105 and 41, respectively). Therefore, we expected our swing-state forecasts to reflect an acceptable level of accuracy.

WHO WILL WIN THE 2024 US PRESIDENTIAL ELECTION?

This study examined whether citizen forecasts could predict the winner of the national popular vote and the winner in seven key swing states. This section presents the results from our citizen-forecasting model of the 2024 US presidential election. We also compared those results to a publicly available citizen forecast conducted in June 2024. Our citizen-forecasting model produced two forecasts. First, a national-level forecast described the percentage of the national popular vote that citizens expected Donald Trump and Joe Biden to win at the time of data collection. Second, we presented a vote-share forecast that identified the party expected to win in seven competitive swing states. We first examined the share of the national popular-vote share that we expected each candidate to receive. Figure 1 shows that citizens expected Trump and Biden would receive 55% and 45% of the national popular vote, respectively, during the data-collection period. When we used the median, Trump would receive 53% and Biden 47%. Our citizen forecast predicted results similar to the April 2024 Verasight Midwest Political Science Association Omnibus Study that forecasted Trump to win 50% of the national popular vote and Biden to receive 38% (Leiter and Lewis-Beck 2024).

Figure 1 Vote-Share Expectations at the National Level

In the state-level forecast shown in figure 2, our citizen forecasters clearly expected a Trump victory in all seven swing states under study. Regardless of whether we calculated state-level vote shares using the average or the median likelihood, respondents across all seven swing states also expected Trump to garner a sizeable share of the vote in their state. When we calculated vote shares using the average likelihood, citizens’ forecasts of Trump’s vote shares ranged from a minimum of 53% in Virginia to a maximum of 67% in Florida. When we calculated vote shares using the median likelihood, citizens’ forecasts of Trump’s vote shares ranged from a minimum of 50% in Pennsylvania—tied with Biden—to a maximum of 80% in North Carolina.

Figure 2 Vote-Share Expectations in Selected Swing States

This study provided several lessons for conducting future citizen forecasts. First, future survey questions should ask citizens about parties’ chances of winning an election, not candidates’ chances. Although replacing candidates in the middle of a presidential campaign remains rare in US presidential elections, the 2024 election demonstrated that this can occur. Therefore, future researchers should ensure that their expectation measures ask about the likelihood that a party will win the presidency rather than a candidate. Second, future researchers should ensure that every state has at least 30 respondents. Previous citizen-forecasting studies aimed to include this number of respondents because it reduces the uncertainty in state-level forecasts (Murr and Lewis-Beck 2020, 92). Our study had at least 30 respondents in swing states but did not achieve this sample size in every state. Third, future researchers should delegate national- and state-level forecasts to the most competent citizens in the sample. To accomplish this, they should include items on future surveys that measure respondents’ levels of political knowledge. Previous research finds that delegating the citizen-forecasting task to those with higher levels of political knowledge increases forecasting competence (Mongrain 2021b, 721).

This project provided several lessons for conducting future citizen forecasts.