A Political History Forecast of the 2024 US Congressional Elections

Abstract

Statesman and scholar Alexis de Tocqueville (1876) once noted, “History is a gallery of pictures in which there are few originals and many copies.” In other words, history has a habit of repeating itself, and we can deduce cycles and patterns that likely will recur. Such stability and inertia should bode well for prediction. Nevertheless, when it comes to election forecasting, especially in the United States, most prognostications rely on short-term political fundamentals measuring macroeconomic performance or government or leader popularity. In this contribution, we adopt a structural approach but depart from existing literature by focusing on historical party and governance dynamics in the vein of de Tocqueville to establish whether they offer solid guidance regarding the performance of Democrats in US congressional elections. Our ex-post political history models provide solid predictions of which party will control Congress and the Democrats’ seat tally in each chamber between 1946 and 2022. This creates conditions to assume that political history may help us forecast Campaign 2024. Our study applied this political history model to predict the 2024 congressional elections. It forecast Democrats to lose control of the Senate with a net loss of three seats and estimated an exceptionally close race for House control, with the point estimate for the House suggesting that the Democrats would fall short of winning control.

“The history book on the shelf is always repeating itself.” ~ABBA (1974)

An infamous line from the winning Eurovision hit song Waterloo epitomizes that events and behaviors often recur and human nature and institutions are arguably largely stable and persistent. This observation is hardly new. Scholars have long recognized the idea of historical cycles and patterns and their repeatability—the concept of “the road taken” and how decisions and events can foster enduring constellations (Pierson 2000). Others have championed the “laws of politics” (Cuzan 2015) giving birth to aphorisms like governments losing votes in second-order contests or the advantage of candidate incumbency. Such notions about institutions and citizens augur well for election prediction.

“The history book on the shelf is always repeating itself.” ~ABBA 1974

Election forecasting has three principal strands: polls, markets, and models. Academics conventionally devise models that traditionally rely on theory for projection, incorporating “fundamental” indicators to prognosticate. Consider, for example, the long-standing model of Lewis-Beck and Tien (2004) which, following Tufte (1975), argues that elections are plebiscites of how the president’s party has performed in office, primarily through their stewardship of the economy. Other scholars have followed suit, and such fundamental-inspired models have been used to forecast elections globally (e.g., Dassonneville, Lewis-Beck, and Mongrain 2017; Nadeau and Lewis-Beck 2020). All of these models primarily rely on short-term aspects such as government or leader popularity, longevity in office, and macroeconomic indicators.

Our contribution is in the modeling tradition, but we take a different tack, eschewing any measure of public opinion or macroeconomic context close to the race. Instead, we focus on political structure and history, with the method assuming that election performance is shaped partly by enduring patterns in a polity. We test whether historical-critical junctures, party strength at the state level, and institutional and electoral forces can provide solid guides to the performance of Democrats in US congressional elections. This collection of dynamics forms what we classify as a political history (PH) model. Although some US forecasting endeavors, in part, customarily have incorporated structural dynamics into their predictions (Keilis-Borok and Lichtman 1981), sole reliance on these features to forecast elections generally is a recent phenomenon (Quinlan and Lewis-Beck 2024; Quinlan, Schnaudt, and Lewis-Beck 2021). As this article demonstrates, the US PH model provides credible and competitive estimates of how Democrats fared in congressional elections between 1946 and 2022, with out-of-sample estimates correctly predicting control of the House of Representatives in 81% to 87% of instances and for the Senate in about seven out of 10 instances.

Some may consider that devising a PH model that relies on structural components to foretell the 2024 elections is valiant, given that the current political climate is in flux and these congressional contests are being held concurrently with a presidential election with many novel features. President Biden’s late withdrawal from the race—sparked by a widely recognized poor debate performance against Donald Trump in June 2024 that highlighted concerns over the president’s age—means that he is the first eligible incumbent not to seek reelection since 1968. Democrats coalesced around Kamala Harris as their nominee, hoping that she would become only the second sitting vice president to be elected directly to the presidency since 1836. The Harris campaign was significant in many respects: the first contender in 56 years to gain the nomination without winning a primary, only the second female to headline a presidential ticket, and the first woman of Black and South Asian heritage to contest the office.

“Some may consider that devising a PH model that relies on structural components to foretell the 2024 elections is valiant, given that the current political climate is in flux and these congressional contests are being held concurrently with a presidential election with many novel features.”

On the Republican side, there is the distinctness of Donald Trump, a former president who was twice impeached, with a bombastic character and unconventional political approach, who continues to make mendacious claims about the propriety of the 2020 election. It is the first time a former president contests since 1912, raising the specter of only the second nonconsecutive second-term presidency since Grover Cleveland. Trump’s third tilt at the White House is notable for him, being the first US president convicted of a crime, and during the campaign surviving an attempt on his life. Several commentators have noted his campaign’s controversial rhetoric (Colvin and Barrow 2024; Homans 2024), with some warning that a Trump win could threaten democracy (Brownstein 2023; West 2022). If this were not enough, 2024 marks the first post–COVID-19 election, while abortion looms large after the 2022 Supreme Court decision limiting the practice. All of these factors could cast doubt on whether a PH model that takes little direct heed of this has much to offer.

We recognize that 2024 is a stern test of the PH approach. The 2022 contests proved a challenge for the model, where it overstated Democrat losses and, against its ex-ante projection, the Democrats retained Senate control (Quinlan and Lewis-Beck 2023). The PH model was not alone—other well-traveled prediction models also understated the performance of the Democrats (e.g., Tien and Lewis-Beck 2023). Defects such as 2022 mean that the model “learns” by incorporating these misses into future projections. Nevertheless, despite the unsettled political environment, the PH model historically has performed solidly, including correctly calling many recent elections. ¹ Moreover, we contend that the PH approach can be seen as a historical benchmark testing how these contests conventionally unfold. Suppose the PH model comes close to predicting the results in 2024. In that case, the contest could mark a “return to normalcy”—to invoke former President Warren Harding—for which historical and institutional features remain a solid guide to how congressional elections would play out despite the recent political tumult. If the PH model were to fall significantly short, we contend that it could be viewed as a yardstick for understanding how different the 2024 contest was compared to the races of yesteryear.

Based on a point-estimate forecast of 46 seats formulated in July 2024, four months before Election Day, the PH model forecasted that the Democrats would lose Senate control in 2024, with a net loss of three seats. This indicated that Democrats were apparent but not staggering favorites to lose control, with a 70% chance of being in the minority. The PH model for the House predicted a knife-edge race. The point-estimate projection of 215 seats implied that Democrats would fall short of a majority by three seats. However, with the average error of the forecast between 12 and 14 seats, this was no sure bet, with the PH model indicating a small margin of seats would determine House control.

“The PH model forecasted that the Democrats would lose Senate control in 2024, with a net loss of three seats….The PH model for the House predicted a knife-edge race. The point-estimate projection of 215 seats implied that Democrats would fall short of a majority by three seats.”

THE POLITICAL HISTORY MODEL

This section discusses the theoretical grounds of the PH model and defines the research strategy underlying our empirical analysis.

Theory

The PH model approach rests on structural features assumed to be autoregressive—that is, they have long-standing predictive power. The specific dynamics will differ by polity and by contest. Its bedrock in the United States includes governance, federal–state electoral dynamics, and historical junctures. Applying this intuition to forecasting elections is evolving. It had mixed results when applied to the 2022 US congressional contests, for which it underestimated the Democrats (Quinlan and Lewis-Beck 2023). However, it was not alone in doing so (Lockerbie 2023; Tien and Lewis-Beck 2023) and evidence suggests that 2022 was an atypical midterm contest (Quinlan and Lewis-Beck 2023). Yet, the PH model in 2022 correctly forecasted that the Democrats would lose control of the House, the baseline yardstick for judging the model’s accuracy. In Germany, a PH model forecast of the 2021 federal race (Quinlan, Schnaudt, and Lewis-Beck 2021) did not anticipate that the Social Democratic Party would win the most votes; neither did other established models (e.g., Jérôme, Jérôme-Speziari, and Lewis-Beck 2022). However, it accurately predicted the vote share of the Others Bloc, against poll expectations.

The starting point for the 2024 PH models is how much a party controls the federal government. We contend that when Democrats hold complete control of the federal government (i.e., the presidency and control of the Senate and House simultaneously), they will lose seats. The cost of governing has achieved ubiquity as a “law” of elections, with several studies noting that parties in power lose votes (Cuzan 2015; Norpoth 1991). This is driven by the governing party straying from the median voter and alienating certain groups with governing decisions (Nannestad and Paldam 2002; Wlezien 2016). Between 1946 and 2022, we observed patterns of one party gaining outright federal government control. Although such power is often short-lived, we posit that it exacerbates the cost of the governing penalty because voters have one party to hold responsible.

In Federalist Papers No. 39 and No. 45, James Madison (1788a, 1788b) articulated the symbiotic relationship between the federal government and the states. We posited that such reciprocity translates into the electoral arena and that the state political context influences federal elections. Specifically, for both chambers, we include the number of Democrat governors six months before the contest and the overall Republican Party strength in the state. The mechanism supposed is that with control of the governorship, structural advantages will accrue, such as campaign infrastructure and greater coverage in the media. Furthermore, the costs and benefits of office flow to the party that controls governorships. Conversely, we expect that when Senate races occur in states where Republicans are electorally strong, Democrats will win fewer seats because Republicans will have an inbuilt advantage of support and infrastructure.

We posit that Senate election dynamics impact both the Senate and the House elections. Approximately one third of Senate seats are contested in each cycle. To recognize the influence of the electoral calendar, we codify the number of Democrat seats not up for election, and we expect that the fewer Democrat seats that were, the more seats they would win in the Senate. We suppose that Senate elections can have ripple effects on House contests by the number of incumbent Democrat Senators who are retiring. The power of Senate incumbency has been widely recognized (Matland and Studlar 2004); an incumbent senator’s retirement means a standard bearer lost. Open Senate seats sometimes involve brutal primary battles and attract more significant resources, potentially diverting them from House contests. These races often appeal to sitting House members, thereby opening more House seats. Thus, we suppose that the more retiring Democrat senators in a cycle will be negatively associated with the number of seats that Democrats win in the House.

Recurring events are “standard coin” in model forecasts; however, they do not go far enough. Game changers or shocks to the system often bring about long-standing shifts that endure. New patterns are, so to speak, “locked in.” Such realignments can come from different emerging issues prominent in political competition and structure or a change in the power balance between actors (Key 1959; Petrocik 1987). Including these realignments in a model is critical because these transformative events shift the constant and slope terms; if not considered, this will lead to model misspecification and systematic forecasting error.

We contend that two significant realignments are relevant. With its shifting balance of governmental powers, the Civil Rights legislation passed in August 1965 represented such a critical juncture because it ensured the enfranchisement of minorities. It played a pivotal role in the realignment of the South—where Democrats traditionally had dominated—as Republicans began to gain ground, drawing more support from Southern white conservatives. We suggest that this is associated with the decline of the Democrat Party Senate firewall, underpinned by the South (Pildes 2011). We expect it to negatively impact the performance of Democrats in the Senate.

The 1994 Republican Revolution based on its Contract with America witnessed the party win control of the House for the first time in 40 years and make substantial Senate gains. It now is widely considered a realigning contest (Abramowitz and Saunders 1998; Campbell 2006) as partisanship reasserted its power, social conservative issues became more prominent, and the Republicans consolidated their electoral stranglehold of the South, which coincided with the rise of cable news television. We assume that this will be negatively associated with the Democrats’ performance in the House.

Data and Model Operationalizations

We constructed the dataset (Quinlan 2024) based on data from the Office of the Clerk, US House of Representatives (2022); the Ballotpedia database (2022); and the Brookings Institution. We focused on the Democrats because, historically, they have been the dominant force in Congress since 1946. Of the 39 contests from 1946, Democrats controlled the House 26 times (~67%) and had a Senate majority 24 times (~62%).

We used ordinary least squares (OLS) regressions with two dependent variables: the total number of seats the Democrats win in the Senate and the House. We have seven independent variables across our two models. Three are common to both: Democrat dominance of federal government, total number of Democrat governors, and GOP strength in a state. We codify dominance of federal government with a dichotomy: coded 1 if the Democrats controlled all three branches of the federal government and 0 otherwise. We classify the number of Democrat governors nationwide six months before Election Day. We measure the strength of the GOP in states by counting the number of states with two GOP senators who voted for the Republican presidential candidate in the most recent presidential election and where a Senate contest is taking place in the election year. For the Senate model, we codify the number of Democrat Senate seats not up for election. For the House model, we count the incumbent Senate Democrats who were not contesting the election as of five months before polling day. We measure the two realignments—the Voting Rights Act of 1965 and the 1994 Republican Revolution—with two dummy variables, codifying elections post-1966 for the former and all contests post-1994 as 1. All other contests are coded 0. Online appendixes A–C include summary data, parameters informing the 2024 forecast, and ancillary analyses.

MODELS PERFORMANCE 1946–2022

As shown in table 1, our slope estimates align with theoretical expectations. Federal government dominance negatively correlates with the Democrats’ performance. When they control all branches of the federal government, on average, Democrats will lose seats. The number of Democrat governors positively correlates with the party’s performance. Every governorship that the party holds is associated with an additional 2.9 House seats, on average, and for every five governorships held, the party can expect to gain one Senate seat ceteris paribus. The more Senate seats being contested in states where the Republican Party is strong is associated with fewer Democrat Senate and House seats. The more holdover Senate seats the Democrats have, the more Senate seats the party can anticipate winning. In contrast, the more incumbent Democrats retire from the Senate, the fewer seats they can expect to win in the House. Finally, we observe a strong association between the Democrats’ performance and two critical junctures. On average, Democrats win fewer Senate seats in elections after the Voting Rights Act of 1965. Similarly, ceteris paribus, since the 1994 Republican Revolution, Democrats can be assumed to win 13 fewer House seats in elections post-1994 than pre-1994.

Table 1 Political History Model: OLS Regression Models Exploring the Democrat Number of Seats in US Senate and House Elections, 1946–2022

Notes: Entries are unstandardized coefficients of OLS regression with standard errors in parentheses. *=p<0.05; **=p<0.01; ***=p<0.001. For more information, see online appendix C, tables C5–C9.

In adjudicating the models’ prediction capacity, four criteria are essential: lead time, replication, parsimony, and accuracy (Lewis-Beck 2005). On the first three, both models stand up reasonably. We can estimate the models at least five months in advance. They are easily reproducible, based on official data readily available, and both are relatively efficient with five variables a piece, which has the advantage not only of relative simplicity but reduces the risk of overfitting.

The El Dorado of forecasting is accuracy, and several means exist to explore this. First, regarding the model’s fit to the data: both fit adequately, with an adjusted R² of 0.72 for the Senate and 0.74 for the House model, respectively. Second, we tested how the PH model compared to a naïve model in which seat performance in the previous contests was used as the sole predictor. This shows that the PH model performs stronger (see online appendix C, table C4). Third, we looked at the within-sample mean absolute error (MAE), which treats all errors equally and provides a benchmark for the typical prediction error. For the Senate model, the MAE was 2.7; for the House, it was 12. Fourth, the root mean square error (RMSE) is a stricter test of average error because it gives more weight to more significant errors from the model. Unsurprisingly, it is greater than the within-sample MAE: 3.684 for the Senate and 15.732 for the House, indicating that, on average, the Senate point forecast for Democrats will be within four seats, and the House prognosis, on average, will be within 16 seats of the result. Finally, we deciphered how often the model correctly predicts which party will control each chamber, the ultimate test of the model’s accuracy. Encouragingly, control of the House is accurately forecast in 87% of instances. For the Senate, the PH model correctly predicts control 72% of the time.

Within-sample predictions are known to be optimistic because they rely on data that are available retrospectively. Out-of-sample tests are stricter because they involve prophesying without information about the election in question. The most common out-of-sample diagnostic is the jackknife method. Under this procedure, the Senate model’s MAE is 2.7, reassuringly the same as the within-sample estimate. Soberingly, the largest seat error is 10, three more than the within-sample largest error. However, and more comforting, control of the Senate is correctly forecast in 72% of instances, identical to within-sample estimates. For the House, the jackknife MAE is duplicated to the within-sample estimate (12.0), but the largest seat error rises to 38 (33 for the within-sample estimate). Promisingly, this stress test sees control of the House correctly projected 87% of the time, identical to the within-sample prediction.

The one-step-ahead method is arguably the strictest out-of-sample test but perhaps the most apt because it focuses solely on data that would have been available to the forecaster for an ex-ante prediction. We apply this procedure to projecting elections from 1972 onward (i.e., N=26; ~67% of data). The MAE for the Senate model under this scenario is marginally larger than the within-sample MAE (2.9 versus 2.7). The most significant absolute seat error was 10, identical to the jackknife method and three higher than the within-sample estimate. Promisingly, the models under this specification correctly predict Senate control 77% of the time. For the House, the MAE under this condition is 14, higher than the within-sample MAE (12), while the largest error in seats is 35 (compared to 33 for within-sample) Nevertheless, this specification accurately predicts House control in 81% of instances, which is somewhat lower than the within-sample and jackknife estimates, but still competitive.

2024 EX-ANTE FORECAST AND POST-ELECTION REFLECTIONS

This study demonstrates that the PH model approach for the Senate and the House, when tested empirically in two separate regression equations across 39 elections, shows promise in predicting the performance of Democrats and which party will control each chamber. The coefficient signs of the PH predictors are consistently in the expected direction. The models exhibited solid statistical fit, are easily replicable, and are parsimonious. Moreover, the approach correctly forecasted which party would control the Senate 72%-77% of the time time and the House of Representatives 81% to 87% of the time. These results are remarkable, given that they are based on structural and institutional features, with no mention of issues of the day. In other words, the model sets the bar high by incorporating only these parameters.

Although the PH model approach only tells part of the story and may have less precision than other models, this study shows that it provides a solid idea of where the US electorate will land in most instances. For the 2024 election, we maintain that whether right or wrong, the PH model could act as a barometer of normalcy, allowing us to situate Campaign 2024, which, at face, seems unusual given the volatile and febrile political environment and novel given the presidential contest dynamics, in a broader historical context.

Using the PH models from table 1 to forecast the composition of the 119th Congress in July 2024, four months before the election, it forecasts that Democrats will lose Senate seats, with the point estimate suggesting that they would have 46 seats and a 70% chance of losing control of the chamber. For the House, the prognosis of the PH model approach was a cliffhanger contest. The point-estimate prediction of 215 seats for the Democrats, three short of a majority, was remediated because, with an average error of 12 to 14 seats anticipated, the model predicted a race that was “too close to call” and would be settled by a small margin. In summary, the PH model for 2024 suggested that Democrats were unlikely to achieve a clean sweep of Congress.

At the time the PH forecast was devised in July 2024, it was notable that PH predictions were in sync with other renowned forecasters and poll aggregators in predicting Democrat losses in the Senate and likely minority status (see online appendix D). Polling averages on the generic ballot of the two parties showed a dead heat, with these prognosticators rating the House contest as “too close to call,” with Republicans having a slight edge.

When the votes were tabulated in November 2024, we see that the PH model performed strongly in projecting Campaign 2024. It accurately predicted Senate control for the Republicans and projected the number of Democrat mandates in the Senate to within one seat (i.e., Democrats won 45 seats, the PH model called 46). Remarkably, the PH model point estimate for the number of Democrat seats in the House was spot on, with Democrats winning 215 seats (+2 on 2022) and Republicans retaining control by a five-seat margin. Whereas other model congressional forecasts indicated Republican control, the PH model was the closest in terms of calling the Democrat number of seats (Algara et al. 2024; Lockerbie 2024). This demonstrates that the PH model has solid predictive power and that political history has some promise in predicting elections, even in a volatile political climate.