As We Like It: Did the UK’s 2016 EU Referendum Reveal the “Will of the People?”

Rhetoric surrounding the United Kingdom’s 2016 referendum on continued European Union (EU) membership frequently has invoked the “will of the people.” Addressing the House of Commons in March 2019, then–Prime Minster Theresa May stated that “my sense of responsibility and duty has meant that I have kept working to ensure that we deliver on the result and the will of the people” (March 27, 2019). ¹ May’s successor, Boris Johnson, appealed to the same notion when suggesting in the Daily Telegraph (September 15, 2019) that opposition parties were “united in wanting to cancel the referendum result…and overturn the will of the people.” On the other side of the debate, Caroline Lucas (currently the sole Member of Parliament for the UK’s Green Party) stated that “[e]very recent opinion poll shows that the will of the people has changed since [the referendum]” (December 4, 2018).

These statements raise a key question: What precisely is “the will of the people” in electoral contexts? This article contends that voters’ true preferences—and, in the aggregate, the “will of the people”—comprise the preferences that would have been reported had people been more fully informed on relevant matters (Ahlstrom-Vij 2022a; Harsanyi 1997). As such, given widespread public ignorance on politically relevant matters (Achen and Bartels 2016; Delli Carpini and Keeter 1996), there likely are gaps between the choices made by the electorate at the polls and what is in their collective “will.” Such gaps have been investigated in various electoral contexts under the banner of “information effects,” including in the United States (Ahlstrom-Vij 2021; Althaus 2003; Bartels 1996; Delli Carpini and Keeter 1996); Canada (Blais et al. 2008); Denmark (Hansen 2009); Sweden (Oscarsson 2007); and European Parliamentary elections (Bhatti 2010). Crucially, reading across the results of this body of work does not suggest that more fully informed voters consistently prefer any particular ideological or party-political position.

Building on this research, we examined whether the results of the UK’s 2016 referendum on leaving the EU—51.9% for “Leave” and 48.1% for “Remain”—were potentially sensitive to differences in voters’ levels of political information. We did so by developing and applying two counterfactual models of how information influences vote choice to the 2017 British Election Study (BES) face-to-face survey wave (N=2,067)—that is, the wave following the referendum. In a more fully informed electorate, and across different sets of theoretical assumptions about how what people know is related to how they vote, we find that support for leaving the EU likely would have decreased by up to 10.6 percentage points, producing a 58.7%–41.3% split in favor of Remain. We view the modeled estimates as good reasons for taking the extent to which voters are informed more seriously in interpreting electoral outcomes. Moreover, given widespread misinformation about consequential political issues, we suggest that they offer cautionary lessons for politicians and policy makers who confidently interpret electoral results as reflecting their constituents’ collective and definitive will. ²

WHAT IS THE “WILL OF THE PEOPLE”?

The “will of the people” consists of the preferences that would have been reported had we been informed. ³ We consider the preferences of an individual first. Although the most straightforward way of finding out what people want is to ask them, interpreting their answers is difficult when they are mistaken about the nature or implications of the options in question. To illustrate, for example, consider two environmental policies. Policy A is disastrous for the environment and Policy B is good for the environment—yet, in all other ways, they are identical in line with their outcomes. As it happens, we have it the wrong way around: although we care deeply about the environment, we choose Policy A. However, this does not necessarily mean that Policy A is preferred over Policy B. Indeed, imagine that we realize our mistake after choosing Policy A. At that point, we likely would deny that we ever wanted Policy A: the only reason we chose it was because we thought it had the properties actually possessed by Policy B.

This is what underlies Goodin’s (1995, 137) observation that, under circumstances of ignorance, “we can serve a person’s ‘real’ preferences only by censoring the misleading indication of his preferences that is revealed in his choices.” Harsanyi (1997, 133) made a similar point when distinguishing “those choices of a person that really express his true preferences…from those choices of his that fail to do so because they are based on incorrect information.” Generalizing this point to the collective, Delli Carpini and Keeter (1996, 5) suggested that “the real interest of an individual—and by extension of a group and of the polity as a whole—are reflected in the choices one would make if he or she were fully informed about the consequences.” Similarly, we hold that the “will of the people” comprises the preferences its constituent individuals would hold if relevant yet false beliefs were corrected. Assuming a majoritarian electoral system, the will of a population thereby is indicated by what the majority prefers (if anything) under conditions of having full information.

When examining that will, we are squarely in the political context—a context in which mistaken beliefs are in fact rife (Achen and Bartels 2016; Friedman 1998). ⁴ To be sure, if the public reports wanting something, there usually is a strong case for government to act accordingly. However, in settings in which what voters say they want might be due to their not having had the time or opportunity to consider all relevant information—arguably, a central feature of the division of political labor involved in any representative democracy—directly reading the “will of the people” from electoral results becomes difficult. It would be convenient, then, if there were a way to identify diagnostically whether there were such gaps between what people say they want and what they would have said if they had possessed fuller information. The next section argues that work on “information effects” offers just such a way.

MEASURING THE SENSITIVITY OF ELECTORAL OUTCOMES TO INFORMATION

We argue that a long tradition of political scientists using statistical models to examine “information effects”—that is, gaps between actual outcomes or distributions of attitudes and the counterfactual outcomes or attitudes that we likely would have seen under full information—provides a way of diagnosing the presence of such gaps.

The Intuition Behind Counterfactually Modeling Information Effects

In electoral contexts, the procedure follows four steps. First, we construct a knowledge scale and place respondents on that scale. Such scales (see, e.g., Delli Carpini and Keeter 1996) typically are built from questions that we might see on a civics exam—the roles that named people hold or the platforms of political parties—and are meant to indicate generalist political knowledge. ⁵ Second, using data on reported votes, we fit a model that estimates the probability of someone reporting having voted a particular way as a function of that person’s demographics and level of knowledge. Third, we increase each person’s placement on the knowledge scale to the maximum level—that is, the level at which they answer all of the questions correctly—while leaving everything else as is, and then use the fitted model to “predict” how each respondent would vote. This simulates how that person would have voted had they been “fully informed.” (If we want to relax this assumption of being fully informed, we can move individuals’ knowledge scores to a different threshold. Reporting and justifying this choice is good practice.) Fourth, by noting the difference between the actual electoral outcome and the model’s estimated outcome, we obtain a measure of the aggregate information effect. A substantial difference indicates high sensitivity to voters’ level of information and a greater risk that the result might be an artifact of mistaken beliefs to a non-trivial degree.

What counts as “substantial” in the context of elections? Bartels (1996, 220) found information effects in the range of 2 to 5 percentage points in US presidential elections. Blais and colleagues (2008) found an average information effect of 2.3 percentage points across parties in six Canadian elections. Oscarsson (2007) found an average net gain of 2.7 percentage points for right parties in six Swedish elections. Bhatti (2010) modeled three European Parliament elections (i.e., in Denmark, Finland, and Sweden) with an average information effect of 3.5 percentage points across parties and elections. Collectively, these studies suggest that finding information effects of more than approximately 3 percentage points would be substantial.

Modeling a More Informed EU Referendum

In contrast to previous scholars’ focus on regularly scheduled elections, we considered whether, to what extent, and in which direction the UK’s EU referendum result may have displayed an information effect. This was motivated not only by observing politicians’ invocations of the “will of the people,” as described previously, but also by referenda representing a relatively rare event in British politics—and one in which information conveyed through campaign efforts has been shown experimentally to have significant effects on attitudes (Morisi 2018).

To this end, we used the 2017 BES face-to-face survey wave (N=2,194), which contains demographic information, responses to six questions measuring political knowledge, and respondents’ reported vote in the referendum (Fieldhouse et al. 2018). Survey fieldwork occurred between June 26 and October 1, 2017. Omitting 127 observations that lacked survey weights left 2,067 observations, of which 2.6% had missing values that we imputed with multiple imputation using aregImpute in R ’s Hmisc package (Harrell et al. 2019; R Core Team 2017). ⁶ By way of creating a knowledge scale, we then fitted a dichotomous item response theory (IRT) model using mirt (Chalmers 2012) to estimate the underlying knowledge of respondents based on the six knowledge items (Ahlstrom-Vij and Allen 2023). ⁷

Because we were looking to model a counterfactual—that is, how the electorate likely would have voted had we somehow been able to intervene on their level of knowledge by increasing its value—we used an explicitly causal model. ⁸ In terms of likely confounders (i.e., variables that likely have an effect on a person’s voting behavior and their level of knowledge), we controlled for gender (Plutzer 2020; vanHeerde-Hudson 2020), level of education (Hebbelstrup and Rasmussen 2016), income (Plutzer 2020; Vowles 2020), and age (Plutzer 2020). To reduce noise in the models, we also controlled for variables that likely influence voting behavior but not necessarily a person’s level of knowledge: ethnicity (Dawson 1994), religion and social class (Evans and Northmore-Ball 2020), and marital status (Denver 2008).

We viewed partisanship as a mediator because it likely is affected by political knowledge—specifically, knowledge of party and candidate positioning (Brader and Tucker 2018). Controlling for a mediator—that is, a causal node located on a direct or indirect pathway between political knowledge and political preference—would misestimate the relevant causal effect. ⁹ Nevertheless, in the interest of robustness, we fitted two multinomial logistic models: (1) a demographic model, containing all variables mentioned previously except partisanship; and (2) a partisanship model, containing all variables including partisanship. In both cases, an individual’s reported vote (i.e., “Leave,” “Remain,” or “Did Not Vote”) was the dependent variable. The models were fitted with R ’s nnet package (Venables and Ripley 2002) using “doubly robust” estimation of causal effects (Morgan and Winship 2015). For purposes of this estimation, we identified the knowledge score given by the IRT model as corresponding to having answered correctly all knowledge items. We then recoded the knowledge variable as a binary variable, with all observations meeting that threshold coded as 1 and everyone else as 0. That threshold occurred at a knowledge score of 1.036 and was met by 26% of the sample. IRT scores do not have any intrinsic meaning; however, because they can be interpreted as Z-scores, a knowledge score of 1.036 corresponds to someone at about one standard deviation above the estimated mean level of knowledge. We then estimated propensity scores using logistic regression via the informationeffects package (Ahlstrom-Vij 2022b). In line with the assumptions about the causal determinants of political knowledge discussed previously, the model estimated the association between being fully informed (as given by the binary knowledge variable) and gender, age, income, and level of education. Diagnostic plots from the cobalt package (Greifer 2022) confirmed improved balance between the two groups (i.e., those who were and were not fully informed) across these covariates. ¹⁰ We then used these propensity scores as weights in fitting the models. ¹¹

Finally, to estimate the distribution of support we likely would see for “Leave” and “Remain” in this counterfactually informed electorate, we compared the actual outcome with the distribution estimated by the two models. We did this after setting the (binary) knowledge variable for each respondent to 1 (which represents being fully informed, in the sense of answering correctly all of the knowledge items), with each observation weighted using the survey weights included with the dataset, to approximate representativeness. ¹² Figure 1 reports the actual referendum outcome alongside the estimated outcomes on the two models.

Figure 1 Actual and Estimated Referendum Results Under Full Information Using Two Different Models, Including Survey Weights

On the partisanship model, the proportion in support for “Leave” decreased by 9.7 percentage points, from 51.9% to 42.2%. Using the demographic model, which assumes partisanship acting as a mediator, the “Leave” vote decreased by 10.6 percentage points to 41.3%. Reading across the models, not only does the aggregate result switch from “Leave” to “Remain,” the information effect also exceeds the standard set for a substantial information effect (i.e., 3 percentage points) by a fairly wide margin. This suggests that the results likely were sensitive to voters’ levels of information.

DISCUSSION

What does this exercise of simulating electoral results meaningfully contribute to the understanding of politics and political behavior? When supported by a robust set of causal models, this approach provides an empirical and replicable measure of the degree to which an outcome is sensitive to shifts in voters’ political knowledge. This should be of interest to political scientists who are concerned with the role of information in voters’ choices. Moreover, from a methodological point of view and as with causal modeling generally, this approach also forces researchers to be explicit and transparent about the mechanisms we think are at work when people vote, behave, and make sense of political issues. This invites a healthy skepticism of causal assumptions and enables us to pinpoint which assumptions matter. In our case, for example, conducting robustness checks involving partisanship—clearly, an important factor for political choices (De Vries, Hobolt, and Tilley 2018)—enables us to gauge the sensitivity of information effects to different model specifications.

The outcomes of such models also should be of interest to policy makers who must translate electoral outcomes into political policy, especially because such modeling does not presume a priori that additional information benefits any particular side. This is a helpful and welcome feature in political contexts characterized by motivated reasoning and affective polarization. In the particular case of the 2016 UK referendum on EU membership, however, it might be objected that there actually was an abundance of information in the lead-up to the campaign: the problem was that this information, for whatever reason, did not influence people. Perhaps voters were sensitive to the partisan and ideological positions of either the messengers of factual information or the political elites who endorsed or dismissed those sources of evidence; studies about cues suggest that this happens in political domains (Aarøe 2012; Darmofal 2005). This explanation may be particularly relevant in the UK referendum context in which divisions between “Leave” and “Remain” voters seemingly have hardened into identities in their own right (Hobolt, Leeper, and Tilley 2021).

Yet, whether this is true does not detract from what we attempt to do in this article. For example, it might be true that had we known X, we would not have believed Y or done Z. However, that counterfactual holding also is compatible with motivated reasoning preventing us from believing (and therefore knowing) X in the first place, and that someone telling us that X therefore will not have us relinquish our belief in Y or stop doing Z. For that reason, the point of the type of causal modeling conducted in this article is not to offer a recipe for attitude change. Rather, it is to isolate the effect of knowledge on political choices and attitudes in order to provide a diagnostic tool to stress-test claims about what is in the “will of the people.” In particular, having empirical evidence of substantial information effects should give policy makers pause in directly and conclusively inferring the will of the people from any result, including in contexts of relatively rare electoral events such as referenda in the United Kingdom. Specifically, to the extent that we believe that electoral outcomes must be sensitive to the will of the people to be legitimate—at least within reasonable liberal constraints on majority rule—substantial information effects offer valuable datapoints on whether acting on the relevant outcomes would be politically legitimate.

…evidence of substantial information effects should give policy makers pause in directly and conclusively inferring the will of the people from any result, including in contexts of relatively rare electoral events such as referenda in the United Kingdom.

Would applying such a diagnostic tool mean that policy makers pay attention only to the informed segment of the electorate and, as such, be elitist? On the contrary, it would be more egalitarian than simply catering to the stated preferences of the public, whether via traditional opinion polls, elections, or referenda. If our real preferences are those that we would have had if we had been fully informed (as we contended at the outset), then we only will tend to successfully express our real preferences insofar as we are politically informed. If we are not informed, by contrast, we are less likely to know what those preferences are. Given the correlation between privilege and political information, paying attention to stated preferences therefore will consistently—even if not deliberately—prioritize the preferences of the privileged and ignore those of the less privileged. By contrast, policy makers paying attention to information effects likely will be in a better position to listen to everyone’s true preferences—at least in the sense of being able to measure whether electoral outcomes likely mischaracterize the less privileged part of the population and, consequently, risk ignoring the injustices and concerns that disproportionally affect them.