Hostname: page-component-586b7cd67f-gb8f7 Total loading time: 0 Render date: 2024-11-23T19:32:08.520Z Has data issue: false hasContentIssue false
  • English
  • Français

Why Are American Presidential Election Campaign Polls So Variable When Votes Are So Predictable?

Published online by Cambridge University Press:  27 January 2009

Andrew Gelman  and
Gary King
Get access Rights & Permissions [Opens in a new window]

Extract

As most political scientists know, the outcome of the American presidential election can be predicted within a few percentage points (in the popular vote), based on information available months before the election. Thus, the general campaign for president seems irrelevant to the outcome (except in very close elections), despite all the media coverage of campaign strategy. However, it is also well known that the pre-election opinion polls can vary wildly over the campaign, and this variation is generally attributed to events in the campaign. How can campaign events affect people's opinions on whom they plan to vote for, and yet not affect the outcome of the election? For that matter, why do voters consistently increase their support for a candidate during his nominating convention, even though the conventions are almost entirely predictable events whose effects can be rationally forecast?

In this exploratory study, we consider several intuitively appealing, but ultimately wrong, resolutions to this puzzle and discuss our current understanding of what causes opinion polls to fluctuate while reaching a predictable outcome. Our evidence is based on graphical presentation and analysis of over 67,000 individual-level responses from forty-nine commercial polls during the 1988 campaign and many other aggregate poll results from the 1952–92 campaigns.

We show that responses to pollsters during the campaign are not generally informed or even, in a sense we describe, ‘rational’. In contrast, voters decide, based on their enlightened preferences, as formed by the information they have learned during the campaign, as well as basic political cues such as ideology and party identification, which candidate to support eventually. We cannot prove this conclusion, but we do show that it is consistent with the aggregate forecasts and individual-level opinion poll responses. Based on the enlightened preferences hypothesis, we conclude that the news media have an important effect on the outcome of presidential elections – not through misleading advertisements, sound bites, or spin doctors, but rather by conveying candidates' positions on important issues.

Type
Articles
Information
British Journal of Political Science , Volume 23 , Issue 4 , October 1993 , pp. 409 - 451
Copyright
Copyright © Cambridge University Press 1993

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

1 Achen, Christopher, ‘Mass Political Attitudes and the Survey Response’, American Political Science Review, 69 (1975), 1218–23CrossRefGoogle Scholar; Feldman, Stanley, ‘What Do Survey Questions Really Measure?’ Political Methodologist, 4 (1991), 8–12Google Scholar; Piazza, T., Sniderman, Paul and Tetlock, Phillip, ‘Analysis of the Dynamics of Political Reasoning: A General-Purpose Computer Assisted Methodology’, Political Analysis I (1989), 99–120.CrossRefGoogle Scholar

2 Dahl, Robert, Democracy and Its Critics (New Haven, Conn.: Yale University Press, 1989), p. 181.Google Scholar

3 Rosenstone, Steven J., Forecasting Presidential Elections (New Haven, Conn.: Yale University Press, 1983).CrossRefGoogle Scholar

4 Lewis-Beck, Michael S., ‘Election Forecasts in 1984: How Accurate Were They?’ PS, 18 (1985), 53–62CrossRefGoogle Scholar, and Lewis-Beck, Michael S. and Rice, Tom W., Forecasting Elections (Washington, DC: Congressional Quarterly Press, 1992)Google Scholar review many other statistical forecasting models. Lichtman, Allan J. and DeCell, Ken, The Thirteen Keys to the Presidency (Lanham, NY: Madison Books, 1990)Google Scholar and Forsythe, Robert, Nelson, Forrest, Neumann, George and Wright, Jack, ‘The Iowa Presidential Stock Market: A Field Experiment’, Research in Experimental Economics, 4 (1991), 1–43Google Scholar, present some non-statistical approaches to forecasting presidential elections. Social scientists have been explaining and forecasting individual votes and aggregate election outcomes almost since the start of the discipline. The first quantitative article published in a political science journal (about political science) was on voting behaviour (Ogburn, William and Goltra, Inez, ‘How Women Vote: A Study of an Election in Portland, Oregon’, Political Science Quarterly, 34 (1919), 413–33CrossRefGoogle Scholar), and voting, particularly in presidential elections, has almost always remained a lively area of research.

5 Rosenstone, Steven J., ‘Predicting Elections’ (Ann Arbor: University of Michigan, unpublished manuscript, 1990).Google Scholar In Forecasting Presidential Elections, p. 122Google Scholar, Rosenstone also reports sending letters on 14 October 1980 to twenty scholars with his forecasts of the November 1980 election.

6 See, for example, Budge, Ian and Farlie, Dennis, Voting and Party Competition (New York: Wiley, 1977)Google Scholar; Tufte, Edward R., Political Control of the Economy (Princeton, NJ: Princeton University Press, 1978)Google Scholar; Fair, Ray C., ‘The Effect of Economic Events on Votes for President’, Review of Economics and Statistics, 60 (1978), 159–73CrossRefGoogle Scholar; and ‘The Effect of Economic Events on Votes for President: 1980 Update’, Review of Economics and Statistics, 64 (1982), 322–5Google Scholar; and ‘The Effect of Economic Events on Votes for President: 1984 Update’, Political Behavior, 10 (1988), 168–79Google Scholar; Campbell, James E., ‘Forecasting the Presidential Vote in the States’, American Journal of Political Science, 36 (1992), 386–407CrossRefGoogle Scholar; Lewis-Beck, and Rice, , Forecasting Elections.Google Scholar

7 See Lewis-Beck, and Rice, , Forecasting Elections, chap. 1.Google Scholar

8 Details appear in Gelman, Andrew and King, Gary, ‘Forecasting the 1992 US Presidential Election’, manuscript, in progress.Google Scholar

9 Campbell, , ‘Forecasting the Presidential Vote in the States’.Google Scholar

10 The 1992 presidential election campaign drew an unusually large number of political scientists to make forecasts. The quality of these forecasts were quite uneven, as was their success. Models which ignored features of voter decision making that the political science literature has demonstrated to be important – especially candidate ideology and presidential approval – seemed to do especially poorly. (For summaries, see Beck, Nathaniel, ‘Forecasting the 1992 Presidential Election: The Message is in the Conference Interval’, Public Perspective, 3, No. 6 (1992), 32–3Google Scholar; Political Methodologist, 04 1993Google Scholar; Greene, Jay P., ‘Forewarned Before Forecast: Presidential Election Forecasting Models and the 1992 Election’, PS, 26 (1993), 17–21.)Google Scholar It is easy to be too hard on all the forecasters of 1992, however, since this was a year without precedent: no president since Truman in 1948 has ever run for re-election with such low public approval. Fortunately, extreme observations such as occurred in 1992 should help substantially in making future forecasts. Of course, one should be especially wary of forecasting ‘models’ that are not precise enough to be replicatale. For example, one co-authored method was applied by each co-author in different television interviews: according to one, the method picked Clinton as the likely winner; according to the other, it picked Bush.

11 Rosenstone, , Forecasting Presidential Elections.Google Scholar

12 Gelman, and King, , ‘Forecasting the 1992 US Presidential Election’.Google Scholar

13 We presented these forecasts several weeks before the election in public lectures at Harvard University and the University of California, Berkeley, as well as in communications with several others.

14 Our extensive analyses, some of which are reported below, indicate that one can safely merge the data from the different polling organizations in order to study trends in candidate support but not the percentage undecided or not responding.

15 We chose the 1988 election because it was the most recent when we began our analyses. We completed all but the final draft of this article before the 1992 election.

16 These polls are a vast and relatively untapped data source for election studies. As the Appendix describes most of the surveys also include a number of useful explanatory variables. Although each poll does not always include the exact question we would prefer, these data do contain a considerable amount of data – considerably more interviews from 1988 alone than the sum total of all the interviews from every presidential National Election Survey since 1952. See Asher, Herbert, Polling and the Public: What Every Citizen Should Know (Washington, DC: Congressional Quarterly Press, 1988)Google Scholar, for a general review of polls and the public.

17 The survey question asked most often was, ‘If the 1988 Presidential election were being held today, would you vote for George Bush for President and Dan Quayle for Vice President, the Republican candidates, or for Michael Dukakis for President and Lloyd Bentsen for Vice President, the Democratic candidates?’ Analogous questions were asked in the other years. We confront potential problems of question wording below.

18 Lewis-Beck, and Rice, , Forecasting Elections.Google Scholar

19 See Kessel, John, Presidential Campaign Politics (Belmont, Calif.: Dorsey Press, 1988).Google Scholar

20 See Markus, Gregory B., ‘The Impact of Personal and National Economic Conditions on the Presidential Vote: A Pooled Cross-Sectional Analysis’, American Journal of Political Science, 32 (1988), 137–54.CrossRefGoogle Scholar

21 Lazarsfeld, Paul F., Berleson, Bernard and Gaudet, Hazel, The People's Choice: How The Voter Makes Up His Mind in a Presidential Campaign (New York: Duell, Sloan and Pearce, 1944).Google Scholar

22 Bartels, Larry, ‘Stability and Change in American Electoral Polities’, in Butler, David and Ranney, Austin, eds, Electioneering (New York: Oxford University Press, in press).Google Scholar

23 See Franklin, Charles H. and Jackson, John E., ‘The Dynamics of Party Identification’, American Political Science Review, 77 (1983), 957–73.CrossRefGoogle Scholar We can distinguish between two kinds of fundamental variables: (1) characteristics of the voter and his or her situation, including their position on issues, party identification, ideology, economic conditions etc.; and (2) voters' perceived characteristics of the candidates, such as the candidates' ideology and positions on issues. There are also variables like incumbency which modulate the effect of the second category of fundamental variable: if you run a stronger campaign, you are most likely to convey a positive message about yourself relative to the other candidates. Variables in the first category change very little over the campaign, while variables in the second are directly influenced by the campaign.

24 Profiles, 12 1991, p. 21.Google Scholar

25 Newsweek, 14 10 1991, p. 29.Google Scholar

26 Stahl, Lesley, CBS News broadcast, 22 07 1988Google Scholar, during the Democratic convention; Newsweek, 5 09 1988Google Scholar; Newsweek, 19 09 1988.Google Scholar

27 Editorial, Washington Post, 14–20 10 1991.Google Scholar

28 See Buchanan, William, ‘Election Predictions: An Empirical Assessment’, Public Opinion Quarterly, 50 (1986), 222–7.CrossRefGoogle Scholar

29 The responses to the standard question wording refer to Gallup's poll conducted 15 June 1988. The responses to the non-standard wording refer to Gallup's poll conducted on 22 June. The standard question wording and the unusual question wording are given in the notes to Figure 2.

30 These proportions are corrected for differences due to varying survey methodologies across the different survey organizations.

31 Other variables also give similar results. We show in the Appendix that party identification and ideology are largely exogenous variables, not responding much to changes in voter preferences or anything else that changes during the campaign.

32 Lewis-Beck, , ‘Election Forecasts in 1984: How Accurate Were They?’Google Scholar

33 See, for example, Fair, , ‘The Effect of Economic Events on Votes for President’ and updates.Google Scholar

34 The two models are also inconsistent with one another about the evidence they provide on who ran a better campaign in 1988. Contrary to the journalists' claims (and even Dukakis himself), most political science models showed Dukakis doing as well or even better than expected, perhaps because Dukakis's vice-presidential selection was better (from an electoral perspective) than Bush's.

35 The Appendix shows that party identification and ideology in the population are roughly constant during the campaign.

36 According to Condorcet's ‘jury theorem’, if some voters have incomplete information, then, under certain conditions, a majority-rule electoral system will produce outcomes equivalent to the situation that would exist if all voters were informed. This is obviously relevant to our inquiry, except that the assumptions required to prove this theorem are far too restrictive. Scholars have recently been quite successful at dropping some of these restrictive assumptions, so perhaps in the near future the two lines of research might converge. (See Miller, Nicholas R., ‘Information, Electorates, and Democracy: Some Extensions and Interpretations of the Condorcet Jury Theorem’, in Grofman, Bernard and Owen, Guillermo, eds, Information Pooling and Group Decision Making (Greenwich, Conn.: Jai Press, 1986)Google Scholar; Ladha, Krishna, ‘Condorcet's Jury Theorem, Free Speech and Correlated Votes’, American Journal of Political Science, forthcoming.Google Scholar) Related work in experimental economics has studied how markets proceed on the road to various types of equilibria. (See Plott, Charles R., ‘An Updated Review of Industrial Organization: Applications of Experimental Methods’, in Schmalensee, R. and Willig, R. D., eds, Handbook of Industrial Organization, Volume II (Amsterdam: Elsevier Science Publishers, 1989)Google Scholar; and ‘Industrial Organization Theory and Experimental Economics’, Journal of Economic Literature, 20 (1982), 1485–1527.)Google Scholar

37 Dahl, , Democracy and Its Critics.Google Scholar

38 See Popkin, Samuel, The Reasoning Voter: Communication and Persuasion in Presidential Campaigns (Chicago: University of Chicago Press, 1991).Google Scholar

39 Kahneman, Daniel, Slovic, Paul and Tversky, Amos, eds, Judgment Under Uncertainty: Heuristics and Biases (New York: Cambridge University Press, 1982).CrossRefGoogle Scholar

40 Designing surveys so as to reduce this embarrassment, making it easy to report ‘no opinion’, would not necessarily improve the forecasting ability of the polls, since those voters who express a ‘certain’ opinion seem to mirror the survey population as a whole; see the discussion of question wording in Section 3.1 and Figure 2. A very useful future research project would be to design a survey or experiment to encourage voters to account rationally for their uncertainty (perhaps by giving them more time or financial incentives to give the ‘right’ answer), and see if it makes a difference to their reply.

41 Some of the most important variables forecasters use do not change over the course of the campaign, such as incumbency status and some other national variables. That we have no information on these does not affect our inferences because they are effectively controlled by being held constant. The remaining variables that might have some effect include perceived economic well-being and perceived ideological distances between voters and candidates, both of which might change over the campaign.

42 We omit 1952–60 from Figure 5 because Gallup did not list polls between the two conventions for those years.

43 Campbell, James E., Cherry, Lynna L. and Wink, Kenneth A., ‘The Convention Bump’, American Politics Quarterly (1993, forthcoming) also discuss poll movements during conventions.Google Scholar

44 A small amount of uncertainty is reduced by the conventions, but this could not account for the systematically predictable shifts in voter support in Figures 1 and 5.

45 We also tried the following analyses with all three-way interactions and obtained similar results, except that the many groups with small numbers of voters increased sampling error and thus made the results much more variable and more difficult to interpret.

46 We have conducted extensive analyses, not presented here, searching for identifiable groups of respondents who become relatively more ‘informed’ or ‘enlightened’ as the campaign progresses. Even using education and many other variables, we have found no clear evidence for differences across groups in the speed with which they learn during the campaign.

47 Indeed, this concept should be useful for predicting changes in group support over the campaign. In general, groups that are more divided at the start of the campaign will move the most as the campaign progresses.

48 For each of the three ‘parties’ (Democratic, Republican and Independent) and each poll, let x 1, x 2, x 3, x 4 … be the average support for Bush in all the subgroups of that party (for example white Democrats, non-white Democrats, liberal Democrats, moderate Democrats, etc.), and let n 1, n 2, n 3, n 4 … be the survey weights of the respondents in each subgroup of the party. The mean support for Bush in the party is , the observed variance across subgroups is: and the expected sampling variance is: Σin ix i(1 − xi)/Σin i. Each circle on Figure 7 plots the difference between the observed variance and the expected sampling variance, set to zero if the difference is negative.

49 Each point on the ‘lowess’ curve is calculated by weighted least squares, with the points in closest proximity on the horizontal axis given the highest weights. See Cleveland, William, ‘Robust Locally Weighted Regression and Smoothing Scatterplots’, Journal of the American Statistical Association, 74 (1979), 829–36.CrossRefGoogle Scholar

50 In fact, we do have many additional survey questions aside from those we analyse, but these were not asked in as many polls. Thus far, our auxiliary studies of these questions do not suggest any changes in the conclusions presented in this article.

51 See King, Gary, Unifying Political Methodology: The Likelihood Theory of Statistical Inference (New York: Cambridge University Press, 1989).Google Scholar

52 Because we used only surveys which had all of our covariates, there are fewer points in this figure. With this smaller sample size, the ‘lowess’ estimates used in Figure 7 were less useful here.

53 From July to October, the proportion of the public who saw Bush as a conservative (rather than liberal or moderate) increased only from 54 per cent to 59 per cent, while the proportion who saw Dukakis as a liberal increased from 35 per cent to 51 per cent. From our perspective, it seems clear that Dukakis's actual degree of ‘liberalness’ is closer to his October than July rating.

54 One should be careful in drawing conclusions from this figure. At worst it shows that the levels of the fundamental variables did not change much over the campaign, only their relative weights. Overall, the figure is one final observable implication consistent with our hypothesis about voter enlightenment.

55 Of course, we have shown only that voters base their decisions on the variables which political scientists call ‘fundamental’. However, these are not trivial variables from a normative perspective, such as the candidates' personalities or good looks; they are at least a good portion of the variables on which voters ‘should’ base their decisions in order to fulfil general notions of democratic citizenship.

56 Reporting the polls does not seem to influence the outcome, since there is no evidence of a ‘halo effect’ – the winner in the early polls does not inevitably win the election – although it may work strongly in primaries.

57 For example, Franklin, and Jackson, , ‘The Dynamics of Party Identification’.Google Scholar