The Civic Option? Using Experiments to Estimate the Effects of Consuming Information in Local Elections

Abstract

Political parties and civic organizations disseminate information to improve citizen decision making in local elections. Do citizens choose to consume such information and, if so, how does it affect their decisions? We conduct a survey experiment during a real-world local election that randomly assigns 1) political party endorsements, 2) a voter guide, 3) no information, or 4) a choice among these options. Respondents assigned to receive party endorsements and a voter guide are more likely than respondents receiving no information to choose candidates who share their policy views. When given a choice, a majority opts to receive information (including many with low levels of political interest), with most respondents preferring a voter guide. Using an instrumental variable approach, we show that the effect of information on those who choose to receive it is substantial. These results offer hope that voter education efforts can succeed despite widespread political disinterest.

The extent to which government activities reflect citizens’ preferences is a central question in the study of democratic politics. The occurrence of regular elections offers citizens opportunities to staff elective offices with representatives who share their policy views. Such opportunities, however, do not guarantee that election outcomes will reflect citizens’ policy interests. Indeed, citizens often lack information that might help them connect their policy views to their candidate choices (Converse 1964). Such information deficits are particularly acute in local elections where media coverage is sparse compared to prominent state and national elections. Local elections typically involve a bewildering array of issues affecting neighborhoods, groups of citizens, local functions like policing and transportation, and entrenched institutional actors. Moreover, because elections in most large American cities are dominated by Democrats, candidates’ partisanship is often uninformative in signaling differences between them. This deprives citizens of a reliable information shortcut.

Nonetheless, while many local elections are accurately described as low-information settings, efforts by political parties and civic organizations can provide citizens with low-cost sources of information to learn about candidates. Parties, for example, signal their preferences for candidates through endorsements, usually accompanied by resources from the local party organization. Such endorsements, especially from local Democratic organizations, can help citizens sort through lists of co-partisans running for local offices. Newspapers and nonpartisan civic organizations also disseminate voter guides that provide detailed information about candidates’ positions on local policy issues. The efficacy of such voter education efforts depends on citizens’ interest in and ability to use them.

Will citizens take advantage of opportunities to receive information, such as party endorsements and voter guides, in local elections? Does such political information help citizens choose candidates whose policy views resemble their own? We address these questions by conducting a survey experiment during the 2018 mayoral special election in San Francisco. We randomly assign respondents to receive either 1) the Democratic and Republican parties’ endorsements of candidates (party cues), 2) a voter guide summarizing the candidates’ policy positions, or 3) neither type of information. We also include a fourth group where respondents choose whether to receive party cues, a voter guide, or neither type of information. In each group, we assess the extent to which respondents choose candidates whose policy views resemble their own (spatial voting). Within the “information choice” treatment group, we also examine what information respondents choose to receive, as well as its effects on spatial voting.

By randomly assigning party cues, voter guides, and a choice of which type of information (if any) to receive, we overcome two limitations of previous research. First, experiments that manipulate party cues and/or voter guides typically use “forced exposure” designs that assign all respondents within a treatment group to receive the information. Forced exposure designs are unable to determine whether respondents will choose to receive information and what kind of information they will select. Second, if some respondents who receive information in a forced exposure design ignore it, the estimated average effect of the treatment may understate its impact on those who actually assimilate the information. By including an information choice treatment group and using an instrumental variable analysis, we are able to estimate the causal effect of information on those who choose to consume it.

Our results demonstrate that both party cues and voter guides help respondents choose candidates who share their policy views when they are assigned to receive them. When given a choice, a majority of respondents, including many with low levels of political interest, take advantage of the opportunity to receive political information, with most selecting a voter guide over the easier-to-use party cues. Our instrumental variable analysis indicates that the effects of information on those who choose to receive it are substantial. Together, these results indicate that those most in need of information are frequently among those who opt to receive and then benefit from it, offering hope that voter education efforts can succeed despite widespread political disinterest.

Theory and hypotheses

Previous research offers a foundation for theorizing about party cues and voter guides in forced exposure settings. In the one-party contexts that characterize many big city elections, party cues come in the form of endorsements, which frequently are advertised in mailers or on websites. These endorsements identify which candidates have the support of the party organization and, perhaps, many of its members. As Sniderman and Stiglitz (2012) argue, such endorsements can help citizens identify candidates who share their policy views when: 1) there are meaningful ideological differences between the candidates, 2) political parties have well-known ideological reputations, and 3) the parties send ideologically “correct” signals about which candidates to support (i.e., the Democratic [Republican] Party endorses the more liberal [conservative] candidate). Given that these conditions are met in the 2018 mayoral election we study, we expect party cues to send clear signals about the candidates’ ideological positions and facilitate spatial voting (Hypothesis 1).

In contrast to party cues, voter guides typically do not advocate for particular candidates or issues. Rather, they supply detailed information about candidates’ policy positions, backgrounds, and priorities, often in a table format that facilitates candidate comparisons. Such voter guides are regularly disseminated by nonpartisan civic organizations in order to reduce the cognitive costs of forming opinions about candidates (Mummolo and Peterson 2017; Boudreau, Elmendorf, and MacKenzie 2019). Theoretically, voter guides can supply information about candidates’ policy positions that spatial models assume citizens have (Enelow and Hinich 1984), but empirical studies suggest they lack. Consistent with previous research (Boudreau, Elmendorf, and MacKenzie 2015, 2019), we expect voter guides to strengthen spatial voting (Hypothesis 2).

Previous research offers less guidance about information effects outside of forced exposure settings. Experimental studies nearly always assign respondents to receive party cues and policy-related information, as opposed to letting them choose whether and what type of information to consume (Arceneaux 2008; Bullock 2011; Nicholson 2011; Boudreau and MacKenzie 2014; Boudreau, Elmendorf, and MacKenzie 2015, 2019). In real-world contexts, such forced exposure to political information is atypical. Citizens can ignore information they receive in the mail and turn the channel when undesired content appears on their television or computer screens, as studies documenting the “selective exposure” to political news attest (Prior 2007; Stroud 2008; Arceneaux, Johnson, and Murphy 2012).

To our knowledge, ours is the first experimental study allowing citizens to choose whether to receive party cues, a voter guide, or no information in a real-world election setting. Will citizens prefer party cues, a voter guide, or no information? What effects will information have on those who choose to receive it? These questions cannot be answered with a forced exposure design. We theorize that many citizens will take advantage of low-cost opportunities to receive political information that bears on their choices in local elections. First, studies show that demand for information increases as Election Day approaches (Gelman and King 1993; Rosenstone and Hansen 1993), when political parties and civic organizations circulate endorsements and voter guides. Further, empirical studies of voter guides indicate that citizens find their content useful (Boudreau, Elmendorf, and MacKenzie 2019). Studies also show that low-propensity voters respond to both partisan direct mail and voter guides containing procedural information (Wolfinger, Highton, and Mullin 2005).

Assuming that some citizens choose to receive party cues or voter guides, previous research offers conflicting predictions about what type of information citizens will select, who will choose to receive it, and what effects information will have on their decisions. Based on dual process models of attitude change, we might predict that citizens’ low levels of political interest will lead them to choose party cues over a voter guide requiring systematic processing (Hypothesis 3; see Eagly and Chaiken 1993). With respect to who will choose information, studies of news consumption indicate that citizens with high levels of political interest opt to receive political news, while less interested citizens choose entertainment or no political news at all (Prior 2007). To the extent that party cues and voter guides resemble political news, we might expect that only politically interested citizens will choose to receive them (Mummolo and Peterson 2017). If so, then information will have minimal effects on those who choose to receive it (Hypothesis 4). That is, the information will reach only those (politically interested citizens) whose opinions are already formed and unlikely to change. If instead, those who need political information choose to receive it, then the effects of information will be substantial (Hypothesis 5). That is, the information will reach those who are likely to use it to form their opinions.

Study design

The 2018 mayoral special election in San Francisco provides an opportunity to test our hypotheses. Like many large American cities, San Francisco is a one-party (Democratic) setting in terms of party registration and voting patterns. ¹ As a result, 5 of the 8 official candidates in this election were Democrats. Nonetheless, these candidates took different positions on important local policies that reflect the ideological division between so-called “progressives” (the local left) and “moderates” (the local right). Recently, progressives have advocated giving cash grants to the homeless and opposed tax breaks for local businesses, while moderates have taken the opposite positions. Such policy-based differences, in the absence of partisan differences, help us to isolate the effects of ideology on citizens’ reactions to party cues and voter guides.

San Francisco also features a unique convention that allows us to reliably measure candidates’ policy positions. Political party organizations, newspapers, and interest groups in the city distribute questionnaires to candidates for local offices. Answers to questionnaires are often made public and scrutinized for inconsistencies. Candidates who refuse to answer or dissemble do so at their peril. These questionnaires enable us to ascertain candidates’ support for or opposition to various local policies and measure their local ideological positions.

Specifically, we estimate candidate ideal points based on their responses to the policy questions included in publicly available questionnaires. To measure citizens’ ideological positions on the same scale, we included ten of these policy questions in an online survey we conducted one week before Election Day (June 5, 2018). We then estimated citizens’ ideal points based on their responses. Table 1 summarizes these policy questions, as well as the candidates’ and citizens’ answers.

Table 1. Policy questions with mayoral candidates’ and voters’ answers

To recruit citizens, we sent emails to 66,645 San Francisco residents selected at random from the city’s list of registered voters with an email address on file. ² The email invited recipients to take an online survey developed by researchers at the University of California, Davis in exchange for a $5 Amazon gift card. We used Qualtrics to administer the survey. We received 977 completed surveys, 766 of which included the treatment/control groups relevant to this study. Despite the low response rate (which is typical for this method of contact), our sample resembles San Francisco’s voting and general populations in many respects (see Table A2 in the Online Appendix [OA]).

To test our hypotheses, we randomly assigned respondents to control or treatment groups. To measure our outcome of interest, we asked respondents to express their preferences for four candidates, considered pairwise. Respondents were asked to indicate which candidate in each pair they would prefer to be the mayor. We use answers to these questions, together with our measures of respondents’ and candidates’ ideal points, to identify whether respondents choose the candidate whose ideological position is closer to their own in each pair.

In the control group, respondents do not receive any information about the candidates. Respondents assigned to the “party cues” treatment group receive information about the political party endorsements that the four candidates actually received. As shown in Figure 1, respondents receive a grid that lists the Democratic and Republican parties’ endorsements. In this election, the Democratic Party endorsed two candidates (both progressives) and ranked them in order of preference. The Republican Party endorsed a candidate whose ideological position is well to the right of others (see Figure A1 in the OA). Thus, the political party endorsements sent ideologically “correct” signals about the candidates’ policy views.

Figure 1. Party Cues Treatment

Respondents assigned to the “voter guide” treatment group receive a table that summarizes the four candidates’ actual positions on seven local policy issues. Figure 2 displays the information that these respondents received, which mimics real-world voter guides and provides information about the candidates’ policy views. Question marks were used to indicate nonanswers to policy questions. ³

Figure 2. Voter Guide Treatment

Respondents assigned to the “information choice” treatment group are able to choose whether or not to receive information about the candidates. If they choose to receive information, they select either party cues or the voter guide. They then receive the information they selected. This treatment is analogous to real-world settings where citizens choose whether to receive information and, if so, what type of information to consume. We use this treatment to assess the effects of information on those who choose to consume it.

Methods

To test our hypotheses, we first conduct difference of means tests to assess the effects of party cues and voter guides on spatial voting. Our dependent variable, Vote_Spatial _ij, takes the value 1 for each respondent i who prefers the candidate whose ideal point is closer to their own in each pair j, and zero otherwise. To estimate candidates’ and respondents’ ideal points, we use the item-response model developed by Clinton, Jackman, and Rivers (2004). ⁴ We compare this outcome across our treatment and control groups. Because assignment to these groups is randomized (i.e., unconfounded by pre-existing differences between respondents), any differences between the treatment and control groups can be ascribed to the causal effects of information. Nonetheless, it is possible that some respondents assigned to receive either party cues or voter guides ignore the information (i.e., noncompliers). Because we cannot observe whether respondents in these treatment groups actually consume the information, our comparison of these groups is considered an intention-to-treat (ITT) analysis. The disadvantage of this approach is that it cannot determine the causal effects of the receipt of information, only the effects of assignment to receive information.

To shed light on the causal effects of information on those who actually choose to receive it, we perform additional analyses using respondents assigned to the information choice treatment group. First, we examine the percentage of respondents who choose to receive party cues, a voter guide, or no information within this group. This reveals whether respondents want to receive information about candidates and, if so, what type they prefer. Second, we use an instrumental variable analysis to estimate the causal effect of information on those who choose to receive it. In doing so, we take advantage of the fact that while choosing to receive either party cues or a voter guide is confounded, assignment to the information choice treatment group is not. Thus, our information choice group is akin to a randomized experiment with one-sided noncompliance (i.e., some respondents choose to forego information). This allows us to estimate the local average treatment effect (LATE) for those who choose to receive information (i.e., the “compliers”).

The primary assumption that enables us to estimate this causal effect is that assignment to our information choice treatment group has no effect on respondents who choose not to get party cues or a voter guide when given an opportunity to do so (“non-compliers”). Given this exclusion restriction, Imbens and Rubin (2015) show that the LATE can be estimated as follows:

$$LATE = {{{{\widehat {ITT}}_Y}} \over {{{\widehat {ITT}}_W}{\rm{\;}}}} = {{\bar Y{{\rm{\;}}_{Info{\rm{\;}}Choice}} - \bar Y{{\rm{\;}}_{Control}}} \over {{\pi _{co}}}}$$

where ITT_Y is the ITT effect on the outcome, ITT_W is the ITT effect for the receipt of treatment, and π_co is the share of compliers. To be clear, this LATE is the average effect of choosing information – either party cues or a voter guide – on respondents who opt for one or the other. Our design does not allow us to separately identify the effect of the voter guide on those who select it, or the effect of party cues on those who select this option.

Results

Our ITT analyses demonstrate that party cues and voter guides improve respondents’ ability to choose candidates whose policy views resemble their own. Figure 3 plots the percentage of respondents who choose the candidate in a pair whose ideal point is closer to their own in our treatment and control groups. In the control group, 53.8 percent of respondents choose the candidate closest to their own ideal point. In the party cues treatment group, 60.0 percent of respondents do so. This difference is statistically significant. The effect of party cues is comparable to that of the voter guide. Indeed, 61.4 percent of respondents in the voter guide treatment group choose the candidate closest to their own ideal point, which is also a significant increase over the control group. Given these effects of party cues and voter guides when respondents are assigned to receive them, it is not surprising that giving respondents a choice to receive either type of information yields similar, albeit slightly smaller, increases in spatial voting.

Figure 3. Effects of Information on Spatial Voting

Numbers are mean percentages from Table A1 in the OA. * difference with control is statistically significant (p < 0.05, two-tailed).

Further analysis of our information choice group indicates that a narrow majority (50.8 percent) of respondents are interested in receiving political information. As Figure 4 shows, of those who choose to receive information about the candidates, 75.3 percent selected the voter guide. The decided preference for a voter guide over easier-to-use party cues is inconsistent with Hypothesis 3. It is possible that the proximity of an election with real consequences inspired respondents’ interest in and motivation to process substantive information about candidates, leading many to choose the voter guide. ⁵ As we show in Figure A2 in the OA, those with low levels of political interest are actually more likely than other respondents to choose to receive information.

Figure 4. Percentage Receiving Information in the Information Choice Treatment

Our instrumental variable analysis estimates the causal effect of information on those who choose to receive it. Table 2 contains the estimate of the local average treatment effect (LATE) of choosing to receive either party cues or a voter guide. ⁶ Choosing to receive information increases the share of respondents who choose the candidate closest to their own ideal point by 0.09. This effect is substantively large and statistically significant. Given that respondents with low levels of political interest are more likely to choose to receive information, this result indicates that, consistent with Hypothesis 5, those most in need of political information do benefit from it (by selecting candidates who share their policy views).

Table 2. Instrumental Variable Estimate of Local Average Treatment Effect

Local average treatment effect (intent-to-treat effect on receipt of treatment) calculated from Table A1 in the OA for respondents who choose to receive any information relative to the control group.

Conclusion

Our study demonstrates the positive effects of party cues and voter guides on spatial voting both among citizens assigned to receive them and among citizens who choose to receive them. The latter effect suggests that many citizens who need information in local elections will take advantage of a low-cost opportunity to learn about candidates and benefit via enhanced spatial voting. Indeed, approximately half of respondents chose to receive information, with most choosing a voter guide requiring systematic processing. That so many respondents chose to receive and effectively used political information offers hope that voter education efforts can succeed despite widespread political disinterest. Nonetheless, given that nearly half of respondents opted to forego information, obstacles to expanding the pool of citizens willing to receive information remain.

Our study of two types of information in one election is, hopefully, just an opening salvo about the efficacy of political information outside forced-exposure contexts. Future research can build on our design by examining the choices of different types of citizens, in different real-world elections, and/or among an expanded array of political information sources. Both scholars and practitioners would benefit from additional research to identify the conditions under which citizens with low levels of political interest and others will choose to receive information in more natural environments than our survey experiment. Experimental studies of citizens’ choices in other elections can determine whether our findings generalize to state and national elections with two-party competition (where citizens’ interest is greater) and nonpartisan elections for other local offices (where citizens’ interest is lower). Finally, while we find that many citizens are willing to process an issue-oriented voter guide, their willingness to do so might change in the presence of other commonly available options, including campaign contributions, horserace coverage, or even infotainment. Understanding which types of information will induce different groups of citizens to opt in rather than out of political discourse is a new frontier with large stakes for citizen competence.