Assessing the relative influence of party unity on vote choice: evidence from a conjoint experiment

Abstract

Observational studies and anecdotal evidence suggest that party unity improves a party's electoral performance. Yet, due to a lack of experimental evidence, the causal standing of these findings remains unclear. Moreover, party unity manifests in various ways and we do not know how much different types of party unity affect the vote. Relying on a conjoint experiment implemented in a probability-based survey of the German population, our study unveils the distinct causal effect of different forms of party unity on vote choice. We further establish that appearing united can compensate for substantive policy distances between parties and voters. These findings have important implications for our understanding of how citizens vote and how intra-party politics affects the political representation of citizens in democracies.

What are the electoral consequences of conflict within a party? While ample scholarship on intra-party unity emphasizes its importance for party and electoral behavior (e.g., Ceron, 2015a; Greene and Haber, 2015; Lehrer and Lin, 2020), the causal standing of these findings remains elusive as experimental evidence is missing. Furthermore, party unity is frequently simplified to be a unidimensional phenomenon. The various ways in which intra-party dissent manifests itself empirically are neither properly theorized nor well understood. Theoretical arguments on party unity are hardly specific enough to derive precise expectations about the distinct effect of different manifestations of party unity. This theoretical coarseness, the utilization of different measurement approaches, and the lack of causal evidence in general contribute to inconsistent results regarding the effect of party unity on vote choice in published research (Close and Gherghina, 2019).

Against this background, this research note seeks to improve upon the current state of the literature in three important ways. First, we provide experimental evidence for the effect of party unity on the vote. Second, we compare the effect size of party unity to other important determinants such as the ideological distance between voters and parties. Third, we investigate to what extent conclusions drawn about the consequences of intra-party dissent depend on its operationalization.

Different arguments have been presented in the literature to explain how intra-party conflict depresses a party's vote share (e.g., Greene and Haber, 2015; Jung and Somer-Topcu, 2020; Lehrer and Lin, 2020). Empirically, however, only observational studies exist that fail to provide causal evidence for the link between party unity and vote choice. We not only causally investigate the electoral consequences of intra-party conflict but also three of its manifestations that empirical research often employs as general measures of party unity: legislative voting behavior, internal critique of the party leadership, and the behavior of delegates at party congresses.¹

We implement a conjoint experiment in a probability-based survey of the German population. The experimental design not only permits the derivation of causal statements. It also allows us to compare the effect sizes of different forms of party unity to one another as well as to the effect of ideological distance. The results reveal that internal disagreement curtails a party's electoral fortunes. We also find that the electoral consequences of intra-party conflict differ depending on the way the dissent is displayed. While critique from actors within the party has a negligible impact, incohesive voting behavior in parliament and especially publicly displayed conflict at party congresses decrease voters’ likelihood to vote for a party. Finally, party unity is a relevant factor in the calculus of voting that can compensate for some degree of ideological distance.

The intra-party unity dimensions included in the experiment mimic operationalizations that are often used in observational research: Researchers often turn to legislative voting behavior to operationalize intra-party unity. Even though party discipline is usually strong in parliamentary systems, so that parliamentary voting cannot be equated with the sincere expressions of individual preferences (e.g., Carey, 2007), diverging voting patterns or more subtle signals such as abstentions are observable indicators divisions within parties (Ceron, 2015a).

Another distinct manifestation of intra-party dissent is openly voiced critique of the party leadership. In principle, different actors – including rank-and-file members, faction leaders, or former party leaders – may openly voice criticism at any time. The availability of social media facilitates the expressions of criticism toward the own party leadership or the parties’ policies without media gatekeeping. Current or former party elites may also utilize traditional forms of public communication such as interviews or press releases to publicly state criticism. Both classical communication strategies and social media statements are used in empirical studies of party unity (Ceron, 2017; Sältzer, 2020; Somer-Topcu et al., 2020).

Finally, party congresses provide an excellent channel to investigate a party's internal cohesiveness. Speeches at party congresses provide a relatively ungated stage for party members to express their concerns with the party leadership and the party's policy platform. Given that national party congresses are among the most important intra-party activities and regularly receive extensive media coverage, the electorate is exposed to information on the proceedings of national party congresses (e.g., Ceron, 2015a; Greene and Haber, 2016; Ceron and Greene, 2019).

1. Data and research strategy

To causally identify links between (different manifestations of) party unity and vote choice, and to relate their sizes to one another as well as to other determinants of the vote, we conduct a choice-based conjoint experiment (Hainmueller et al., 2014; Ben-Akiva et al., 2019). This design allows us to simultaneously estimate the causal effects of different dimensions of party unity on vote choice. Furthermore, since effect sizes are expressed on a common scale, we can directly compare the influence of different aspects of party unity to one another and to other relevant factors such as ideological distance or different candidate characteristics. The survey experiment was embedded in the November 2019 wave (Blom et al., 2020) of the German Internet Panel (GIP), which is based on a probability-based sample of the German population (Blom et al., 2015). The combination of high-quality survey data and a conjoint experiment enables us to draw conclusions about real-world voter behavior (Hainmueller et al., 2015).

In our experiment, respondents receive randomly varied information about various characteristics of two fictional parties competing against each other in a hypothetical electoral contest. To ease the cognitive burden for respondents, we present the information on the two parties in a table. Respondents are asked to choose one of the parties without an explicit “none”-option (forced-choice design). Yet, they are able to skip the question after an error message asks them to state their choice.² Each respondent consecutively evaluates ten party pairs which is in line with corresponding recommendations for data quality (e.g., Bansak et al., 2018).³

While we provide information on several features of the hypothetical parties, our primary interest lies in the causal effect of party unity and its manifestations that appear in empirical research. Consequently, we specify three party unity attributes and randomly vary their levels. Table 1 shows all attributes and their respective levels. Specifically, the study separates the following dimensions (with attribute levels in brackets): First, critique toward the party leadership (none, rank-and-file members, former party leader, party faction). This mimics operationalizations that are used in research on party factions, for instance by using social media data (e.g., Ceron, 2012; Sältzer, 2020). Second, parliamentary voting (united, divided). This kind of approach resembles approaches in legislative studies (e.g., Sieberer, 2006; Ceron, 2015a). Third, behavior at party congress (united, neither united nor divided, divided) which stems from the literature on party congresses (e.g., Greene and Haber, 2016; Ceron and Greene, 2019).

Table 1. Attributes and attribute levels

We also randomly vary several additional party features. Foremost, we acknowledge the predominant role of ideological distance for voting (e.g., Franchino and Zucchini, 2015) and assign each fictional party one of five ideological positions (left, center-left, center, center-right, right). Since Lehrer and Lin (2020) report that vote choice is not only affected by party unity on its own, but also jointly with a party's policy ambiguity, we include clarity of reform proposals (high, low). To test for incumbency effects that may be particularly relevant in our experiment in which partisanship is muted (Lim and Jr, 2015), we manipulate a party's current role (junior coalition party, prime minister's party, opposition party). We also consider the party's top candidate characteristics by varying gender (female, male), age (38, 56, 74 years), and professional experience (activist, employee, retired employee, lawyer, politician, entrepreneur). Finally, to avoid generating unrealistic profiles, we restrict specific combinations of candidate profiles. This refers to scenarios where a 74 years old top candidate still works as an employee, or where a 38 or 56 years old candidate is already retired.⁴

The research design mimics German elections. Similar to Bundestag elections, respondents choose between parties using information on parties’ policy stances, incumbency status, and their Spitzenkandidaten (Debus et al., 2014). To facilitate the identification of causal effects, however, the experiment also mutes effects of partisanship, limits choices to two parties and presents information to voters in a way that media usually do not use. Germany serves as an interesting starting point to study party unity since its parties are known to be home to party factions. At the same time, these factions do not dominate party politics as they do elsewhere (Sieberer, 2006; Ceron, 2015b). Interestingly, the experiment happened to be conducted when Germany's Social Democrats, at the time a junior coalition partner, chose a new party leadership by membership vote. Hence, Germans may have been more sensitive to intra-party division at the time. These aspects somewhat limit the experiment's external validity which we further discuss in the conclusion.

To discern the effect of each attribute, we estimate a conditional logistic regression model (McFadden, 1974) and compute average marginal component effects (AMCE) as well as predicted vote probabilities.⁵ The levels of each attribute are represented by indicator variables. A party's behavior at the party congress, for instance, is modeled as two indicator variables, one for “neither united nor divided” and one for “divided”, while “united” serves as the reference category. Even though the experiment randomizes party ideology, we expect that ideological distance to the respondent's ideal position and not ideology per se matters. To translate party ideology into ideological distance, we rely on respondents’ self-placements on an ideological left-right scale, which was collected two months prior to the experiment in GIP Wave 43 (Blom et al., 2021). We use these self-placements to compute the ideological distance between every respondent and the hypothetical party.⁶

Upon list-wise deletion, the sample size totals $3687$ respondents. Because respondents perform multiple choice tasks, our analysis includes $26,761$ choices and standard errors are clustered by respondents to account for this.

2. Results

2.1. The effect of party (dis)unity on the vote

Figure 1 shows the attributes’ AMCEs.⁷ These can be interpreted as the expected change in the probability to choose a given party when an attribute value is compared to the reference category (Hainmueller et al., 2014). In Figure 1, the reference levels are indicated by the black dots at zero.

Figure 1. Average marginal effects of different attributes. Estimates are based on a conditional logistic regression model with clustered standard errors; bars represent $95\percnt$ confidence intervals. The points without horizontal bars represent the reference categories.

Unsurprisingly, by far the strongest predictor of vote choice is ideological distance. A party that has an ideological distance of one unit to the respondent, e.g., a center-right party's distance to a centrist respondent, is on average 10.3 percentage points less likely to be chosen than a party with a zero distance (the 95% confidence interval covers [$8.9$; $11.7$]). When the ideological distance equals two units, e.g., a right party and a centrist voter, its effect already exceeds any other attribute's effect. This result confirms the predominant role of ideological distance for voting.

Regarding the possibility of direct effects of different manifestations of party unity, Figure 1 reveals the existence of heterogeneous causal effects across different dimensions of party unity. Overall, internal critique from party actors has almost no effect on the probability that a respondent chooses the party. In comparison to no internal critique, there is a small negative effect when critique is voiced by internal party factions ($-1.8$percentage points $[\! -\!0.4;\; -\!3.2]$) or former party leaders ($-1.5$ percentage points $[\!-\!0.7;-\!2.9]$). The estimates, however, also suggest that there is no difference between the complete absence of critical comments and critique from rank-and-file members.⁸

In contrast, we find that incohesive voting behavior of a party's parliamentary group harms the party electorally. All else equal, a party failing to ensure united voting patterns by its parliamentary group is on average $5.7$ percentage points $[ 4.6;\; 6.7]$ less likely to be chosen by voters.

Finally, our experiment identifies a party's behavior at its congress as the most decisive dimension of party unity with respect to vote choice. A party that successfully manages to convey a cohesive image at its congress to voters is able to obtain an electoral advantage whereas parties that engage in internal fights will be punished. Open disagreements at party congresses harm parties with a loss in vote probability of $14.7$ percentage points $[ 13.5;\; 15.9]$.

Overall, we find clear support for the expectation that intra-party unity boosts a party's electoral performance. The evidence is particularly strong when party unity becomes visible in party congress behavior and to a lesser degree in parliamentary voting.

Besides these main effects of interest, the experiment further reveals the effects of several additional features. With respect to the top candidate, for example, we corroborate previous findings that voters prefer younger and female candidates with a primary occupation as employees (e.g., Carnes and Lupu, 2016). Moreover, our experiment clearly supports observational evidence suggesting that voters punish parties for low reform clarity, i.e. vague policy proposals (Lehrer and Lin, 2020).

2.2. How important is party unity for electoral competition?

To compare the effect sizes of the different party unity manifestations to ideological distances and to determine when party unity is pivotal for vote choice, we use the estimates from our conditional logistic regression model to simulate hypothetical electoral contests. Consider a voter who is faced with a choice between two hypothetical parties, Party 1 and Party 2. Both of these parties are identical, except that Party 1 is ideologically somewhat closer to the voter than Party 2. The parties’ full characteristics are given at the top of Figure 2. Following the reasoning of spatial voting, the model predicts a probability of $60.3\% [ 60.1;\; 60.7]$ that the hypothetical voter chooses Party 1 which is ideologically closer (see the top plot in Figure 2).

Figure 2. Hypothetical competition scenarios.

Now imagine that Party 1 competes with Party 3 which manages to present itself as united at its party congress but is otherwise identical to Party 2. In particular, it also positions a unit away from the voter, is divided in parliamentary voting, and its factions voice critique toward the leadership. If only ideological distance mattered to the voter, we would expect Party 1 to prevail because ideological distances did not change. However, because party unity matters, Party 3 is more likely to be chosen. In fact, Party 1 is now chosen by the hypothetical voter with only $45.3\% [ 45.3;\; 45.5]$ probability, which means that we expect the voter to choose Party 3 over Party 1 (center plot in Figure 2).

A similar pattern emerges when Party 1 instead competes with Party 4. This party produces no internal critique, exhibits unified parliamentary voting and neither a divided nor a united behavior at its party congress. Otherwise it is like Party 2. As the bottom plot in Figure 2 shows, in an electoral contest between these parties, Party 1 is roughly as unlikely to be chosen as in its competition against Party 3 (42% [41.8; 42.6]).

These examples present causal evidence that party unity can compensate for greater ideological distance which may, for example, be the ideological difference between center-left and center. They also reveal that while party unity with respect to parliamentary voting and critique of the party leadership may exert rather small effects on voting probabilities, their joint effect is similar in strength to a change in party congress behavior from neither united nor divided to fully united.

3. Additional results: simulate your own electoral competition

Overall, our experimental design gives rise to $21, 600$ distinct party profiles and $466,\; 560,\; 000$ possible party pairings. To facilitate the analysis of interesting hypothetical electoral competitions despite the large number of potential pairings, we provide a free simulation tool that allows researchers to vary all dimensions, and investigate how the predicted election outcome changes when amending one or several distinct attributes. The simulation tool is available online at https://party-unity-conjoint.shinyapps.io/simulation/ and requires no technical or statistical knowledge. Effectively, it operates along the lines demonstrated above: Users specify hypothetical parties with specific attribute configurations, and the tool simulates an electoral contest between them by using the results from the regression analysis.

Suppose we are interested in learning whether perfect party unity can compensate for more than a unit of distance on the five-point scale of policy distance. To do so, we would specify two parties that only differ in their party unity and their ideological distance to the voter. One party is ideologically congruent with the voter, yet, fully divided (internal critique from party factions, divided in parliamentary voting and at the party congress), while the other party is perfectly united (no internal critique, united in parliamentary voting and at the party congress), yet, positions two units away from the voter. We present the corresponding results in online Appendix 6. In particular, we show that even the highest levels of party unity cannot compensate for more than a single unit on a five-point scale of policy distance.

Similarly, we find that in the most extreme scenario, an all united party (no internal critique, united in parliamentary voting and at the party congress) is roughly 2.5 times as likely to be chosen than an all divided party (internal critique from party factions, divided in parliamentary voting and at the party congress).⁹

We strongly encourage researchers to think of novel attributes combinations that are meaningful to their research questions, and to use the simulation tool to gain relevant insights.

4. Conclusion

Using a conjoint experiment, this note establishes a causal relationship between party unity and vote choice. Internal dissent decreases a party's electoral performance. It also reveals that party unity can compensate for ideological distance, however, this effect is limited as spatial considerations clearly dominate vote choice. Finally, we find that the electoral consequences of intra-party conflict differ across distinct dimensions of party unity. While publicly stated critique from party actors has a negligible impact, incohesive voting behavior in parliament and especially publicly displayed conflict at party congresses decrease voters’ likelihood to vote for a party.

The experimental evidence presented here has important implications for political representation in democracies. We find that parties face strong electoral incentives to act in unison and appear united (Greene and Haber, 2015). Viewed from a normative perspective, this points to a trade-off parties need to manage: Parties are supposed to be deliberative institutions in which critical voices are raised and leaders are held accountable (Wolkenstein, 2016). At the same time, our findings suggest that parties have strong electoral incentives to not appear divided in public. Future research will need to study whether and how parties address this challenge.

Further, the results show that the effect of party unity depends on how it is operationalized. This has direct implications for the empirical study of party unity because it has, up until now, often singled out a party unity dimension to measure it (Close and Gherghina, 2019). Our findings suggest that results may not be robust to the choice of the measurement approach. This also highlights the necessity to improve the theoretical foundation to allow for the derivation of fine-grained expectations regarding the more subtle effects of party unity.

While our experimental design allows us to uncover the causal effect of party unity on vote choice, it abstracts from several important aspects which future research should scrutinize: First, while the experiment establishes a causal relationship, it is not able to reveal the causal mechanisms underlying the link between party unity and vote choice. Second, by prompting respondents with specific levels of party unity as part of the conjoint design, we are unable to uncover any effect perceptions of party unity may have (Plescia et al., 2020). Third, we conducted the experiment within a panel survey in Germany at a time of potentially heightened sensitivity toward party unity. Hence, future research should not only confirm our findings in other electoral contexts but also with respondents who do not regularly answer surveys on political items. Finally, the experiment abstracts from actual German elections in multiple ways, including a choice between two parties, the omission of party labels, and a presentation of information on parties that do not match the actual ballot. We suspect that these aspects limit the experiment's external validity, yet, we do not fully know to what extent. Therefore, we call on scholars to further investigate the relationship between party unity and vote choice to develop a more sophisticated understanding of this important nexus.