Deradicalization and the Experience of Governing: Evidence from the Transformation of Socialist Parties in Western Europe, 1945–2021

Abstract

The question of deradicalization looms large in the historiography of western European socialism. But in this contested field, the contributions of the New Left historian, Ralph Miliband, have been curiously neglected. Through his work on the British Labour Party, Miliband developed a distinctive account of deradicalization that foregrounds the fact that when parties enter government, party elites find themselves transplanted into new, alien institutions. Over time, he argued, they then come to internalize the worldviews of those institutions and reshape their parties accordingly. This essay presents the first quantitative and cross-national test of this “experience of governing hypothesis,” using Comparative Manifesto Project data from western European socialist parties between 1945 and 2021 and a novel matching technique for panel data. Miliband’s theory is strongly supported by this analysis, which also demonstrates the value of taking a multi-dimensional approach to deradicalization.

By the end of the war, a whole army of Labour representatives were serving on a multitude of official committees, commissions, tribunals and agencies. Nor certainly did they lose the habits of mind engendered by this experience when the war came to an end.

– Miliband (1964: 48)

Introduction

In the historiography of Western European socialism, there is one vital, explanatory problem: How did parties that emerged out of a radical critique of existing political and economic systems, come to stand for something quite different? ¹ How, as Gerassimos Moschonas (2002: 232) put it, did socialist parties eventually opt “not simply for another strategy, but for another identity”?

This deradicalization is now the subject of a vast scholarly literature, which I will try to summarize below (see e.g., Bartolini 2000; Eley 2002; Moschonas 2002; Mudge 2018; Sassoon 1996). But one strand of thinking that has been curiously neglected in this field is the work of the New Left historian Ralph Miliband. In his analysis of the British Labour Party, Miliband (1964, 1969) developed a distinctive account of deradicalization that focussed on the way the experience of governing affected party elites. As Miliband shows, serving in government meant that party leaders and advisors were uprooted from the familiar world of trade unions and party bureaucracies, and transplanted into the alien institutions of the state. This move had profound consequences, as party elites were exposed to new ways of thinking and acting and, gradually, came to adopt the worldviews, ideologies, and “habits of mind” embedded within those state institutions (Miliband 1964: 48). This was not a simple move along a left-right spectrum, but rather the absorption of different sets of ideas and practices that were specific to particular branches of the state (Miliband 1969). It was a complex, domain-specific, and multifaceted process of deradicalization, but one that left the British Labour Party profoundly transformed.

Although Miliband’s work has been highly influential in the British context, it has yet to be tested systematically or applied internationally. That is what this paper sets out to do, examining whether the experience of governing is correlated with deradicalization for the wider family of western European socialist parties. In what follows, I begin by situating Miliband’s work in the broader debates about the transformation of European socialism (Sections 2 and 3). I then explain how I propose to test the “experience of governing hypothesis,” using data from the Comparative Manifesto Project that covers all left-wing parties in western Europe between 1945 and 2021 and employing a novel matching technique for time-series cross-sectional data (Sections 4 and 5). I then present the main results, which show that socialist parties that have recently been in government are more likely to express support for key state institutions and their traditional goals, and less likely to express support for the labor movement (Sections 6 and 7). I also show that the size of this effect varies meaningfully over time. On the one hand, as constitutions and political systems have matured and become more widely accepted, the additional impact of governing on socialist parties’ attitudes towards them tends towards zero. On the other hand, as socialist parties’ electoral base has become less concentrated in the working class and the labor movement, the experience of governing seems to have a larger impact on their attitudes towards trade unions.

I then conclude by arguing that Miliband’s “experience of governing hypothesis” deserves a more central place in the historiography of European socialism. This quantitative confirmation of Miliband’s theory suggests that the deradicalization of socialist parties was intimately connected to their successes. As they won elections and entered into the state, they also came to be enmeshed within it. And, as Miliband (1964, 1969) predicted, becoming embedded in the institutions of the state had profound ideological consequences, not just creating a new political strategy for left-wing parties, but leaving them with a radically different identity.

The transformation of European socialism

Party-state relations

The literature grappling with the deradicalization of European socialism can be roughly divided into two camps: one that focusses on the relationship between parties and the state, and another that focusses on the relationship between parties and wider society. The idea that parties’ relationship with the state could be a source of deradicalization can be found in some of the earliest writings on the party form, with the most famous example being Robert Michels’ (1915) account of the evolution of the German Social Democratic Party. Drawing on classical elite theory and the ideas of his teacher, Max Weber (1958 [1914], 1978 [1917]), Michels argued that as parties grow, they begin to need bureaucrats and specialist leaders in order to function efficiently. But as well as supporting the goals of the party, those bureaucrats and leaders also amass skills and resources and develop a material interest in preserving their own positions of power. They will therefore acquire the means and motives to push their parties into compromises with the state and away from anything that might upset the status quo, creating an irrepressible tendency towards deradicalization: the “iron law of oligarchy.”

Within the socialist movement itself, many of Michels’ contemporaries were also arguing about the relationship between left-wing parties and the state. For revolutionary socialists, the crucial flaw in the reformist strategy of winning elections is that they would then become dependent on the state. And the state, in turn, is dependent on an economic system – capitalism – which has an inbuilt tendency towards crisis and declining rates of profit, making it impossible for a left-wing electoral party to deliver benefits to the working class in the long run (see e.g., Lenin 1902; Luxemburg 1900). There might be moments where there does seem to be space for reformism – such as the marriage of the post-war boom and Keynesianism in Europe, or when global empires allowed the European working class to benefit from the exploitation of workers overseas – but the nature of capitalism ensures that those moments will only ever be temporary and localized.

Echoes of these theories can be found in contemporary studies of the party form. Katz and Mair’s (1995, 2018) account of the “cartelization” of political parties, for example, suggests that as parties have become increasingly disconnected from civil society, they have compensated by embedding themselves more deeply in the state and becoming dependent on it in various ways (Biezen and Kopecký 2014; cf. Koole 1996). This creates a tendency towards deradicalization, with profound implications for vibrancy of the democratic system as a whole (Mair 2013). A similar theme can be seen in the literature on “inclusion-moderation,” which suggests that when parties are “included” in parliament and government they tend to “moderate” their political program (Tepe 2019). Many scholars have used this framework to explain the deradicalization of populist parties (e.g., Akkerman et al. 2016; Capaul and Ewert 2021; Rooduijn et al. 2014). The experiences of the Freedom Party of Austria (Luther 2015), the Swiss People’s Party (Bernhard et al. 2015) and the Geneva Citizen’s Movement (Bernhard 2020) have all been used to argue that after populist parties enter government, they are forced into a series of compromises with coalition partners, which blunts their populist edges and leads them to move rapidly towards the political mainstream.

However, all these accounts face a similar set of problems. First, from Michels’ “iron law” to Katz and Mair’s “cartelization,” they suggest a rigid determinism that is hard to reconcile with the very different trajectories taken by socialist parties in different parts of Europe. Second, these theories do not explain why, when confronted with intransigent coalition partners or the crises of capitalism, socialist parties would choose deradicalization rather than a more confrontational strategy.

Party-society relations

The second set of explanations for the ideological transformation of European socialism has focussed on parties’ connection to society. The central assumption here is that party elites respond to electoral pressure by shifting their ideological commitments in order to win votes (e.g., Downs 1957; Kitschelt 1994). The secular decline in left-wing parties’ vote share over time (see Figure 1 and Benedetto et al. 2020) makes it difficult to assess whether left-wing parties are really winning over voters in this way and, in fact, there is limited evidence to support the idea that rightward ideological shifts lead to durable electoral gains (Loxbo et al. 2021; Polacko 2022). But the argument that shifts in parties’ ideologies can be explained as responses to structural changes in society remains extremely powerful.

Figure 1. The decline of left-wing parties’ vote share (Western Europe, 1945–2021).

Notes: Data from the Comparative Manifesto Project. The top panel displays the election results of the UK Labour Party. In the bottom panel, each point shows the combined electoral strength of all left-wing parties (“Social Democratic,” and “Socialist and other left”) in each election in western Europe since 1945. Falling turnout means that the downward trend would be even more marked if the data tracked share of the electorate, rather than share of voters.

One version of this argument focusses on the numerical decline of the manual working class and the fragmentation of that group in terms of lifestyles and political identity (Hobsbawm 1978; Przeworski and Sprague 1986). In this account, as the power of that voting block (or the party funding apparatus associated with it, Ferguson 1995) declined, left-wing parties were forced to turn to other social groups for electoral support. They then evolved into “catch-all” parties that attempted to represent working- and middle-class groups simultaneously and transformed themselves ideologically to do so (Kirchheimer 1966).

An alternative version of the electoral-responsiveness argument focuses not on shifts in the economic base, but on the massive expansion of education over the twentieth century. This has led to the phenomenon of the “Brahmin Left,” where support for left-wing parties increasingly comes from highly educated voters and where parties are increasingly polarized along socio-cultural, rather than economic, lines (Gethin et al. 2022). One explanation for this tendency is that, while party elites have often been dominated by the highly educated, it was only as education expanded to the masses that appeals to the kind of socio-cultural liberalism traditionally correlated with education (Cavaille and Marshall 2019; d’Hombres and Nunziata 2016) became a viable electoral strategy (Shor 2020).

Both versions of the party-society explanation contain powerful insights. But, as Dylan Riley’s (2015: 183) “neo-Gramscian research program on parties” makes clear, this unidirectional model cannot do justice to the dialectical and reciprocal evolution of classes, parties, and civil society. More narrowly, it also does not explain why party elites interpreted structural changes in society in the particular way they did. According to Stephanie Mudge (2018), the key actors here are the “experts” who do the interpretive and diagnostic work of translating social shifts into new political strategies. From this insight, Mudge then develops an account of the evolution of left-wing party experts: from the “party theoreticians” of the interwar period with their backgrounds in journalism, agitation and party organizing; to the “economist theoreticians” of the post-war settlement who emerged out of the world of professional economics; and finally the “trasnationalized, finance-oriented economists, strategists and policy specialists” of the 1990s (Mudge 2018: 1–43). Each new cohort of experts had a different institutional background and so brought with it a different “ethic” or “habitus.” And it was these new ethics that led to particular interpretations of structural change and to the emergence of new ideological positions.

But Mudge’s (2018) work remains centered on the party-society dyad and does not address the fact that socialization also takes place within the institutions of the state. This is where I propose turning to Miliband (1964, 1969) for a more nuanced and less deterministic account of party-state relationships, and for insight into the workings of a crucial mechanism in the transformation of western European socialism.

Miliband and the experience of governing hypothesis

Miliband’s work on the Labour Party was part of a broader movement within the British New Left, which was concerned by the party’s lack of radicalism and keen to evaluate the prospects for more progressive forces working within it. Many of these accounts focussed on Labour’s idiosyncrasies: the gradualism and empiricism it inherited from the British trade union movement (Nairn 1964a, 1964b), its commitment to parliamentarianism and repudiation of any form of direct action (Miliband 1964), the defensive concern to protect the institution of the party at all costs (Panitch 1979), and the party’s peculiar constitution (Minkin 1978). But Miliband (1964, 1969) also drew attention to a more universal and abstract process: the way the experience of governing led to institutional socialization, which in turn led to deradicalization.

The clearest exposition of this mechanism comes in Miliband’s account of the Labour Party’s early history. This part of his narrative begins with the party joining the wartime government in 1915, as junior partners in a Liberal-Tory coalition (Miliband 1964: 47). As Miliband points out, this had a profound ideological effect because Labour politicians did not “lose the habits of mind engendered by this experience [of government] when the war came to an end” (Miliband 1964: 48). Indeed, this legacy became clear in 1924 when Ramsay MacDonald formed the first minority Labour government. The backbench Labour MP, George Lansbury, during a debate over the Labour government’s reversal of its previous policy on soldier-officer relations in the military, said “I think one of the faults of the system under which affairs are managed in this House is that men, when they accept office, are expected immediately to change their attitude towards great public questions” (quoted in Miliband 1964: 111). A founder of the immensely influential Fabian Society, Beatrice Webb, similarly noted in her diary from March of 1924 that “one of the most unpleasant features of this Government has been the willingness of convinced and even fanatical pacifists to go back on their words once they are on the Treasury Bench as Under-Secretaries for the War Services” (quoted in Miliband 1964: 111).

Despite the ignominious collapse of the first MacDonald government after only ten months in power, by 1929 Labour had become the largest party in Parliament and formed a second minority administration. Drawing on his earlier experience in government, MacDonald ignored a Labour Party conference decision that had given the party the power to appoint Cabinet and asserted his right as Prime Minister to pick his own. But whatever plans he might have had; events soon overtook them. A few months after Labour took office, news of the Wall Street Crash reached Britain and a global depression quickly ensued. There are many different accounts of Labour’s turn to austerity in this moment, and the eventual decision of MacDonald and his allies to abandon the Labour Party for a Tory-Liberal “National Government” (e.g., Howell 2002; Marquand 1977). But Miliband’s (1964: 163–79) distinctive contribution is to draw attention to the ways in which their experience of governing and socialization into the institutions of the state shaped their actions.

Alongside MacDonald in this moment was his key ally Philip Snowden, and Miliband is careful to show how both men embraced the new institutions they found themselves connected to through the offices of Prime Minister and Chancellor. As Miliband describes, Labour ministers did not “lack informed [economic] advice from friendly sources… [The party had established] a National Economic Committee, which would be the ‘eyes and ears’ of the Prime Minister on economic questions… and it included [the economist John Maynard] Keynes, [trade union leader Ernie] Bevin, [Fabian economist G. D. H.] Cole and [socialist economist R. H.] Tawney… [But] responsible Ministers were at all times more ready to listen to advice from industrialists and Treasury officials than from their own friends” (Miliband 1964: 163). Rather than a simple story of “betrayal,” what Miliband is describing here is a complex process of transformation. Many Labour figures were deeply affected by the institutions of government in which they found themselves. They were transformed by the experience of governing, and they then fought to remake the party in their own image.

There are several important points to draw out of Miliband’s experience of governing hypothesis. The first is that it focusses on governing. This marks an important difference with earlier writers who had focussed on the ways socialist politicians were absorbed into various elite social scenes (for the British case, see Owen 2007; Webb 1930). But entering government is a very particular experience (Miliband 1964: 106). Crucially, it involves socialist party elites moving out of one set of institutions (parties, trade unions, socialist organizations) and embedding themselves in the alien institutions of the executive (the civil service, central banks, government committees, the military, international trade boards, etc). This exposes them to new sources of information, new ways of thinking and, ultimately, a new habitus.

The second key point is that the experience of governing hypothesis focusses on institutional practices: the ways-of-doing and taken-for-granted assumptions that are reproduced within particular institutions (Miliband 1969: 119–145; for the famous “Treasury View” see Davis 2022; Peden 2000; Pliatzky 1989). These habits are not just sustained by individuals, but also by departmental procedures, models, and working assumptions. They form institutional practices that often outlast the individuals who implement them. (This does not mean that individuals are totally irrelevant to Miliband’s theory. Most importantly, it is assumed that party elites have the power to reshape the ideology of the party as a whole and, in the quantitative test below, it is assumed that they exert influence over what ends up in their election manifestos. In that sense, Miliband’s is a top-down theory of party change and one that privileges party elites and their response to the experience of governing.)

The third point of clarification is that Miliband’s account of deradicalization is domain-specific, with each branch of the state nurturing a potentially unique worldview and habitus. His view of deradicalization is therefore a more nuanced and multi-dimensional than in most of the social science literature (where it is normally operationalized as movement along a single left-right dimension). Instead, Miliband (1964, 1969) draws attention to the different interests, priorities, and ways of working that are embedded in different institutions. The state is therefore not seen as a monolithic entity but one that is divided and fragmented, with diverging (and at times even opposing) traditions inculcated in the Treasury, the Home Office, the military, and other branches of the state.

The fourth and final point concerns the scope restrictions of this experience of governing hypothesis. Miliband’s example is that of the British Labour Party, and the details of his account make it clear that it cannot be easily generalized beyond the world of left-wing, socialist, and social democratic parties. There are several reasons for this. At the level of ideology, liberal, conservative, and Christian democratic parties often explicitly work within the existing social and political framework. This is very different to socialism that was, at least in theory, committed to moving beyond capitalism and bourgeois democracy. In terms of personnel, those other parties also often recruit leaders who already have connections to the institutions of government, or at least do so at a greater rate than socialist parties (see Alexiadou 2022). The effects of governing on other smaller, party families (green, regional/separatist, far-right) may be pronounced, but they would need separate theorization and testing, which unfortunately lies beyond the remit of this essay.

In sum, Miliband’s work represents a distinctive contribution to our understanding of European socialism. He argues that the experience of governing poses a unique challenge for socialist parties. It embeds left-wing party elites into new institutions, where they are gradually socialized into new ways of thinking and acting. And, ultimately, it leads them to reshape and deradicalize their own parties. As a quantitative proposition, this can be formalized in two hypotheses:

H1. Socialist parties that have recently been in government will be more supportive and more ideologically aligned with the institutions of the state.

H2. Socialist parties that have recently been in government will be less supportive and less ideologically aligned with the institutions of the labor movement.

The challenge taken up in the rest of this essay is to subject these two hypotheses to systematic, cross-national testing, demonstrating the importance of Miliband’s work to scholars outside of the narrow confines of British political history.

Data

Quantitative tests of deradicalization require systematic measures of party ideology to serve as the dependent variable. At present, there are four main sources of such data: surveys of party members or supporters, surveys of party elites, expert surveys, and content analysis of manifestos. Each of these methods has its own advantages and disadvantages, and much work has been done to evaluate how well the measures produced in these different ways correlate with one another (Ecker et al. 2021; Norris 2020). But in my case, the options are somewhat limited. Surveys of party members and supporters can be discounted on theoretical grounds: the experience of governing hypothesis is explicitly top-down as it is party elites who are exposed to the practice of government and who then reshape party ideology, not grassroots members. Of the remaining options (surveys of party elites, expert surveys, and manifesto data), my choice was guided by the variables they record and their temporal/geographic coverage. Unfortunately, surveys of party elites have tended to be country-specific with limited temporal coverage, while no expert surveys include measures of parties’ embeddedness in the institutions of state (instead they tend to include more general measures of left-right, libertarian-authoritarian, and populist ideology). This leaves us with content analysis of manifestos, of which by far the most significant is the Comparative Manifesto Project (CMP) (Volkens et al. 2021).

The CMP is a large and very well-established dataset within political science, covering 67 countries, 849 elections, 1373 parties, and 5089 manifestos. The variables are produced by breaking down party manifestos into discrete statements, which are then sorted into various semantic categories. For example, the following sentence from the Democratic Party platform from 2012 can be broken down into three statements: “This approach includes tough spending cuts that will bring annual domestic spending to its lowest level as a share of the economy in 50 years [Economic Orthodoxy] // while still allowing us to make investments that benefit the middle class now [Middle Class and Professional Groups] // and reduce our deficit over a decade [Economic Orthodoxy]” (Manifesto Project Team 2018: np). Across a given manifesto, it is then possible to count the number of phrases in support of the Economic Orthodoxy, Freedom and Human Rights, Traditional Morality, and other variables. Important nuance is no doubt lost in this process. But it has the central advantage of creating a systematic, cross-national measure of party ideology, allowing researchers to easily test, for example, whether a party has made more statements in support of Welfare State Expansion than it used to or than its rivals have done.

There are two important issues with the CMP data for the purposes of this research. First, parties may say one thing in a manifesto but then go on to do something different in government. This represents a substantial limitation to my argument and cannot be discounted, although there is good evidence that most parties in mature democracies do fulfill their promises (Thomson et al. 2017). Second, what parties say in a manifesto may be driven by what they believe voters want to hear and so not accurately reflect their real ideological position. This would only be problematic if parties who were in government in the previous election cycle were more likely to misrepresent their ideologies in their manifestos (so that the measurement error was correlated with the treatment). To evaluate this possibility, I repeat the matching procedure described below but use the absolute difference between each party’s score and that of the average voter ² as the dependent variable. I find no statistically significant differences between treated and untreated groups, suggesting that parties that have recently been in government are no closer to the average voter than their rivals from the opposition (Supplementary Material A9). ³

Using the CMP to measure the ideological affinity of political parties with respect to key state institutions (the dependent variable), I operationalize my two hypotheses as follows:

H1. Socialist parties that have recently gone into government will be more supportive and more ideologically aligned with the institutions of the state, by making:

1. a. More positive references to the military and their traditional goals (CMP variable 104)

2. b. Fewer negative references to the military and their traditional goals (CMP variable 105)

3. c. More positive references to the constitution and its importance (CMP variable 203)

4. d. Fewer negative references to the constitution and its importance (CMP variable 204)

5. e. More positive references to the economic orthodoxy and institutions like the stock market and banking system (CMP variable 414)

H2. Socialist parties that have recently gone into government will be less supportive and less ideologically aligned with the institutions of the labor movement, by making:

1. a. Fewer positive references to labor groups and their traditional goals (CMP variable 701) ⁴

Full descriptions of each variable and the CMP’s coding procedures are available in Supplementary Material A2.

The independent variable is a binary indicator of whether a party has been in government at any point since the last election (thus capturing their “experience of governing”). As this is not recorded in the CMP, I linked the CMP data to two other datasets: ParlGov (Döring and Manow 2021) and Party Facts (Döring and Regel 2019). There are a handful of inconsistencies between these datasets (largely driven by decisions about how to code new parties formed out of splits), all of which were resolved manually. The governing experience of western European left-wing parties captured in this data is depicted in Figure 2.

Figure 2. Left-wing governments (Western Europe, 1900–2021).

Notes: Data from ParlGov. The shading of the tiles indicates the number of days spent under left-wing cabinets per year, weighted by the proportion of seats held by left-wing parties.

Finally, two variables are used as controls: the percentage of the vote attained by the party in that election, and their overall left-right score ⁵ at the previous election. These are both drawn from the CMP. Density plots for all key variables are displayed in Figure 3.

Figure 3. Density plots for key variables (n = 473 party-election observations).

Notes: Data from the Comparative Manifesto Project, ParlGov, and Party Facts. Sample includes all western European left-wing parties that have ever been in government (n = 37) resulting in 473 party-election observations between 1945 and 2021. Plots display density curves for each variable.

The argument in this essay focusses on left-wing parties in western Europe. ⁶ However, I also limit the quantitative analysis to left-wing parties that have ever been in government (n = 37, listed in Supplementary Material A1). This proviso is designed to better facilitate comparison across different socialist parties and assumes that small, fringe parties that have never entered government are subject to fundamentally different pressures than large, electorally competitive parties. (In any case, the main results are robust to dropping this condition and using all the data available in the CMP, see Supplementary Material A8.) The final dataset is therefore composed of 473 party-election observations between 1945 and 2021.

Methodology

Until very recently, the standard tools for quantitative analysis of this sort were two-way fixed effect (TWFE) models. However, recent methodological advances have revealed some serious problems with this approach (see de Chaisemartin and D’Haultfoeuille 2022; Goodman-Bacon 2021; Imai and Kim 2021). Crucially, the coefficients generated by TWFE regressions are not robust to heterogeneous effects over time or across units and can be easily contaminated by other treatments. The fact that TWFE models can contain negative regression weights also makes it very difficult to untangle the implicit comparison being made by the model.

Taking onboard these critiques, I borrow from and extend Imai’s et al. (2023) matching approach for time-series cross-sectional data. Their work builds on the framework of causal identification, in which the central challenge is to identify a suitable “control group” against which we can compare those who receive the “treatment.” (The language used in the causal identification literature has its origins in experimental science but is now commonly used for observational data. In essence, the “treatment” refers to the independent variable and the “outcome” to the dependent variable.) Imai et al.’s (2023) contribution is to suggest a standardized way of identifying a control group in settings with many units and repeated observations over time (time-series cross-sectional, or panel, data). Their central intuition is that each treated unit can be matched to a control group that share a similar trajectory in the independent variable up to the moment where one of them receives the “treatment.” For example, their method suggests that a suitable comparator for a country that was run by center-left governments from 1970 before switching to a far-right government in 1985 (received the “treatment”) would be a different country that was also governed by the center-left from 1970 to 1985 but that did not then switch to the far-right (control).

More formally, there are three stages to Imai et al.’s (2023) approach. In the first, each treated unit is matched to a control group of untreated units that otherwise share an identical treatment history over the recent past. Adapting one of their original examples, country A that, in 1970, transitioned to democracy after ten years of authoritarian rule (received the “treatment”) would be matched to a control group of countries X, Y, and Z that share a history of authoritarian rule from 1960 to 1970 and have not yet made the transition to democracy (not received the “treatment”). Imai et al. recommend using a series of control variables to further refine the control group and ensure that it matches the treated unit as closely as possible. Continuing the same example, countries from the control group (countries X, Y, and Z) that have a similar Gross Domestic Product and population size to the treated unit (country A) could be given greater weight in the subsequent analysis. With each treated unit matched to a control group, the final stage is to estimate the overall effect of the treatment through a difference-in-difference calculation, which compares the trajectory in the dependent variable for the treated unit with that of the control group. Developing the same example, a researcher might calculate whether the change in economic growth (outcome) after country A transitioned to democracy was greater than the change in growth for those similar countries that did not make the transition (control group). (Difference-in-difference calculations implicitly include fixed effects for time and unit and are therefore often seen as analogous to TWFE, but Imai et al.’s approach is much more robust to the kinds of problems outlined above.) Across many treated units and many control groups, Imai et al. then take a simple average of the various difference-in-difference coefficients with bootstrapped standard errors, producing a robust, non-parametric estimate of the causal impact of the treatment.

Imai et al. (2023) present their approach in the context of regular and balanced panel data (e.g., observations for every year and every country). But their framework is clearly generalizable to other settings. In the case of socialist parties competing in elections, elections rarely take place in the same year across countries, while parties are continually emerging and disappearing. I have therefore adapted their framework to accommodate irregular panel data and developed an R package to allow other researchers to follow this more general approach in other empirical settings (Tiratelli 2024). The R package implements the following four-stage procedure:

1. 1. Defining the contemporary time-period: This is the only addition to Imai et al.’s (2023) original approach and involves matching each treated observation to all untreated observations occurring within a user-specified period. In this case, I focus on socialist parties competing in an election within a five-year window. ⁷

2. 2. Exact matching: Find a subset of those contemporary, untreated observations that have the same treatment history over the last n observations. In this case, I match parties who been out of government over the last three elections cycle. ⁸

3. 3. Refinement matching: To control for potential confounders, Imai et al. recommend further matching using propensity scores, covariate balancing propensity scores or Mahalanobis distance. In the first two cases, they can be used to produce weights that are used to calculate a weighted mean for the control group (giving more weight to more similar cases). All three can also be used to limit the size of the control group to the n most similar observations. In the analysis below I control for two variables (standardized vote share this election and standardized overall right-left position in the previous election) and set n to five.

4. 4. Estimate treatment effects: For each matched and refined set j: $\widehat {{b_j}} = \Delta T\; - \overline {\Delta U} $ ., where $\Delta T$ is the change in outcome for the treated unit and $\overline {\Delta U} $ is the (weighted) mean change in outcome for the control group. The R package then takes the average of those difference-in-difference coefficients as the final estimand: $\hat \beta = \mathop \sum \limits_{j = 1}^n \widehat {{b_j}}\;/n$ . Standard errors are calculated using a block bootstrap procedure (resampling across parties with 1000 iterations).

To make my procedure more concrete: when the Socialist Workers’ Party of Luxembourg competed in the 1989 election, it had just come out of a five-year period in government (i.e., it had received the “treatment”). I therefore match it to a control group of other socialist parties who competed in elections between 1987 and 1991 from opposition, and that otherwise share a similar history of being in and out of government over the previous three elections (in this case, the matches are: the Finnish People’s Democratic Union, the Icelandic People’s Alliance, the Icelandic Social Democratic Party, and the Portuguese Socialist Party). This matching process is then repeated for every “treated” unit, producing 59 matched sets. I then refine the control groups by controlling for vote share and overall right-left position and, finally, calculate the change in the dependent variables for the treated and control groups from the last election to this one. For example, I compare the change in the economic orthodoxy score (H1e) from 1984 to 1989 for the Socialist Workers’ Party of Luxembourg, to the (weighted) average change for the four parties in the control group.

The average trajectories for positive mentions of the economic orthodoxy (H1e) are depicted in Figure 4. In the first period (pre-treatment), none of the parties had been in government in the previous period and the treated and control groups both have very similar scores. But by the second period (post-treatment), there is a clear divergence, with parties who have been in government on average making far more positive mentions of the economic orthodoxy than those who remained in opposition. The difference between these two average trajectories is captured by the final difference-in-difference coefficient, which gives a robust estimate of the impact of the experience of governing on party ideology.

Figure 4. Average economic orthodoxy score for treated and control groups.

Notes: Data from the Comparative Manifesto Project, ParlGov, and Party Facts. Sample includes all western European socialist parties that have ever been in government (37) resulting in 473 party-election observations between 1945 and 2021. The analysis follows Imai et al. (2023): I match each treated observation with untreated observations that were also involved in an election within a 5-year time window and that have the same treatment history over the last 3 election cycles. This reduces the effective sample size to n = 234 with 59 matched sets. Further matching was then conducted to control for standardized vote share this election and their overall right-left position (lagged and standardized), using covariate balancing propensity score weights to produce weighted averages for the control group.

Results

Descriptive results

The CMP data provides several illustrations of socialist parties who go into government and are soon deradicalised by the experience. One telling example concerns the communist-adjacent Icelandic People’s Alliance. In the late 1980s, the People’s Alliance served in Steingrímur Hermannsson’s coalition government, which was dominated by the agrarian Progressive Party. After this fairly unusual spell in power, the People’s Alliance manifesto for 1991 included an unprecedented number of positive mentions of the economic orthodoxy (its standardized CMP score increased from −0.4 to 3.4, confirming H1e). This marked a considerable departure from other People’s Alliance manifestos across the 1980s and into the 2000s. It is also unusual when compared to other socialist parties competing in elections in that era.

Another example shows positive references to the labor movement falling after parties serve in government (H2a). In the late 1970s, the far-left Finnish People’s Democratic Union (SKDL) was twice called on to prop up the Social Democratic government of Kalevi Sorsa. In 1979, after serving in two coalitions over the five-year parliament, the new SKDL manifesto made almost no reference to trade union movement, which had previously sustained this communist-dominated alliance (the standardized CMP score dropped from 0.9 to −0.4).

There are, of course, also counterexamples. To take just one, when the Belgian Socialist Party formed a government in 1954, after a rare defeat for the dominant Christian Social Party, their manifesto was strongly pro-military. But four years later, after leading a “purple” coalition with the Liberal Party, the Belgian socialists’ new manifesto made almost no positive references to the military and to external security (the standardized CMP score fell from 2.5 to −0.7, contradicting H1a). Instead, the election was fought on the anti-clerical agenda pursued by the purple coalition and the broader relationship between church and state.

As the Belgian example shows, contingent factors and idiosyncratic party histories matter enormously. But the quantitative analysis presented here aims to trace out the common patterns behind that variation and, in particular, to identify the average effect of serving in government on party ideology.

Quantitative results

The main results of the matched difference-in-difference analysis are shown in Figure 5. Across six different refinement methods (listed above in Methodology), I find that left-wing parties that were in government in the previous period made more positive references to the military, to the constitution, and to the economic orthodoxy (H1a, H1c, He). They also made fewer positive references to labor groups and their goals (H2a). The results are all presented as standardized coefficients, which means that being in government moves a party’s ideological position by about half a standard deviation in all four cases. This suggests that the experience of governing does lead to deradicalization: it turns socialist parties away from their allies in the organized labor movement and renders them more supportive of the military, the constitution, and the traditional economic orthodoxy. However, the results for negative references to the military and of constitutionalism are not statistically significant (H1b, H1d). One plausible interpretation is that support and criticism are not symmetrical forms of political rhetoric and so are subject to different causal processes, but further research would be needed to confirm that speculation.

Figure 5. The effect of the experience of governing on socialist party ideology.

Notes: Data from the Comparative Manifesto Project, ParlGov, and Party Facts. Sample includes all western European socialist parties that have ever been in government (37) resulting in 473 party-election observations between 1945 and 2021. The treatment is a dummy variable indicating whether the party has been in government since the last election. The analysis follows Imai et al. (2023): I estimate the average treatment effect on the treated by comparing each treated observation with untreated observations that were also involved in an election within a 5-year time window and that have the same treatment history over the last 3 election cycles. This reduces the effective sample size to n = 234 with 59 matched sets. Further matching was then conducted to balance the party’s standardized vote share during the current election and their overall right-left position (lagged and standardized). I present results using five such matching methods: propensity score weights/matches, covariate balancing propensity score weights/matches, and Mahalanobis distance matches. Coefficients are produced via a difference-in-difference estimator with block bootstrapped standard errors. Points represent standardized coefficients. Lines represent 95 per cent confidence intervals. Gray lines indicate non-significance.

These results are robust across a variety of alternative specifications (see Supplementary Material A4–8). Daniel Ho et al. (2007) recommend using matching procedures as a preprocessing technique before parametric estimation. In that spirit, I use the matched data in a weighted, OLS regression and control for country-level fixed effects, standardized vote share, and (lagged and standardized) overall right-left position with cluster-robust standard errors. This produces extremely similar results. Repeating the OLS approach with a continuous treatment variable (number of years in government) again produces very similar findings, although the coefficients for economic orthodoxy are no longer significant at the 95 per cent level. The results are also robust to changing the parameters of the matching procedure. Relaxing the exact matching criteria by only using two lags of the treatment variable produces extremely similar results. Making it stricter by using four treatment lags produces similar coefficient estimates but, in part because of reduced sample size, the results are not significant at the 95 per cent level. Finally, the findings are robust to using a 10-year time window, the full sample of all west European left parties contained in the CMP, and a traditional TWFE model without matching (Supplementary Material A17).

I also conduct a battery of placebo tests to demonstrate the soundness of the research design (Eggers et al. 2021). First, I deploy a series of placebo treatment tests, which involve repeating the main analysis but replacing the independent (treatment) variable with something that should theoretically have no effect on the outcome (the equivalent of the sugar pill in classical medical experiments). I try three such placebo treatments: (i) the second lead of the main treatment variable (following the simple logic that future events cannot affect the past), (ii) random reassignments of the original treatment variable (repeated 1000 times), and (iii) reversing the treatment condition to look at the effect of a party not being in government. In all three cases, the tests are successful and these placebos (which, by design, should not have any effect on the dependent variables) return no significant results. Next, I implement a placebo population test, where one repeats the core analysis on a population that theory suggests should not be affected by the treatment. For reasons explained above, I repeat the analysis looking at liberal, conservative, and Christian democratic parties and, as expected, find that the experience of governing does not seem to affect those kinds of parties (i.e., I find no statistically significant results). Taken together, these various placebo tests suggest that the research design is sound and that the effects shown in Figure 5 are not simply artifacts of my methodological choices but reflect real patterns in the underlying data (Supplementary Material A11–14).

Variation over time

Although these results seem to hold for large and small, as well as radical and moderate socialist parties, there are reasons to think the significance of Miliband’s arguments might have changed over time. As is well known, socialism has not stood still in Europe over the past seventy-five years and left-wing parties have been buffeted by powerful historical headwinds. The most significant and universal of these were the broader ideological reorientation of European politics away from Keynesianism towards other forms of economic management (Harvey 2007), the eclipse of the radical, anti-system politics of revolutionary socialism (Moschonas 2002), and the decrease in socialist parties’ reliance on working-class voters and the institutions of organized labor (Marks et al. 2023). Focussing on those variables where I observe a significant main effect, I therefore hypothesize that (i) the shift away from anti-system politics will have reduced the effect of the experience of governing on how socialist parties relate to the constitution, the economic orthodoxy, and traditional military themes (i.e., the coefficients for H1a, H1c, and H1e should tend towards zero over time), and (ii) that their increasing distance from the working class and trade unions will have increased the effect of the experience of governing on socialist parties’ attitudes towards the labor movement (i.e., the coefficient for H2a should get stronger over time).

It is possible to test this hypothesis by comparing the individual difference-in-difference estimates from each of the 59 matched sets (each $\widehat {{{{b}}_{{j}}}}$ coefficient). The results shown in Figure 6 and Table 1 partially confirm my theoretical expectations. The effect of governing on positive mentions of the constitution tends towards zero over time (and indeed the variance of ConstPos also decreases), indicating that constitutions have become more widely accepted by socialist parties over this period and that there is therefore less room for the experience of governing to have an effect. The opposite is true of the relationship to the labor movement, where the negative impact of governing has grown stronger, particularly in the last ten years. This suggests that the experience of governing has, over the decades, become a more powerful force in shaping how socialist parties relate to the labor movement.

Figure 6. Variation in effect size over time.

Notes: Points are individual difference-in-difference estimates for each matched set, plotted against the election year of the treated observation. Matching is done using only the exact matching procedure described above.

Table 1. Linear bivariate models for effect size over time

Notes: Linear model Y = X + e, where Y is the difference-in-difference coefficient for each matched set, X is a linear time trend set to 0 in 1945, and e is the error term.

** p < 0.01.

*** p < 0.001.

However, there is no trend for the effect on attitudes towards the military or the economic orthodoxy, implying that those remain live areas of debate and issues for which the experience of governing still matters today.

Alternative mechanisms and limitations

Miliband’s (1964, 1969) experience of governing hypothesis foregrounds institutional socialization as the key causal mechanism linking serving in government with deradicalization. But scholars of populism have suggested an alternative explanation: the necessity of compromising as part of a coalition (e.g., Bernhard 2020; Capaul and Ewert 2021). If true, this alternative theory would imply that the causal effect will be smaller for parties that have less need to compromise. To test this, I mirror the analysis in Figure 6 and Table 1 and compare the individual difference-in-difference estimates for (a) parties ruling on their own vs those in coalition, and (b) parties that hold the position of prime minister vs those that don’t. This analysis produces no statistically significant results and so supports the original theory that institutional socialization is driving the associations seen in Figure 5 (Supplementary Material A15–16).

Another alternative explanation could be extrapolated from work on “cartelization.” If governing parties are more dependent on the state for resources than opposition parties, the former might also be more likely to adopt the ideologies of the state for purely instrumental reasons, rather than because of socialization. However, this argument runs against a central tenet of Katz and Mair’s (1995) original thesis, which was that the spoils of power are shared out more widely in a cartelized system than in a genuinely competitive one. Cartelization should therefore have reduced the impact of governing on party ideology, whereas my results suggest that this is only true for one dimension of deradicalization (parties’ relationship with the constitution). This is not decisive evidence against a focus on state resources but, on balance, the argument for socialization seems more strongly supported.

There are, however, important limitations to the results presented here. First, the analysis focusses on the effect across one electoral cycle, and it is not easy to determine how long those effects persist. In part, this is a question of methodology. Difference-in-difference designs (in fact most causal identification strategies) tend to be better at detecting credible causal effects over shorter time periods. This might therefore be an area that benefits from careful historical work and in-depth case studies. Second, I was unable to detect any variation by institutional context. As I argued above, this supports my focus on socialization rather than coalition-building as the key causal mechanism. But scholars may in future want to examine whether the experience of governing is the same for presidential and parliamentary executives, or for proportional representation and first-past-the-post systems. Third, a related area for future research concerns parties’ internal structures. One hypothesis, which follows Seymour Martin Lipset et al.’s (1956) suggestion that robust internal democracy allows organizations to evade the “iron law of oligarchy,” is that the effect would be smaller in more democratic party structures. But there are also other organization-level variables that could be examined, such as funding structures and the degree of party members’ control over elected officials.

Conclusion

Most modern accounts of the evolution of socialism have focussed on parties’ relationships with society, whether that means the electorate, networks of policy experts, or organizations capable of funding political action (Ferguson 1995; Mudge 2018; Przeworski and Sprague 1986). But, as an earlier generation of scholars (e.g., Michels 1915) pointed out, parties’ relationship with the state also matters. For Miliband (1964, 1969), the crucial mechanism is the way that the experience of governing forces socialist party elites into a new institutional context, one that has profound effects on their beliefs and, through them, on the ideological positions of the parties they lead. Testing this link between the experience of governing and deradicalization against CMP data from all western European socialist parties from 1945 to 2021, I find strong evidence in support of Miliband’s theory. Compared to similar parties that were in opposition, parties that have recently been in government tend to express more positive attitudes towards the military, constitution, and economic orthodoxy, while being less supportive of the trade unions. These effects also vary meaningfully over time. As political constitutions have become more universally accepted, the additional impact of being in government has tended towards zero. Meanwhile, as socialist parties have become less reliant on the organized labor movement, the effect of serving in government on their attitudes towards trade unions has grown more pronounced.

So, what does this confirmation of Miliband’s experience of government hypothesis tell us about the wider history of European socialism? The first lesson is that the deradicalization of socialist parties was in some ways a by-product of their successes. As left-wing parties won elections and entered into government, they began to imitate the patterns of behavior that characterized their predecessors. So rather than representing a radical challenge to the established logics of statecraft, they found themselves learning from and being socialized into the existing institutions of the state. The second lesson is that this process of deradicalization was and is domain specific. Socialist party elites discovered a different habitus in each of the various branches of the state and, while scholars cannot ignore the general ideological shift from left to right, neither should they forget about the particular ethics and ways-of-thinking that characterize particular institutions. The third lesson is that, while Miliband’s thesis clearly has applicability beyond his original case study of the British Labour Party, the experience of governing will affect different parties in different ways at different moments in history. This suggests that rather than searching for one master variables that explains the evolution of European socialism, scholars should instead try to provide a more comprehensive account of the many different pressures weighing on those parties. The experience of governing was, in that sense, yet another factor pulling left-wing parties away from their distinctive and radical origins, and one which the mainstream of historical and social scientific scholarship has ignored for too long.