Mini-Publics, (Lack of) Representativeness, and Legitimacy Beliefs

Abstract

The participants in deliberative mini-publics are typically randomly selected; therefore, mini-publics are often marketed as representative of the wider population. However, in practice, mini-publics are unlikely to be fully representative due to their small size and non-response bias. I report the results of a pre-registered survey experiment designed to assess the implications of deviations from statistical representativeness for citizens’ legitimacy beliefs (N = 1,308). Consistent with prior research, I find that the involvement of a mini-public in democratic decision-making can lead to substantial increases in perceptions of process legitimacy; however, even minor biases in the composition of mini-publics substantially decrease those gains while larger biases can wipe them out entirely. The results of this study temper hopes that mini-publics offer an easy fix to perceptions of low democratic legitimacy.

Introduction

According to a recent data collection, more than 500 deliberative mini-publics have been held since the turn of the millennium (OECD 2021). A key reason for their increasing popularity is that mini-publics are often seen as a promising response to several worrying trends, including waning political trust, the rise of anti-pluralist actors, and affective polarization (for example, Diamond 2015; Fukuyama 2018; Levitsky and Ziblatt 2018). ¹ More specifically, advocates have suggested that mini-publics could help to revitalize modern democracies by increasing the argumentative quality in political debates and descriptive representation. As a result, mini-publics may increase the quality of policy outputs and strengthen perceptions of democratic legitimacy (Cohen 1989; Dryzek et al. 2019; Goodin and Dryzek 2006). Yet, to what extent mini-publics live up to those promises remains unclear.

A major social scientific effort is currently underway to better understand the possible contributions of mini-publics. Traditionally, this literature has concentrated on the participants in mini-publics, with studies focusing on questions such as the extent to which ordinary citizens are willing (Jacquet 2017; Neblo et al. 2010) or competent (Gerber et al. 2018; Muradova 2021; Niemeyer et al. 2024) to participate in political deliberation. However, in recent years there has been a growing recognition that it is important to extend the focus beyond the relatively small number of people who participate in mini-publics and (also) study how members of the broader public react to them (van der Does and Vincent 2023). Heeding this call, recent studies have investigated citizens’ preferences regarding the design of mini-publics (Christensen 2020; Goldberg and Bächtiger 2023) and the extent to which hearing about their recommendations influences policy support (Boulianne 2018). Furthermore, several studies reported promising evidence suggesting that mini-publics tend to increase political trust and legitimacy beliefs (Boulianne 2018; Jacobs and Kaufmann 2021; Werner and Marien 2022). However, this literature has tended to study a somewhat idealized notion of mini-publics rather than mini-publics as they actually unfold in practice. An important exception is two recent studies that have considered how people react if policy recommendations by mini-publics are subsequently not implemented – as often happens in reality (Font et al. 2018). Their findings suggest that legitimacy gains can vanish (Germann, Marien, and Muradova 2024) or even reverse (van Dijk and Lefevere 2023) if the policy recommendations of mini-publics are not honoured. In this research note, I contribute to this line of research by studying a second possible reason why citizens may come to reject mini-publics: deviations from statistical representativeness.

According to the standard academic definition, the participants in mini-publics should be randomly selected through a ‘civic lottery’ (Curato et al. 2021, 3f; also see Lafont 2015). However, despite frequent claims to the contrary, random selection cannot guarantee representativeness, for two main reasons. First, most mini-publics are relatively small, with typical numbers ranging from somewhere around a dozen to 250 participants (OECD 2021). As is well-established, small samples such as these inevitably have large error margins and are likely to feature substantial sampling biases (Peixoto and Spada 2023). Second, experiences from around the world suggest that citizens frequently turn down invitations to participate in mini-publics (OECD 2021). For example, only 3 per cent of those invited to participate in Belgium’s G1000 mini-public agreed to attend, and only 70 per cent of those selected for participation actually showed up (Caluwaerts and Reuchamps 2015, 159). For the Citizens’ Assembly on Electoral Reform in British Columbia, the corresponding figures were 7 per cent and 63 per cent (Fournier et al. 2011: 32), and for the Dutch Civic Forum, 8 per cent and 52 per cent (Renwick et al. 2017, 26). Since willingness to participate in mini-publics is unlikely to be randomly distributed (Jacquet 2017; Neblo et al. 2010), they are likely to suffer from substantial non-response bias. ² In recognition of these basic statistical facts, the organizers of mini-publics increasingly rely on stratified forms of sampling in an effort to counter resulting sampling biases. Unfortunately, though, it is not possible to stratify based on more than a small number of strata in small samples (Peixoto and Spada 2023). Even with stratified sampling, mini-publics are therefore unlikely to perfectly mirror the wider population. Accordingly, deviations from the representative ideal have been relatively widely reported in the literature. For example, existing evidence suggests that mini-publics frequently over-represent citizens with high educational attainment, older citizens, and males (Farrell et al. 2021; Font and Blanco 2007; Fournier et al. 2011; French and Laver 2009; Goidel et al. 2008; Griffin et al. 2015).

Deviations from statistical representativeness could have significant consequences for citizens’ legitimacy beliefs. Scholars of descriptive representation have long argued that perceptions of political legitimacy crucially depend on the extent to which representative bodies collectively mirror the society they are meant to represent (Mansbridge 1999; Pitkin 1972). As is well-established, real-world legislatures often fare poorly in that regard (Giger, Rosset, and Bernauer 2012). If mini-publics are representative of the wider population, they could therefore help to increase legitimacy perceptions among the broader citizenry (H1). However, if mini-publics end up reproducing the same, or similar, biases as elections and referendums, this beneficial scenario may not come to pass. Recent empirical studies suggest that citizens see legislative committees or international fact-finding missions as significantly less legitimate if they underrepresent groups such as women or ethnic minorities (Arnesen and Peters 2018; Chow and Han 2023; Christensen et al. 2023; Clayton, O’Brien, and Piscopo 2019; Kao et al. 2024). Analogously, legitimacy gains may decrease if mini-publics are not fully representative (H2), especially when representative biases are large (H3). Indeed, a severely unrepresentative mini-public could even backfire and reduce legitimacy perceptions rather than increase them.

That said, it is also possible that deviations from the representative ideal are much less consequential for citizens’ legitimacy beliefs. While some proponents of deliberative mini-publics emphasize the promise of improved descriptive representation (for example, Dahl 1989; Pow, van Dijk, and Marien 2020), others see their most important benefit in the space that is provided for the careful deliberation of policy issues (Cohen 1989; Dryzek et al. 2019; Fishkin and Mansbridge 2017). And, critically, a common view among deliberation scholars is that while good deliberation requires that all initial viewpoints are represented, it does not require statistical representativeness (Mansbridge 1999). To the extent that citizens similarly prioritize deliberative quality in their process evaluations, mini-publics could make substantial contributions to perceptions of political legitimacy even in the absence of full representativeness. Indeed, as long as mini-publics provide a space for high-quality deliberation, deviations from the representative ideal may not be harmful to legitimacy perceptions at all.

This research note provides the first empirical test of the implications of deviations from statistical representativeness for citizens’ perceptions of deliberative mini-publics. It does so based on a scenario experiment (N = 1,308) in the Republic of Ireland, a country with recent experience with several high-profile mini-publics. Following previous studies (Germann, Marien, and Muradova 2024; Jacobs and Kaufmann 2021; van Dijk and Lefevere 2023; Werner and Marien 2022), the experiment involves a fictitious political decision-making process that randomizes whether or not a mini-public is involved in democratic decision-making. Yet, going beyond previous studies, I also randomize the extent to which a mini-public is representative of the electorate. Specifically, I consider two types of representative bias: first, a bias in terms of the mini-public’s demographic composition; and, second, an attitudinal bias in terms of participants’ initial viewpoints on the policy at stake. In keeping with prior research, I find that mini-publics can substantially increase perceptions of democratic legitimacy; however, those gains are cut roughly in half in the presence of minor deviations from representativeness, while larger biases can wipe them out entirely.

Experimental Design

The experiment took the form of a single-factor, web-based scenario experiment in which subjects were asked to read and rate a hypothetical political decision-making process. Scenario experiments combine the internal validity of randomized controlled trials with the external validity of surveys (Werner and Marien 2022). Prior research on the legitimacy of political decision-making mechanisms suggests good correspondence between findings in scenario experiments, field experiments, and observational studies (for example, Esaiasson, Gilljam, and Persson 2012; Esaiasson et al. 2019; Olken 2010; Torgler 2005). The hypotheses, design and analysis were pre-registered with AsPredicted (#111066). ³ Minor departures from the pre-analysis plan are reported in SI Appendix §1. The vignette, outcome questions, and other survey materials can be found in SI Appendix §2.

Location

Participants in scenario experiments should have basic familiarity with the concepts involved, as this is likely to strengthen generalizability to real-world behaviour (Schmuckler 2001). Hence, I chose to conduct my experiment in the Republic of Ireland. Mini-publics have become increasingly common in recent years in a range of countries, but they often have limited public visibility (Germann, Marien, and Muradova 2024; Setälä 2011). Ireland is different due to its recent experience with several high-profile, government-sponsored mini-publics including the 2013–2014 Irish Constitutional Convention, the 2016–2018 Irish Citizens’ Assembly, the 2020–2021 Assembly on Gender Equality, the 2022 Dublin Citizens’ Assembly, the 2022 Assembly on Biodiversity Loss, and the 2023 Assembly on Drugs Use (Farrell, Suiter, and Harris 2019). ⁴ Survey evidence suggests that a majority of Irish voters have basic familiarity with the concept of a mini-public, including the idea of sortition (Elkink et al. 2020), and 88 per cent of the participants in the present study indicated they had previously heard of the concept of a mini-public. According to a recent study, the 2016–2018 Irish Citizens’ Assembly was significantly skewed in favour of the middle and upper classes and over-represented citizens with high educational attainment (Farrell et al. 2021). Perhaps, surprisingly, these results have however not been widely discussed in Irish media. Mini-publics with varying levels of representativeness should, therefore, appear plausible to experimental subjects.

Sample

The sample includes 1,308 Irish residents aged 18 or older. The data was collected through an online access panel (Ireland Thinks) in November–December 2022. Quotas were used to broadly match the sample to the population in terms of age, gender, and region of residence. The sample size was informed by an a priori power analysis (see SI Appendix §3). SI Appendix §4 includes descriptive statistics.

Scenario

Respondents were first introduced to the policy matter at stake: whether or not Ireland should introduce a universal basic income scheme (UBI). Two major Irish parties (Fianna Fáil and the Green Party) have proposed UBI schemes in recent years and, similar to other European countries, the pros and cons of UBI schemes have been widely debated in Ireland. The specific proposal mentioned in the vignette was modelled after proposals made in the Irish context and involved weekly payments to every adult of €200 (plus an additional €30 per dependent child). The proposal suggested that this scheme would replace most other welfare benefits, and be paid for by increasing taxes. To date, the introduction of UBI has not been debated by an Irish mini-public, nor has there been a vote on UBI in the Irish parliament or by Irish citizens in a referendum. Survey evidence suggests that Irish voters are about equally split on the issue, with 55 per cent supporting and 45 per cent opposing such a scheme, according to the 2016 edition of the European Social Survey (ESS). In the current sample, similar shares of respondents supported and opposed the scheme (52 per cent vs 48 per cent). Different decision processes and decision outcomes should, therefore, be plausible to experimental subjects.

Existing literature suggests that it is important to raise the saliency of procedural considerations in procedural justice experiments (van den Bos 2001). Hence, respondents were asked to consider that the decision to introduce the UBI scheme could be made in different ways after the introduction of the policy issue. In particular, respondents were told that some people argue that the issue should be referred to a mini-public before a final decision is made by the Irish parliament. As part of this discussion, respondents were also reminded about the basic features of a mini-public, including that participants are randomly selected. Broadly similar prompts have been used in several prior studies (for example, Esaiasson et al. 2019; Germann, Marien, and Muradova 2024).

Next, respondents were randomly assigned to one of six experimental conditions (see Fig. 1). When evaluating democratic innovations, it makes sense to draw comparisons with the institutional status quo (Werner and Marien 2022). Respondents in the baseline condition were therefore informed that the Irish parliament made the decision without the prior involvement of a mini-public. Respondents in the second condition were told that the issue was referred to a mini-public before the final decision and that the mini-public closely mirrored the Irish adult population. In the remaining four conditions, respondents were similarly told that a mini-public was involved in the decision-making process; however, the mini-public featured different types and amounts of sampling bias.

Figure 1. Experimental design.

First, the respondents were informed that the mini-public either had a demographic or an attitudinal bias. Demographic bias, on the one hand, refers to a situation in which a mini-public differs from the population in terms of socio-demographic factors, such as class, age, or gender (Fishkin 2018a). For this experiment, I chose to focus on educational attainment since education is one of the most consistent predictors of participation in mini-publics (Jacquet 2017). While it would have been interesting to include additional demographic factors, this would have undercut statistical power (see SI Appendix §3). I therefore leave comparisons of the consequences of different types of demographic biases to future research. To maximize experimental realism, people with high educational attainment (that is, a university degree) were always over-represented in conditions with demographic bias.

On the other hand, attitudinal bias refers to a situation in which supporters or opponents of a policy are over-represented in a mini-public (Fishkin 2018a). Since citizens are most likely to react negatively when people from the ‘other’ side are over-represented, all conditions involving attitudinal bias featured a scenario in which people with different initial viewpoints on the policy (from the perspective of the experimental subject) were over-represented. For example, if a subject suggested that they support the basic income scheme, they were informed that opponents of the scheme were over-represented. Setting up the experiment in this way captures the litmus test for mini-publics in terms of representative bias.

Finally, respondents were told that there was either a small or a large representative bias. To fixate understandings, respondents were informed about the exact numerical deviations from population figures (the latter were drawn from census and survey (ESS) data for educational attainment and policy support, respectively). More specifically, the conditions with a small bias featured a 5 percentage point deviation from population figures, while the conditions with a large representative bias featured a 20 percentage point deviation. Deviations of 5 to 20 percentage points from population figures have been commonly reported in the literature on mini-publics (Font and Blanco 2007; Fournier et al. 2011; French and Laver 2009; Goidel et al. 2008; Griffin et al. 2015), including in the Irish context (Farrell et al. 2021). Therefore, they constitute realistic figures.

After reading about the process, the respondents were informed about the decision outcome. In all conditions, parliament took the final decision and, critically, the final decision was always counter to the respondents’ stated policy preference. Analogously, if there was a mini-public, it always recommended in favour of the UBI scheme if respondents were opposed, and against the UBI scheme if respondents were in favour. I focus on the acceptance of negative political decisions because decision winners are likely to see political decision as legitimate irrespective of how they come about (Christensen, Himmelroos, and Setälä 2020; Esaiasson et al. 2019). The study of legitimacy perceptions should, therefore, focus primarily on decision losers (Werner and Marien 2022).

Outcomes

I assess the effects of my treatments on (1) respondents’ evaluations of the fairness of the decision procedure and (2) their willingness to accept the decision. Procedural fairness and decision acceptance are both important components of political legitimacy and can contribute to broader, system-level perceptions of political legitimacy (Tyler 2006). I measure procedural fairness evaluations using three survey questions (for example, ‘How fair do you think matters were when the decision was taken?’) and decision acceptance using two survey questions (for example, ‘How willing are you to accept the decision?’). Respondents provided their answers on scales from 0 to 10. All outcome questions have been used in the same or similar form in prior studies (Esaiasson et al. 2019; Germann, Marien, and Muradova 2024). I aggregate the five outcome questions into two summated rating scales. Both the procedural fairness and the decision acceptance scales can be considered unidimensional (Loevinger’s H = 0.88 and 0.77, respectively) and have high scale reliability (α = 0.95 and 0.87, respectively). SI Appendix §5 contains all item wordings and the complete results of the scaling analysis.

Results

Figure 2 shows the results of linear regressions estimating the differences between experimental conditions in terms of (1) procedural fairness evaluations and (2) decision acceptance. The results suggest that, in line with H1, the involvement of a mini-public that closely mirrors the population increases people’s perceptions of process fairness (+0.17, p < 0.001) and decision acceptance (+0.17, p < 0.001). These are substantially sized effects that imply increases of around 60 per cent of a standard deviation compared to a ‘standard’ political process in which parliament decides without the prior involvement of a mini-public. However, consistent with H2, the legitimacy gains decrease, or even vanish entirely, if mini-publics are biased in their composition.

Figure 2. Main results.

Note: All results are based on linear regression. The spikes represent 95% confidence intervals. N = 1,308. For the complete numerical results see Table S5 in SI Appendix §6.

Indeed, relatively minor demographic or attitudinal biases are sufficient for legitimacy gains to drop substantially. If holders of a university degree are over-represented by 5 percentage points, fairness perceptions increase by only 0.09 when a mini-public is involved (p < 0.001) and decision acceptance by just 0.05 (p = 0.06). The results for attitudinal bias are similar: if the mini-public over-represents citizens with a different initial viewpoint on the UBI scheme by 5 percentage points, legitimacy gains are cut roughly in half (+0.08, p < 0.01). According to Wald tests, the reductions in legitimacy perceptions compared to the representative mini-public are statistically significant at or below the 0.1 per cent level (see Table S6 in SI Appendix §6).

Larger, but still realistic, representative biases can have even more pronounced consequences. Most strikingly, if citizens with a different initial viewpoint on the UBI scheme are over-represented by 20 percentage points, subjects no longer see a process as more legitimate if a mini-public is involved. In fact, at −0.01, the point estimates are even slightly negative, though they cannot be statistically distinguished from zero (p > 0.50). In keeping with H3, a large over-representation of university degree holders also further reduces legitimacy perceptions. However, according to Wald tests, only the legitimacy reductions resulting from a large attitudinal bias are statistically significant (see Table S6 in SI Appendix §6). ⁵

Robustness Checks

A possible concern with intention-to-treat effects is that not all experimental subjects pay attention to experimental treatments and other information. The experiment included several factual manipulation checks to assess respondents’ levels of attentiveness (Kane and Barabas 2018). The results suggest that 86 per cent of all respondents correctly remembered the final decision, while 85 per cent of respondents exposed to a mini-public correctly remembered its policy recommendation. A somewhat lower share (67 per cent) correctly remembered the exact experimental condition they were assigned to. However, most of the erroneous recollections were relatively minor. For example, 93 per cent of subjects correctly remembered whether there was a mini-public, and 87 per cent of those exposed to a mini-public correctly recalled whether there was some form of representative bias. Overall, those values suggest acceptable levels of attentiveness. For descriptive purposes, I re-ran both regression models while dropping subjects who incorrectly recalled the exact decision process and outcome. The results remain similar (see SI Appendix §7).

I report several additional robustness checks in SI Appendix §7. First, I re-estimate both models including additional predictors of legitimacy perceptions (respondents’ age, gender, educational attainment, past vote choice, political trust, and satisfaction with democracy). Second, I restrict the analysis to respondents who reported that they had heard of the concept of a mini-public before taking part in the experiment. Third, I dropped speeders who rushed through the survey. Finally, I restrict the analysis to the first 1,200 respondents and, therefore, the number of respondents stated in the pre-registration (I received 1,308 responses despite requesting 1,200). The results are always similar.

Finally, the experimental treatments are not fully equivalent for (1) respondents with and without a university degree and (2) respondents who support or oppose the UBI scheme. In SI Appendix §8, I explore whether the treatment effects differ depending on respondents’ educational attainment and policy support. The results suggest that supporters of the basic income scheme are significantly more appreciative if a mini-public is held. A possible explanation is that UBI supporters feel generally less well-represented by parliament. Besides this, the results are comparable; that is, legitimacy perceptions decrease among both supporters and opponents of the basic income scheme if the composition of the mini-public is biased. Finally, I find no evidence for effect heterogeneity due to educational attainment. This is an interesting finding since it suggests that people with high educational attainment prefer proportional representation even if the counterfactual is that people ‘like them’ are over-represented.

Conclusion

Political legitimacy is often studied in normative terms. However, legitimacy fundamentally constitutes a belief, and scholars have, therefore, increasingly started to study empirical determinants of legitimacy beliefs. One of the key findings of this literature has been that citizens see political decision-making as more legitimate if a mini-public is involved in the decision process. This study was able to replicate this result; however, it also demonstrated a caveat : the gains in legitimacy perceptions decrease substantially if mini-publics are not fully representative of the population they are meant to represent, and can even vanish entirely when the deviations are large.

This is an important finding. In practice, representativeness is difficult to achieve due to the small size of most mini-publics and non-response bias. Perhaps in part due to this, advocates of mini-publics sometimes downplay the role of representativeness while highlighting other possible benefits, such as improvements in debate quality and better policies. However, while these are all important promises in their own right, the results of this study suggest that the extent of representativeness critically shapes citizens’ acceptance of the involvement of mini-publics in democratic decision-making.

An interesting observation is that even though substantial representative biases have repeatedly been demonstrated in the literature, they remain rarely discussed in the public sphere. Notably, this applies even in the case of Ireland, where mini-publics have paved the way for major policy changes, such as the legalization of abortion. A possible objection to the results of this study is, therefore, that citizens are unlikely to care about something they are unlikely to know about in the first place. However, if mini-publics continue to proliferate, political actors are likely to become more familiar with their workings over time, and better understand (and communicate) potential shortcomings, such as deviations from representativeness. Thus, an important policy implication of this study is that the organizers of mini-publics should consider measures to increase representativeness. For example, organizers could consider increasing the size of mini-publics – a proposal that would be consistent with survey evidence suggesting that citizens tend to prefer larger over smaller mini-publics (Goldberg and Bächtiger 2023). An important precedent is offered by Deliberative Polling, which has tended to involve substantially higher numbers of participants compared to other mini-publics (Fishkin 2018b).

Furthermore, to tackle non-response bias, the organizers of mini-publics could consider more generous forms of remuneration and/or think about ways to reduce the time commitment necessary for participation. A more radical proposal would be the introduction of fines for those who do not wish to serve in a mini-public (Barber 2003, 293). Finally, this study found that attitudinal biases can be particularly harmful to perceptions of the legitimacy of mini-publics, suggesting that the organizers of mini-publics should always include strata for citizens’ policy views in the random selection process. Perhaps surprisingly, this is something that is currently only rarely done (Paulis et al. 2021).

Yet, short of the introduction of a legal requirement to participate in mini-publics, statistical representativeness is likely to remain an elusive goal, even with carefully designed recruitment processes. Fundamentally, many citizens are not interested in politics, and for various reasons prefer not to participate in extended discussions on politics with strangers. Pervasive non-response bias makes it hard for mini-publics to achieve true representativeness, and this study suggests that deviations from full representativeness can harm perceptions of their legitimacy. Of course, this is but one study and its results need to be replicated in other contexts and with other methods. Still, the evidence presented in this article points to the conclusion that mini-publics may not be the easy fix to perceptions of low democratic legitimacy they are sometimes made out to be.