Hostname: page-component-cd9895bd7-dzt6s Total loading time: 0 Render date: 2024-12-25T07:39:28.616Z Has data issue: false hasContentIssue false
  • English
  • Français

Disaggregating Deliberation’s Effects: An Experiment within a Deliberative Poll

Published online by Cambridge University Press:  22 March 2010

Cynthia Farrar ,
James S. Fishkin ,
Donald P. Green ,
Christian List ,
Robert C. Luskin  and
Elizabeth Levy Paluck
Get access Rights & Permissions [Opens in a new window]

Abstract

Using data from a randomized field experiment within a Deliberative Poll, this paper examines deliberation’s effects on both policy attitudes and the extent to which ordinal rankings of policy options approach single-peakedness (a help in avoiding cyclical majorities). The setting was New Haven, Connecticut, and its surrounding towns; the issues were airport expansion and revenue sharing – the former highly salient, the latter not at all. Half the participants deliberated revenue sharing, then the airport; the other half the reverse. This split-half design helps distinguish the effects of the formal on-site deliberations from those of other aspects of the treatment. As expected, the highly salient airport issue saw only a slight effect, while much less salient revenue-sharing issue saw a much larger one.

Type
Research Article
Information
British Journal of Political Science , Volume 40 , Issue 2 , April 2010 , pp. 333 - 347
Copyright
Copyright © Cambridge University Press 2010

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

1 See, e.g., Luskin, Robert C., Fishkin, James S. and Jowell, Roger, ‘Considered Opinions: Deliberative Polling in Britain’, British Journal of Political Science, 32 (2002), 455–487CrossRefGoogle Scholar; Barabas, Jason, ‘How Deliberation Affects Policy Opinions’, American Political Science Review, 98 (2004), 687–701CrossRefGoogle Scholar; Druckman, James N., ‘Political Preference Formation: Competition, Deliberation, and the (Ir)relevance of Framing Effects’, American Political Science Review, 98 (2004), 671–686CrossRefGoogle Scholar.

2 See Fishkin, James S., The Voice of the People: Public Opinion and Democracy (New Haven, Conn.: Yale University Press, expanded paperback edition, 1997)Google Scholar; Luskin, Robert C., Fishkin, James S. and Jowell, Roger, ‘Considered Opinions: Deliberative Polling in Britain’, British Journal of Political Science, 32 (2002), 455–487CrossRefGoogle Scholar; Fishkin, James S. and Luskin, Robert C., ‘Experimenting with a Democratic Ideal: Deliberative Polling and Public Opinion’, Acta Politica, 40 (2005), 284–298CrossRefGoogle Scholar.

3 Marie Jean Antoine Nicolas Caritat (the Marquis de) Condorcet, Essai sur l’Application de l’Analyse à la Probabilité des Décisions Rendues à la Pluralité des Voix (Paris, 1785); Arrow, Kenneth, Social Choice and Individual Values (New York: Wiley, 1953)Google Scholar.

4 See Luskin, Robert C., Fishkin, James S. and Hahn, Kyu S., ‘Deliberation and Net Attitude Change’(paper presented at the biennial General Conference of the European Consortium for Political Research, Pisa, Italy, 2007)Google Scholar; List, Christian, Luskin, Robert C., Fishkin, James S. and McLean, Iain, ‘Deliberation, Single-Peakedness, and the Possibility of Meaningful Democracy: Evidence from Deliberative Polls’ (unpublished, London School of Economics, 2007)Google Scholar.

5 See Luskin, Fishkin and Hahn, ‘Deliberation and Net Attitude Change’; List, Luskin, Fishkin and McLean, ‘Deliberation, Single-Peakedness, and the Possibility of Meaningful Democracy’.

6 This is a plausible worry for DPs conducted during and about referendum or election campaigns, as well as in scattered other cases, including DPs about educational policy in Northern Ireland, just as the current accord was being reached, and about foreign policy in the United States on the eve of the Iraq war.

7 Of course, both samples are corroded – unequally – by non-response/non-participation, depriving these comparisons of the full authority of true random assignment. Barabas’s analysis of much the same question involves broadly similar quasi-control groups (created in his case by matching), though with a highly non-random participant sample and hence lesser external validity.

8 See also Fishkin, James S. and Luskin, Robert C., ‘Bringing Deliberation to the Democratic Dialogue: The NIC and Beyond’, in Maxwell McCombs and Amy Reynolds, eds, A Poll with a Human Face: The National Issues Convention Experiment in Political Communication (Mahwah, NJ: Lawrence Erlbaum, 1999)Google Scholar; Luskin, Robert C., Fishkin, James S., McAllister, Ian, Higley, John and Ryan, Pamela, ‘Information Effects in Referendum Voting: Evidence from the Australian Deliberative Poll’ (paper presented at the annual meeting of the American Political Science Association, Boston, Mass.; 2002)Google Scholar; Barabas, ‘How Deliberation Affects Policy Outcomes’.

9 The changes could conceivably if implausibly stem simply from the stimulation of the initial interview, which even without the invitation to and anticipation of the DP should produce some heightened attention and thought. Luskin, Fishkin, McAllister, Higley, and Ryan, ‘Information Effects in Referendum Voting: Evidence from the Australian Deliberative Poll,’; Luskin, Robert C. and Fishkin, James S., ‘Deliberative Polling, Public Opinion and Democracy: the Case of the National Issues Convention’ (paper presented at the annual meeting of the American Association for Public Opinion Research, 1998)Google Scholar; and Luskin, Robert C., Iyengar, Shanto and Fishkin, James S., ‘Considered Opinions on U.S. Foreign Policy: Face-to-Face versus Online Deliberative Polling’ (Center for Deliberative Democracy, Stanford University, 2006)Google Scholar address this concern by comparing to quasi control groups likewise interviewed before as well as after the deliberations.

10 See, e.g., Luskin, Fishkin and Jowell, ‘Considered Opinions’; Luskin, Robert C., Fishkin, James S., Jowell, Roger and Park, Allison, ‘Learning and Voting in Britain: Insights from the Deliberative Poll’ (paper presented at the annual meeting of the American Political Science Association, Atlanta, Ga. 1999)Google Scholar; Luskin, Robert C., Fishkin, James S. and Plane, Dennis L., ‘Deliberative Polling and Policy Outcomes: Electric Utility Issues in Texas’ (paper presented at the annual meeting of the Association for Public Policy Analysis, 1999)Google Scholar; Luskin, Fishkin, Higley and Ryan, ‘Information Effects in Referendum Voting’.

11 See, e.g., Luskin, , Fishkin, and Jowell, , ‘Considered Opinions’Google Scholar; and Luskin, , Fishkin, , Higley, and Ryan, , ‘Information Effects in Referendum Voting’Google Scholar.

12 See List, , Luskin, , Fishkin, and McLean, , ‘Deliberation, Single-Peakedness, and the Possibility of Meaningful Democracy’.Google Scholar

13 Converse, Philip E., ‘The Nature of Belief Systems in Mass Publics’, in David E. Apter, ed., Ideology and Discontent (New York: Free Press, 1964)Google Scholar. Cf. Hill, Jennifer and Kriesi, Hanspeter, ‘An Extension and Test of Converse’s “Black-and-White” Model of Response Stability’, American Political Science Review, 95 (2001), 397–414CrossRefGoogle Scholar.

14 To the extent that the first mechanism predominates, we should expect to see attitudes change so as to increase their predictability from socio-demographic variables, proxying interests; to the extent that the second predominates, they may change so as to decrease it. See Luskin, Fishkin and Jowell, ‘Considered Opinions’; Luskin, Robert C., ‘The Heavenly Public: What Would the Ideal Citizenry Be Like?’, in Michael B. MacKuen and George Rabinowitz, eds, Electoral Democracy (Ann Arbor: University of Michigan Press, 2003).Google Scholar

15 Across fifty policy attitude indices in seven DPs, the mean absolute net change of opinion ranges from 0.068 to 0.117, averaging 0.092, impressively large for net attitude change (for reasons sketched in Luskin 2002). Thirty-four (68 per cent) of the fifty indices show statistically significant net change. See Luskin, Fishkin and Hahn, ‘Deliberation and Net Attitude Change’.

16 For example, consider a population/sample of four individuals, with (declining) preference orderings (x, y, z), (z, y, x), (y, x, z), and (y, z, x). These preference orderings are single-peaked, with structuring dimension [x, y, z]. Each individual has a first preference, a second preference on one side of it, and a third preference no ‘closer’ (in terms of ordinal slots) to it. By contrast, if a fifth individual with the preference ordering (x, z, y) is added to this population/sample, the preferences are no longer single-peaked. They are not single-peaked with respect to the structuring dimension [x, y, z], because the fifth individual prefers x, then z, which is ‘further’ than y from x on this dimension. And there is no other possible structuring dimension (left–right ordering of x, y and z) with respect to which all five preference orderings are single-peaked. For a more precise definition, see List, Luskin, Fishkin and McLean, ‘Deliberation, Single-Peakedness, and the Possibility of Meaningful Democracy’.

17 Black, Duncan, ‘On the Rationale of Group Decision-Making’, Journal of Political Economy, 56 (1948), 23–34CrossRefGoogle Scholar and Arrow, Kenneth, Social Choice and Individual ValuesGoogle Scholar.

18 See Brams, Steven J., Jones, Michael A. and Kilgour, D. Marc, ‘Single-Peakedness and Disconnected Coalitions’, Journal of Theoretical Politics, 14 (2002), 359–383CrossRefGoogle Scholar. Cardinal single-peakedness is sufficient but unnecessary for ordinal single-peakedness if the space is one-dimensional, but neither sufficient nor necessary if the space is multi-dimensional.

19 Condorcet, , Essai sur l’Application de l’Analyse à la Probabilité des Décisions Rendues à la Pluralité des VoixGoogle Scholar.

20 For example, Arrow, Social Choice and Individual Values, and McKelvey, Richard, ‘General Conditions for Global Intransitivities in Formal Voting Models’, Econometrica, 47 (1979), 1085–1111.CrossRefGoogle Scholar

21 Riker, William H., Liberalism against Populism (San Francisco: W. H. Freeman, 1982).Google Scholar

22 Black, , ‘On the Rationale of Group Decision-Making’.Google Scholar

23 List, , Luskin, , Fishkin, and McLean, , ‘Deliberation, Single-Peakedness, and the Possibility of Meaningful Democracy’.Google Scholar

24 In the second example of fn. 16, the largest subset of the population/sample of five whose preferences are single-peaked contains four members (the four of the first example, with respect to the structuring dimension [x, y, z]). Thus S = 4/5 = 0.8. For a more precise definition and discussion, see List, Luskin, Fishkin and McLean, ‘Deliberation, Single-Peakedness, and the Possibility of Meaningful Democracy’. See also Niemi, Richard G., ‘Majority Decision-Making with Partial Unidimensionality’, American Political Science Review, 63 (1969), 488–497.CrossRefGoogle Scholar

25 Note that S concerns the aggregate patterning of preferences across individuals, not necessarily the cognitive organization of individuals’ preferences (although it may partly reflect that). See the further discussion below and in List, , Luskin, , Fishkin, and McLean, , ‘Deliberation, Single-Peakedness, and the Possibility of Meaningful Democracy’.Google Scholar

26 Niemi, , ‘Majority Decision-Making with Partial Unidimensionality’Google Scholar. This is also supported by computer simulations reported in List, , Luskin, , Fishkin, , and McLean, , ‘Deliberation, Single-Peakedness, and the Possibility of Meaningful Democracy’Google Scholar.

27 See Miller, David, ‘Deliberative Democracy and Social Choice’, Political Studies, 40 (1992: special issue):54–67CrossRefGoogle Scholar; Knight, Jack and Johnson, James, ‘Aggregation and Deliberation: On the Possibility of Democratic Legitimacy’, Political Theory, 22 (1994), 277–296CrossRefGoogle Scholar; List, Christian, ‘Two Concepts of Agreement’, PEGS: The Good Society, 11 (2003), 72–79Google Scholar; List, Luskin, Fishkin and McLean, ‘Deliberation, Single-Peakedness, and the Possibility of Meaningful Democracy’. Cf. Van Mill, David, ‘The Possibility of Rational Outcomes from Democratic Discourse and Procedures’, Journal of Politics, 58 (1996), 734–752CrossRefGoogle Scholar, countered by Dryzek, John and List, Christian, ‘Social Choice Theory and Deliberative Democracy: A Reconciliation’, British Journal of Political Science, 33 (2003), 1–28CrossRefGoogle Scholar.

28 This does not necessarily mean that deliberation will lead people to converge on some particular ranking, what List calls ‘agreement at a substantive level’. In fact, the results in List, Luskin, Fishkin and McLean, ‘Deliberation, Single-Peakedness, and the Possibility of Meaningful Democracy’, suggest that it may decrease agreement of this sort. Our only claim here is that deliberation increases proximity to single-peakedness, an instance of what List calls ‘agreement at a meta-level’. See List, ‘Two Concepts of Agreement’.

29 Across thirteen issues, the index of ‘proximity to single-peakedness’, defined below, increases by an average of 0.101, on a scale that runs from a minimum always > 0 to 1.0. Across the ten low to moderate salience issues, the average was 0.134 (computed from Table 2 in List, Luskin, Fishkin and McLean, ‘Deliberation, Single-Peakedness, and the Possibility of Meaningful Democracy’).

30 Across seven DPs, the correlation between pre-deliberation knowledge, proxying salience, and the mean absolute net change for all the policy attitude indices in that DP is −.61. See Luskin, Fishkin, and Hahn, ‘Deliberation and Net Attitude Change’. Across thirteen DPs, the correlation between pre-deliberation knowledge and the increase in “proximity to single-peakedness” (defined below) is −.59 (computed from Tables 1 and 3 in List, Luskin, Fishkin, and McLean, ‘Deliberation, Single-Peakedness, and the Possibility of Meaningful Democracy’).

31 There was much discussion of residential property taxes and property tax relief for the elderly, disabled and those on a low-income, but regional revenue sharing was only mentioned briefly in one editorial and as a low-profile issue in the platform of one unsuccessful mayoral candidate.

32 We shall thus be referring to two sorts of ‘groups’ – the small groups, within which the issues are discussed, and the treatment groups, each consisting of eight small groups, which tackle the issues in different sequences.

33 The A-first and R-first groups thus had different plenary sessions on Saturday but shared a plenary session on Sunday.

34 One might expect preferences to reflect attitudes – more after deliberation than before and more, at least initially, on the airport than on revenue-sharing – and the results bear this out. Maximum likelihood estimation of binomial logit models expressing preferences for expanding airport service over ending it, for non-mandatory revenue sharing over local control, and for non-mandatory over mandatory revenue sharing as functions of the respective attitude indices produce pre- and post-deliberation (McFadden’s) pseudo-R 2s of 0.479 and 0.680, 0.377 and 0.345, and 0.099 and 0.371, respectively. Preferences are indeed relatively well predicted by attitudes by the end of the process, are indeed initially better predicted on the airport than on revenue sharing, and are indeed better predicted after than before deliberation in two of the three cases. More detailed results are available on request.

35 Niemi, , ‘Majority Decision-Making with Partial Unidimensionality’.Google Scholar

36 A thousand random samples of the relevant subjects were drawn with replacement. The standard deviation of the resampled S provides the estimate of the standard error.

37 The bootstrapped standard errors, moreover, are for S, not the difference between the values of S at different times, which may be somewhat larger or (less likely) smaller, depending on the sign and magnitude of the covariance.

38 List, , Luskin, , Fishkin, and McLean, , ‘Deliberation, Single-Peakedness, and the Possibility of Meaningful Democracy’.Google Scholar

39 On the airport, the dimension along which the largest sub-sample is single-peaked remains the same throughout, ordering the alternatives as [A2 A1 A3]. The Condorcet winner, throughout, is A2 (expanding service). On revenue sharing, the dimension along which the largest sub-sample is single-peaked remains the same throughout, ordering the alternatives as [R1 R2 R3 R4], although the Condorcet winner changes from one non-mandatory sharing option to the other – from R3 (state-encouraged sharing) at T1 to R2 (voluntary sharing) at T3.

40 In each treatment group, the Condorcet winner changes from R3 to R2 over the interval during which the group deliberates the issue – from T1 to T2 in the R-first group and from T2 to T3 in the A-first group.

41 In words, S′ is the extent to which S exceeds its minimum, expressed as a fraction of the extent to which it could do so. Note that S′ necessarily ⩽ S (indeed < S for S < 1) and that because Sm is only approximate, S′ should be taken as heuristic. For more on S and S′, see List, , Luskin, , Fishkin, and McLean, , ‘Deliberation, Single-Peakedness, and the Possibility of Meaningful Democracy’.Google Scholar

42 The T1-T2 gains tend to exceed their T2-T3 counterparts, no doubt because T1-T2 is a much longer interval, bracketing the receipt of the briefing materials and the anticipatory, off-site deliberations, as well as the first on-site deliberative session.

43 By contrast, the general knowledge items show significant but roughly equal gains in both the A-first and R-first groups from T1 to T2 and thus from T1 to T3 (results not shown). On these more general matters, there is no reason to expect the members of either treatment group to learn more during either interval, and they do not.