No CrossRef data available.
Article contents
Behavioural Insights in Consultation Design: A Dialogical Architecture
Published online by Cambridge University Press: 27 December 2018
Abstract1
In this paper, we examine consultation procedures in the light of behavioural sciences. We feature consultation as a dialogue between the administration and the participants in the form of a public good game with one or more tournaments. Our focus is on the architecture of the dialogue and its design. We propose three models characterised by the varying degrees of the interaction among participants, and between participants and the administration, occurring during the consultation process. We suggest that mapping stakeholders according to homogeneity of interest influences the structure and affects the dialogue taking place during the consultation process. We then examine the levels of efforts parties would engage in, defining models that maximise efforts and adjust for different cognitive stakeholders’ capabilities, advocating an empirical approach. The paper concludes with policy recommendations on how to improve the current consultation design deployed at EU and national level.
- Type
- Articles
- Information
- Copyright
- © Cambridge University Press
Footnotes
*
Consiglio di Stato.
**
Luiss Guido Carli, Rome.
1
This paper was inspired by and drawn from joint lectures at Luiss Guido Carli, Rome. We are grateful to the participants who helped us in identifying the issues and refining the hypotheses. In particular we would like to thank Laura D’Aprile, who contributed to the definition of issues that sparked the design of experiments and the ensuing theoretical discussion that followed suit. Corresponding author: Giacomo Sillari ([email protected]).
References
2 For EU see J Mendes, Participation in EU Rule-making: A Rights-based Approach (Oxford, Oxford University Press 2011); for the international setting see A Berman, “Participation in international governance 2.0” (unpublished manuscript on file with the author, 2017).
3 See Alemanno, A, “Stakeholder Engagement in Regulatory Policy” in Regulatory Policy Outlook (OECD Publishing 2015)Google Scholar .
4 See OECD, “The Public Consultation on the draft OECD Recommendation of the Council on Open Government” (2017) available at <goo.gl/C49ZxD> (last accessed on 25 September 2018): “Grant all stakeholders equal and fair opportunities to be informed and consulted, and to actively engage in policymaking and service design and delivery, at a minimal cost, avoiding duplication to minimise consultation fatigue, with adequate time, dedicating specific efforts to reach out to the most relevant, vulnerable, underrepresented, or marginalised segments of society, while avoiding policy capture by interest groups …”.
5 See OECD, “OECD best practice principles on stakeholder engagement in regulatory policy” (2017) available at <goo.gl/mWftM4> (last accessed on 25 September 2018): “10. Open and inclusive policy making as promoted by the OECD is a culture of governance that builds upon the idea of opening up policy-making processes to stakeholders beyond the public administration to better design policies by broadening the evidence base.
It recognises that the public administration does not hold the monopoly of expertise but that other stakeholders (citizens, civil society, private sector etc) have valuable information and ought to express their needs and expertise.
It emphasises the responsiveness of policies and services in actively involving those that will be affected by the policy; it is user-centred.
It relies on an inclusive approach where all relevant actors are involved and attention is paid to marginalised, disadvantaged or less powerful groups.
It can be conducted in different degrees and different modalities, ranging from providing information to consulting and to active engagement in the design, implementation and evaluation stage of a policy”.
6 See EU Commission, “Guidelines on stakeholder consultation” (2014) available at <ec.europa.eu/info/sites/info/files/better-regulation-guidelines-stakeholder-consultation.pdf> (last accessed on 25 September 2018).
7 See Stewart, RB, “Remedying disregard in global regulatory governance: accountability, participation, and responsiveness” (2014) 108(2) American Journal of International Law 211 Google Scholar .
8 See for example UNDP, Multi-Stakeholder Decision-Making: A Guidebook for Establishing a Multi-Stakeholder Decision-Making Process to Support Green, Low-Emission and Climate-Resilient Development Strategies (2012) available at <goo.gl/5fNAMx> (last accessed on 2 November 2018).
9 Applying behavioural insights analysis to regulators rather than to citizens is an often neglected topic. For an example of it, see the last chapter in World Bank, World Development Report 2015: Mind, Society, and Behavior (World Bank Group 2015). See below.
10 See Cafaggi, F, “The Many Features of Transnational Private Rule-Making: Unexplored Relationships between Custom, Jura Mercatorum and Global Private Regulation” (2015) University of Pennsylvania Journal of International Law 875 Google Scholar , available at <scholarship.law.upenn.edu/jil/vol36/iss4/2> (last accessed on 25 September 2018).
11 See F Cafaggi, “A comparative analysis of transnational private regulation: legitimacy, quality, effectiveness and enforcement” (2014), available at <ssrn.com/abstract=2449223> or <dx.doi.org/10.2139/ssrn.2449223> (last accessed on 25 September 2018).
12 See Cafaggi, supra, note 10.
13 See OECD, “Open Government: The Global Context and the Way Forward” (2016), available at <goo.gl/jRKxYT> (last accessed on 25 September 2018).
14 See for example the distinction made in OECD, supra, note 4.
“Stakeholders’ participation: all the ways in which stakeholders (ie citizens, private sector, and civil society organisations) can be involved in policymaking and service design and delivery, including:
Information: a one-way relationship in which government produces and delivers information to stakeholders. It covers both “passive” provision of information, upon demand, and “active” measures by government to disseminate information. This relationship could also encompass information that citizens provide to governments.
Consultation: a two-way relationship in which people provide feedback to government. It is based on the prior definition by government of the issue on which stakeholders’ views are being sought and requires the provision of information to them.
Engagement: a relationship based on partnership with government, in which stakeholders are given the opportunity and the necessary resources (information, data, digital tools etc.) to actively engage in defining the process and content of policy making and service design and delivery. It acknowledges the right of stakeholders to take part in setting the government’s agenda, proposing and weighing different options, and shaping the dialogue. Although the responsibility for the final decision generally rests with government, stakeholders hold the government responsible through the political process.”
15 See OECD, supra, note 4; OECD, supra, note 5; EU Commission, supra, note 6; OECD, supra, note 13.
16 See EU Commission, supra, note 6, where guidelines are put forth in which it is specified that consultation design has to include:
“the objective of the consultation;
the elements for which this is necessary (nature of the problem, policy options, etc);
the target group (general public or a special category of stakeholders, etc);
the appropriate consultation tool (consultative committees, expert groups, ad hoc meetings, consultation via internet, etc);
and the appropriate time frame for consultation.”
17 See C Sunstein, “Cost-Benefit Analysis and the Knowledge Problem” (2014), available at <ssrn.com/abstract=2508965> (last accessed on 25 September 2018). In the paper, Sunstein tackles the issue of asymmetric information between regulators and citizens tracing its intellectual history to Hayek, but focusing only on one direction of the asymmetry (the knowledge problem) rather than on both (cognitive empowerment), as we do in this article.
18 In fact, equal participation might mean that weaker stakeholders should be over represented in the consultation process. We consider equality and fairness in results as the endpoint of a consultation process that might be achieved by unequal means, for instance by giving more voice to weaker stakeholders, or by weighing and correcting opinions that display cognitive bias favouring particular stakeholders.
19 See OECD, supra, note 5:
“26. Despite the wide recognition of the importance of stakeholder engagement, there are still many challenges connected with its application. The most important ones that are often mentioned include:
The risks of stakeholder engagement activities being captured by organised interest and pressure groups;
Difficulties in reaching out to some groups stakeholders and wider society in general;
Engaging stakeholders too late in the regulatory process, ie when the decision has been actually made and there is little will to change it, resulting in low public participation rates in the future,
Engaging too often, particularly in academic debates or with insufficiently precise plans and information, and/or not responding or reflecting stakeholder input in the final outcome, engendering ‘consultation fatigue’”.
20 Also, in this article we do not address the decision-making process of the administration, either at the level of resources, timing, politics, or at the level of cognitive bias potentially involved in it (with the exception, see section III.1 below, of considering the administration’s confirmation bias). While crucial, these aspects are best considered after the behavioural elements that pertain to stakeholder engagement and participations are laid on the table, which remains the aim of the current paper.
21 EU Commission, supra, note 6: “Open public consultation reaches a wide spectrum of respondents without, however, ensuring full representativeness. The relevance of opinions collected needs, therefore, to be thoroughly assessed. Open public consultations can foster transparency and accountability and ensure broadest public validation and support for an initiative”.
22 On the differences between a pool of strangers and a pool of partners in the production of public good see Andreoni, J and Croson, R, “Partners versus strangers: random rematching in public goods experiments” in C Plott and VL Smith (eds), Handbook of Experimental Economics Results (Amsterdam, North-Holland 2008) 776 CrossRefGoogle Scholar ; for a summary of the literature see Camerer, C, Behavioral Game Theory: Experiments in Strategic Interaction (Princeton, Princeton University Press 2004)Google Scholar ; for a more recent overview see Chaudhuri, A, “Sustaining cooperation in laboratory public goods experiments: a selective survey of the literature” (2011) 14(1) Experimental Economics 47 CrossRefGoogle Scholar . If the consutation procedure is determined to be a repeated interaction, in which issues of reputation or reciprocity matter, then partner design would be preferred.
23 Among the participants there might be also other administrations.
24 The predominant approach used by the EU Commission is that publication should occur at the end of the consultation, cf EU Commission, supra, note 5: “Publication of contributions on the webpage. After a consultation has ended, the contributions made by stakeholders should be published. Contributions should be published in the languages in which they were submitted and/or the language used for the consultation activity. Written contributions should be made public on the dedicated consultation webpage. In the case of stakeholder consultation events (meetings, hearings, conferences, etc), summary minutes and speeches or presentations provided during the event should be made public on the consultation webpage”.
25 For prospect theory, see Tversky, A and Kahneman, D, “Prospect Theory: An Analysis of Decision under Risk” (1979) 47(2) Econometrica 263 Google Scholar ; for heuristics and bias, most of the relevant literature is collected in Kahneman, D, Slovic, P, and Tversky, A (eds), Judgment Under Uncertainty: Heuristics and Bias (Cambridge, Cambridge University Press 1982)CrossRefGoogle Scholar ; see also Gilovich, T, Griffin, D and Kahneman, D, Heuristics and Biases: The Psychology of Intuitive Judgment (Cambridge, Cambridge University Press 2002)CrossRefGoogle Scholar , Kahneman, D and Tversky, A (eds), Choices, Values and Frames (Cambridge, Cambridge University Press 2000)Google Scholar , Camerer, C, Rabin, M and Loewenstein, G (eds), Advances in Behavioral Economics (New York, Russell Sage Foundation 2004)Google Scholar ; for ecological rationality, see the work of Gigerenzer and the ABC group (eg G Gigerenzer and P Todd, Ecological Rationality: Intelligence in the World (New York, Oxford University Press 1999). For a general account of behavioural economics, see Angner, E and Loewenstein, G, “Behavioral Economics” in U Mäki (ed), Handbook of Philosophy of Science vol 13: Philosophy of Economics (Amsterdam, Elsevier 2012)Google Scholar . For popularised accounts, see Kahneman, D, Thinking, Fast and Slow (New York, Macmillan 2011)Google Scholar , Thaler, R, Misbehaving: The Making of Behavioral Economics (New York, WW Norton & Company 2015)Google Scholar and Thaler, R and Sunstein, C, Nudge: Improving Decisions about Health, Wealth, and Happiness (New Haven, Yale University Press 2008)Google Scholar . For behavioural game theory, the main reference continues to be Camerer, supra, note 18. See also specific references in section III.2 below.
26 See Nickerson, RS, “Confirmation bias: a ubiquitous phenomenon in many guises” (1998) 2(2) Review of General Psychology 175 CrossRefGoogle Scholar .
27 See eg Tuchman, B, The March of Folly: From Troy to Vietnam (New York, Ballantine Books 1984)Google Scholar , claiming that an administration will be subject to confirmation bias once a policy has been issued, so that evidence will be sought or interpreted in ways that favour the policy that has been made, or Jervis, R, Perception and Misperception in World Politics (Princeton, Princeton University Press 1976)Google Scholar , on the tendency to “explain away” undesirable evidence.
28 Tversky, A and Kahneman, D, “The framing of decisions and the psychology of choice” (1981) 211 (4481) Science 453 CrossRefGoogle Scholar .
29 Druckman, JN, “The implications of framing effects for citizen competence” (2001) 23(3) Political Behavior 225 Google Scholar .
30 See, for instance, Sniderman, P and Theriault, S, “The structure of political argument and the logic of issue framing” in W Saris and P Sniderman (eds), Studies in Public Opinion: Attitudes, Nonattitudes, Measurement Error, and Change (Princeton, Princeton University Press 2004) 133 Google Scholar ; Brewer, P and Gross, K, “Values, framing, and citizens’ thoughts about policy issues: effects on content and quantity” (2005) 26(6) Political Psychology 929 Google Scholar ; van Londen, M, Coenders, M, and Scheepers, P, “Effects of issue frames on aversion to ethnic-targeted school policies” (2010) 6(3) Methodology: European Journal of Research Methods for the Behavioral and Social Sciences 96 Google Scholar ; van Gorp, B, “Strategies to take subjectivity out of framing analysis” in P D’Angelo and J Kuypers (eds), Doing News Framing Analysis: Empirical and Theoretical Perspectives (Routledge 2010) 84 Google Scholar .
31 See Groves, R, Cialdini, R and Couper, M, “Understanding the decision to participate in a survey” (1992) 56(4) Public Opinion Quarterly 475 CrossRefGoogle Scholar .
32 See Thaler and Sunstein, supra, note 21; Sunstein, C, “The council of psychological advisers” (2016) 67 Annual Review of Psychology 713 CrossRefGoogle ScholarPubMed ; Sunstein, C, “‘Better Off, as Judged by Themselves’: Bounded Rationality and Nudging” in R Viale (ed), Routledge Handbook on Bounded Rationality (Routledge 2018)Google Scholar .
33 Nickerson, DW and Rogers, T, “Do you have a voting plan? Implementation intentions, voter turnout, and organic plan making” (2010) 21(2) Psychological Science 194 CrossRefGoogle ScholarPubMed ; Bayer, U et al. “Responding to subliminal cues: do if-then plans facilitate action preparation and initiation without conscious intent?” (2009) 27(2) Social Cognition 183 CrossRefGoogle Scholar . The result extends to domains other than political participation.
34 On reward substitution, see Milkman, KL, Minson, JA and Volpp, KG, “Holding the Hunger Games hostage at the gym: an evaluation of temptation bundling” (2013) 60(2) Management Science 283 CrossRefGoogle Scholar .
35 Besides the role of behavioural biases, it is quite relevant to take into account the role of social proof (see eg Cialdini, RB, “Harnessing the science of persuasion” (2001) 79(9) Harvard Business Review 72 Google Scholar and Cialdini, RB, Kallgren, CA and Reno, RR, “A focus theory of normative conduct: A theoretical refinement and reevaluation of the role of norms in human behavior” (1991) 24 Advances in Experimental Social Psychology (201)Google Scholar and, in general, of social norms and social norms engineering to foster participation, see C Bicchieri and R Muldoon, “Social Norms” in EN Zalta (ed), The Stanford Encyclopedia of Philosophy, available at <plato.stanford.edu/archives/spr2014/entries/social-norms/> (last accessed on 25 September 2018). For a game-theoretic account see Bicchieri, C, The Grammar of Society (Cambridge, Cambridge University Press 2006)Google Scholar , Sillari, G, “Rule-following as coordination: a game-theoretic approach” (2013) 190(5) Synthese 871 Google Scholar ; for social norms change and its behavioural relevance see Bicchieri, C, Norms in the Wild: How to Diagnose, Measure, and Change Social Norms. (New York, Oxford University Press 2016)Google Scholar and Cass Sunstein, “Unleashed”, available at <ssrn.com/abstract=3025749> (last accessed on 25 September 2018).
36 Notice that risk attitudes shift also in relation to ther factors, such as, for instance, gender. It might therefore be advisable, in some situations, to split a stakeholder group by gender in order to better deal with shifting risk-attitudes.
37 Tversky and Kahneman, supra, note 25.
38 For instance, people generally prefer to bet on drawing a red ball from an urn that contains 50 red and 50 black balls than from an urn that contains 100 red and black balls in some unspecified proportion, even though the bets are probabilistically equivalent.
39 Cf Kunreuther, H, “Mitigating disaster losses through insurance” (1996) 12(2) Journal of Risk and Uncertainty 171 Google Scholar .
40 On social dilemmas and cooperation, see eg Dawes, RM and Thaler, RH, “Anomalies: cooperation” (1988) 2(3) The Journal of Economic Perspectives 187 CrossRefGoogle Scholar ; Dawes, RM et al., “Organizing groups for collective action” (1986) 8 American Political Science Review 1171 CrossRefGoogle Scholar ; Dawes, R, “Social dilemmas” (1980) 31 Annual Review of Psychology 169 CrossRefGoogle Scholar . For a thorough survey of the experimental literature on such problems of cooperation, see Ledyard, JO, “Public Goods: A Survey of Experimental Research” in J Kagel and A Roth (eds), Handbook of Experimental Economics (Princeton, Princeton University Press 1995)Google Scholar and for the subsequent literature, see Chaudhuri, supra, note 22.
41 There may be further strategic elements in the consultation mechanism, such as costly signalling. For instance, a stakeholder is willing to incur a cost in making some private information explicit to signal their competence and their position. Formal models based on signalling interactions are well suited to capture the strategic elements underlying notice-and-comment procedures in the United States system, in which a third party (the Court) can effectively veto a policy if found inconsistent with the information provided in public comments, see eg Gailmard, S and Patty, JW, “Participation, process and policy: the informational value of politicised judicial review” (2017) 37(3) Journal of Public Policy 233 CrossRefGoogle Scholar .
42 See Fehr, E and Falk, A, “Psychological foundations of incentives” (2002) 46(4) European Economic Review 687 CrossRefGoogle Scholar ; Fehr, E and Gächter, S, “Reciprocity and economics: the economic implications of homo reciprocans” (1988) 42(3) European Economic Review 845 CrossRefGoogle Scholar .
43 Cognitive bias can disrupt productive efforts to convey information. This is why the kind of analysis we offered in section III.1 is highly relevant for consultation design. Behavioural analysis of that kind is needed to identify the risk of parties to the consultation conveying biased information. Particular behavioural insights, thus, should be taken into account to better the quality of the information provided, through specific mechanisms of cognitive empowerment (see Rangone, N and Di Porto, F, “Behavioural Sciences in Practice: Lessons for EU Policymakers” in A Alemanno and A-L Sibony (eds), Nudge and the Law: A European Perspective (Oxford, Bloomsbury 2015)Google Scholar .
44 See Bornstein, G and Ben-Yossef, M, “Cooperation in intergroup and single-group social dilemmas” (1994) 30(1) Journal of Experimental Social Psychology 52 CrossRefGoogle Scholar ; Bornstein, G, “Intergroup conflict: Individual, group, and collective interest” (2003) 7(2) Personality and Social Psychology Review 129 CrossRefGoogle Scholar .
45 For the simultaneaous competitive case, see Bornstein and Ben-Yossef, supra, note 44. The intuition, spelled out above, is that in competition teams face a series of problems of cooperation within each other, one such problem for each level of contribution of the competing team.
46 The only exception is sequential threshold public good games (see Duffy, J, Ochs, J and Vesterlund, L, “Giving little by little: dynamic voluntary contribution games” (2007) 91(9) Journal of Public Economics 1708)CrossRefGoogle Scholar , in which the payoff from the public good increases discontinuosly only if a certain threshold of contributions is met; in the sequential case there might be strategic considerations making it convenient for players to contribute the amount necessary on their part so that the group could achieve the threshold.
47 See Ledyard, supra, note 40, and Chaudhuri, supra, note 22.
48 See Bornstein, supra, note 44, and Bornstein and Ben-Yossef, supra, note 44, for the simultaneous case.
49 See for example a questionnaire in the field of chemicals where the different options are stated in the following way: “2. Received contributions may be published on the Commission’s website, with the identity of the contributor. Please state your preference with regard to the publication of your contribution: (Please note that regardless the option chosen, your contribution may be subject to a request for access to documents under Regulation 1049/2001 (http://eur-lex.europa.eu/legal-content/EN/TXT/?qid=1454925130412&uri=CELEX:32001R1049) on public access to European Parliament, Council and Commission documents. In this case the request will be assessed against the conditions set out in the Regulation and in accordance with applicable data protection rules (http://ec.europa.eu/justice/dataprotection/)) My contribution may be published under the name indicated; I declare that none of it is subject to copyright restrictions that prevent publication My contribution may be published but should be kept anonymous; I declare that none of it is subject to copyright restrictions that prevent publication I do not agree that my contribution will be published at all”.
50 Tversky and Kahneman, supra, note 25.
51 Loewenstein, GF and Prelec, D, “Preferences for sequences of outcomes” (1993) 100(1) Psychological Review 91 CrossRefGoogle Scholar .
52 Frederick, S, “Measuring intergenerational time preference: are future lives valued less?” (2003) 26(1) Journal of Risk and Uncertainty 39 CrossRefGoogle Scholar .
53 Tversky, A, Slovic, P and Kahneman, D, “The causes of preference reversal” (1990) The American Economic Review 204 Google Scholar .
54 Schneider, SL, “Framing and conflict: aspiration level contingency, the status quo, and current theories of risky choice” (1992) 18(5) Journal of Experimental Psychology: Learning, Memory, and Cognition 1040 Google ScholarPubMed .
55 Kahneman, D, “Reference points, anchors, norms, and mixed feelings” (1992) 51(2) Organizational Behavior and Human Decision Processes 296 CrossRefGoogle Scholar .
56 Levin, IP, Schneider, SL and Gaeth, GJ, “All frames are not created equal: a typology and critical analysis of framing effects” (1998) 76(2) Organizational Behavior and Human Decision Processes 149 CrossRefGoogle Scholar .
57 See section V.3 for an example of how a de-biasing consultation frame could work.
58 See REACH regulation.
59 Confirmation bias pushes decision makers to consider only evidence supporting their pre-existing beliefs; overconfidence referes to the fact that people tend to be mistaken more often that they think they might be, even when taking into account the fact that most people miscalibrate the confidence in their beliefs; hindsight bias refers to the fact that people tend to ascribe a higher probability to possibility of an event occurring after the event has occurred than before.
60 For public good games studying this kind of sequentiality and leader-follower effects, see, among others, Güth, W et al., “Leading by example with and without exclusion power in voluntary contribution experiments” (2007) 91(5) Journal of Public Economics 1023 CrossRefGoogle Scholar , in which it is shown that leader effects exist, though they are stronger when leaders are chosen by other participants.
61 A similar result in the context of weak-link coordination can be found in Devetag, G, Hosni, H and Sillari, G, “You better play 7: mutual versus common knowledge of advice in a weak-link experiment” (2013) 190(8) Synthese 1351 CrossRefGoogle Scholar .
62 Notice that this consideration lends support to a model in which the endowments of industry and CSOs are comparable: while industry has presumably more resources, revealing information is quite costly, reducing the advantage stemming from abundant resources.
63 This is reflected in the design of a competitive public good game, in which a group that fears a high level of contributions by the competing group, has an incentive, collectively, to try and match the opposing group’s contribution level, as otherwise the resulting public good will be severely negative for them.
64 In terms of our game-theoretic model: consider a basic payoff function in which the initial public good is 0, and parties influence the administration by offering information, so that their payoff increases (decreases) linearly if the public good increases (decreases). Offering more information could be valuable if I know that the administration will otherwise be swayed by competitors whose interests diverge from mine.
65 TC Schelling in Schelling, TC, The Strategy of Conflict (Cambridge, Harvard University Press 1960)Google Scholar was the first to strategise this type of segmentation.
66 See Marx, LM and Matthews, SA, “Dynamic voluntary contribution to a public project” (2000) 67(2) The Review of Economic Studies 327 Google Scholar ; Duffy, Ochs and Versterlund, supra, note 46, and the subsequent literature of threshold public good games.
67 Rules governing access to subsequent rounds need to be designed with particular care towards both fairness of access and perceived fairness of access, so that the democratic, open, and fair nature of the consultation design is not infringed.
68 It is, however, important to think in terms of ex-post analysis. In this case, model 1 could lead to a lesser information transfer and hence to a lesser quality regulation and hence end up becoming much costlier than model 2 or even model 3, if though more economical ex-ante. It is important to notice, moreover, that there are two lines of considerations justifying the ex-ante more expensive (in terms of time and resources) model 3. One is the ex-post argument just mentioned. The other is that through model 3 the administration can exercise more capillary control both on the information being offered and on the stakeholders offering it, therefore having more and better opportunities to counterbalance inequalities in terms of participation, effectiveness, or inclusion.
69 See EU Commission, supra, note 6.
70 It would be extremely difficult for the administration to make assumptions about the endowments whereas it is much easier to define different level of endowments and ask the different stakeholders to associate themselves with one.
71 See EU Commission, supra, note 6, at 76. “The choice of the consultation method will determine the consultation tools. The consultation tools most commonly used are written consultations via consultation documents or questionnaires as well as direct interactions with stakeholders via meetings, conferences, hearings or other events. The selection of the most appropriate consultation tool should take into account ∙ Proportionality; ∙ The degree of interactivity needed (eg written consultation versus stakeholder events/online discussion fora/other internet based tools); ∙ Accessibility considerations (language regime, disability etc); and ∙ Timing requirements. In practice, effective consultation often requires a combination of written consultation tools (used for both open public and targeted consultations) and more direct interactions with stakeholders. If the consultation should provide statistically representative results, then particular tools should be foreseen, such as surveys (eg Eurobarometer)”.