Global social indicators and the concept of legitimacy

Abstract

Global social indicators have become a core point of interest of scholarship in law and other social sciences. The term ‘legitimacy’ is occasionally mentioned in this literature but without in-depth discussion. This paper aims to fill this gap by way of exploring the relationship between global social indicators and the concept of legitimacy. The crucial issue of such indicators is that, being drafted in a general fashion and based on a quantitative metric, they can be regarded as somehow ‘law-like’, thus raising questions about their legitimacy. This paper addresses these issues at both the general level of social theory and the operation of indicators in action.

I. Introduction

Global social indicators and governance standards have proliferated in recent years with profound implications for states, organisations and individuals. They have also become a core point of interest of scholarship: on the one hand, critics often focus on the substantive dimension of those indicators which are seen as one-sided, for example, in their emphasis on the promotion of business interests. On the other hand, supporters of indicators then often respond that what is needed are procedural rules that ensure a balance of the respective interests in the design of these indicators (for the discussion, see e.g. the contributions in Rottenburg et al., 2015; Merry et al., 2015; Cooley and Snyder, 2015; Frydman and Van Waeyenberge, 2013; Davis et al., 2012a). ¹

The term ‘legitimacy’ is occasionally mentioned in this literature but without in-depth discussion. Sometimes, ‘legitimacy’ is also highlighted as an important topic for future work. According to Davis et al.:

‘A study designed to understand the interactions between indicators and law would devote more attention to law. A study of this kind might make considerable progress on questions such as: Will indicators become law-like, in terms of their influence and perceived legitimacy?’ (Davis et al., 2015, p. 22)

This paper and the subsequent contributions of this special issue aim to fill this gap by way of exploring the legitimacy of global social indicators. Social and cultural forces shape human behaviour in a variety of ways. So the crucial issue that indicators raise is why certain forces – namely those that are drafted in a general fashion and that are based on a quantitative metric – can be regarded as somehow ‘law-like’, ² thus raising questions about their legitimacy and the need for accountability.

In the following, to set the scene, Section II explains what we mean by ‘global social indicators’ and Section III discusses the concept of ‘legitimacy’. In Section IV, we relate both to questions of social theory while Section V addresses the legitimacy of indicators ‘in action’. Section VI concludes.

II. What are ‘global social indicators’?

While there is no full consensus about the meaning of the term ‘indicators’, it is helpful to present some definitions suggested in practice and academia. A good starting point is the definition by the OECD:

‘In general terms, an indicator is a quantitative or a qualitative measure derived from a series of observed facts that can reveal relative positions (e.g. of a country) in a given area. When evaluated at regular intervals, an indicator can point out the direction of change across different units and through time. In the context of policy analysis … indicators are useful in identifying trends and drawing attention to particular issues. They can also be helpful in setting policy priorities and in benchmarking or monitoring performance.’ (OECD, 2013, p. 13)

This emphasis on the practical dimension of indicators is a common feature of most definitions. However, the alternatives ‘quantitative or qualitative’ are not shared by everyone as, usually, the emphasis is on quantitative measures, often by way of rankings. For example, according to Merry (2011, p. 84), ‘indicators are a special use of statistics to develop quantifiable ways of assessing and comparing characteristics among groups, organizations and nations’ and, according to Davis et al.:

‘An indicator is a named collection of rank-ordered data that purports to represent the past or projected performance of different units. The data are generated through a process that simplifies raw data about a complex social phenomenon. The data, in this simplified and processed form, are capable of being used to compare particular units of analysis (such as countries or institutions or corporations), synchronically or over time, and to evaluate their performance by reference to one or more standards’ (Davis et al., 2015, p. 4; see also Davis et al., 2012b).

Turning to the practical relevance of indicators, international organisations that create and use such indicators are bound to emphasise their virtues and attractiveness across a wide range of fields. According to the UN Rule of Law Indicators guide:

‘Indicators are indirect measures of elements that, taken together, can be used repeatedly and over time to assess progress towards specific goals and objectives. They often have the dual role of spurring reform and holding agencies and individuals accountable for their past performance. The use of indicators is becoming increasingly popular in many areas of public policy, science and business as an accountability mechanism for public institutions and private companies, and as an early warning sign of disease, famine, economic crisis and other problems with widespread impact. For example, commonly used indicators include aggregate stock prices, the gross domestic product of countries or regions and changes in the rates of infectious disease. Indicators allow the synthesis of complex information to produce easily interpreted measures that are well suited to tracking changes over time and drawing comparisons between places.’ (UN, 2011, p. 1)

This paper is specifically concerned with global social indicators. While broad terms such as ‘global’ (and ‘globalisation’) are subject to criticism (see Twining, 2011), there is now also a growing literature that accepts that, at least, there may be ‘intimations of global law’ (Walker, 2015). In the following, the focus is on indicators that have a global ambition. These types of indicators raise legitimacy concerns different from local, national and regional indicators, since global indicators face specific problems, for example, due to their need to accommodate different socio-economic systems and the lack of global democratic structures (discussed in more detail in the subsequent sections).

For the term ‘social indicators’, it is helpful to provide another definition, this time from a US federal agency:

‘Social indicators are numerical measures that describe the well-being of individuals or communities. Indicators are comprised of one variable or several components combined into an index. They are used to describe and evaluate community well-being in terms of social, economic, and psychological welfare.’ ³

In this wide range of social indicators, the focus of this paper will be on indicators that raise general legitimacy concerns. Table 1 , below, presents a representative list of such indicators categorised under the headings ‘Good governance and rule of law’, ‘Personal rights and economic freedom’, ‘Human development and political stability’ and ‘Performance of educational and commercial entities’.

Table 1 Examples of global social indicators

The current literature (see the citations at the beginning of this paper) typically focuses on the specific operation of those and other individual indicators. By contrast, it is the aim of this paper to discuss global social indicators at a more general level. We are therefore interested in the processes which lie behind the creation of ostensibly global principles, the expectations and the methods used so as to monitor and enforce them and, in particular, the legitimacy issues that arise when these are used in the context of shaping behaviour.

III. ‘Legitimacy’ as a goal or means

3.1 Normative and sociological concepts

There is a rich literature on legitimacy in many academic fields. This literature often makes a broad distinction between legitimacy as a normative concept (i.e. legitimacy as a goals) and legitimacy as a question of sociology (i.e. legitimacy as a means). To be sure, both of these perspectives also overlap, which, as we will see, has particular relevance for the legitimacy of global social indicators.

A common starting point for the normative variant of legitimacy is the introduction of Rousseau's Social Contract:

‘Man is born free; and everywhere he is in chains. One thinks himself the master of others, and still remains a greater slave than they. How did this change come about? I do not know. What can make it legitimate? That question I think I can answer.’ (Rousseau, 1762, book 1)

Thus, the question about legitimacy is what justifies why persons in power have the authority to make decisions that affect others. Here, Beetham (1991) suggests that the ‘legitimation of power’ requires that (1) the power confirms to established rules, (2) that these rules need to be based on shared beliefs by both parties and (3) that the subordinate party has consented to this power relationship.

Another conceptualisation distinguishes between forms of ‘input’, ‘output’ and ‘throughput’ legitimacy (Beisheim and Dingwerth, 2008; similar Bodansky, 1999: ‘source-based’, ‘substantive’ and ‘procedural’). Input legitimacy refers to aspects such as the expertise of the law-making authority and sufficient forms of accountability; output legitimacy may be assessed by theories of justice as well as criteria of efficiency; and throughput legitimacy refers to fair and transparent procedures. Often, it is also said that democratic legitimacy should be the main foundation of such an assessment (Barnard, 2001).

The sociological (or empirical) perspective of legitimacy appears to be very different. The usual starting point is here a reference to Max Weber, who stated that ‘social action … may be guided by the belief in the existence of a legitimate order’ (Weber, [1921] 1968, p. 81). Thus, having laws that the public believes to be legitimate is important, since the state may not be able or willing to enforce those by force and since other reasons for compliance (such as habit and expediency) may not always be sufficient.

It follows that this ‘relevance of people's perceptions of the rightfulness and appropriateness of authority for their acceptance and support for political and social order’ (Quack, 2010, p. 8) poses the empirical question how far individuals really act in such a way. Hyde (1983) is doubtful; however, recent psychological research suggests that:

‘perceptions of system legitimacy do shape everyday compliance with the law, which is a conclusion that is also supported by more recent studies …. Furthermore, perceived legitimacy seems to have more influence on compliance than do subjective assessments of the likely risk of punishment. When people perceive the system as legitimate, they feel an intrinsic moral obligation to comply with its demands.’ (Tyler, 2008, p. 717)

As far as state law is concerned, the sociological and the normative positions are linked because a high degree of voluntary compliance means that the state is best able to pursue the long term goals of society (Tyler, 2008, p. 716). An even stronger connection exists for pure private forms of governance, since – due to the lack of state enforcement – the success of such a private scheme crucially depends on its perception as legitimate (Beisheim and Dingwerth, 2008, p. 3). This observation will also be important for the discussion of the legitimacy of global social indicators in the subsequent sections.

3.2 The international context in particular

Another consideration relevant for the present paper is that the recent literature has frequently discussed the relevance of ‘legitimacy’ in the context of international, transnational, and global law and governance (e.g. Brunnée and Toope, 2010; Karlsson-Vinkhuyzen and Vihma, 2009; Black, 2008; Buchanan and Keohane, 2006; Clark, 2005; Zürn, 2004; Bodansky, 1999). To some extent, the general concepts about legitimacy simply apply here. For example, in the transnational context, the distinction between input, output and throughput legitimacy can be understood as a distinction between inclusiveness of wide stakeholder participation, expertise-based effectiveness in solving international public policy problems, and validation of transnational governance through procedural fairness and impartiality (summary in Quack, 2010, pp. 6–8).

But it can also be said that concepts of legitimacy of domestic laws require more fundamental adjustments. For example, researchers have explored how to implement democratic legitimacy in a global constitutional order (O'Donoghue, 2014; Zumbansen, 2012), how shared social norms based and adherence to a practice of legality can produce legitimacy in international law (Brunnée and Toope, 2010) and how the ‘democratic, results-based, order-derived, systemic, deliberative, and procedural’ values of a global administrative law can create legitimacy at a supranational scale (Esty, 2006).

The complexity of the many interests in the international sphere can also lead to further complications. For instance, it can be noted that ‘too much’ input legitimacy can be counter-productive for output legitimacy, since numerous diverse interests make it difficult to achieve a meaningful consensus (Kelly, 2008, p. 623). The question ‘legitimacy for whom?’ is also more complex here, since it requires to give consideration to more than the representatives of one state (Karlsson-Vinkhuyzen and Vihma, 2009, pp. 408–409). Indeed, to put it in more general terms, in the international and transnational sphere ‘mandates are uncertain, and it is not clear on whose behalf they purport to act and to whom accountability should be owed’ (Black, 2008, p. 143).

It is suggested that the various dimensions of legitimacy make it a useful concept for the analysis of global social indicators. It also raises a number of questions such as: What type of legitimacy can such indicators have? Do indicators turn the goals of normative legitimacy into the means to sociological legitimacy (or vice versa)? Who are the relevant groups that need to be considered for their legitimacy (markets, targets, users, etc.)? How does it compare to alternative ways of achieving the same functions? The following will address these and other questions based on considerations of social theory and the perspective of indicators ‘in action’.

IV. Legitimacy of indicators and social theory

4.1 The nature of global social indicators

At a theoretical level, a preliminary question is when and how statistics and data can be said to be transformed into indicators. This is not an entirely new discussion. Ewald (1990), using examples from insurance and industrial standardisation, reflects on ‘normalisation’ as a process: it means that, today, norms emerge not simply through formal expressions (constitutions, codes and laws), but through a process where common standards form a normative and objective basis for judgment. More concretely, Shore and Wright observe that:

‘Quantification and statistics have long served as instruments of state power. However, a key development over the past three decades has been the extraordinary rise of international performance indicators and rankings and their involvement in new forms of governance that operate both within and beyond the state. Virtually every aspect of contemporary professional life and organisational behaviour is now subject to elaborate systems of audit and inspection: everything from the provision of public services, education, policing and security, to health care, safety, energy conservation, information systems and the performance of individuals. Even intangible phenomena such as levels of “trust”, “perceptions of corruption”, “quality of life years” and “gross national happiness” are now routinely quantified, measured and ranked in competitive national and international league tables that purport to offer at-a-glance comparisons.’ (Shore and Wright, 2015, p. 22)

A concrete example for the way ‘rankings’ and ‘audits’ can transform data into indicators is the World Bank's Doing Business Report (DBR) (see Table 1). Its origins go back to academic work by La Porta et al. and Djankov et al., which ‘merely’ tried to understand the relationship between legal rules and financial development. But, when these studies were incorporated into the DBR, they were transformed into a ranking system and became one of the benchmarks for World Bank funding.

This transformation can also be seen as an example of a process whereby the underlying information becomes less meaningful and, possibly, illegitimate. Many countries care about their performance in the DBR ranking and some, such as Georgia (cf. Schueth, 2011), implement policies in order to rise in the ranking. The problem is then that the DBR indicators are only proxies for good law and can only cover a small proportion of relevant laws for doing business. Thus, a country that changes its law according these indicators quickly rises in the ranking, but this is irrelevant as far as the remainder of their legal system remains unchanged.

Conceptually, such a process can be described as a version of ‘Goodhart's law’ – a variant of Heisenberg's uncertainty principle in the social sciences. According to Goodhart's law, originally developed for questions of macro-economic policy, an indicator that is employed as a target for a particular policy makes it unsuitable to fulfil this role (Chrystal and Mizen, 2003). This is a problem for the DBR, but potentially other indicators as well, since measurement and impact of indicators are inherently interlinked – thus, to a larger or lesser extent allowing the ‘gaming’ of the corresponding results.

Another way to describe the paradox of global social indicators is that they are torn between their belongings to the fields of ‘sociology of knowledge’ and ‘regulation and governance’ (e.g. Davis et al., 2015, pp. 1–3). It is an inherent feature of indicators that they aim to be ‘successful’ due to their persuasive power and the expertise and power of the drafting institutions. It can also be asked how to conceptualise the governance function of global social indicators – and how far this should reduce, or accept, the complexity of the new global landscape (cf. Goodwin, 2013, p. 696).

A possible approach is Krisch's ongoing project on ‘liquid authority’. ⁴ The concept of liquidity aims to characterise the dimensions of ‘informality, ideationality, dynamism, and multiplicity’, in particular referring to forms of governance ‘without a clear centre, dispersed over a multitude of actors and institutions (public and private) without a formal ascription of authority’. Typically, indicators can said to belong to this category. For details, of course, the precise shape of the indicator needs to be considered, such as whether it is drafted by public or private institutions; whether it is aimed at states or private bodies; how the global, transnational, national and local levels interact with each other; and, therefore also, whether it is really truly global or connected to a particular local context (see also Section V, below).

4.2 Contrasting Habermasian and Luhmannian positions

An understanding of indicators can also benefit from juxtaposing positions by Habermas and Luhmann. ⁵ It can be suggested that, for Habermas, the problem posed by indicators would be connected to the need to get beyond what he calls systematically distorted communication. For Luhmann, on the other hand, the issue would be rather how indicators illustrate the necessary limits of communications in conditions of highly differentiated social systems.

This difference is particularly relevant for the relationship between the contestation and the legitimacy of indicators. According to a Habermasian, the role of indicators can be justified if it helps the community come to terms with essentially contested concepts, such as if the indicator channels the discussion of how an indicator about ‘rule of law’ is to be understood. But repoliticisation through contestation may also be seen as problematic. The very appeal of indicators can be that they are presented as mere techniques of measurement, thus staying away from controversial political choices, complex social realities about which there is little agreement and the corresponding risk of ‘gridlock’ in global governance (for such gridlock, see Hale et al., 2013).

From the point of view of Luhmannian systems theory, the maintenance of social complexity produces and requires differentiation of codes and programmes. On this view, there is little to be gained by repoliticising what law and indicators have depoliticised. Public consensus is a chimera at a time of increasing (and competing) types of expertise – nor are the resources available that would be needed to allow members of civil society to double check on indicators. In other words, too much contestation can have bad effects, because, as with law, indicators need to have some means of imposing closure at some point if they are to be of practical use. Whether and when contestation can be seen as a sign of success – or of failure – will therefore depend on how far a given indicator needs to provide certainty to users and targets.

4.3 What does ‘law’ have to do with it?

A further question is whether and how law-makers should react to global social indicators. In general studies of ‘law and regulation’, it is often said that law can play an important role for legitimacy claims, though, in the international context, it may mainly be the case that it can structure regulatory conversations (Morgan and Yeung, 2007, pp. 244, 330–331). While there are some similarities between indicators and international legal instruments, it is also clear that there are differences, such as at the production stage (Davis et al., 2015, pp. 18–19). Here, an important consideration is that legitimate indicators need to ‘get the numbers right’: thus, at the level of soft law, international codes of good practice and ethics for statistics may be needed (Lievesley, 2001). It may also be possible to regulate some of the procedural aspects of indicators (Nelken, 2015, p. 328), aiming to improve their procedural legitimacy.

Alternatively, it may be said that law-makers should simply not allow the proliferation of global social indicators. For example, critics of indicators refer to a lack of democratic participation and accountability and, in the international sphere, a ‘risk of imperialism’ (Scamardella, 2015). They also argue that indicators camouflage political considerations and provide false specificity (Merry, 2016, pp. 19–20) and may therefore be seen as a form of ‘expert rule’ that is detached from its distributive consequences (Kennedy, 2016). It is also possible to relate indicators to the growing use of mathematical models in modern life, the problem being that such models are said to be ‘opaque, unregulated, and incontestable, even when they're wrong’ (O'Neil, 2016, as ‘weapons of math destruction’). Thus, a possible response may be that law-makers themselves should take responsibility for such matters of public concern.

However, the reason global social indicators have emerged is often that local and national legislation would not be sufficient and that agreement on international legal norms is not feasible. Thus, in those circumstances, the choice of formal law is not available. In addition, some of the criticism may go too far. As the next section will discuss, power imbalances are possible but not inevitable for all indicators. It is also contentious to criticise the fact that indicators give more power to experts than conventional law-making: the political events of 2016 show rather the opposite problem, namely that of a ‘post-truth’ political climate in which people are said to have ‘enough of experts’. ⁶

Finally, it is not just the case that law can regulate or restrict indicators but that legal rules may foster their success as they can provide an ‘aura’ of legitimacy (Bernstein, 2004, pp. 10–14: ‘legitimacy as legalisation’). This has both a sociological and a normative dimension. Once a particular indicator is legally embedded, users and other stakeholders are more likely to consider it as legitimate. Law-makers also contribute to the normative legitimacy of indicators, since, with ‘legalisation’, it is then not simply the power or persuasion of the originator of the indicator that matters, but also the law-maker who acts on behalf – and, in democratic societies, with representation – of the users of those indicators.

V. Legitimacy of indicators in action

5.1 Previous research

It has been said that ‘approaches to legitimacy have generally paid little attention’ to the study of such claims and ‘beliefs in real-life situations’ (Quack, 2010, pp. 7–8). To be fair, some of the previous studies have addressed the legitimacy of specific indicators. Beisheim and Dingwerth (2008) assess the Global Reporting Initiative (GRI) and identify ‘the development of ownership based on inclusive, fair and representative participation; social learning and persuasion based on deliberative procedures; and social control based on transparency and accountability’ as factors of procedural legitimacy. Dubois and Nowlan (2010) put the initiatives of the World Bank in the context of global administrative law, which is said to explain legitimacy as a function of the bank's transparency and structured decision-making. Niemann (2013) deals in more detail with the World Bank's Doing Business Report (DBR) and identifies how the enrolment of experts, professionals and academics creates legitimacy and how the bank keeps tracks of how ‘prestige and media attention plays a large role in the popularity and legitimacy of the DB programme’.

These three studies address important issues. However, considering the wider range of indicators of Table 1 (see Section II, above), the following identifies further dimensions of global social indicators in action and their implications for legitimacy. Doing so, it suggests that we have to consider that indicators differ in many respects, starting with the data, drafters, addressees and beneficiaries, and then dealing with further dimensions.

5.2 Data, drafters, addressees and beneficiaries

Indicators differ in the way they make use of objective data (e.g. the Indicators of Employment Protection), subjective data (e.g. the Corruption Perception Index) or both (e.g. the Worldwide Governance Indicators (WGI)). Regardless of the method, many indicators have been criticised due to problems of measurement and aggregation (e.g. Arndt and Oman, 2006, pp. 49–76, for the WGI) or simple arithmetic flaws, even in the most high-profile reports (Pogge, 2004, for the UN-Millennium Development Goals). Naturally, such flaws are detrimental to the legitimacy of the indicator in question. Apart from such general measurement problems, subjective data may have a slight advantage in terms of procedural legitimacy, since surveys can involve the potential users of the indicator in an early stage.

As we have seen, the effectiveness of an indicator is correlated with its sociological legitimacy. A further distinction can therefore be made between indicators by drafters that are economically and politically powerful (such as the UN or the World Bank, cf. Darian-Smith, 2016; Perry-Kessaris, 2011) and by others that merely have to rely on persuasion (e.g. the Good Country Index). However, from the perspective of normative legitimacy, it can also be said that these latter indicators may be preferable, since they raise fewer concerns about the deficit of democratic governance structures at the global level.

The drafters of the indicators can be international public bodies (such as the World Bank, the UN, the OECD, the EU) or private ones. The former can have an advantage as far their structures reflect the input legitimacy of their members. However, some private bodies may also benefit from their high reputation (e.g. the ISO). It can also be the case that the independence and neutrality of a private body foster its credibility and therefore the input and output legitimacy of the indicator (e.g. this may be the case for the Press Freedom Index by Reporters without Borders).

Next, many indicators take the country as the unit of comparison while others address private entities such as universities or corporations (e.g. the global university rankings; the GRI guidelines). In terms of legitimacy, the former indicators may be criticised as far they bypass principles of international law and democratic participation at the domestic level. Similar concerns about the latter indicators may be more indirect, such as when the indicator determines the behaviour of the unit and there is little that the state in which the unit is based can do to prevent it (e.g. one may think about the global university rankings).

Indicators can also be divided into those that aim to benefit specific groups and those that have a wider range of potential beneficiaries. Often this may derive from the content of the indicators: for example, the Global Competitiveness Report is more focused on economic interests while the Happy Planet Index is a wide-ranging one. Drafters influence how narrow or wide they design their indicators. For example, having a wide scope can be beneficial to the procedural and substantive legitimacy of an indicator as far as it manages to balance all of the relevant interests. However, there is also a trade-off, since an indicator that tries to please everyone may become too diffuse to be of use for any of the beneficiaries.

5.3 Further dimensions

Going beyond the specific drafters, addressees and beneficiaries, some indicators are embedded in a broader network of institutions which support the indicator, while others have a ‘slimmer’ support structure (such as many of the indicators drafted by academics). An example of the former are the World Bank indicators, since the main shareholders of the World Bank, other donor institutions, financial institutions, corporations, non-governmental organisations (NGOs), etc. all influence the role these indicators play in donor countries. The legitimacy of such indicators may benefit from such a network as far it makes the indicator more effective and, possibly, accountable to a larger constituency.

Next, while all of the indicators discussed here have a global ambition, not all of them can said to be relevant at a global level. For example, the Index of Legal Certainty has the main aim to defend the French civil law model of the world, the global university rankings are mainly focused on Anglophone institutions and the Failed State Index is mainly relevant for the assessment of countries at the bottom of this list. Broad indicators such as the Rule of Law Index are ‘more global’ in their ambition but face the challenge of international comparability. There is also the issue of possible mismatches between the indicator and the local context: while it is unrealistic to expect full agreement about the values that underlie global indicators (Frydman and Twining, 2015, p. 4), the legitimacy of an indicator is bound to suffer if it is frequently not accepted at the local level. Indeed, it may therefore be said that indicators need to be ‘paired with context-rich qualitative accounts grounded in local knowledge’ (Merry, 2016).

A further distinction can be made according to the focus of indicators on economic or wider social goals, such as the Global Competitive Report on the hand and the UN Global Compact on the other. From those examples, it can be seen that indicators are not necessarily ‘left-wing’ or ‘right-wing’. This is not to say that there may not be a differences: for example, indicators that support economic interests have a natural advantage as far as they are supported by influential multinational corporations (Krever, 2013).

Some indicators are rather unique while others have to compete with indicators that cover similar topics. Examples of the first group are the Global Slavery Index and the Index of Legal Certainty, while the indicators on rule of law and university rankings compete in a crowded field. The general expectation is that indicators that operate in a well-functioning market for indicators are more likely to be legitimate than ‘monopolistic indicators’, since their drafters face pressure to fix possible technical flaws. But it is also possible that multiple organisations that work on the same topic operate in a cartel-like structure, with the result that flaws are not fixed – or even proliferate (parallel to the indicators of the ‘big three’ credit rating agencies; see Infantino, 2016, p. 363).

Finally, indicators differ in the extent to which they have a system of checks and balances. For example, the World Bank's DBR initially included an employment law sub-index that penalised countries with high levels of employment protection. After considerable protest, the World Bank decided to drop this sub-index from its DBR rankings (see McCann, 2015). In preparation for the recent versions of the DBR, the World Bank also engaged with some of the critics of its report (including one of the authors of this paper). This openness to criticism is certainly beneficial for the legitimacy of an indicator. However, it is also rare. In particular, it is unlikely that the drafters of indicators will listen to radical critics who reject, for example, the ‘mathematical turn in law’ (for the discussion, see Restrepo Amariles, 2015, pp. 14–19) or the possibility of university and journal rankings (for the discussion, see van Gestel, 2015).

Overall, it can be seen that the legitimacy of global social indicators ‘in action’ shows various nuances which drafters and users of indicators – as well as domestic and international law-makers – need to consider. It should also be noted that the dimensions identified here are not independent of each other; often there may be trade-offs between increasing the legitimacy in one of those dimensions while decreasing it in another one.

VI. Conclusion

This paper has discussed how the concept of legitimacy can be relevant for the discussion about the use (and abuse) of global social indicators. It addressed these issues at both the general level of social theory and the operation of indicators in action. In this concluding section, it is worth returning to the general questions about legitimacy ⁷ : What type of legitimacy can such indicators have? Do indicators turn the goals of normative legitimacy into the means to sociological legitimacy (or vice versa)? Who are the relevant groups that need to be considered for their legitimacy (markets, targets, users, etc.)? How does it compare to alternative ways of achieving the same functions?

The first two questions concern types of legitimacy and the relationship between sociological and normative perspectives of legitimacy. It was found that, for global social indicators, types and perspectives of legitimacy often overlap. But, more specifically, it can also be said that aspects of procedural legitimacy can more easily be addressed than substantive ones, since the drafters of indicators do not have to balance interests in the same way as democratic law-makers would have to do. The normative legitimacy of indicators may sometimes also be questionable, in particular as far as global indicators challenge national sovereignty.

With respect to the relevant groups that indicators consider, it has to be acknowledged that some indicators are skewed towards economic interests. However, today, there are also global social indicators dealing with a wide variety of topics – and correspondingly aimed at a wide variety of different interests. For questions of legitimacy, it is difficult to say whether a narrow (economic or otherwise) or broader approach is preferable. While the input legitimacy may benefit from the involvement of many groups, there is also the trade-off that the very success of some of the indicators depends on their limited ambition as mainly technical tools.

Finally, we suggest that indicators can be an effective way to organise a particular field. Due to the soft (or ‘liquid’) nature of these indicators, it could be argued that they have an inbuilt mechanism of checks and balances: if a drafter were to ignore all criticism, the perceived legitimacy of its indicator would suffer. But this idealist position may be rare in practice, since indicators often construct ‘the truth’ so that it is difficult to find a lever to move them. There can also be circumstances where legal intervention may be necessary: for example, where an indicator is contrary to certain values (human rights, etc.), law-makers may want to (or indeed need to) intervene and enact mandatory laws in order to counter the effect of the indicator in question.