I. Introduction
Scholars from different disciplines increasingly point out the significant role indicators tend to play in contemporary societies. Indicators have been described as ‘integral to the fabric of global governance’ (Cooley, Reference Cooley, Cooley and Snyder2015, p. 17), a ‘technology of governance’ (Davis et al., Reference Davis, Kingsbury, Merry, Davis, Fisher, Kingsbury and Merry2012b, pp. 10–21), ‘tools in the global struggle for law’ (Lewkowicz, Reference Lewkowicz, Frydman and van Waeyenberge2014, pp. 167–168), ‘a mode of governance’ (Frydman, Reference Frydman, Frydman and van Waeyenberge2014b, p. 58; Merry, Reference Merry2011, p. 85), ‘normative devices in a non-sovereign environment’ (Restrepo Amariles, Reference Restrepo Amariles2015, p. 14) and even as ‘weapons of choice for the knights of investment climate discourse’ promoted globally (Perry-Kessaris, Reference Perry-Kessaris2011, p. 402). Although the existing literature has been prolific in unveiling the power indicators exercise at various levels of decision-making, only a few studies have attempted to characterise the concrete mode of governance they rely upon (Le Texier, Reference Le Texier2013, pp. 162–164). Gaining a deeper understanding of how indicators become entrenched in institutional practices and interconnect with other normative devices should help in revealing new insights about their reactivity (Espeland and Sauder, Reference Espeland and Sauder2007; Espeland and Sauder, Reference Espeland, Sauder, Davis, Fisher, Kingsbury and Merry2012, pp. 95–97) and legitimacy (Siems and Nelken, Reference Siems and Nelken2017). It should also help in shedding light on the most effective ways to improve, resist or challenge their power (Frydman and Twining, Reference Frydman and Twining2015, p. 5; Merry, Reference Merry2011, p. 85; Twining, Reference Twining2000, pp. 157–165).
This paper attempts to explore the legitimacy of global legal indicators as performance measures deployed in a system of management control and distributed governance. In this context, the paper shows that legitimacy is contingent on an indicator's capacity to generate subjectification, self-knowledge by comparison, accountability and stimulus for action throughout its cycle of production and implementation. The paper also shows that indicators with high degrees of legitimacy become entrenched in institutional practices and decision-making processes, eventually driving the institutional behaviour of organisations within a defined global regulatory space. The paper concludes that, as legal indicators and their underlying managerial rationality pervade legal processes, law's authority and democratic rationality are undermined.
The contours of indicators as a subject matter of scholarly interest for academic lawyers and socio-legal scholars remains fairly undefined. As Twining noted, there are not only different kinds of relationships among indicators, but also many kinds of indicators (Frydman and Twining, Reference Frydman and Twining2015, p. 7). In this special issue, Siems and Nelken (Reference Siems and Nelken2017) explore the legitimacy of indicators with global ambitions and related to societal concerns, as opposed to merely local and economic indicators. Sally Merry (Reference Merry2011, p. 89) has also predominately looked at non-economic global indicators, which has not prevented her from exploring local roots and contestations. Others scholars such as Borges Fortes (Reference Borges Fortes2015, pp. 45–48) have analysed local implementations of global indicators, whereas Frydman (Reference Frydman, Frydman and van Waeyenberge2014b, pp. 41–65) and Lewkowicz (Reference Lewkowicz, Frydman and van Waeyenberge2014, p. 145) have focused on indicators that complement, compete or challenge state law in the real world irrespective of their presumptive field of concern or geographical scope.
This paper focuses on global legal indicators as a subcategory of social indicators, which includes measures, standards and rankings of the quality of law and legal institutions across the entire world, particular regions, or selected jurisdictions (Twining, Reference Twining2009, p. 254). In contrast to social and economic indicators, legal indicators are concerned with measuring institutional performance that is legally relevant. They may belong to specialised legal indexes such as the Legal Certainty Index or the Financial Secrecy Index, or be part of larger sets of socio-economic data such as the Global Competitiveness Index or the World Governance Indicators (WGI).
The conception of legal indicators adopted here is significantly broader than the notion of transnational legal indicators, which covers measurements of the performance of state legal systems or parts thereof in a comparative perspective (Restrepo Amariles, Reference Restrepo Amariles, Fortes, Boratti, Lleras and Daly2017, p. 95; Reference Restrepo Amariles, Frydman and van Waeyenberge2014, pp. 9, 11). As a consequence, it is also broader than the conception put forth by Kevin Davis, who defines legal indicators by reference to whether they measure some component of state legal systems (Davis, Reference Kevin2014, p. 39). In addition to covering state law, the notion of global legal indicators also includes measures of legal performance from non-state actors operating across jurisdictions. This comprises indicators such as the CRShub and the Corporate Human Rights Benchmark (CHRB), which measure, among others, the extent to which multinational corporations respect human rights conventions and labour regulations, but tend to go unnoticed under narrower conceptions of legal indicators.
In this work, legitimacy is defined as the quality of an indicator by means of which organisations and individuals subject to measurement – namely the addressees – feel compelled to acknowledge its results and to use them as a basis for taking action. This notion of legitimacy largely overlaps with the idea of reactivity of social measures studied by Espeland and Sauder (2007, p. 6). For them, reactivity defines a situation in which individuals ‘alter their behavior in reaction to being evaluated, observed, or measured’ (Espeland and Sauder, Reference Espeland and Sauder2007, p. 6), which is related with the fact that, as pointed by Siems and Nelken (Reference Siems and Nelken2017), indicators collapse the normative and sociological dimensions of legitimacy.
The change in the behaviour of organisations in reaction to a systematic assessment of their legal performance may also be defined in terms of what Dimaggio and Powell (Reference Dimaggio and Powell1983, p. 150) called a source of isomorphic organisational change and, in particular, a type of coercive isomorphism. Indeed, organisations feel compelled to get in line with the precepts underlying indicators because they are believed to convey certain societal expectations. Hence, compliance with them may help an organisation gain support in the long run among certain key stakeholders. Nonetheless, one should be aware that to predicate the legitimacy of an indicator on these terms does not entail that addressees endorse the standpoints and assumptions of the indicator. The notion of legitimacy adopted here is defined entirely from a pragmatic perspective (Frydman, Reference Frydman, Watt and Arroyo2014a, pp. 181–185; James, Reference James1975, p. 29). A pragmatic definition of legitimacy emphasises the effects of indicators over their origin or soundness, by looking at it in terms of whether addressees are compelled to change their behaviour with the purpose of improving their scores or position in the rankings.
This paper is divided in eight sections. The next section introduces the notions of managerial rationality and key performance indicators (KPIs), and Section III analyses the process of legitimation of legal indicators in a system of management control. Sections IV–VII discuss the cycle of production and implementation of indicators and the related processes through which indicators gain legitimacy, namely subjectification, self-knowledge by comparison, accountability and stimulus for action. Section VIII concludes with some critical remarks.
II. Managerial rationality and legal performance
The increasing attention legal indicators are receiving by policy-makers, private decision-makers and academics is a worrying signal of the extent to which managerial rationality has pervaded law in our society, and of the capacity of other forms of rationality to resist this influx. Thibault Le Texier has provided an insightful account of the managerial rationality overtaking bureaucracies of public organisations. He argues that management is not a science or an ideology, but a type of rationality, namely an ensemble of structured and formalised concepts and mental schemes that shape the representation and principles of judgment and action (Le Texier, Reference Le Texier2016, p. 113). Legal indicators seem to be spearheading the progression of such rationality in the legal sphere. They redefine the premises on the basis of which institutional behaviour is discussed and assessed by framing the analysis in classic managerial terms such as quality, efficacy and efficiency (Le Texier, Reference Le Texier2016, pp. 114, 149).
Indeed, indicators have been broadly referred as part of a culture of performance (Davis et al., Reference Davis, Kingsbury, Merry, Davis, Fisher, Kingsbury and Merry2012b, p. 9; Davis et al., Reference Davis, Kingsbury and Merry2012a, p. 77) and auditing (Shore and Wright, Reference Shore and Wright2015, p. 24; Parleea and Wiber, Reference Parleea and Wiber2015, p. 97; Merry, Reference Merry2011, p. 84). If both assertions are doubtlessly right, more clarity is needed about what kind of performance measures legal indicators are and what role auditing plays in constructing their legitimacy. I will discuss the question of performance here and leave the discussion on auditing for a later section.
Producers of indicators often proclaim to be measuring performance. For instance, the Rule of Law Index claims to measure the extent to which countries attain the rule of law in practice by means of performance indicators (World Justice Project (WJP), 2016, p. 9), whereas the CSRHub seeks to provide ‘consistent ratings of Corporate Social Responsibility (CSR) performance for as broad a range of companies as possible’.Footnote 1 Similarly, the CHRB claims to be the ‘first-ever ranking of the world's largest publicly listed companies on their human rights performance’.Footnote 2 Finally, the Doing Business Index has recently introduced a measure of distance to frontier to capture ‘absolute regulatory performance and how it improves over time’ (The World Bank, 2016, p. 14), as opposed to capturing only relative performance through rankings.
A performance measure is broadly defined as a ‘general term applied to a part of the conduct of the activities of an organisation over a period of time, often with reference to some standard or base, with emphasis on management responsibility and accountability, or the like’ (Edwards, Reference Edwards1986, p. 5). Measuring performance is nothing else than assessing the extent to which the organisational objectives have been achieved, whatever their nature and diversity. In management practice, a performance measure is deemed successful if it facilitates the management control process through which managers assure that resources are obtained and used in an efficient and effective manner to accomplish the objectives of the organisation (Edwards, Reference Edwards1986, p. 2).
Global legal indicators may be characterised as non-financial KPIs measuring the legal performance of organisations. KPIs have traditionally been used in corporate settings to help managers to understand whether the company is heading in the right direction or is veering off the promising path (Löning et al., Reference Löning, Aalleret, Aéric and Pesquex2016, p. 118; Marr, Reference Marr2012, p. xxv). Similarly, legal indicators allow decision-makers to keep track of the legal performance of organisations in two main aspects.
On the one hand, some indicators measure the legal and socio-economic performance of an organisation's own institutions and rules. For instance, the Doing Business Index measures the extent to which state legal systems or parts thereof facilitate the creation and operation of businesses. On the other hand, some indicators measure the extent to which organisations respect and abide by third-party rules, whether these are public or private. For instance, the CSRHub indicators measure the extent to which corporations abide by international human rights law and implement good practices in the field of corporate social responsibility.
KPIs should be distinguished from Key Risk Indicators (KRIs). KPIs are important for general management because they are concerned with the company's historical performance. Instead, KRIs provide real-time measures about the risks of an activity or investment project. As Scarlat et al. argue, the key difference between KPIs and KRIs is that ‘KPIs tell us if we will achieve our goals, and KRIs help us understand changes in risk profile, impact and likelihood to achieve our goals’ (Scarlat et al., Reference Scarlat, Chirita and Bradea2012, p. 7). Since most global legal indicators are at best updated annually (Restrepo Amariles and Maclahan, Reference Restrepo Amariles, McLachlan, Deffains and Kessedjian2015, pp. 9–13) and look at organisations’ performance in terms of conformity with certain objectives rather than in terms of risk, it would be inaccurate to take them as risk measures by themselves. However, legal indicators may be used for legal risk management in combination with other types of data.
The development of global legal indicators seems to follow in the footsteps of other non-financial indicators, which complement the array of financial KPIs measuring the performance of states and corporations. Corporations and states used to focus attention only on financial KPIs. While corporations focused on measuring EBIDTA, total shareholder return (TSR) or market growth rate, states emphasised the Gross Domestic Product (GDP) and the Consumer Price Index (CPI). Corporations progressively adopted non-financial KPIs to account for customer and employee satisfaction, innovation and other intangible assets needed for continuous growth (Kaplan and Norton, Reference Kaplan and Norton1996, p. 2). States also put in place non-financial indicators in the last two decades to cover a broader set of societal concerns ranging from human development (Sen, Reference Sen2000) to happiness (Cavalieri, Reference Cavalieri2015). Recently, the report of the Stiglitz-Sen-Fitoussi Commission confirmed the need to shift focus on the way public measurements are designed. It called to emphasise people's well-being rather than economic production to account more realistically for social progress (Stiglitz et al., Reference Stiglitz, Sen and Fitoussi2010, pp. 12–13).
The means through which global legal indicators achieve legitimacy and reactivity may differ from traditional KPIs. Global legal indicators are not internal organisational measures, neither are they designed to assess progress towards internal goals or objectives. Additionally, producers of global legal indicators are diverse, and rarely have a role of management oversight and strategic direction over the organisations subject to measurement. This contrasts sharply with the role of the board of directors and executive managers issuing and using KPIs in corporate settings. Moreover, producers lack a direct chain of authority vis-à-vis states and corporations, and are only occasionally involved in direct project implementation. These particular circumstances require us to explore the legitimacy of legal indicators by combining theories of management and governance in the next sections.
III. Performance management and control
Management scholarship considers performance indicators to be integral to a system of management control. In turn, management control is defined as one of the four modes of control used to ensure goal convergence (Löning et al., Reference Löning, Aalleret, Aéric and Pesquex2016, pp. 8–9). The other three are: (1) bureaucratic control by regulations and procedures, to which arguably law belongs, (2) human control by satisfaction and (3) social control by supporting common values. Management control was developed to support the decentralisation required by major American corporations in which decentralised managers needed to take decisions quickly to adapt to and be competitive in the local context (Löning et al., Reference Löning, Aalleret, Aéric and Pesquex2016, p. 18). Hence, central management sought to ensure goal convergence by delegating decision rights and resources, while setting objectives to decentralised managers and monitoring results to ensure they were achieved (Löning et al., Reference Löning, Aalleret, Aéric and Pesquex2016, p. 9).
The idea that performance measures can help driving behaviour as part of a system of control is not particularly new. In Principles of Scientific Management, Frederick Taylor (Reference Taylor1919, pp. 117–121) describes the different stages through which quantification, systematisation and what we would call today benchmarking lead to the definition of single standards that can then be replicated to ensure quality in engineering and industrial processes. Michael Power (Reference Power, Hopwood and Miller1994b, p. 299) defines the transformation of the concept of ‘quality’ from an engineering to a management concept as a ‘managerial turn’, because it has moved quality assurance from its so-called ‘hard’ technical base towards the ‘cultural’ emphasis that one finds in the idea of Total Quality Management (TQM). Since the 1970s, performance measures have become broadly used to manage the behaviour of individuals and private organisations, and to direct them towards certain objectives (Daniels and Rosen, Reference Daniels and Rosen2004). In the 1990s, Christopher Hood (1991) theorised the notion of New Public Management, showing the extent to which the introduction of performance measures and related management practices to government agencies and public services led to an efficiency-driven and outcome-oriented model of administration (Hood, Reference Hood1991, pp. 15–16).
Global legal indicators are not standalone managerial devices in global governance. They are integral to a system of management control based on a broader set of institutional practices and networks. As Löning et al. argue:
‘Performance management and measurement relies on a simple and fundamental assumption: ‘You get what you measure’. However, simply measuring performance is not enough as measures can be manipulated and almost always mask dysfunctional behaviour effects. Measurement must therefore be integrated into a management system, for which it can serve as an objective and clear starting point, but which it cannot replace.’ (Löning et al., Reference Löning, Aalleret, Aéric and Pesquex2016, pp. 157–158, bold in original)
Keeping sight of the entire managerial approach is meaningful because it explains comprehensively how indicators define standards of performance, determine thresholds of compliance, gain reactivity and exercise compliance audit. Instead, an audit approach may be too narrowly focused. It is likely to emphasis exclusively the monitoring stage, but can hardly tell us where these standards come from, what are the underlying assumptions and why organisations change their behaviour to meet them. A managerial approach also pushes us to enquire about the consequences of the spread of a managerial rationality in the legal sphere, and about how it can be corrected, resisted and contested. Martti Koskenniemi (Reference Koskenniemi2007, p. 29; 2011, p. 359) argues that managerial practices are a means of driving behaviour while masking the underlying rationale. He claims that, when managerial practices reach international law, the politics behind it are shadowed. From this perspective, the managerial rationality spreading to the legal sphere would not be in competition with politics, but a substitute for it (Koskenniemi, Reference Koskenniemi2007, p. 29; 2011, p. 359).
Nonetheless, global legal indicators are not classic managerial measures. As opposed to KPIs in corporate and public management, they are not internal organisational measures. Indeed, their capacity to generate reactivity is not attached to internal authority, hierarchy or management culture. They thrive by ensuring appropriation and awareness of both the measurement process and the results. Hence, to understand how legal indicators achieve legitimacy, one needs to study how addressees in specific contexts develop an identity with their cycle of production and implementation. This cycle can be divided into four processes: data-producing, benchmarking, auditing and attributing incentives (Löning et al., Reference Löning, Aalleret, Aéric and Pesquex2016, pp. 7, 147–164). Each of them participates in the legitimacy of legal indicators by respectively generating subjectification, self-knowledge by comparison, accountability and stimulus for action.
IV. Data production as subjectification
The first step in the production of performance measures consists of the design and implementation of a database (Edwards, Reference Edwards1986, pp. 5–7). Producers of global legal indicators run extensive databases providing information about the legal performance of organisations. For instance, Kaufmann et al. claim that the World Bank's WGI enhance the quality of information on governance and provide solid data to donors, risk-rating agencies and policy-makers (Kaufmann et al., Reference Kaufmann, Kraay and Mastruzzi2011, pp. 242–243). Similarly, the WJP explicitly claims that the Rule of Law Index's approach ‘provides practitioners with performance information about the outcomes they ultimately want to influence through reform’ (Botero and Ponce, Reference Botero and Ponce2010, p. 16). Finally, the Doing Business Index argues that its database facilitates access to managers and legal reformers to relevant information about the performance of business law and regulation of countries (The World Bank, 2004a, pp. 12, 19, 105). Overall, global legal indicators are designed to collect and make data available in their concrete domains of interest.
The production of a database involves two main steps: first, to determine the proxies or conceptual categories that will be used to measure the objectives pursued; second, to actually collect performance data. This twofold process of data production contributes to the legitimation of the indicator by generating subjectification. Subjectification allows the reification of the subject by generating awareness of how one becomes subject of knowledge, power and moral actions (Foucault, Reference Foucault, Defert, Edward and Lagrange1994, p. 576). As a subject of knowledge, the subject is created not by the utterance of labels (mots) or the visibility of things (choses), but, as Foucault claims, by the ‘median region’, namely ‘by a pre-coded observation and reflexive knowledge’ (Foucault, Reference Foucault1966, p. 12). In other words, subjectification is a reflexive manner of constructing an image about ourselves by looking inwards through pre-defined categories.
Global legal indicators generate subjectification by creating conceptual categories about legal performance and by applying them to collect empirical data. Indicators become mirrors where organisations look at themselves to acquire self-knowledge and a representation of themselves, eventually incorporating the built-in categories into internal processes. Borges Fortes describes clearly this situation in Brazil with the programme justice in numbers. He explains:
‘calculation of the length of procedures was uncommon in the Brazilian justice system, but the necessity of producing legal indicators for the “rule of law index” and “Doing business” radically transformed the panorama of justice administration. Nowadays, with the widespread belief that judicial delay generates negative externalities and profound social costs to plaintiffs, all tribunals are expected to produce regular statistical measurements of various aspects of the justice system.’ (Borges Fortes, Reference Borges Fortes2015, p. 45)
Concretely, indicators trigger subjectification by either getting organisations to report data according to a pre-defined grid or set of categories, or by collecting directly the data and then submitting it, individually or publicly, to the organisation. A telling example is the joint initiative of the UN and Standard & Poor's to provide twenty-five African countries with a sovereign debt rating and to monitor their performance (Krause Hansen and Mühlen-Schulte, Reference Krause Hansen and Mühlen-Schulte2012). As Lewkowicz (Reference Lewkowicz, Frydman and van Waeyenberge2014, pp. 1, 56) explains, this public initiative sought to enlarge the scope of action of rating agencies to countries that either did not have an economy large enough to receive the rating free of charge or were not rich enough to pay for it. By obtaining a sovereign debt rating, argues Lewkowicz, they gained access to the global capitals market (Lewkowicz, Reference Lewkowicz, Frydman and van Waeyenberge2014). In terms of subjectification, the ratings provide states with knowledge they previously lacked about themselves, such as, of course, the risk of default, but also about all the other categories included in Standard & Poor's grid of analysis for which these states had never gathered data. These include, among others, the effectiveness, stability and predictability of policy-making and political institutions, as well as the transparency and accountability of institutions, data and processes, which comprise the rule of law and contract enforcement (Standard & Poor's, 2011, pp. 9, 13).
These categories are eventually institutionalised by the organisations as they collect and report the data to the issuer, if required to do so, or as they become acquaint with the indicator's results and seek internal validation or contestation. This individual engagement of the addressees with the categories and data-production process of the indicator ends up contributing to the reactivity of its measures.
V. Benchmarking as self-knowledge by comparison
As Löning et al. note, indicators require benchmarks to put performance in a comparative perspective. They argue:
‘A quantified indicator without a basis for comparison is pointless since it cannot be interpreted. “A figure on its own is meaningless” and must therefore always be accompanied by another one that allows the reader to infer whether the situation is good or not.’ (Löning et al., Reference Löning, Aalleret, Aéric and Pesquex2016, p. 153)
In the corporate sector, some of the most common sources of benchmarking include a competitor, a comparable business belonging to the same company or a non-rival company with which information exchange agreements have been concluded (Löning et al., Reference Löning, Aalleret, Aéric and Pesquex2016, p. 153).
The underlying rationale of a benchmark is that the best way to know yourself is by comparing your performance to that of similar others. Competitive benchmarking carries the idea that ‘it is not enough to achieve the targets set: it is also essential to do as well, if not better, than our competitors’ (Löning et al., Reference Löning, Aalleret, Aéric and Pesquex2016, p. 154). Benchmarking shapes the structure and processes of organisations by placing them in a context of relative performance. Self-knowledge by comparison contributes to the legitimacy of global legal indicators because it renders available the performance of organisations competing for resources and reputation, while creating an implicit agreement on the goals and objectives they should strive to achieve. Moreover, it contributes to the institutionalisation of the categories used in the stage of data collection by establishing them as pertinent comparators.
In contrast to subjectification, which is about knowing yourself individually by looking inwards, self-knowledge by comparison is about getting to know yourself by looking outwards. Hence, benchmarking allows organisations to identify weak and strong performance beyond internal standards and beliefs. The perverse effect is the implicit incorporation of the underlying values and objectives of the benchmark as part of the organisation's own processes and goals, often disregarding its own context and internal culture. In the end, benchmarking practices not only helps organisations to know how well they perform comparatively, but also pushes them to align with the goals and objectives of their peers. Benchmarking exemplifies as much as it contributes to collapse the sociological and normative dimensions of legitimacy of global legal indicators.
Global legal indicators benchmark legal performance through two main techniques, namely rankings and best practices.
5.1 Rankings
The ranking system of global legal indicators operates much like what William Twining calls ‘league tables’ in the field of educational rankings (Twining, Reference Twining2000, p. 157). ‘League tables’ define the way in which rankings such as US News and World Report organise and report statistical data about the quality of universities and professional schools. Twining also calls to attention the dubious methodologies and criteria on the basis of which they compare and rank universities, which eventually lead to establishing hierarchies among them (Twining, Reference Twining2000, pp. 161–163). He also warns us about dangerous uses and, sometimes unavoidable, misuses of them (Twining, Reference Twining2000, p. 157). Despite their lack of sophistication and precision, educational rankings became successful tools organising the US higher education. In the same process, they set ‘Ivy League schools’ as benchmarks. Today, educational rankings are recognised to be highly influential in the election prospective students make of universities (Espeland and Sauder, Reference Espeland and Sauder2007, p. 21).
Similarly to educational rankings, global legal indicators emphasise the position of an organisation in the ranking. Rankings are measures of relative performance because they establish the performance of an organisation in relation to the performance of other organisations. This means that the governing standard – or benchmark – of a ranking is dynamic. For state legal systems, it changes along with the legal reforms implemented by other countries and the performance improvements or deteriorations they undergo. The ‘league table’ approach increases the reactivity of indicators, while exacerbating their pernicious effects, by facilitating access to comparative performance information by potential users of legal data.
For instance, business lawyer Al-Fadhel relies on the Doing Business Index to point out loopholes in legislation and the need to improve contract enforcement in Saudi Arabia, without looking at context or evaluating potential harmful effects. He claims that:
‘when one scrutinizes the indicators used to analyze the economic outcomes and identify which reforms have worked, where, and why, one may be surprised to discover that out of the 183 economies, Saudi Arabia is ranked 140 in the “Enforcing Contracts” indicator. In the period ending May 2008, Saudi Arabia ranked 136. These rankings confirm that the country still lags behind in contract enforcement and the implementation of foreign decisions. They also clearly indicate the need for sweeping reforms in the judicial system of Saudi Arabia and a corresponding amendment in the Saudi Arbitration Law.’ (Al-Fadhel, Reference Al-Fadhel2010, p. 46)
Al-Fadhel provides us with an example of how scholars, lawyers, firms, individuals and reformers may dangerously rely on rankings to assess legal performance. Al-Fadhel provides some words of caution by explaining that one should identify ‘which reforms have worked, where, and why’ (Al-Fadhel, Reference Al-Fadhel2010, p. 46). In any case, this type of analysis ends up reinforcing the criteria of assessment and the hierarchies resulting from the ranking. As Borges Fortes and Sam Schueth show with their respective analysis of the programme ‘Justice in Numbers’ in Brazil and the implementation of Doing Business and USAID programmes in Georgia, organisations eventually implement internal reforms and performance measures to attempt improving their position in the ranking (Borges Fortes, Reference Borges Fortes2015, p. 45; Schueth, Reference Schueth, Cooley and Snyder2015). More worryingly, Borges Fortes also reports changes in the individual behaviour of judges in response to the statistical measurement of judicial activity by the Brazilian National Council of Justice (Reference Borges Fortes2015, pp. 50–53). By using legal indicators to compare with others, organisations and individuals end up sharing their underlying values and objectives, and reinforcing, often against their own will, their legitimacy.
5.2 Best practices
But indicators not only produce information to facilitate straightforward comparisons. When used as part of a system of management control, they also ensure good practices are identified and shared. Benchmarking as a tool of legal performance management provides evidence encouraging low performing organisations to reproduce or transplant good practices from high-performing organisations as a means to improve positions in the ranking. Mark Brown sets out this dimension of benchmarking clearly:
‘Borrowing the good ideas of others is what benchmarking is all about. This is how you can save a great deal of time and effort in coming with good strategies. Rather than inventing your own, find out what has worked in other organizations and implement it in yours.’ (Brown, Reference Brown1996, p. 89)
Most producers of global legal indicators use the knowledge collected and organised in indicators to point out deficient legal performance. Subsequently, they promote reforms on the basis of the good practices identified among the best-performing organisations. Countries and corporations increasingly tend to adopt those recommendations without critical scrutiny for the plain fact that they result from a benchmarking exercise and are correlated with better performance. Okonjo-Iweala, former Minister of Finance and Foreign affairs of Nigeria, cheered the World Bank for providing reformers with indicators to guide their work on governance. He says: ‘Now we can benchmark performance in various governance areas and measure progress’ (The World Bank, 2007, p. 27). Similarly, Canadian organisations have relied on the Rule of Law Index to assess the quality of Canadian legal institutions. For instance, the annual report of the Quebec Bar Association assesses the legislative and judicial activity in Quebec and issues several reform recommendations on the basis of the criteria developed by the Rule of Law Index (Barreau du Québec, 2013, p. 7). The Canadian Bar Association has also followed up on the results of the Rule of Law Index to point out poor performance by Canada in the indicator ‘access to justice’ and to suggest the need for reforms (Canadian Bar Association, 2013).
More recently, the Supreme Court of Canada in its landmark case Hryniak v. Mauldin (7 A.C.S. 24 (2014)) embraced the evaluation conducted by the Rule of Law Index of Canada's judicial system. According to the court, the poor performance of Canada in comparison to equivalent countries in North America and Europe shows that Canadian average citizens undergo greater difficulties to access the civil justice system. The court relies on the Rule of Law Index to highlight that bad performance is mainly due to the high fees of lawyers and legal counselling, and the duration of disputes in court. For the Supreme Court, this results shows that the principle of fair and equitable treatment underpinning the administration of justice is undermined in practice.
Among all global legal indicators, the Doing Business Index has possibly the most comprehensive, sophisticated and influential benchmarking system. Its policy is simple: ‘repeated benchmarking creates healthy competition’ (The World Bank, 2013, p. 18). The benchmarking system of Doing Business is composed of both rankings and good practices. First, following individual assessments, it establishes a comparative ranking that points out to low- and high-performing regulatory areas of each country. This is a benchmark of relative performance. Second, it looks for correlations and patterns between, on one side, legal and regulatory reforms and, on the other hand, performance scores. Finally, it publishes a list of countries implementing good practices in each area of business law and regulation (The World Bank, 2016, pp. 20–21) and makes available an up-to-date ‘law library’ containing more than 150 samples of laws adopted by countries around the world for each regulatory area measured.Footnote 3 By matching the type of legal reform with the performance of countries, and by compiling these results into its database, Doing Business not only assists countries in adopting laws with a ‘proven record’ of facilitating business, but also flags up reforms with a ‘proven record’ of making Doing Business more difficult.
All the above examples show that benchmarking legal performance through rankings and good practices is core to the legitimacy of legal indicators. In the context of a system of management control, issuers of global legal indicators use benchmarking to promote regulatory convergence (The World Bank, 2016, p. v). States and corporations subject to measurement are not only provided with an assessment of their legal performance, with a guide of how to improve it. In these circumstances, addressees find themselves in the difficult position of either rejecting the indicators as a whole, or playing by its rules, adopting the benchmarks as standards of performance they should strive to achieve. As they do so, the goals embedded in the indicators become organisational goals and, consequently, internal drivers of behaviour.
VI. Auditing as accountability
Today, auditing practices are everywhere. They range from classic forms of internal accounting controls in private corporations (Power, Reference Power and Power1994a, pp. 19–22) and quality verification in industrial processes (Frydman, Reference Frydman, Frydman and van Waeyenberge2014b, pp. 27–28; Brown, Reference Brown1996, pp. 83–93) to total quality assessments of private and public institutions (Power, Reference Power1996, pp. 299–304). What Michael Power (Reference Power1997; Reference Power, Hopwood and Miller1994b) called in the 1990s ‘the audit society’ has become commonplace. Auditing is nowadays a lifestyle. Individuals, as users or consumers of services and goods, whether public or private, request and trust audits because they believe auditing can ensure transparency, enhance accountability and guarantee safety (Power, Reference Power1997, pp. 1–3; Power, Reference Power and Power1994a, p. 124). It is telling, paraphrasing Ronald Dworkin that even the sovereigns of Law's Empire, namely parliaments and the administration of justice, are today subject to audit controls (Frydman and Jeuland, Reference Frydman and Jeuland2011). In the audited society we live in, legal indicators are used to conduct systematic and periodic audits of private organisations’ compliance with the law, and to monitor the performance of states’ legal systems. Sally Merry clearly describes the audit role of indicators in the field of human rights:
‘The turn to indicator creation marks a shift in the way the administration of human rights law takes place. Instead of pressuring countries to conform to human rights laws on the basis of ambiguous and contextualized accounts in country reports or case studies – reports in which each country is presented as shaped by its history, social structure, wealth, and political agendas – indicators provide comparable information in numerical terms. The burden of assessment rests on the indicator itself, with its agreed-on standards and means of measurement … . The reliance on numbers, with their apparently simple and straightforward meanings, produces an unambiguous and easily replicated field for judgment. Compliance becomes far more open to inspection and assessment.’ (Merry, Reference Merry2011, p. 88)
If auditing is often used as a shorthand term for describing the managerial rationality underpinning indicators (Shore and Wright, Reference Shore and Wright2015, p. 24; Parleea and Wiber, Reference Parleea and Wiber2015, p. 97; Merry, Reference Merry2011, p. 84; Rosga and Satterhwaite, Reference Rosga and Satterhwaite2009, pp. 279–281), it does not exhaust the array of managerial practices they deploy. As mentioned earlier, indicators also create conceptual categories for data collection, set benchmarks and, of course, monitor compliance with them. The pressure indicators exercise on institutions and individuals as a means to drive their behaviour and bring it in line with their underlying precepts is better understood as part of a wider system of management control, in which auditing has the function of ensuring accountability.
Espeland and Sauder have observed that performance indicators transformed the notion of accountability. They argue that ‘[w]here accountability once included many different practices, making institutions accountable now usually means making them “auditable”, which often involves devising indicators to measure performance’ (Espeland and Sauder, Reference Espeland and Sauder2007, p. 2). Power has suggested that auditing is not a natural property of things; it is constructed as a valid and faithful mechanism of verification (Power, Reference Power1996, pp. 289–291). In his view, ‘making things auditable’ implies the creation of both a knowledge base (techniques, cues, etc.) and the environment that supports such knowledge as appropriate and reliable to represent the quality or performance of the thing audited (Power, Reference Power1996, p. 291). Power goes on to define the key features characterising audit practices: ‘independence from the matter being audited; technical work in the form of evidence gathering and the examination of documentation; the expression of a view based on this evidence; [and] a clearly defined object of the audit process’ (Power, Reference Power1997, pp. 4–5, emphasis in original). Global legal indicators meet all these criteria.
First, all issuers of global legal indicators are external to the organisations measured and can claim reasonable independence, without this meaning that they do not adopt a particular political or economic standpoint. Second, it has been shown earlier that indicators produce evidence, either through direct empirical research methods such as the Rule of Law Index, Doing Business and the Global Competitiveness Index, or through the aggregation of secondary data such as the WGI Rule of Law Indicator or the CSRHub. Third, indicators express a clear view about the legal performance of organisations. For instance, the Doing Business Index and the WGI Rule of Law Indicator operationalise private-led growth strategies of development and efficient and effective-based models of law (Restrepo Amariles, Reference Restrepo Amariles, Fortes, Boratti, Lleras and Daly2017, p. 104). The CRSHub and CHRB make corporations accountable for human rights violations, distributing the obligation of protection beyond the state. Finally, all global legal indicators have relatively well-defined objects of audit, which include the rule of law, business regulations and human rights.
Interestingly, Power has also observed that auditing is not only a way of gathering data and building up evidence, but also a ‘system of values and goals which are inscribed in the official programmes which demand it’ (Power, Reference Power1997, p. 5). This programmatic dimension of auditing is one of the dangers associated to the expansion of global legal indicators. Auditing legal systems from a global perspective is seen as a way to coordinate efforts towards more efficient rules and legal institutions, enhance accountability of national legal operators, and ensure prosperity and economic development. However, in this process, many indicators tend to promote only one model of development and one model of law in development (Restrepo Amariles, Reference Restrepo Amariles, Fortes, Boratti, Lleras and Daly2017, p. 104). The values spread through auditing practices eventually contribute to reinforce the categories of subjectification and self-knowledge, which in turn reinforce the legitimacy of indicators.
A positive effect of auditing practices promoted by legal indicators is to place organisations in a ‘comply or explain’ position vis-à-vis a larger class of stakeholders, including but not limited to shareholders or employees of a corporation, the press, citizens of a given state, NGOs, donors and investors. The essential idea behind ‘comply or explain’ in corporate governance ‘is to make the board accountable for what has been or not been done’ (Keay, Reference Keay2014, p. 282). Similarly, legal indicators allow stakeholders to hold to account those responsible for organisational performance – whether governments, boards of directors, politicians, managers, etc. This process is central to reinforce the legitimacy of indicators. Indeed, as performance happens to be below expectations or lower than that of similar organisations, states and corporations will need to engage with the metrics to explain the results, establish internal responsibilities and review their operation in the view of future amelioration. This process creates a commitment to design action plans and improve the overall performance based on the indicator's agreed-upon standards, reinforcing the legitimacy of its categories and the indicator itself as an instrument of control.
VII. Incentives as stimulus for action
Incentives are core to a management control system in corporations. They are one of the main mechanisms used to drive the behaviour of individuals and to push them towards the company's objectives, facilitating goal convergence (Löning et al., Reference Löning, Aalleret, Aéric and Pesquex2016, p. 8). As Löning et al. explain:
‘management control generally relies on an incentive system, in other words a reward (or punishment) system. Rewards and incentives might be financial –extrinsic rewards – but they might also be intrinsic, based on recognition and enhancement of processional or social standing.’ (Löning et al., Reference Löning, Aalleret, Aéric and Pesquex2016, p. 7)
However, to explain the reactivity and legitimacy of legal indicators as management tools, one cannot rely exclusively on a classic system of management control, but should place the latter as part of the architecture of a distributed system of global governance. The reason is that producers of legal indicators are not generally concerned with the direct execution and allocation of incentives, but instead they participate in larger, often informal and modular, networks, alliances and partnerships with actors dispensing them. Distributed governance highlights that, while centralised and hierarchical decisional architectures continue to exist at the global level, informal and non-centralised forms of decision-making are developing based on a diffused pattern of power (Kingsbury, Reference Kingsbury2015; Paquet, Reference Paquet2001; Reference Paquet and Courchene1997).
Commentators often observe that, while the globalisation of the market economy has for instance entailed the development of new roles for financial markets and a new distribution of production and labour, international law remains structured around the sovereignty of states and their voluntarism to adopt, ratify and abide to international legal instruments. In the absence of a global ruling power, global regulatory dynamics are straying away from state-centred positive law towards new forms of regulations that include, among others, informal rules (Pauwelyn, Reference Pauwelyn and Jan Wouters2012; Pejovich and Colombatto, Reference Pejovich and Colombatto2008), co-regulatory models (Kadelbach and Günther, Reference Kadelbach, Günther, Kadelbach and Günther2011; Calliess and Zumbansen, Reference Calliess and Zumbansen2010; Berns et al., Reference Berns, Docquir, Frydman, Frydman and Lewkowicz2007), unidentified normative objects (Frydman, Reference Frydman, Frydman and van Waeyenberge2014b, p. 183) and administrative-alike regulatory devices (Kingsbury et al., Reference Kingsbury, Krisch and Stewart2005). Indicators are often described as part of these new forms of regulation (Frydman and Twining, Reference Frydman and Twining2015, p. 4; Davis et al., Reference Davis, Kingsbury, Merry, Merry, Davis and Kingsbury2015, p. 2). The co-existence and interrelation of the above regulatory tools is shifting the centre of gravity of governance from a ‘bureaucratic focus to a market-cum-clan focus’ (Paquet, Reference Paquet2001, pp. 192, 201). As Paquet explains, following Boisot's grid of the information space:
‘The new dynamic involves more complex mixes of intertwined relations, networks and regimes, and governance capabilities that are more diverse and seemingly more disconnected than has been the case in the old nation-state centered governance world…This sort of strategy calls for lighter, more horizontal and modular structures, networks and informal clan-like rapports in units freer from procedural morass, empowered to define their mission and their clienteles more precisely, and to invent different performance indicators.’ (Paquet, Reference Paquet2001, p. 184)
Legal indicators participate to this distributed model of global governance (Figure 1). They allocate incentives through a set of strategies and alliances that are not organised under a hierarchical structure, but whose interplay creates regulatory spaces (Lange, Reference Lange2003; Scott, Reference Scott2001; Hancher and Moran, Reference Hancher, Moran, Hancher and Moran1989). These spaces become arenas where the relationships between producers of indicators, addressees and stakeholders unfold. The configuration of regulatory spaces is flexible. They allow the combination of new forms of regulations and different modes of governance such as hierarchies, markets and networks (Mols et al., Reference Mols, Haslam, Jetten and Steffens2015, p. 81). They can be shaped by sectorial concerns, societal interests or simply defined in relation to specific projects or policies. As such, they embody arrangements of power that instil new expectations of conduct, and new mechanisms of rewards and sanctions. This view reinforces the idea that the global society is not a space of non-droit, neither one regulated by the sole law of the market (Frydman, Reference Frydman2012, p. 36). Instead, it is depicted as a society that evolves through compartmental but generally interconnected regulatory spaces framing individual and institutional behaviour.
Figure 1 Global legal indicators and the formation of regulatory spaces
In these regulatory spaces, stakeholders with varied interests distribute incentives to drive the behaviour of organisations and promote their own conception of the good. This makes out of regulatory spaces arenas of struggle in which the dynamics of regulation, efficacy and institutional power are contingent. In this context, the legitimacy of legal indicators strongly depends on their capacity to obtain endorsements and establish alliances and partnerships with organisations capable of delivering valuable incentives.
Take the following example. The World Bank produces the Doing Business Index and the WGI, both of which include several legal indicators. The Doing Business Index measures, among others, the regulations relating to paying taxes, protecting minority shareholders, resolving insolvency and enforcing contracts. The WGI includes an indicator measuring the rule-of-law performance of legal systems. In contrast to Doing Business, which relies on primary data collection, the WGI constructs its indicators on the basis of data aggregation from third-party sources (Restrepo Amariles and Mclachlan, Reference Restrepo Amariles, McLachlan, Deffains and Kessedjian2015, pp. 23, 26). For instance, the WGI Rule of Law Indicator relies on sources such as the Rule of Law Index and the Global Competitiveness Index (Restrepo Amariles and Mclachlan, Reference Restrepo Amariles, McLachlan, Deffains and Kessedjian2015, pp. 9–13).Footnote 4 Hence, composite indicators such as the WGI clearly contribute to the diffusion of values of other producers of indicators by endorsing the assumptions and objectives they rely upon. However, none of these indicators delivers direct incentives to addressees.
Now, take the case of the Standard and Poor's sovereign ratings (S&P), which may in principle seem alien to the measurement of legal performance. Standard & Poor's assesses the creditworthiness of sovereigns on the basis of their performance in five key areas: political, economic, external, fiscal and monetary (Standard & Poor's, 2011). If law is not included as a self-standing area of evaluation, Standard & Poor's uses nonetheless the scores of both the WGI Rule of Law Indicator and the Doing Business Index to calculate the primary factor of political performance of states, namely ‘the effectiveness, stability, and predictability of the sovereign's policymaking and political institutions’ (Standard & Poor's, 2011, p. 10).
Standard & Poor's includes law in its evaluation. It considers unbiased enforcement of contract and respect for the rule of law (especially in the area of property rights) to be closely correlated to respect for creditors’ and investors’ interests (Standard & Poor's, 2011, p. 13). Additionally, Standard & Poor's also assesses whether policy-making and legal reforms may ‘weaken support for sustainable public finances and balances economic growth’ (Standard & Poor's, 2011, p. 12). Standard & Poor's does not allocate directly incentives to countries with good sovereign ratings. However, it has been well documented that a drop in the creditworthiness of states as measured by Standard and Poor's exacerbates, for these countries, the cost of borrowing in the international capitals market (Gaillard, Reference Gaillard2014).
This example brings to light the emergence of a new regulatory space made up of, on the one hand, the combination and mutual reliance of different indicators and, on the other hand, the incentives embedded in the structure of the international capitals market. These dynamics endow legal indicators with greater reactivity than they would be able to convey individually. States need now to pay attention to legal performance as measured by the Doing Business Index, the Rule of Law Index and the WGI Rule of Law Indicator if they wish to maintain a good sovereign rating and, hence, a favourable borrowing rate. The intertwinement of different forms of regulations (e.g. indicators) and modes of governance (e.g. markets and networks) ends up driving the institutional behaviour of states.
Another example of incentives allocated in a distributed system of governance relates to flows of foreign direct investment (FDI), which commentators note tend to favour countries with high-performing legal systems and a strong rule of law (Anyanwu, Reference Anyanwu2012, p. 448). At the turn of the century, the World Bank argued that developing countries in transition to market economies could not encourage domestic and foreign private investment ‘without modifying or overhauling the legal and institutional framework and firmly establishing the rule of law to create the necessary climate of stability and predictability’ (The World Bank, 2004b, p. 2). In this context, legal indicators became central to law reform and investment decision-making. For instance, the Doing Business Index claims on the basis of empirical evidence that ‘better business regulation – as measured by Doing Business – is associated with higher levels of foreign direct investment’ (World Bank, 2016, pp. 21–22). This type of information becomes relevant to policy and decision-making, and calls on states to implement, often dubious, legal reforms should they wish to attract FDI. As Fauvarque-Cosson and Kerhuel put it, this approach transforms law into an ‘economic contest’ (Reference Fauvarque-Cosson and Kerhuel2009, p. 811), where it is treated as a commodity competing for the attention of foreign investors (Perry-Kessaries, 2011, p. 402).
Addressees of indicators may also respond to non-economic incentives, such as those having positive effects on their reputation or giving them access to a ‘club’ of good performers. The Organization of Economic Cooperation and Development (OECD) and the EU are probably the best examples of what I call the ‘club effect’. The OECD produces a considerable number of indicators aimed at measuring the performance of its Member States, often referred as ‘developed economies’. Many countries that strive to join the club may feel compelled to embrace OECD indicators or emulate the behaviour of high-performing countries in the belief that they will also become high performers and eventually gain access to the organisation.
The EU runs possibly the most successful system of management control and distributed governance aiming at driving the institutional behaviour of states. The best example is Article 140 of the Treaty on the Functioning of the EU (TFEU), originally set up in 1992 by the Maastricht Treaty and Protocol No. 13 on the convergence criteria. The paper lays down five indicators on economic governance to evaluate the performance of Candidate States to the Economic and Monetary Union (EMU). Before being granted membership, Candidate States need to achieve certain benchmarks in five indicators signalling convergence in economic performance with already Member States of the EMU. For instance, a candidate state meets the first criterion, namely a high degree of price stability, when its rate of inflation is close to that of, at most, the three best-performing Member States.
Performance-based and comparative indicators were also set up to assesses the other four criteria, namely sustainability of the government financial position, stability of exchange rate, stable long-term interest-rate levels and limited budget deficit. Before the current crisis flagged up deficiencies of economic governance in the EMU, the success story of the Euro zone was attributed largely to the economic convergence achieved by its Member States through performance indicators (Gotz, Reference Gotz2012, p. 20). With the implementation of the Open Method of Coordination (OMC), the EU dangerously expanded the use of performance-based governance to many other policy areas including employment, social protection, education, youth and vocational training (van Waeyenberge, Reference Van Waeyenberge2015, pp. 64–75).
As the EU becomes increasingly involved in the production of legal indicators, it may reasonably consider implementing a similar managerial system to assess legal convergence. The EU has repeatedly held consultations to develop a rule-of-law indicator with the aim of assessing Member States’ compliance with the EU rule-of-law standards (Hiil, 2013; 2012; Pech, Reference Pech2012, p. 9). Following the experience of Protocol No. 13, and considering that adherence to the rule of law is a condition of membership to the EU – pursuant to Articles 2 and 29 of the Treaty on European Union (TEU) and the Copenhagen criteria – it would not be surprising that such an indicator would become a means to assess compliance of Candidate States to EU rule-of-law standards. The EU should take precautions to avoid possible detrimental and unintended consequences for both Candidate and Member States. For instance, an EU rule-of-law indicator may end up driving states to focus their efforts on improving only the elements assessed by the indicator at the expenses of other valuable aspects left unmeasured by it.
As the above examples show, incentives are critical to the legitimacy of indicators because they ensure that adherence or deviance from them is met with consequences in the real world. Hence, indicators that are reused, endorsed or in any case become successfully entrenched in the institutional practices of organisations delivering incentives are not only most likely to exhibit higher degrees of legitimacy, but also to impose their values and views. Incentives provide organisations with the ultimate determinants to conduct their activities and operations in conformity with the predicaments of an indicator, allowing the latter to accomplish its objective to organise behaviour in a given regulatory space and promote goal convergence.
VIII. Conclusions
This paper attempted to characterise the managerial rationality underlying indicators on the belief that, to understand, contest and resist their pervasive power in the legal field, it is first necessary to comprehend their deep managerial roots. Through concrete examples, it was shown that indicators are rapidly developing as an effective way to drive the institutional behaviour of organisations, including those of states and corporations. The power legal indicators have gained in the past decade need also to be met with critical and systematic scrutiny from academics, lawyers and policy-makers, not only in relation to their methodological quality and in-built values, but also in relation to their damaging, and often unavoidable, uses and misuses. This is particularly relevant considering that the quality of indicators in terms of theoretical premises, accuracy and methodological robustness is not always proportional to their actual influence in the real world (Schnyder, Reference Schnyder2012, pp. 4–7). Indeed, methodologically flawed indicators may in certain cases be more influential in driving the behaviour of organisations than methodologically sound indicators.
Scrutiny of indicators needs to be complemented with the repoliticisation of regulatory spaces. Legal indicators convey a mathematical proceduralisation of values and concepts (Restrepo Amariles, Reference Restrepo Amariles, Fortes, Boratti, Lleras and Daly2017, pp. 106–107), which drifts assessment and decision-making away from political debate to statistical considerations, eventually stripping legal processes from democratic legitimacy. Repoliticising regulatory spaces means flagging up mechanisms and processes where managerial practices are taken for granted and criticism dismissed on grounds of efficacy and efficiency. Löning et al. (2016, p. 149) explain that managerial practices cannot be successful without a managerial culture. If this is correct, politicisation should allow halting the expansion of careless, and often unconscious, managerial rationality to areas where societal interest requires public exposure and democratic debate.
