Numbers in an emergency: the many roles of indicators in the COVID-19 crisis

1 Introduction

The COVID-19 crisis has shaken the world. Yet, what may be its lasting consequences? Jared Diamond offers the optimistic view that: ‘If the world's peoples join together, under compulsion, to defeat COVID-19, they may learn a lesson. They may become motivated to join together to combat more dangerous global threats like climate change, resource depletion, and inequality’ (Diamond, 2020) By contrast, Michel Houellebecq is both sceptical and pessimistic as he suggests that: ‘We will not wake up after the lockdown in a new world. It will be the same, just a bit worse.’¹ In this Special Issue, we are specifically interested in the many roles of indicators in the COVID-19 crisis. Thus, reflecting on the statements by Diamond and Houellebecq, it may be asked, for example, whether the extensive use of indicators in the COVID-19 crisis should be seen as a positive or negative development, and how far the profound social role of indicators in this crisis is the continuation of developments that have preceded the pandemic.

The contributions of this Special Issue address a variety of indicators that have played a role in the COVID-19 crisis. The most obvious ones are the infection and fatality rates used as proxies for the spread and effect of the virus by governments and newspapers, but also the World Health Organisation (WHO).² There are also indicators about the health-care capacities of countries (often preceding the emergence of the COVID-19 crisis),³ government responses,⁴ as well as combined ranked indicators.⁵ Finally, the start of the vaccinations in late 2020 has been accompanied by data that compare and rank countries, possibly also reflecting variations in government effectiveness.⁶

The contributions of this Special Issue cover a variety of topics related to the many roles these indicators play. It is also both about learning from the current literature on indicators (see references in the following section) and revising that literature on the basis of this case-study. Doing so, a summary of the role and significance of COVID-19 indicators, as discussed in this issue, would include key words such as: expertise, legitimacy, health as a human right, trust and risk, democracy and the handling of crisis, law and technology, soft law, law as engineering, compliance and many others.

The remainder of this introductory note is structured as follows. Section 2 outlines how this Special Issue builds on prior socio-legal research on indicators. Section 3 presents an overview of the contributions to this Special Issue. Finally, section 4 reflects on the possible implications of these findings for a (possible) post-COVID-19 world.

2 Relationship to prior socio-legal research on indicators

Recent years have seen the publication of a large body of literature dealing with indicators from a socio-legal perspective. This includes, for example, English-language books on The Power of Global Performance Indicators (Kelley and Simmons, 2020), The Palgrave Handbook of Indicators in Global Governance (Malito et al., 2018), Governance by Numbers: The Making of a Legal Model of Allegiance (Supiot, 2018), The Quiet Power of Indicators: Measuring Governance, Corruption, and Rule of Law (Merry et al., 2015), The World of Indicators: The Making of Government Knowledge through Quantification (Rottenburg et al., 2015), Ranking the World: Grading States as a Tool of Global Governance (Cooley and Snyder, 2015), Governance by Indicators: Global Power through Quantification and Rankings (Davis et al., 2012) and Uses and Abuses of Governance Indicators (Arndt and Oman, 2006). In the non-Anglophone literature, there have been, for example, the books on Numera et impera: Gli indicatori giuridici globali e il diritto comparato (Infantino, 2019) and Gouverner par les standards et les indicateurs: De Hume aux rankings (Frydman and Van Waeyenberge, 2013).

A number of Special Issues of journals have explored further details.⁷ The two editors of this Special Issue (and some of its authors) have also contributed to the debate in this form, namely with a prior Special Issue of this Journal, on the ‘Global social indicators: constructing transnational legitimacy’.⁸ In this latter publication, the term ‘global social indicators’ was used to refer to numerical measures that describe the well-being of individuals or communities (Siems and Nelken, 2017). It explained that such indicators have reached across many societal fields: for example, some deal with good governance and the rule of law (e.g. the Corruption Perception Index and the various indicators by the World Bank), some with personal rights and economic freedom (e.g. the World Press Freedom Index and the Global Competitiveness Report), some with human development and political stability (e.g. the Human Development Index and the Country Risk Index) and some with the performance of commercial and educational entities (e.g. the UN Global Compact and the various global university rankings).

The growing use of indicators provides various links to socio-legal research in general and research on ‘law in context’ in particular. The literature has discussed, for example, that indicators do not just provide information, but can be regarded as somehow ‘law-like’, as they are often drafted in a general fashion and have a profound influence on society due to their quantitative metric (Davis et al., 2015).⁹ In the international sphere, it may therefore be argued that they are often a form of transnational soft law (Siems and Nelken, 2021). There are also some situations in which indicators have been explicitly included in the law itself, while, in other instances, they are of interest to socio-legal researchers, as indicators are a form of control that both affect and are affected by regulation.¹⁰

The precise relationship between indicators and the role of context has sometimes been phrased as a paradox. For example, this line of reasoning states that indicators aim to be ‘globally commensurate’ and thus ‘cannot be rooted in local contexts, but in order to accurately reflect local situations, they need to be’ (Merry and Wood, 2015, p. 217). Yet, it can also be argued that this ‘paradox’ simply shows that law-in-context research needs to be aware that indicators can have different functions: some of them have the explicit aim to deal with standards and solutions that have to be applied evenly even if the context is different, while others benefit from more consideration of ‘context’ in order to make them more ‘truthful’ for purposes of comparison.¹¹

Some of the prior literature on indicators has already examined issues of public health and diseases. For example, three chapters of the aforementioned books deal with the way in which WHO and UNICEF immunisation-coverage indicators have been used (Fisher, 2012), the application of a scorecard in the fight against malaria in Tanzania (Gerrets, 2015) and the application of Millennium Development Goals (MDGs) to HIV treatment in Uganda (Park, 2015). Further specialised journal publications include, for example, an analysis of the MDG's goal to improve maternal health (Yamin and Boulanger, 2014), the role of private actors for the assessment of the health-related Sustainable Development Goal indicators (Mahajan, 2019) and critical historical perspectives on the evolution of global health metrics (Gorsky and Sirrs, 2017; Shiffman and Shawar, 2020; Aue, 2021). Finally, just as COVID-19 emerged, a monograph on The Uncounted: Politics of Data in Global Health specifically focused on the role of for health data for AIDS response (Davis, 2020).

With respect to the COVID-19 crisis, this Special Issue is the first to discuss the role of indicators in detail.¹² In contrast to the aforementioned previous Special Issue of this Journal, we consider all possible ‘indicators’ (i.e. not only ‘global social indicators’) within the scope of analysis. As already noted in the introduction, this includes therefore, for example, indicators of government responses, but also infection and fatality rates. A further distinction is that some of these indicators may be regarded as ‘global’ (e.g. because they aim to rank the performance of countries in the COVID crises) while others are used ‘locally’ (e.g. as they track COVID infections across time in a particular place).

The specific topics included in this Special Issue partly reflect an e-mail correspondence in which we indicated a number of questions that the authors could address, namely: How can COVID-19-related indicators be classified (descriptive, explanatory, normative, etc.)? What can the prior debates about the strengths and weaknesses of indicators add to the discussion and uses of indicators in the COVID-19-pandemic? Conversely, what can the way in which these indicators were made and used add to the academic discussion on indicators? How far do these indicators compare things that are comparable, in particular in a cross-country context? What are the advantages and disadvantages (or uses and abuses) of these indicators? How far do (and should) these indicators guide social interventions and change behaviour? What is the role of law in terms of allowing, restricting or incorporating such indicators? What is the role of technology in this field? What are the relevant ethical considerations? The following will outline the contributions to this Special Issue as they address many of these questions (while others leave scope for future research).

3 Overview of contributions to this Special Issue

The first paper of this Special Issue is by Marta Infantino on ‘Hazards and fallacies of social measurements: global indicators in the pandemic’. It presents an overview of different types of COVID-19-related indicators. It also discusses the hazards and fallacies associated with global attempts to frame the social world in numbers. Thus, this paper attempts to verify whether and to what extent these hazards and fallacies affect numerical representations of the pandemic and its effects. Its aim is therefore to gain a deeper understanding of global measurements of health and related law-and-policy measures and to suggest caution about their use as a basis for knowledge and action in the context of the pandemic.

Second, the paper by Nathan Genicot on ‘Epidemiological surveillance and performance assessment: the two roles of health indicators during the COVID-19 pandemic’ draws on the history of international public health and of the management of infectious disease. It shows that the normative power acquired by metrics during the pandemic can be understood in light of two forms of quantification that gained in importance over the last decades – namely epidemiological surveillance and performance assessment. Thus, it suggests that, on the one hand, indicators are established to evaluate and rank countries’ responses to the outbreak; on the other hand, the evolution of indicators has a direct influence on the content of public health policies, also noting the inherent partiality and precarity of the information provided by health indicators.

Third, Manjari Mahajan's paper on ‘Casualties of preparedness: a case of the Global Health Security Index’ also presents a historical and critical perspective. Specifically, it analyses the 2019 Global Health Security Index by two leading US think-tanks that ranked 195 countries on a range of metrics of preparedness. It notes that the US and the UK were assessed as the two states best prepared to address a catastrophic health event, while, in the COVID pandemic, we now know that these countries have had some of the worst public health outcomes among wealthy countries. Thus, this paper interrogates the dominant schemas from which global health security indices have emerged and the narratives and governance that they facilitated. In particular, it argues that there is a need not simply for ‘better indicators’, but also for a different framing of public health and what constitutes pandemic preparedness.

Fourth, David Nelken's paper on ‘Between comparison and commensuration: a case-study of COVID-19 rankings’ discusses the problem of indicators to compare and rank the relative performance of states or other units. Thus, it raises the question of how far indicators can, or should, be required to be faithful to the requirement to ‘compare like with like’. Specifically, using the role of indicators in documenting and responding to the COVID-19 epidemic as an example, the paper investigates in detail how the hybrid and sometimes inconsistent commitment to both comparison and commensuration may help to account for the difficulty they have had so far at establishing stable rankings of best practice.

Fifth, the paper by Mathias Siems on ‘Mapping a causal scheme of indicators in the COVID-19 crisis’ also presents a comparative (as well as empirical) perspective. It focuses on two main issues. First, it develops and maps a general causal scheme of indicators and their underlying real-world phenomena in the COVID crisis. Second, it shows how such a causal scheme has been, and can be, applied in comparative empirical legal research. Yet, the paper also notes the difficulties of proving causal relationships. Thus, it suggests that, while it is useful to conduct empirical research on the effects of COVID policies, the limitation remains that any comparative facts about a particular causal relationship do not answer the ultimate decision of what this means for the right policies in a particular place.

Sixth, John Harrington's ‘Indicators, security and sovereignty during COVID-19 in the Global South’ continues with a critical analysis specifically addressing the contest over health governance and sovereignty in countries of the Global South. It notes that indicators have been a vehicle for the government of states through the external imposition and internal self-application of standards and benchmarks. It then contextualises their use, in particular identifying a decline in Third World solidarity and its replacement by neoliberalism and global governance mechanisms in health, as in other sectors. The interaction between these modes and their relative prominence during COVID-19 is also studied through a case-study of developments in Kenya during the early months of the pandemic.

Finally, the paper by David Restrepo Amariles is entitled ‘From computational indicators to law into technologies: the Internet of Things, data analytics and encoding in COVID-19 contact-tracing apps’. It investigates the data life cycle of contact-tracing apps as well as the socio-legal implications resulting from the design and technology choices that software developers inevitably made. In particular, the paper shows, first, that technology-harvest data do not reflect an objective representation of reality, and therefore require additional context to be understood and interpreted, and, second, that the use of data analytics to extract insights from these data makes possible the production of computational indicators, which in turn enable the development of novel computational applications in the legal domain.

4 Where do we go from here?

At the moment of writing this introduction, it is possible, but by no means certain, that COVID-19 will be history in the near future.¹³ Yet, beyond COVID-19, a pessimistic view may be that we have entered a ‘pandemic era’ given that, ‘as human societies grow in size and complexity, we create an endless variety of opportunities for genetically unstable infectious agents to emerge into the unfilled ecologic niches we continue to create’ (Morens and Fauci, 2020, p. 1089) with climate change as a possible ‘threat multiplier’ (Mann, 2021). There are also further reasons why the impact of the use of indicators in the COVID-19 crisis is likely to remain relevant in the medium to long term. For example, the ongoing quantification of the social world (and its awareness by the general public),¹⁴ the growing regulatory role of technologies¹⁵ and possibly also the apparent success of China in containing the pandemic will certainly have a continuing impact on domestic and international policy debates.

It is also clear that COVID-19 has many implications for research. According to Tim Besley, ‘COVID-19 will be with us for a long time as social scientists long after the crisis’, also noting the need for ‘an integrated social science’ (Besley, 2020). Specifically for research on indicators, the contributions of this Special Issue show that such research requires a ‘law-in-context’ perspective. As some of the COVID-19 indicators relate directly to legal rules (e.g. the coding of the stringency of government responses),¹⁶ their operation crucially depends on the context in which these indicators operate. Moreover, the contributions of this Special Issue show various overlapping in-context perspectives of indicators in the COVID-19 crisis, such as historical critiques of the origins of such indicators or perspectives focusing on the way in which such indicators interact with socio-political and technological elements of society.¹⁷

A further key issue is what the experience from the COVID-19 crisis may tell us about the success in dealing with indicators. It may be argued that some of the countries that have fared well in the pandemic have been those that, for example, were quick in their policy reactions and were the able to contain the virus through contact-tracing and other means (e.g. Taiwan, South Korea). However, it is also clear that any country's reaction to COVID-19 did not happen independently of other country characteristics, such as the degrees of state capacity and trust.¹⁸ Thus, it may also be argued that, despite the global nature of the pandemic, COVID-19 is also a case-study showing that the context of a country (or indeed any particular place) has often made all the difference.

A final note of caution follows from the unique nature of any emergency. For example, even a superficial of comparison of COVID-19 and HIV shows that differences related to the transmission mechanism of the virus, the age profile of the victims and the possibility of a vaccine mean that any lessons may not be transferable – and, of course, this may even be less likely for unknown viruses, or indeed any other ‘unknown unknowns’¹⁹ that cause emergencies in the future. Finally, as regards the role of indicators, it is also best to conclude that both their use and desirability depend on the circumstances: while the COVID-19 crisis has shown that some of the indicators are clearly indispensable, other indicators are more dubious and thus require improvements in design and application²⁰ or, if this is not feasible or appropriate, replacement by legal rules or other forms of social ordering.²¹