4 - And an Algorithm to Entangle Them All?

Summary

This chapter considers the emergence of data-driven analytics and its algorithmic techniques of imposing consequences as defining not just new modalities of governance but also as redefining governance forms, and consequently the ways that legalities entangle. The first step is to understand how data-driven governance differs from conventional forms of governance. One can then examine the points of entanglement and their character, whether within, between or beyond the conventional public law containers of legal space. At the centre is an examination of entanglement of law-norm systems and data-driven or social credit systems. That entanglement requires rationalizing a disjunction in how law/regulation systems understand data-driven metrics-based governance and vice versa, with the disjunction complicating entanglement. The first section briefly sketches the emerging legalities that neither embrace the form nor the language of law. The two sections that follow examine the nature of intersystemic entanglements when data-driven governance legal orders are thrown into the already plural mix. The first is the Chinese ‘social credit’ initiative. The second is US and Western private initiatives around emerging markets for data. Each exhibit distinct characteristics and therefore different forms and qualities of entanglement that suggest a more complex fracture and interacting among ever more different systems of legalities emerging around global trade regimes.

In the sixth month, Gabriel the angel was sent from God to a town in Galilee called Nazareth, to a virgin engaged to a man called Joseph, from the family of David. […] ‘Don’t be afraid, Mary’, said the angel to her. ‘You’re in favour with God. Listen: you will conceive in your womb and will have a son; and you shall call his name Jesus […] The Lord God will give him the throne of David his father, and he shall reign over the house of Jacob for ever. His kingdom will never come to an end.’¹

4.1 Introduction

The annunciation by Jessup² of the birth of what would become transnational law conceived within the womb of Western (and then global) jurisprudence imagined a transformed juridical order in which the notion of law was broadened beyond the state (at least with respect to a definable set of activities). These legalities, in turn, were to be entangled³ to solve what before had been issues the resolution of which could be undertaken solely by reference to the law of a state.⁴ But this ‘good news’ did not immediately produce transformations in the halls of the priestly castes charged with the preservation of the jurisprudential order carefully nurtured in recognizable form from the time of Martin Luther and centred on the state as the principal expression of regulatory power through law.⁵ For this caste and the states they served, the solution was, is and remains the law of conflicts and traditional private international law.⁶ However, the twenty-first century has seen a reluctant acceptance in theory of the decentring of the state, and consequentially, of the recognition of the rise of multiple centres of governance with multiple forms of law.⁷ That conceptual recognition comes at least a century behind early modern studies of its realities in some states.⁸ The reluctance arises from the ideological consequences of such a conceptual acceptance. At its limit, the fear produces a modernist panic⁹ about the state of the state, and of law as its official language. Despite the need expressed by some,¹⁰ there remains a reluctance to give up ‘the ultimately law-focused epistemological mechanism still at work’,¹¹ and thus to forgo the post-1945 ambitions for a transformation of global politics based on the self-reflexive state as the highest legitimate form of communal political expression,¹² the expression of which could be made legitimate only when undertaken through the language of law.¹³ Indeed, ‘[t]he main shortcoming of the extended conception of law advocated by anthropological and sociological approaches is the one pointed at by the so-called pan-legalist objection: the problem of the distinctiveness of law from other social normative orderings has been haunting the theorists of legal pluralism until today’.¹⁴ In short, what factions of the leader class feared most in the twenty-first century was precisely the consequences of legal entanglements that might bring down their tightly woven conceptual house.¹⁵

Yet that very rejection provides strong evidence not merely of its existence but also of its effects, principal of which are the ruptures that entanglement produces between these emerging centres of law/norms/governance,¹⁶ both within and among the conventional nation state.¹⁷ It is within these spaces that one might seek both the meaning and manner in which what Luhmann once described as structural coupling might occur.¹⁸ These spaces without a space, these in-between spaces of law (and governance), these fragmented but entangled legalities, have assumed a spatial dimension.¹⁹ It is now understood as both a connector (the trames²⁰ through which spaces connect and communicate) but also as its own normative territory within which those communications and connections are not merely mediated but managed through complex entanglements,²¹ or dynamic processes of communicative irritations.²²

The budding focus on the interspatial carries with it both promise and challenge. The promise: an interspatial gap filling in all of its complexities and theoretical possibilities. This has been the central exploration of entanglements among law, norms, rules and habits with coercive effect.²³ The challenge is centred on the risk of boundaries without end: of the permanent and quite dynamic cacophony of borders that is produced by the obsession with the interfaces between bodies of norms that themselves create borders within the space between norm systems for which other interfaces are necessary.²⁴ The problem of the never-ending spaces between spaces, where every law system defines its own inter-spaces, becomes self-entangling, and might itself be undergoing an extra-spatial transformation. That extra-spatial form of governance – in which space loses its centrality and law changes its forms and function – is the object of the exploration here.

More specifically, the chapter considers the emergence of data-driven analytics and the algorithmic techniques of imposing consequences (some of it machine driven, on the basis of artificial intelligence (AI))²⁵ as defining not just new modalities of governance but also reshaping the conception of spatiality within which entangled governance happens, national and transnational, public and private. So reshaped, these data-driven governance legalities entangle with traditional modes of governance through law in what may be new and interesting ways. These ‘social credit’ ratings, risk management, assessment, accountability or compliance systems have been established as a means to aid traditional governance;²⁶ yet they have the potential to displace the structures of governance they are meant to serve. But more than that, they may well change the landscape and language in which one encounters legal entanglements and its operational effects. The thesis of this chapter is simple: legal entanglement has moved beyond the two-dimensional space for engagement envisioned by Jessup and his successor. Entanglement was once confined to states, other governance institutions and between them all. That engagement was built around and with the traditional language of law and jurisprudence. Increasingly, the emerging quantitative legalities built around AI and big data management systems (e.g. social credit initiatives) that are neither dependent on the forms and techniques of law nor on the bureaucratic apparatus of state entangle more decisively with conventional and plural law and norm systems. These add an additional layer of actor (the ‘market’, the analyst, the modeller, the systems engineer, the coder) and seek to displace the language of law and jurisprudence with the language of data-based analytics applied towards a comprehensive management of behaviour. The resulting entanglement may reshape the meaning and making of entanglement itself. At its limit, this reshaping will have a particular effect on the way in which conventional plural legalities, until now the singular feature of globalized law frameworks, may be assembled through dynamic and sometimes unstable entangled linkages and rationalized by a regulatory algorithm that may come to manage them all.²⁷

This contribution, then, considers governance entanglement between the entangled legalities of law-norm systems and the legalities emerging from data-driven systems of behaviour management. The realities of emerging legalities that exceed the capacity of law to express their form require a three-dimensional analysis of entanglement and a broader view of legality, one that reconsiders data-driven, machine-administered regulatory systems more than an enhanced form of property.²⁸ One deals here not just with the flattened inter-legalities of the traditional structures of ordering power by rules. Instead, one must now understand the way those clusters of entanglement themselves are entangled with emerging modalities of law/regulation/norms which have come to form the centre of what is understood as plural legalities.

Section 4.2 briefly sketches the characteristics of emerging legalities that neither embrace the form nor the language of law. The two sections that follow examine the nature of inter-systemic entanglements which occur when data-driven governance legal orders²⁹ are thrown into the already plural mix of legalities. Two principal forms of data-driven governance have emerged that are increasingly linking to traditional legalities.³⁰ The first is the Chinese ‘social credit’ initiative, which emerged in its current form in 2014, and is an undertaking by the present administration of the Chinese government that is meant to produce an all-around approach to ensuring compliance with law and social responsibility under the guidance of the state. The second are US and Western private initiatives around emerging markets for data. These are framed around principles of governance, risk management and compliance. With respect to each, the nature and textures of entanglement that encounters between traditional plural legalities and data-driven governance systems produce is considered. These entanglements present a distinct challenge, the challenge of linguistic disjunction, for the management of human organization. While law- and norm-based systems speak the same language, data-driven governance does not. Communicative disjunction may have profound effects on the nature and quality of entanglement, producing a competition for the lingua franca that may affect the way in which law is expressed, and may threaten the plurality of law.³¹ Each also exhibits quite distinct characteristics and therefore quite different forms and qualities of entanglement, suggesting a more complex fracture and interaction among ever more different systems of legalities that are emerging in fractured political systems arranged around global trade regimes.

4.2 The Construction of Data-Driven Operating Systems

The governance consequences of data-driven compliance and risk management systems, and of the informal systems of ratings and assessment, has only recently exploded into the popular imagination of academics and others.³² What started as an effort to rationalize the emerging techniques of indicators,³³ as a means of governance, quickly became a study of these techniques, increasingly systematized, as governance.³⁴ The recognition of algorithmic governance appeared like a direct and hidden threat to the carefully constructed public structures of law and governance systems.³⁵ Just as it seemed that theory could come to grips with the possibility of law (governance) systems beyond the state, the entire framework of law systems themselves seemed to be sidelined by data-based algorithmic systems to which law and norms appeared to be a stranger. And these challenges have come to the West in forms that appear to conflate the operations of Marxist-Leninist government with Western democratic markets-driven polycentric (plural) governance.³⁶ Yet these judgements ought not to deter from considering the (inevitable it seems) rise of these systems in both China and the West, and the additional layer of entanglement they add to the emerging formal and public systems of governance that constitute global legal polycentric (pluralist) governance.³⁷ To that end, a brief exploration of the shape of social credit initiatives fuelled by AI and machine learning-enabled algorithms is worth considering.

The rise of social credit initiatives (ratings and data-driven governance in the West) and big data management systems (as a means to implement these governance frameworks) appears to further entangle legalities in perhaps unexpected ways. It takes as its starting point the significant drive towards accountability and measurement in governance incarnated through rankings and ratings. Administration of rankings and ratings then devolves to the institutions that administer and along with that devolution goes the power to determine what behaviours will be valued, and in what amount, to rank and rate. In effect, the indicator acquires regulatory autonomy. Thus characterized as a regulatory system, it is then entangled with traditional law-norm systems.³⁸ If law and norms – traditional governance – are grounded in the supremacy of space, of territory, then how is one to approach governance orders that might themselves be detached from traditional spatial limitations? That is the principal focus of this chapter. It considers the emergence of what might eventually be understood as a legal/normative order that has arisen not from the need to organize a territory, within which people and things are corralled, but generated from and centred on the objects of regulation themselves. The emerging systems that have been referenced in this chapter as ‘social credit’ initiatives offer a glimpse at a governance operating system quite distinct from the legal/normative systems that have served as the building blocks first of the Westphalian state system (in its domestic and international elements), and thereafter the polycentric systems that mark this transnational age.³⁹

In place of legal/normative systems driven through the construction of an apparatus of government within territories (physical or abstract) it fashions a system driven through data generated by people and things.⁴⁰ Data substitute for custom and tradition; data-driven governance substitutes the language of counting for the more qualitative descriptive language of practice. Governing organs substitute, in place of custom and tradition, the quite precise data warehouse, one with a vocabulary and rule of normativity all of its own.⁴¹ It can be filled constantly with the detritus of daily activity – at the market, on the web, in the car, on the street, in the store, etc.⁴² These can be used for a variety of ends, some of them public, but in the process entangling the normative decisions about character of data for the content of norms.⁴³ In the place of policy and principle one encounters data analytics. Choice ceases to be politically potent only within the electoral field; all choices become political as they point behaviour managers to desires and habits of the masses producing data. In lieu of debate there is analytics. The framing of that analysis, that is the judgements and principles embedded in those analytics, is not subsumed within the algorithm. The algorithm itself is the expression of the sum of the objectives and perspectives of those for whose objectives the algorithm is deployed. In lieu of principles, then, there are presumptions and the self-created limitations of data fields that create the boundaries within which choices are cabined. These are the structures of conventional governance, but now deployed in a quite different space.

But these are not normative orders in the traditional sense. That fundamental characteristic suggests the opportunities and challenges that these emerging systems represent. Social credit initiatives – data and algorithmic orders – change the relation among governance actors, decentring the judge, the regulator and the social actors, in favour of those who can manage, organize and apply metrics to some end or other (that is to the same ends of law). Yet they do not mediate relations among norms so much as absorb them all in furtherance of extra-spatial objectives – to focus on behaviours in a self-reflexive loop founded on behaviour-generated data, the vectors of which may be modified through systems of real-time rewards and punishments (of individuals and institutions, including governmental and economic institutions). These orders, then, are not so much enmeshed as they are ubiquitous – serving both as techniques of management, and as the primary structures of regulation themselves. Absorption does not mean elimination; it posits an entanglement of a different order – between conventional governmental orders, the entanglements of which continue to produce norms, but their interpretation and application, that is the way these norms are given meaning, entangle conventional legalities with the analytic legalities of the algorithm and the entanglement of choices made between techno-experts, coders and public or private officials. These characteristics became much more visible in the context of the Covid-19 pandemic where the entanglement of simulation and conventional political legalities became the central regulatory element of institutional responses to the virus.⁴⁴

It follows then that the anticipated entanglements become more confused where one set of systems (law/regulation) continues to posit that metrics-based governance is merely technique, rather than a regulatory system in its own right. The interspatial, and its data-driven forms, produces a quite distinctive template for the conceptual construction of law/norms between law/norm systems; and it emerges with its own language and sensibilities in ways that are not yet completely clear. Every political culture approaches the emerging realities of ‘artificial intelligence’, machine learning and the data-driven governance that is sometimes its object, in ways that tend to affirm cultural expectations and points of view. Where cultures are obsessed with particular conceptions of privacy and property, the result is an engagement that is centred on political constraints against states and enterprises, on the management of markets for data and at the same time on an enhanced methodology of data-driven analytics to deepen cultures of compliance, assessment and accountability.

In liberal democratic states, that focus also has constitutional and political dimensions.⁴⁵ Thus, the regulatory machinery of the European Union has lately been tasked to manage data under the presumption of a hierarchy of authority that further presumes the normative nature of law and the mechanical nature of data and their analytics. Data and data processing ‘should be designed to serve mankind. The right to the protection of personal data is not an absolute right; it must be considered in relation to its function in society and be balanced against other fundamental rights, in accordance with the principle of proportionality.’⁴⁶ There is a particular aversion to the forms of data-driven intrusions from the economic sector (the ‘for hire’ fields) on the formalities and rituals of exogenous democratic expression.⁴⁷ And, of course, there is an equal obsession with theft of information⁴⁸ and its misuse.⁴⁹ And yet, even as its elites offer the protection of law, it manages that data which itself manages behaviour.

A tiny microchip inserted under the skin can replace the need to carry keys, credit cards and train tickets. […] The small implants were first used in 2015 in Sweden – initially confidentially – and several other countries. Swedes have gone on to be very active in microchipping, with scant debate about issues surrounding its use, in a country keen on new technology and where the sharing of personal information is held up as a sign of a transparent society.⁵⁰

Thus combined, there is a tension between the centrality of the individual as the incarnation of data, and the utility of data in the production of economic goods with great disciplinary effects. This debate has been refined in the context of the Covid-19 pandemic, which has produced a substantial entanglement between data-driven indicator systems, national and international law and regulation in ways that will take some time to untangle.⁵¹

On the other hand, where cultures are obsessed with communal solidarity and the protection of stability around socio-political core values as the basis for legitimate social organization and operation, then the result is an engagement that is centred on issues of surveillance, of detection, and of management of behaviour around key societal principles. There is a particular aversion to antisocial behaviour as a politically destabilizing force.⁵² Feng Xiang, a law professor at Tsinghua University, recently spoke about the end of markets: ‘[i]f AI rationally allocates resources through big data analysis, and if robust feedback loops can supplant the imperfections of “the invisible hand” while fairly sharing the vast wealth it creates, a planned economy that actually works could at last be achievable’.⁵³ He might not be wrong about AI and markets; even the Cubans have sought to turn their economic planning into the rudiments of complex algorithms.⁵⁴ Yet this culture shares with the other a mania for observation – and for punishment and reward based on that power to observe,⁵⁵ one which is incompatible with the values of property and rights-based legal orderings.⁵⁶ And yet, even as there is a movement towards the overt management of behaviour through data, there is as well an obsession to ground that regulatory approach within the structures of regulation – in this case a regulation that protects the integrity of data and its use by the state and its delegees. Data and consequence (accountability for business, incentive to encourage preferred behaviours and punish violations of behaviour norms) thus combined, focus on stability and order in social development.

4.3 Chinese ‘Social Credit’ Systems: The State at the Centre

Social credit is a deeply entangled system; it is a system that will eventually apply to all legal actors, persons, institutions, officials and foreigners in or seeking to engage with China and Chinese actors.⁵⁷ It is more than that – it has been created as a nexus point between the constituting role of law (as well as the normative guides that are its rules) and the evaluative function of data-driven analytics.⁵⁸ It sits at the borderlands between a rating-evaluation system and systems of legal regulation. It is also entangled with the core political premises that drive Marxist-Leninist political economy in China under the leadership of the Communist Party of China (CPC).⁵⁹ Its autonomous and systemic character was emphasized from the beginning:

It is founded on laws, regulations, standards and charters, it is based on a complete network covering the credit records of members of society and credit infrastructure, it is supported by the lawful application of credit information and a credit services system, its inherent requirements are establishing the idea of an sincerity culture, and carrying forward sincerity and traditional virtues, it uses encouragement to keep trust and constraints against breaking trust as incentive mechanisms, and its objective is raising the honest mentality and credit levels of the entire society.⁶⁰

Yet one notes the entanglement – the system is constituted through law, but is established as autonomous from the law system around which it operates. The object is to establish a structure for managed entanglement within the state. Each, in turn, requires the production of a self-reflexive legality whose operation is entangled with those of the normative political, societal and economic order. Its mechanism, machine learning and AI, the entanglement of a rule system for AI and AI as law itself was also elaborated by the State Council in 2017.⁶¹ ‘Social credit systems are centered on ratings. Ratings are derived, in turn, from data generated by what is being rated—individuals, businesses, public and private institutions, and eventually even [CPC] members. To that end, it is necessary to manage data production as it is to manage the analytics and consequences drawn from the data.’⁶²

Social credit initiatives have as their object the development of a national reputation system, assigning a rating that reflects a qualitative judgement of relevant data gathered about the subject. Reputation, itself, embraces notions of sincerity, and of integrity and compliance, in accordance with the standards and objectives overseen by the state. Four areas are identified: ‘sincerity in government affairs’ (政务诚信), ‘commercial sincerity’ (商务诚信), ‘societal sincerity’ (社会诚信) and ‘judicial credibility’ (司法公信). AI and machine learning focus on the generation of mechanisms for the management and integrity of data and for its marketization.⁶³

The objectives of both are grounded in economic and social development and support for national security.⁶⁴ The project of a comprehensive and nationally integrated programme of credit ratings of virtually all aspects of organized life in China, built pursuant to rules and laws, administered by public and private bodies and overseen by the CPC remains a work in progress, though one increasingly structured by the highest Chinese state organs.⁶⁵ It is self-consciously constructed as an alternative legality, both within China and against the forms of plural legalities in the West that the Chinese leadership increasingly find difficult to enmesh with their own.⁶⁶

Nearly from its inception, social credit was understood as a new kind of law, separate from but entangled with traditional law systems, which had been themselves the product of the fusion of traditional approaches to law and Western concepts of law, rule of law and legal mechanics.⁶⁷ At the same time, social credit expressed a form of governance that also deeply entangled systems of social ordering through rules (laws) and the paramount authority of the CPC to guide such lawmaking and its application through its own political-economic model.⁶⁸ The determination to adopt a social credit initiative (and its operationalization through AI-enhanced algorithmic governance techniques) was itself a function of the determination that such data-driven governance would enhance the long-term Chinese political-economic objectives embedded in their concept of socialist modernization, a core policy of the Chinese state since the era of Deng Xiaoping. Social credit was to evidence the transformation of techniques that marked a new phase of the socialist market economy system and of the social governance system, one which received its more definitive political form after the announcement of the ‘New Era’ political line of the CPC in the wake of the 19th CPC Congress of October 2017.⁶⁹

But its genesis also represented a practical response to a long-term problem that both state and private elements of society found increasingly burdensome.⁷⁰ These included a number of issues that threatened not merely the orderly progress of socialist modernization, but also inhibited the progressive advancement of social and cultural objectives. These included grave production safety accidents, food and drug security incidents, commercial swindles, the manufacture and sale of counterfeit products, tax evasion, fraudulent financial claims, academic impropriety and gaps between the extent of integrity in government affairs and judicial credibility and the expectations of the popular masses. If neither law nor regulation appeared to produce conformity, and if the transaction costs of deploying a vast enforcement network was counterproductive to the long-term goal of a self-regulating society that was efficient and productive in socially approved ways, then a different approach to the management of societal factors was necessary.

This ambitious set of objectives was to be guided by a set of core premises and constraints. Social credit initiatives must conform to and advance the objectives of the CPC Basic Line, including the development of economic forces for policy ends.⁷¹ The Chinese social credit system also has a moral dimension, which deeply informs its regulatory and enforcement dimensions. Its object is to steer the culture and practices of people in virtually every aspect of their lives. To those ends, the Twelve Core Socialist Values unveiled in 2012 play an important role.⁷² Entanglement is meant to be conscious and coordinated rather than organic and serendipitous⁷³ – highlighting the difference between a central-planning (public and administrative) versus a markets-based (private and demand driven) political order.

The social credit initiative was comprehensive. It was envisioned that when completed, social credit systems would manage key operations in four sectors: government, commercial activities, social integrity and judicial credibility.⁷⁴ With respect to the role of social credit in government, the focus was on the use of data-driven analytics tied to algorithms that produced the basis for accountability to result in rewards or punishments around a variety of governmental actions. These included: administrative permissions, government procurement, tendering and bidding, labour and employment, social security, scientific research management, cadre promotion and appointment, management and supervision, application for government financial support and other such areas and fostering the development of a credit services market. Clearly, data-driven analytics and its resulting assessment system would produce an immediate effect on those whose conduct triggers measured responses. For those who met credit minima, access to benefits would be enhanced. Those assessed at a low enough level would be placed on an effective blacklist that would make functioning in a modern society substantially more difficult and costly, absent readjustment. Moreover, social credit was also to be used as a tool for intra-governmental accountability, and to monitor civil servants.

Social credit mechanisms were also directed towards a very broad range of commercial activities – whether by state-owned enterprises or the private sector. Private behaviour by individuals was also to be managed through social credit systems under the umbrella of enhancing ‘Social Integrity’. An area of particular note for the application of data-driven analytics was that of the judicial system. The specific focus was on judicial creditability and on the integrity of the judicial function and the performance (and accountability) of judges for their work. The State Council identified a number of areas: proceedings transparency, prosecutorial and public security services conduct, the operation of judicial administrative systems and law enforcement standardization. Here one notes the likelihood of a vertical entanglement. Judges are the objects of law and the agents of procedures reflected in the legalities of law and regulation. Yet the content of the way those functions are assessed (and consequentially the way that the implementation of legal duties is understood and measured, and thus interpreted) become a function of a distinct legality – that of the social credit system applied to the judge. Data-driven projects might well include the production of data-driven algorithms to guide judicial decision-making or to develop guidelines for charging and prosecution. Here one encounters an entanglement in which the discretionary scope of one system is constrained by the operation of the other.

The societal effects of social credit programmes were to be enhanced through the application of these mechanisms on education and culture projects. Social credit in the development of education system reform was to be tied to the construction of the socialist core value system. Education was to be a means for the socialization of social credit mentalities and its general acceptance.⁷⁵ Assessment would be built around parameters for judging the development and operation of ‘moral’ classrooms. These are to be built around the establishment of models of appropriate conduct.⁷⁶ Appropriate conduct, in turn, is to be assessed against the twelve socialist values that have been established after the CPC’s 18th Congress: ‘Core socialist values comprise a set of moral principles summarized by central authorities as prosperity, democracy, civility, harmony, freedom, equality, justice, the rule of law, patriotism, dedication, integrity and friendliness.’⁷⁷ To that end, state organs are encouraged to oversee special campaigns in focus sectors, and to ‘persist in correcting unhealthy trends and evil practices of abusing power for personal gain, lying and cheating, forgetting integrity when tempted by gains, benefiting oneself at others’ expense, etc., and establish trends of sectoral sincerity and integrity’.⁷⁸

All of this is to be accomplished by building social credit baseline systems⁷⁹ and their mechanisms.⁸⁰ These systems were then to be operationalized through blacklists. Blacklists – made up of the names of people whose social credit scores fall below certain thresholds – have already begun to have a substantial effect in everyday life. The system is manifested through ratings, and more importantly from the collection of blacklists produced as a function of ratings. Blacklists then affect the availability of goods and services.⁸¹

These, then, will serve as the systems through which entanglement will be coordinated. Baseline systems were identified as (1) sectoral credit information systems; (2) local information systems; (3) credit investigation systems; and (4) uniform credit investigation platforms in the financial sector. In addition, government in cooperation with the private sector were to develop credit information exchange and sharing. These shall provide the regulatory structures for the operationalization of data-driven assessment and punishment/reward systems. The forms of entanglement are also identified as (1) incentive structures and punishments for deviations; and (2) legal, regulatory and standards systems for credit.

The entanglements of social credit within the Chinese context are as comprehensive as its ambitions. Social credit is meant to provide a new language for law, at least as it is meant to serve to control behaviour. At the same time, the constitution of social credit is driven by law. That is, law serves as a constituting element of regulatory systems that themselves are grounded in forms and practices that are not law. At the same time, those forms and practices of social credit then drive law as it is applied. They do so by giving meaning to the objectives and expectations written into law by the way that social credit is administered through the process of identifying behaviour, analysing its meaning and attaching consequences. At the same time, social credit provides a bridge between law, compliance and assessment systems. In the process it also helps shape the cultures within which expectations are shaped and cultural habits formed, which then lend themselves to expression in the normative content of law. Here social credit moves entanglement into an ecology of interrelated subsystems all deployed to move forward the political project of the CPC.

4.4 ‘Social Credit’ in the West: A Governmentalized Private Sector around Markets for Data

To speak of ‘social credit initiatives’ in the West is to consider an initiative that does not exist – as such. And yet, a more careful consideration reveals the outlines of the forms of social credit outside of the state. To speak to the development of social credit and its operationalization through machine learning and AI-enhanced algorithms, is to understand how the fracture and diffusion of power has produced something more than the aggregation of governance spaces and their ‘in-between’ spaces arranged in some manner or other. It evidences the organization of power beyond the orthodox space of law, in the sense that these new data-driven legalities use conventional polycentric governance spaces themselves as fuel for the generation of behaviour-managing ‘incentives’ or ‘punishments’ that function as regulation while avoiding anything like its traditional forms.

Social credit in the West is made possible not merely by the availability of spaces within spaces, but by the effective borderlessness of the market itself. Those extra-spatial zones have been enhanced through the governmentalization of the private sphere, the privatization of the public sphere and the migration from law and regulation to systems and systems management.⁸² To speak about social credit in the West is to identify a host of fractured and market-driven projects by a large number of actors.⁸³ These actors also exercise governance authority in traditional ways (through rules, laws and the exercise of political and economic power). But for them, increasingly, assessment and accountability regimes, born of compliance objectives within the spaces and inter-spaces of conventional law/normative systems,⁸⁴ provide incentives to manage populations above the multiple governance spaces to which they might have to account.

What emerges in the West are systems of ‘governance, risk management, and compliance’.⁸⁵ Where in China the government (under the guidance of the CPC) pushes data-driven governance, in the West it is the market, and the delegation of managerial authority (compliance), that tends to drive these oversight and control systems – the literature highlighting this dynamic is by now well developed.⁸⁶ Anyone can rate and assess – but there are markets for rating as well; the most successful producers of rating enhance their profitability in markets for ratings. They focus on creating everything from hierarchies of value to assessment of conformity to a variety of corporate social responsibility obligations.⁸⁷ Less well developed is the conception of data, and the ratings that draw on them, as a system with regulatory effect. ‘Simply put, because of big data, managers can measure, and hence know, radically more about their businesses, and directly translate that knowledge into improved decision making and performance.’⁸⁸ Data-driven governance is essential for a variety of private sphere activities, and has become a business in its own right.⁸⁹

For both, data-driven analytics and the algorithms through which data-based judgements can be formed and consequences processed have become important elements of risk and compliance systems in five critical areas. The first is law enforcement (state entities) and compliance (private entities). Data-driven analytics here is presented as a method for complying with legal duty or responsibility. Administrative regulation and law provide the objectives, but the implementation occurs within automized data-driven systems. Examples include regimes for the distribution of governmental funds through revenue sharing and other programmes.⁹⁰ The second is transparency regimes: ‘It is used within an organization or community to enhance its operation and discipline its members; it is used externally to enhance legitimacy (norm) and accountability (technique) among stakeholders who have an interest in but not a direct participation in the operation of the enterprise.’⁹¹ The third is in controlling behaviour. For economic enterprises this was driven in part by law,⁹² and in part by changes in the way that administrative officials exercised authority.⁹³ Data-based analytics may be essential in the exercise of prosecutorial discretion in the USA⁹⁴ and the UK.⁹⁵ Businesses are increasingly using data-driven analytics to control behaviours through health and wellness programmes. The fourth, shaping cultures, is possible when monitoring (micro-surveillance) is tied to transparency and enforcement. Smoking campaigns are a well-known example.⁹⁶ Here one entangles moral value systems (about smoking) into political action (anti-smoking regulation) which is then entangled within medical and quantitative measures of harm which contribute to health rankings that affect private markets for insurance, and rankings that may affect individual access to credit or education.⁹⁷ Fifth is accountability as assessment, self-assessment and accountability regimes at the individual and entity levels.⁹⁸ These entangle both legal systems through compliance regimes and markets-based systems through data-driven ratings systems.

Entanglements between data and conventional governance orders have only recently emerged more clearly. The recent controversy over the inclusion of questions about citizenship on the US census provides a case in point.⁹⁹ Inclusion of the question would generate data, a core operational principle of census taking. That generation is coupled with a host of conventional law which relied on the governance of census data generation for its own operation. But the generation of data has significant consequences precisely because the system of regulation into which those data are injected is itself based on the way data are curated. Here is the point of entanglement between census as a system of data generation, census as the jurisdiction of compiling a quantitative narrative image of the American population, and census as a necessary predicate for the operationalization of a number of conflicting political objectives expressed though the law of states and the federal government. It’s no surprise to see census questions shifting with the times: ‘Lots of questions go off the census when they’re not very important anymore.’¹⁰⁰ The conflict within data governance became famously entangled with the legal regulation of the administrative state in a US Supreme Court opinion most notable for its recognition of conflict expressed in rules governing the exercise of political discretion within a web of legislation where the interests of secondary sovereigns are affected.¹⁰¹

Another important point of entanglement between data-driven and traditional legalities centres on the scope and principles through which data may be harvested. Most aggressive – again voluntary and on a bargained for basis – are chip implants for employees¹⁰² by merchants (including the state). This is a system grounded in consent, in value added as an inducement for participation. Increasingly, however, states have sought to use their legislative authority to restrict, or at least manage, this essential feature of data-based rule systems, by legislating the legal effects (and limits) of consent, especially within an employment relationship.¹⁰³ More passive are the seamless systems of cameras, credit card transactions, turnstiles, passes for highway tolls, key stroke and internet tracking systems and the like that can effectively track individuals and record their activities on a continuous basis. Their regulatory effect is now well known.¹⁰⁴ Yet their entanglements with plural legal systems have only just invaded the consciousness of regulatory stakeholders. Emerging issues include privacy rights;¹⁰⁵ and rights to be forgotten.¹⁰⁶ But the focus has also turned to the application of issues of race, gender and other non-discrimination law and policy to the structures of data-driven governance bound up in the extraction of information for behaviour management ends.¹⁰⁷ Conversely, data-driven systems themselves have entangled with law systems in the area of discrimination law by providing a complementary system, the products of which are used by law systems as techniques of proof of discriminatory intent or in forming or assessing policy and regulations.¹⁰⁸

Compliance itself is a data-driven exercise, but one in which the parameters are set by legal systems, administered through the actors onto which compliance is imposed, and assessed and disciplined either by the state or private actors.¹⁰⁹ Compliance is expressed as the private law internal governance systems that implement the delegation of responsibility (in part) from the state effected through law or regulatory directive. These include implementing administrative guidance,¹¹⁰ to compliance systems built around disclosure and reporting systems, for example, the French Supply Chain Due Diligence Law¹¹¹ or the Australian Modern Slavery Law.¹¹² The second is crafted through data-driven assessment systems that take their objectives from the policies of private law internal governance and which rely heavily on markets for external disciplining.¹¹³ These entanglements are clearest in the context of corporate governance.¹¹⁴

Of all of the forms of data-driven governance, perhaps the closest the West has to the emerging Chinese social credit initiative are credit rating agencies. Financial credit rating agencies, for example, become actors in the governance of financial markets through their production of a standardized rating of risk (creditworthiness), which is then used for making investment decisions, and consequently to make the regulation of financial markets dependent on the risk assessed.¹¹⁵ These operate within a market for services with a few big players rating business and public credit. Their profit derives from subscriptions or issuer-pays models of income. This produces a markets-driven model that mimics the effects of Chinese centralizing and public control models. These credit agencies’ businesses are grounded in data-based analytics applied to objectives and their effect is to discipline behaviours through reward–punishment systems derived from their analytics. Their systems are functionally differentiated, and they exhibit only a necessary unification even within the same field. At the same time, their activities serve as the basis for regulation, and are, to some extent, regulated by the state whose finances they in turn rate.

4.5 Conclusion

The relationship between traditional governance orders, in a context of spatially distinct but intermeshed legal/normative orders, remains to be fully explored. This chapter suggested some points of entry and on fundamental approaches. First, the use of metrics and the quantification of accountability has moved beyond an increasingly sophisticated palette of rankings and inducement to become a regulatory space in its own right. Second, that regulatory space embeds politics and law within the construction of its analytics and the determination of the meaning of rankings. China is building a centralized system tied to its political organization. Most of the elements of social credit have already been developed in the West. But the unification of the various elements, and their seamless operation, would be a great innovation. While the West approaches data-driven governance entanglements through the lens of privatization and markets,¹¹⁶ China inverts the trajectories of entanglement, focusing instead on recreating within the state the universe of data-driven governance in which law becomes the instruction manual for the operation of social ordering through data-driven analytics. Third, these systems pose a challenge for the conventional understanding of entanglement among systems all characterized by qualitative approaches to regulation. Perhaps most interesting of all the consequences of these social credit systems may be their ability to absorb traditional systems, and in that process of absorption to reduce the centrality of borderlands between systems.

Hints of this trajectory are evident in the movement towards compliance and accountability. These increasingly data-driven exercises turn traditional governance systems, and the governance systems arising between and within them (conventional spatiality of governance), into the generators of data that themselves can be subject to management in accordance with principles and objectives. These objectives may themselves be drawn from the political-cultural assumptions of society or themselves may be dynamic expressions monitored through the aggregate conduct of data generators themselves. The multiplicity of legal regimes, then, is itself a source of data useful for data-driven analytics that can manage these as well as aid in the management of systems.¹¹⁷ This suggests entanglement of a different order, between qualitative and quantitative regulatory measures which increasingly fold one into the other while retaining an element of autonomy based on the different regulatory spaces from which they are sourced.

What is clear is that entanglement can no longer ignore the legalities of data-driven governance even as it seeks to embed its language and structure its communication across systems that do not speak the same language. The entanglements of a law after modernity¹¹⁸ become more complicated – not merely as between distinct and polycentric rule systems sharing common characteristics (the forms and functions of rules), but now between systems that do not speak the same language (words versus metrics, compliance versus assessment, etc.). Governance evolves from the language and conceptual universe of politics and principles to the language of the operating system grounded in systemic objectives. In a world of algorithms, those who would devise them will be king. And those who would be kings in Western democratic republics may well soon be scientists and not lawyers.¹¹⁹ ‘When you look at the most important issues facing our country, it is climate change or healthcare policy or cyber security, the integrity of our elections. Who better to address these issues than a scientist?’¹²⁰ Who better indeed, when the state is a container for accountability systems through data-driven algorithms? That, ultimately, suggests the great challenge for legalities and their entanglements where society continues to move beyond the problems of entangling legal systems to those for which the econometrician, the statistician and the moralist¹²¹ may have as great a voice as the lawyer or the judge, and the politician or administrator.¹²²