Over the past decade, a movement for algorithmic accountability has highlighted the power of firms and government agencies determining persons’ and entities’ reputations, visibility in searches, financial standing, and even liberty. A promising line of work has compared the methods and power of governments and financial institutions. This new edited collection joins this established conversation, and advances it significantly.
Money, Power, and AI is an ambitious collection that sheds light on how artificial intelligence (AI) and automated decision-making (ADM), across diverse legal systems, bring new sources of profits and power to financial firms and governments. Several chapters offer approaches for holding powerful institutions accountable. The authors exemplify diverse perspectives, informed by their background as public or private lawyers, social scientists, and public officials, working on various aspects of AI and automation in different jurisdictions. In this way, they demonstrate an important point about algorithmic accountability: contributions from attorneys, social scientists, computer scientists, journalists, and many other professions are critical.
Eight years on from the publication of The Black Box Society,Footnote 1 many unnecessarily opaque governmental and corporate practices persist and proliferate, while few transparency gains have been achieved. Secrecy permeates the use of many critical AI and ADM tools by both public institutions and private corporations. The net effect is something like a one-way mirror, which shields powerful businesses and government agencies from scrutiny, while permitting these same advantaged entities to engage in ubiquitous surveillance of consumers, citizens, and communities. Moreover, from a law and political economy perspective, the intertwining of business and government frustrates accountability.
The vast edifice of currency exchanges, derivatives, swaps, options, and countless other financial instruments rests on a foundation of law – or, to be more precise, rests on the relative power of one party to force another to obey the terms of contracts they have made. Though law to some extent shapes all markets, in finance it is fundamental. In fact, the ‘products’ traded are little more than legal recognitions of obligations to buy or sell, own or owe. This means that we deeply need books such as Money, Power, and AI which illuminate relationships among finance and government that are often obscured by legal and computational complexity.
The collection uniquely enriches our understanding of what it insightfully deems (in parallel) Automated Banks and Automated States, and proposes solutions to prevent, or at least mitigate, dangers that the use of AI and ADM poses in Automated Banks and Automated States. Reminiscent of Katharina Pistor’s The Code of Capital,Footnote 2 several authors examine how legal systems, economies, and polities allow financial firms to use ADM to their advantage. Others focus on governments taking advantage of technology, noting misuse, overuse, and bias, as Ryan Calo and Danielle Citron have observed in their article ‘The Automated Administrative State: A Crisis of Legitimacy’. Encouragingly, many of the same authors generally also acknowledge the opportunities that ADM can bring. The dose makes the poison, as Paracelsus advised, and sometimes better computing is needed to undo what worse computing has done.
By focusing on the use of AI for mitigation of financial crime and the role of the legal system in enabling and encouraging opaque AI, some authors raise deep questions for the legal profession. They rightly call for greater transparency and accountability in the use of automation in the financial industry, pointing to diverse harms and inefficiencies that occur even in seemingly low-risk contexts. The purported benefits of ADM in the financial industry may be outweighed by risks in some contexts, as the authors argue, and so careful and nuanced regulatory attention is needed.
However, this may not be easily achievable, giving the appeal of automated tools to government agencies, as discussed in Part II. A paradox may arise: just as the use of ADM in finance makes nuanced and personalised regulation all the more necessary, regulators may be encouraged to use ‘regtech’ (regulatory technology) and ‘suptech’ (supervisory technology) that is not up to the task. This is not to condemn regtech and suptech generally; in many cases, it may be capable of recognising problems far faster than human analysts. But several contributors to this volume paint a rather grim picture of state capacity across various jurisdictions, suggesting current personnel may not be poised to appropriately control the potential for abuse and misuse of powerful tools.
The final part invites the readers to consider various possible safeguards preventing the harms raised in the previous parts. Mere procedural safeguards will not be enough to counter the unprecedented power of Automated Banks and Automated States. Substantive regulation, and a renewed commitment to human-centred empathy towards vulnerable citizens and consumers are also necessary to address harms arising in increasingly automated environments.
Money, power, and AI intersect, and experts have long called for suspect practices of industry and governments to be uncovered and addressed. This book answers the call admirably. The collection answers questions that are crucial for scholars, policymakers, and regulators to consider in the current socio-technical context. The book draws important parallels between financial industry ‘private governance’ and states’ ‘public governance’, renewing aspects of the ADM research agenda. Similar automated tools have sparked forms of institutional isomorphism that sometimes improve efficiency, but also tend to have detrimental effects on citizens, consumers, and communities in terms of time demands, alienation, and new duties to interact with machines and understand their outputs. Replete with expert contributors addressing critical issues, this collection will shape scholarly and public understanding of Automated Banks and Automated States for years to come.
1 Frank Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information (Cambridge: Harvard University Press, 2016).
2 Katharina Pistor, Code of Capital: How the Law Creates Wealth and Inequality (Princeton: Princeton University Press, 2019).
Open access