Discretion in the Automated Administrative State

Abstract

Automated decision-making takes up an increasingly significant place in the administrative state. This article presents a conception of discretion that is helpful for evaluating the proper place of algorithms in public decision-making. I argue that the algorithm itself is not a site of discretion. The threat is that automated decision-making alters the relationships between traditional actors in a way that can cut down discretion and human commitment. Algorithmic decision-makers can serve to fetter the discretion that the legislature and the populace expect to be exercised. We must strive to maintain discretion, moral agency, deliberative ideals, and human commitment through the system that surrounds the use of an algorithm and to develop a new expertise that can retain and exercise the expected discretion. Backing this argument are traditional legal constraints, public expectations, and administrative law principles, tied together through the organizing principle of discretion.

Introduction

In this project, I seek to present discretion as an organizing principle in public law that will be useful for evaluating the legal and political legitimacy of algorithmic decision-making. ¹ An algorithm may be used in public decision-making in two distinct ways: as a tool of a decision-maker or as a new decision-maker itself. The duties and expectations that we place on these two modes of use are contrasted through this organizing principle of discretion.

I will develop a conception of discretion that I propose is helpful for understanding the role of algorithms across a spectrum of delegation. I am not interested merely in what choices have been left in the hands of administrative decision-makers, but rather, what does it mean for those decision-makers to make those decisions with discretion? Especially interesting is the seemingly unconstrained margin: when the legislature has conferred broad discretion on an administrative decision-maker, what does law require in that setting? What is expected from the decision-maker to fulfill their purpose(s) and to maintain legitimacy in a political system? Answers to these questions will inform where it is appropriate to deploy algorithms in public law decision-making and the system design that is necessary to ensure discretion is being properly exercised.

I argue that sub-delegation to an algorithm is appropriate only in limited circumstances. When legislated delegation contains minimal discretion—where we can break down an algorithm into discrete, expressible rules or operations reflecting the direction of the legislature—the use of an algorithm is unproblematic. However, even when used as a tool, algorithms get between actors that have traditionally been in contact; this presents the real risk of displacing human commitment and moral agency behind the decisions that are ultimately made. My novel claim is that the algorithm itself is not a site of discretion, so when discretion is called for, it must be maintained in the social structures and organizational design surrounding the use of the algorithm.

I begin with an abridged history of the administrative state in Canada and the ontology that has traditionally existed. I then present a contrast between that traditional ontology and the new ontology of an automated administrative state. Finally, I present a collection of expectations and principles that I have extracted from case law and secondary sources, related together by the organizing principle of discretion.

The Old Ontology

An ontology of the administrative state is crucial to understanding algorithms as “analytically distinct” forces and “tools in the governance process.” ² In order to understand the place of algorithms in today’s ontology and the changes they bring, it is helpful to review the traditional ontology and the motivations for its structure. ³ In this review of the old ontology, I recognize that algorithms have been used throughout history to give effect to legislative intent. For example, the Income War Tax Act of 1917 necessarily included a formula that prescribed the amount of income tax to be assessed. ⁴ In the most general sense, an algorithm is simply an “encoded procedure[] for transforming input data into a desired output, based upon specified calculations”; ⁵ these legislated taxation formulae fit that mold. However, much delegation has also included discretion. For example, in that same Act, discretion is involved in determining values for the inputs to the formulae, “reasonable allowance … by the Minister for depreciation,” allowing the Minister to exclude income on a discretionary basis, etc. ⁶ This discretion permeates the administrative state.

The Canadian administrative state finds its roots in the Privy Council and in the constitutional conventions of responsible government and Parliamentary sovereignty. ⁷ Under these traditions, the Privy Council (in practice, Cabinet)—on behalf of the Crown—exercises executive-branch decision-making as delegated by Parliament. ⁸ In the early 1900s, Parliament recognized the need for administrative bodies beyond the literal Cabinet. Parliament created boards of dedicated experts that would handle issues relating to, for instance, railways, grain supply, and international water boundaries. ⁹ Cabinet had neither the capacity nor the expertise to handle these questions and administration efficiently. Throughout the twentieth century, the administrative state has grown to touch nearly every aspect of our lives.

As mentioned, some of this delegated or sub-delegated power is imperative and non-discretionary. Examples of this kind of delegation are found in the front-line administration of large-scale logistics, ¹⁰ registration, ¹¹ or licensing systems. ¹² Vancouver’s current dog-licencing scheme is one such example. ¹³ Once a dog owner provides the required information and payment, a licence will be issued. With delegation like this—with minimal discretion and where we can break down an algorithm into discrete, expressible rules or operations—the use of an algorithm is unproblematic. These rule-like statutory duties “must be performed without unreasonable delay, and this may be enforced by mandamus.” ¹⁴

Of course, in much delegation, the only non-discretionary aspect is the requirement that a decision be made; aside from that element, delegation often allows and expects discretion. Vancouver, for example, has muzzle and security requirements for aggressive dogs: dogs with “a known propensity … to attack without provocation.” ¹⁵ This determination cannot be made without discretion and expertise. Courts supervise this discretion through judicial review. The approach taken by courts in supervising this discretion has evolved over the years. Through judicial review, courts both affirm and shape the ontology of the administrative state. In judicial review, the government is forced to self-identify its motivations for delegation and its requests for deference, and courts announce factual and legal realities regarding this ontology. By following this evolution, we can extract a traditional ontology of the administrative state—one centered on expertise.

Early on, courts were hyper-focused on ensuring the administrative actors had jurisdiction to do what they were doing—that the actors had interpreted their powers and duties correctly and were acting within that sphere of responsibility. ¹⁶ That view eventually gave way to one that recognized that “law making and legal interpretation are shared enterprises in the administrative state” ¹⁷ and that recognized the relative expertise inherent in the tribunals as institutions. One of the most oft-cited justifications for the delegated discretion is “the need for greater specialization and technical or subject-matter expertise.” ¹⁸ The Supreme Court has previously recognized the relative expertise of such decision-makers as a reason to defer to them, even on some questions of law. ¹⁹

Another early feature of Canadian administrative law was a specialized approach to reviewing discretionary decisions. ²⁰ Under that approach, discretionary decisions were only reviewable on grounds of bad faith, improper purpose, or the use of irrelevant considerations. ²¹ But even the reasoning of Justice Rand, in Roncarelli v. Duplessis, ²² spoke about review of discretionary decisions more broadly: “there is always a perspective within which a statute is intended to operate; and any clear departure from its lines or objects is just as objectionable as fraud or corruption.” ²³ And the Supreme Court in Baker explicitly folded the review of discretionary decision-making into the general framework of substantive review. ²⁴

Underlying judicial review in Canadian administrative law is a tension between parliamentary sovereignty and the rule of law. ²⁵ Courts must toe a fine line to both uphold the rule of law and avoid “undue interference … in respect of the matters delegated to administrative bodies by Parliament and legislatures.” ²⁶ On the other hand, judicial review can also be seen as furthering parliamentary sovereignty when courts ensure that a decision-maker’s jurisdiction is “narrowly circumscribed and defined according to the intent of the legislature.” ²⁷

In resolving these tensions, courts have assumed a particular ontology of the administrative state. Differing opinions within the Supreme Court of Canada’s membership can be explained through differing ontologies. These differences pre-Vavilov are best displayed through Alberta (Information and Privacy Commissioner) v. Alberta Teachers Association ²⁸ and Edmonton (City) v. Edmonton East (Capilano) Shopping Centres Ltd. ²⁹

Alberta Teachers displayed a full spectrum of perspectives regarding the role of expertise. Justice Rothstein, writing for the majority, held that when a decision-maker is interpreting or applying their “home statute,” the presumption is that “the Court should afford a measure of deference under the standard of reasonableness.” ³⁰ Under this view, decision makers benefit from assumed expertise when working within their home statute. Justice Cromwell, writing for himself in concurrence, presented a view of judicial review centered on determining legislative intent through a multi-factorial analysis. ³¹ He wrote, “The touchstone of judicial review is legislative intent.” ³² This is a view that is skeptical of whether a decision-maker possesses the expertise that the majority assumes, or alternatively, skeptical of whether the legislature intended any such expertise to be granted deference. Justice Binnie, writing for himself and Justice Deschamps, sets out a compromise position. For Binnie J., deference should be presumed for a decision-maker interpreting or applying their home statute only when it is “within [their] expertise” and when the issue is not of “general legal importance.” ³³ This position keeps expertise central to justifying deference, without assuming it exists simply because a decision-maker is working within their home statute.

By the time of Edmonton East, in 2016, the entire membership of the Court seemed to accept that expertise played a central role in determining whether deference to the administrative decision-maker is warranted. The majority wrote that “the presumption of reasonableness is grounded in the legislature’s choice to give a specialized tribunal responsibility for administering the statutory provisions, and the expertise of the tribunal in so doing.” ³⁴ Under this view, “expertise is something that inheres in a tribunal itself as an institution.” ³⁵ In this one paragraph, the majority both elevates and eliminates the importance of expertise. It becomes the core of the justification of deference to administrative tribunals but also becomes meaningless in practice given that it inheres in the tribunal itself. It is shorthand for the entire justification of deference. The dissent also focused on expertise but left open the possibility that certain tribunals lack expertise in certain areas before them. Their view was that the Board had a “lack of relative expertise in interpreting the law” ³⁶ and that the “legislature … has a role to play in designating and delimiting the presumed expertise of an administrative tribunal.” ³⁷ Even the dissent’s view seemingly premises deference on expertise, albeit expertise that needs to be demonstrated.

So, until very recently, Canada has presented, and the courts have recognized, an ontology of the administrative state centered on the legislature’s considered decision to delegate matters to institutions or people with relative expertise compared to the legislators and the courts. Where the legislature has left uncertainty to be filled in with meaning at the front lines or through expert bureaucracy, the court has been willing to defer. Courts are not the only site of moral agency that might “vindicate private autonomy.” ³⁸ Moving into the present, I show the changing position of expertise in the court’s view of the ontology and the invisibility of an old actor with new roles: the algorithm.

The New Ontology

Recently, in Canada (Minister of Citizenship and Immigration) v. Vavilov, ³⁹ the Supreme Court set out a new vision of the administrative state that relegates expertise to consideration only within reasonableness review: expertise is no longer a factor in selecting the standard of review. While expertise remains part of the justification for reasonableness review as the default, expertise no longer plays any analytical role when selecting the standard of review. The majority in Vavilov viewed an administrative body’s inherent expertise as an unnecessary proxy for legislative intent. In eschewing a contextual analysis for selection of the standard of review, perhaps it was inevitable that expertise would be sidelined in this analysis.

But the majority went further. One of the circumstances that will displace the default standard of reasonableness is when the decision-maker has decided a “general question[] of law of central importance to the legal system as a whole.” ⁴⁰ Previously, this ground for correctness review had required more: that the question also be “outside the adjudicator’s specialized area of expertise.” ⁴¹ The majority says that because consideration of expertise is “folded into the new starting point … namely, the presumption of reasonableness review,” ⁴² it is no longer relevant on this path to correctness review. The logical implication is that there are now questions that formerly would not have attracted correctness review (because they were not outside the adjudicator’s specialized area of expertise) that will now attract correctness review, even though the new starting point includes the assumption that the decision-maker has inherent expertise.

The majority in Vavilov did not merely sidestep the use of expertise as an unnecessary proxy; it also eliminated the possibility that relative expertise (either inherent or demonstrated) could prevent correctness review of questions of law that are of central importance to the legal system as a whole.

This vision set out by Vavilov reflects the Court’s current understanding of the ontology of the administrative state. The actors in this ontology are still the legislature, delegated decision-makers (either located in or associated with the executive branch), subjects of the decisions, and the courts. One primary relationship in this ontology is a reciprocal exchange of deference for justification. The delegated decision-maker obtains legitimacy and deference only when it justifies its decision to the subject of the decision (but reviewed by the courts). The practical expression of expertise in this new ontology is found in the “form and content” of the reasons. ⁴³

Some long-standing aspects of Canada’s administrative ontology are somewhat elided in the decision in Vavilov. Vavilov does recognize a spectrum of decision-makers and functions previously acknowledged by the Court. ⁴⁴ For example, Cabinet’s Orders in Council are to be struck down only on “an egregious case” of “jurisdictional or other compelling grounds.” ⁴⁵ Some actions of the Governor in Council are even considered “legislative action in its purest form.” ⁴⁶ Regulations created by government agencies benefit from a “presumption of validity,” only to be set aside if “irrelevant, extraneous, or completely unrelated to the statutory purpose.” ⁴⁷ The decision of a municipal council to pass a by-law will stand unless it is one that “no reasonable body” would have passed, informed by the wide variety of factors including broad “social, economic, and political factors.” ⁴⁸ But this variation is mentioned only in passing in Vavilov with the apparent goal that it be captured within a single standard of reasonableness review.

A clearer blind spot is the novel place of algorithms in the automated administrative state. The use of algorithms has a causal effect that shapes this new ontology in a way that is potentially problematic, and this has gone unrecognized by the courts. A complete description of the modern ontology cannot ignore how algorithms change relationships between existing actors. We must not only account for the algorithms themselves, but a collection of other entities they bring along with them: algorithm designers, procurers, and the algorithmic output. ⁴⁹ Algorithms get between human actors, displace human commitment, and are jurispathic in that they stymie sensitivity and creativity.

First, algorithms alter the relationship between front-line bureaucrats, the ultimate decision-maker (even when it is that same front-line bureaucrat), and the people subject to the decision. Algorithms get between actors that would have traditionally been in contact. ⁵⁰ Instead of a front-line worker reading someone’s application, an algorithm might pre-process the application, compressing and transforming that information. Those transformations encode policy choices that are often only implicitly made. ⁵¹

Algorithms also displace human commitment. ⁵² The discretion granted to delegated decision-makers is an unavoidable and often intentional choice by the legislature to defer committing to meaning. This meaning is instead intended to be built up at the front lines: through caseworkers in social-assistance programs ⁵³ or immigration officers, ⁵⁴ for example. The resultant meaning—this “legal world”—is created “only to the extent that there are commitments that place bodies on the line.” ⁵⁵ For these decisions and interpretations to create thick meaning, they must be backed by human commitment to the effects they will have on people. Algorithms avoid developing such commitment on the part of the state. The procurers and designers, who are crystalizing interpretation and policy, ⁵⁶ don’t see the front-line effects. The front-line workers may abdicate responsibility to the algorithmic decisions, especially if in an environment that does not foster a sense of moral agency. ⁵⁷ Decisions will be made but without intentional commitment.

The resulting algorithmic decisions likewise do not develop tradition. They cannot, in and of themselves, be a site of jurisgenesis. ⁵⁸ Jurisgenesis (in the sense developed by Robert M. Cover) has a “social basis” ⁵⁹ that is lacking at the site of algorithmic decisions. It requires meaning to be “essentially contested,” ⁶⁰ even if it may be ultimately articulated by an authoritative institution. This front-line flexibility is an essential quality of delegated discretion. ⁶¹ Bernardo Zacka presents this ideal: “Street-level bureaucrats must … resist the pull toward moral dispositions that are overly narrow … they must strive, as a group, to retain a range of dispositions that are sufficiently diverse.” ⁶² I am not arguing that there can be no jurisgenesis in a broader system that includes algorithms as a mere component, but where such jurisgenesis exists, it is found in the other relationships surrounding the algorithm. An algorithm itself contains pre-programmed meaning, killing off all alternatives. The use of an algorithm is thus jurispathic. Once deployed, they constrain creativity, variation, sensitivity, and justifiable divergence from past decisions.

What is discretion?

There are spaces in which the legislature can well specify the decision to be made and the manner in which it should be made: for example, British Columbia’s moose lottery. ⁶³ In such circumstances, where the legislature itself has sufficiently constrained meaning and flexibility, we may be losing nothing by using an algorithm to execute that legislative will. These would be sites where the legislature itself has been jurispathic, leaving no need or jurisdiction for creativity at the front lines. Recognizing this, another question arises: how should we decide which decisions can unproblematically be made by an algorithm? To answer that question, we must first understand the nature of discretion: what does it mean to exercise discretion and what are our expectations on that exercise? These questions motivate the remainder of this piece.

The conception I develop here considers discretion as part of law, contrary to the aphorism: “[w]here law ends, discretion begins.” ⁶⁴ While this conception is largely complementary to Kenneth Culp Davis’s project, which was focused on confining discretion, there are portions of his text that clearly recognize the importance of preserving discretion. ⁶⁵ As a whole, Davis was interested in finding the sweet-spot on the control-discretion spectrum. ⁶⁶ He also emphasized the important role for discretion in the administrative state—that “creativity is impossible without discretion” and that “when discretion shrinks too much, affirmative action is needed to recreate it.” ⁶⁷ His 1969 text was written at a time when computing was in its infancy, and Davis only mentions computers once. ⁶⁸

Decision-making has always been delegated along a spectrum of constraints. ⁶⁹ On one end of the spectrum, there is in effect no decision-making at all: a rule has been successfully specified. To give an example, front-line auto insurance providers in British Columbia have no discretion regarding the basic insurance tariff as it is dictated by the Insurance Corporation of British Columbia (ICBC). ⁷⁰ Other examples include the taxation formulae and the moose lottery mentioned above. At the other end of the spectrum, the legislature seems to have delegated near-unconstrained discretion. For example, under Newfoundland and Labrador’s Fish Inspection Act, the minister “may refuse to issue a licence required under this Act or the regulations without assigning a reason for the refusal.” ⁷¹ Much delegated decision-making falls somewhere in the middle. Consider, for example, the many decisions made by immigration officials when they determine a person’s eligibility for various types of immigration into Canada ⁷² or ICBC’s determination of the formula for the basic insurance tariff. ⁷³ This section aims to describe what is happening when exercising that discretion.

Discretion is not mere choice. It has meaning as a legal term of art. First, there is a sense of the word which connotes jurisdiction, capacity, or power. ⁷⁴ Courts often even use the terms discretion and power interchangeably or as part of the doublet, ‘discretion or power.’ ⁷⁵ In this sense, discretion can range from strong discretion to weak discretion. But in public law, “[t]here is no such thing as absolute and untrammelled ‘discretion.’” ⁷⁶ Discretion as a public-law power is always coupled with duties. ⁷⁷ These duties arise because the people subject to the decision are not mere objects with liabilities to the decision-maker—they also have rights. These rights correlate with the duties of the delegated decision-maker. ⁷⁸

The duty of discretion is not to deliver a particular outcome, but to exercise the discretion in a particular manner. While no outcome is guaranteed, discretion is. It is this second sense, the duty of discretion, that I wish to develop further. Here, discretion also becomes the “name of an intellectual virtue.” ⁷⁹ This discretion does not require legally-trained interpretation of statutes, but it does require informal, personal judgment conducted in good faith. ⁸⁰

I do not adopt Dworkin’s conception of strong discretion, which would mean the decision-maker has “the right to make any decision he wishes” and which would “deny any other participant the right to claim a particular decision.” ⁸¹ Discretion in our administrative state rarely possesses this quality. Dworkin himself argues that discretion in that sense is undesirable. He is notably arguing against that conception of strong discretion in the judicial context, where it has been advanced by others. ⁸²

Dworkin says that it would be “strange to say that a person who seeks to decide a troublesome question of conduct in terms of the moral standards of his community” had discretion. ⁸³ I do not think it so strange, given the conception of discretion that I develop here. Discretion is not a shedding of judicial obligation. ⁸⁴ When exercising discretion, the public decision-maker must “intend[] his reasoning to be based on public, not private, standards of good argument.” ⁸⁵

When an administrative decision-maker faces hard cases of interpretation or application, it may be the case that the ultimate decision is controversial—that reasonable people will disagree about the reasoning, justification, or outcome. It will also be often wrong to say that any party had a right to one outcome versus another. It is nonetheless helpful to recognize this as a site of discretion. “[F]ew, if any, legal rules admit of no element of discretion in their interpretation and application.” ⁸⁶ But it is not unfettered discretion. Only those decisions that are “based on an internally coherent and rational chain of analysis and that [are] justified in relation to the facts and law that constrain the decision maker” are reasonable. ⁸⁷ Canadian law even recognizes that sometimes, this discretion collapses to a single reasonable outcome. ⁸⁸ This is still called discretion and there are still public-law expectations for how that discretion is exercised.

Ultimately, discretion in public law is a process of reasoned decision-making, attentive to relevant factors as directed by the legislature and revealed by context. The exercise of discretion must establish anew a commitment to the validity, interpretation, and application of law in the often-unique circumstances of the subject before the decision-maker.

Where is discretion?

I will now identify several new sites of discretion in the ontology of the automated administrative state.

When an algorithm is deployed in the process of making a decision, there are many preceding and subsequent decisions to consider, aside from the decision or recommendation that might be exercised by the algorithm itself. There is the decision to employ an algorithm in the first place. There is the decision to select a particular algorithm. On the other side of algorithm selection is algorithm design, often performed by engineers and programmers wholly unaccountable to the people affected by the algorithm’s operation. ⁸⁹ Front-line bureaucrats may play a role in shaping the input that is delivered to the algorithm. ⁹⁰ And those same or other bureaucrats will have to interpret or give effect to the algorithmic output. All of these are sites of discretion to be constrained in various degrees by law.

Recognizing that these are sites of discretion is important for assessing the legitimacy and reasonableness of the outcome produced using the algorithm. First, just what is happening when an agency decides to employ an algorithm in its decision-making? If this is a delegation of decision-making power, moving the decision to the algorithm itself, the power to sub-delegate must generally be located in statutory authority. ⁹¹ Without such authority, this sub-delegation would be akin to fettering. ⁹² And even if the authority to sub-delegate can be located in statute or common law, I propose that sub-delegation to an algorithm is appropriate only in limited circumstances.

Specifically, when the constraints presented in the following section require discretion to be exercised, this cannot be delegated to an algorithm because the algorithm itself is not a site of discretion. Sure, algorithms may produce results that are unpredictable ahead of time or results that vary even when presented with the same input data. ⁹³ But this unpredictability or randomness is not discretion. Such variation does not flow from a “reflexive[] refine[ment] of decision criteria.” ⁹⁴ Any variation is limited by “decision boundaries effectively established ex ante.” ⁹⁵ The dispersion of outputs may look from the outside like the dispersion of outputs produced by a person, but the process and reasoning that lead the algorithm to that place are nothing like what we would call discretion. Some people even question whether such disembodied reasoning can ever replicate human cognition. ⁹⁶ Human thoughts, and discretion, are “inextricably associated with perception, action, and emotion, and … our brain and body work together to have cognition.” ⁹⁷

But an algorithm may be used as a tool of the decision-maker rather than being deployed as a decision-maker itself. In such a case, the institutional relationships described above (design, procurement, use) involving the algorithm must be structured in a way that preserves the discretion in the decision-maker, lest the relationship become a site of jurispathos.

Up to this point, I have argued that the algorithm itself should be the decider only in limited circumstances where the legislature has sufficiently constrained discretion. But in all other circumstances, algorithms must be carefully controlled, to be used only as a tool by the ultimate human decision-maker(s). And no matter whether delegating to an algorithm or merely using an algorithm as a tool, the law has recognized constraints on this decision-making. The next section considers those.

What are the constraints?

I have alluded to various expectations placed on the exercise of discretion. In this section, I present the legal constraints that are placed on delegated administrative decision-makers in Canada when exercising discretion. Discretion can be viewed as an organizing principle that ties together various constraints. Violating the duty of non-arbitrariness, conveying a reasonable apprehension of bias, approaching a decision with a closed mind, or fettering one’s decision—these are all different ways in which a decision-maker may fail to make a decision with discretion. This section maps to Kenneth Culp Davis’s question of structuring discretion. ⁹⁸ How must the decision be made in order to be made with discretion? As a starting point, discretion is “never absolute and beyond legal control.” ⁹⁹ Discretion comes with duties. ¹⁰⁰ These duties reflect the typical grounds of review within administrative law.

One duty is a duty to stay within the limits of delegated power. A somewhat outdated view is that the decision-maker has a preliminary duty to stay within their jurisdiction—within the scope of the delegated authority. ¹⁰¹ Under today’s framing, the question of the “limits and contours of [a] decision maker’s authority” is folded into the reasonableness review: “a decision that strays beyond the limits set by the statutory language it is interpreting” will be impossible to justify. ¹⁰²

A second duty is non-arbitrariness. The decision cannot be based on capricious or irrelevant purposes or on bad faith. ¹⁰³ Discretion is not choice based on “fancy or mere whim.” ¹⁰⁴ It is this duty that precludes randomness as an aspect of discretion.

A third duty is that the decision-maker must not convey a reasonable apprehension of bias. Baker established that this duty is a part of procedural fairness. ¹⁰⁵ At the highest level of generality, any person with a significant role in decision-making—no matter whether they are the ultimate decision maker or a subordinate officer—must act in an impartial manner. ¹⁰⁶ The test is usually phrased in terms of what would give rise to an impermissible apprehension of bias:

[T]he apprehension of bias must be a reasonable one, held by reasonable and right minded persons, applying themselves to the question and obtaining thereon the required information.… [T]hat test is “what would an informed person, viewing the matter realistically and practically—and having thought the matter through—conclude. Would he think that it is more likely than not that [the decision-maker], whether consciously or unconsciously, would not decide fairly.” ¹⁰⁷

This duty against a reasonable apprehension of bias has been most developed in the context of internal tribunal consultations with the ostensible goal of consistency. ¹⁰⁸ Consistency in the sense of treating like cases alike is a legitimate goal for an administrative decision maker. ¹⁰⁹ It is reasonable to “foster coherence” and “avoid … conflicting results.” ¹¹⁰ However, this consistency cannot come at the expense of discretion. The jurisprudence in this area has revealed a set of restrictions that tend to protect the decision-maker’s discretion and avoid the imposition of external bias: full board meetings regarding a decision must not be mandatory—they cannot be imposed on a decision-maker by a superior; no attendance or minutes are to be taken; the discussion must remain focused on law and policy; the ultimate decision must be left to the decision-maker(s) who heard the dispute. ¹¹¹

A subset of that duty against bias—or, a relaxation of that duty in certain contexts—is the duty to approach the decision with an open mind. This duty is rooted in the nemo judex principle that applies to decision makers. ¹¹² Some of the case law presents this duty as a prohibition on making statements that would indicate a closed mind. ¹¹³ The reviewing court will ask whether the statements “are the expression of a final opinion on the matter, which cannot be dislodged.” ¹¹⁴ Other case law states this duty more strongly: “the decision-maker must approach the issue submitted to him or her with an open mind, not influenced by personal interests or outside pressure.” ¹¹⁵ A unifying question is whether any “submissions would be futile.” ¹¹⁶

Fourth, there is the basic duty to actually exercise discretion, a breach of which is considered fettering. While fettering is often discussed alongside the duty to maintain an open mind, they are generally considered separate grounds of review. ¹¹⁷ Fettering has been placed under the umbrella of “abuse of discretion.” ¹¹⁸ It has in the past been unclear whether fettering is an independent, nominate ground for judicial review. ¹¹⁹ But it is now clear that Canadian courts conceive of fettering as undermining substantive acceptability and it is encompassed within reasonableness review. ¹²⁰ When the legislature has granted a sphere of discretion, the decision maker “cannot, in a binding way, cut down that scope.” ¹²¹ This issue often arises from the use of guidelines, jurisprudential guides, or leading cases. ¹²² Decision makers, especially agencies or tribunals comprising many distinct individuals, will naturally adopt coherence and consistency as a legitimate goal. ¹²³ But strategies adopted to foster consistency and reduce discord must “not operate to fetter decision making.” ¹²⁴ Fettering is an abdication of discretion. It is refusal to apply the principles of natural justice to the case in front of the decision maker. The exercise of discretion requires one to develop their own opinion on the basis of the particular facts of the case. ¹²⁵ When a decision maker’s discretion is fettered, this precludes the development of internal commitment.

Here is a hypothetical scenario that I suggest complies with the constraints just canvassed. The British Columbia Society for the Prevention of Cruelty to Animals (SPCA) has the power to sell or give abandoned dogs to new owners. ¹²⁶ Interested new owners complete an adoption form that includes biographical information, description of the intended home, how the dog will be cared for, details about previous pets, and willingness to have an SPCA representative do a home visit, among other things. The data from the form could very well be input to an algorithm that has been trained to produce a recommendation to an SPCA employee: approve, reject, or flag for further review. The employee sees this recommendation, turns their attention to any aspects of the adoption form that the algorithm has highlighted, and surveys the remainder of the input for anything suggesting deviation from the algorithm’s recommendation. Seeing none, the employee follows the recommendation of the algorithm. In such a scenario, the employee retained and exercised discretion. The algorithm was an efficiency tool, not a decision-maker.

The analysis changes if an algorithm is left to exercise discretion itself. Certainly, there are several of the duties just presented that the algorithm would meet by design. As a preliminary matter, barring a physical failure, the algorithm would not fail to produce an outcome; that is, it would at least exercise the power to make a decision, often even considering the factors required by statute. And regarding the above duties, the algorithm cannot act in bad faith. But that does not imply that it would be acting in good faith or non-arbitrarily.

The Consolidated-Bathurst/Tremblay/Ellis-Don/Shuttleworth ¹²⁷ arc further suggests that the use of algorithmic tools in discretionary decision-making should not be made mandatory by an agency absent the direction of the legislature. That line of cases establishes that internal agency mechanisms intending to foster consistency must not turn into constraints with outsized influence. Consultative proceedings cannot be imposed on the decision maker. For example, in Tremblay, the Court found that “the referral process [to plenary meetings] … circumvents the will of the legislature by seeking to establish a prior consensus by persons not responsible for deciding the case.” ¹²⁸ Likewise, when an algorithmic tool is provided to a decision-maker, an administrative body should question whether it has maintained an environment in which decision-makers feel free to refuse to use the tool. If an administrative body were to demand that decision-makers achieve a degree of throughput that is unachievable without resort to algorithmic assistance, this may “compel or induce” ¹²⁹ decision-makers to decide against their own opinions, or rather, fail to even form their own opinions. ¹³⁰ The risk is that the algorithm will serve to import and instill a normative world that will “tower in importance over the others.” ¹³¹

Especially widely researched today is the risk of bias within an algorithm. Here, I refer to a conception of bias that is akin to prejudice or discrimination. ¹³² This could potentially run afoul of human rights codes or the Canadian Charter of Rights and Freedoms ¹³³ and may give rise to a reasonable apprehension of bias. However, this is not the focus of this paper. Even if it were possible to craft an algorithm with ‘equitable’ error rates—for example, an algorithm with the same false-rejection rate across all demographics of interest ¹³⁴ —my argument is that if that decision were made without an exercise of discretion, that lack of discretion would be an independent ground of criticism.

Again, although the algorithm may produce decisions, even if predictively accurate when compared to what a human decision-maker may have decided (or at least indistinguishable from the distribution of decisions that a human decision-maker would produce), the reasoning process will “bear little resemblance” ¹³⁵ to that which would be exercised by a human. To achieve such a façade, an algorithm will either be following deterministic instructions, which is a fettering of discretion, or it will incorporate randomness, and that also isn’t what we expect from discretion.

Principles behind the constraints

The above duties are those that we have traditionally placed on human administrative decision-makers. Why do I argue that the constraints I extracted above are the appropriate administrative-law constraints to apply to the use of algorithmic decision-making? I propose this is justified by reference to principles behind the duties, namely: the expectations of the legislature, deliberative ideals, and the central role that expertise has played in our ontology of the administrative state. ¹³⁶ This particular cross-section of principles is tied together by the overarching organizing principle of discretion as a prerequisite to legitimacy for certain decisions. While not exhaustive, this family of principles is especially useful when focusing the lens of judicial review on algorithmic decisions. And I admit that these may merely be motivating principles rather than principles that normatively justify the above constraints.

The decision to deploy an algorithm or otherwise eliminate discretion in public-law decision-making must grapple with a trade-off. On the one hand, as an instance of a decision-making system that necessarily treats its rules as non-defeasible, an algorithm may provide “certainty, predictability, settlement, and stability for stability’s sake.” ¹³⁷ But at the same time, we miss out on the potential advantages of “fairness, equity, and, in theory, reaching the correct result in every instance.” ¹³⁸ It is therefore a political decision, sometimes constitutionally constrained, to determine where the former values of certainty and the like might overtake the latter values of equity, correctness, and the like. When I suggest constitutional constraints, I am talking about, for example, decisions for which the over- or under-inclusiveness produced by an algorithmic decision-maker might deprive a person of a section 7 ¹³⁹ Charter interest—the right to life, liberty and security of person—in a manner that is overbroad or arbitrary; ¹⁴⁰ or, as I suggest above, decisions that could discriminate in a manner contrary to the equality rights as outlined in section 15 ¹⁴¹ of the Charter. ¹⁴² I do not propose to establish where those constitutional constraints lie. But within those constraints, the process of determining where the political and normative lines lie requires us to give effect to the expectations of the legislature, deliberative ideals, and expertise at the front lines. These principles are part of a procedural theory, rather than a substantive theory about precisely where the legislature might be justified in allowing or requiring an algorithm to be used for public decision-making.

These principles are important today as governments and legislatures attempt to deploy algorithms in a way that is consistent with the existing judicial constraints. The European Union has released draft regulation on artificial intelligence which would regulate the entire life-cycle of an AI system. ¹⁴³ The Treasury Board of Canada has published a directive controlling how automated decision-making is used within much of the Canadian government. ¹⁴⁴ And the Law Commission of Ontario is providing recommendations about what concerns regulation should address and some recommendations about how to go about that. ¹⁴⁵ Whether any such regulation adequately constrains the use of algorithms will be up to the populace and the courts. Courts must consider whether the judicial constraints need to evolve in order to give effect to these principles within the new ontology of the automated administrative state. We should give effect to the expectations of the legislators, foster deliberative ideals at the front-line decision sites, and re-capture a pragmatic role for human expertise in the use of algorithms as tools.

First, legislators often expect discretion to be exercised. The nature of the freedom granted to administrative decision-makers is not due merely to the open texture of law and language (which of course, also exists), ¹⁴⁶ but is often due to “avowed discretion.” ¹⁴⁷ Legislatures delegate to administrative decision-makers the power to make rules, determine facts, and announce the rights of parties in particular disputes because of known unknowns. When the legislature is aware of its lack of foresight and aware that any rules it crafts would fall short of the task, it often will grant discretion. ¹⁴⁸ “Interpreting a statute in a way that promotes effective public policy and administration may depend … upon the understanding and insights of the front-line agency.… Administration and interpretation go hand in glove.” ¹⁴⁹ But along with this grant are expectations. The legislature may hope that rules and principles will “ultimately evolve in the course of the discretionary authority’s experience.” ¹⁵⁰ And statutes are not a “one-way projection of authority originating with government and imposing itself on the citizen.” ¹⁵¹ So, the legislature may very well expect to share interpretative responsibility with institutions that are in contact with the subject. ¹⁵² When the legislature expects this work to happen at the front lines, executive agencies should not “cut down that scope” ¹⁵³ of discretion by using automated decision-making. ¹⁵⁴

Deliberative constitutionalism likewise expresses some collective expectations of delegated decision-makers. One premise is that participants be “willing to revise preferences in light of discussion, new information, and claims made by fellow participants.” ¹⁵⁵ Under this theory, decision-makers must take a hard look at the unique situation of the individual affected by the decision. Even frontline decision-makers should be performing self-reflective inquiry about the proper scope and legitimacy of their actions, deciding to what degree they are ready to commit to state action. ¹⁵⁶ The interactions between decision-makers and subjects should be “sites of deliberative empowerment.” ¹⁵⁷ This is reflected in the courts’ emphasis in past decades on duties of procedural fairness, like the duty to be heard, and more recently on reason-giving. Geneviève Cartier suggests that the “centre of legal justification” be found in “collaborative practices of accountability and justification between courts, executive decision-makers and citizens.” ¹⁵⁸ I would emphasize the importance of contingent commitment by a decision maker who can feel the stakes. “Legal interpretation takes place in a field of pain and death.” ¹⁵⁹ But an algorithm does not cry; an algorithm does not die.

This view of governance can be undermined when algorithmic systems are deployed in the decision-making process because they separate humans from action and because they displace human commitment and fail to foster moral agency. But a third principle—a re-centering of human expertise—may operate to help further the previously-mentioned principles. ¹⁶⁰ If systems are designed to retain and foster moral agency in human decision-makers throughout the development, deployment, and use of an automated decision-making tool, the threat to legislative intent and deliberative ideals can be minimized. It may be useful to adopt the keyword ‘tool’ to describe the role of an algorithm in public decision-making. ¹⁶¹

Organizational design that views an algorithm as a tool would place the algorithm in a relationship of feedback with humans. ¹⁶² A human decision-maker can retain and exercise discretion to adopt or reject any suggestion made by the algorithm. They can retain discretion to alter the algorithm’s interactions with subjects, to retrain the algorithm on improved data, to extract additional features from input data, or to discard the use of the algorithm altogether. The human managers of automated decision-making systems should be new experts capable of skeptical interaction with the automated decision-making tools they are using and should retain the agency to alter the way that those tools operate in practice. ¹⁶³ I would caution, though, that skepticism should not become cynicism.

This new expertise will have to be different in content from the expertise that is assumed in the accepted ontology of the administrative state, but not different in purpose. Today, the presumptive posture of courts on judicial review is deferential, in part because of the expertise that is part of the ‘starting point’ the legislature had in mind when delegating certain decisions to an administrative decision-maker. This assumption of expertise lies behind the legislature’s delegation. For this assumption to actually be realized on the ground and continue to justify reasonableness as the default standard of review, decision-makers must not let themselves be displaced from their central position in this ontology.

The expertise required, in addition to the assumed subject-matter expertise, is a meta-expertise: an ongoing, self-reflective assessment of one’s own position in the organizational system that leads to a decision.

Naïve approaches to maintaining human-in-the-loop decision-making often fail to account for the effects that an algorithm can have on human decision-making. ¹⁶⁴ As just one example, empirical research has revealed that even if a risk-assessment algorithm helps a decision-maker come to more accurate risk estimates, it may at the same time induce risk aversion and lead to more negative decisions. ¹⁶⁵ Lorne Sossin has highlighted the important role of system design and training in fostering proper application of Charter values in administrative decision-making. ¹⁶⁶ Such system design and training is just as important for preserving discretion in the decision-maker.

Toward these ends, a great deal of practical work is left to be done on the ground: empirical research to uncover the tensions, pressures, and “impossible situations” faced by front-line decision makers, ¹⁶⁷ identifying the informal but essential tactics that these workers deploy in order to develop and maintain their moral agency. ¹⁶⁸ Algorithmic tools must be designed and evaluated with these goals in mind. ¹⁶⁹

Conclusion

Algorithms and automated decision-making are here to stay. They play a role in automating routine, non-discretionary administrative tasks and assist in discretionary decision-making. The threat is that automated decision-making alters the relationships between traditional actors in a way that cuts down discretion and human commitment. Algorithmic decision makers can serve to fetter the discretion that the legislature and the populace expect to be exercised. And algorithms can undercut ideals of deliberative democracy and moral agency that are hoped to exist in front line decision-making.

Recognizing the above, there are only limited circumstances when delegation to an algorithm is unproblematic: sites with no expected discretion. Otherwise, given that the algorithm itself is not a site of discretion, we must strive to maintain discretion, moral agency, deliberative ideals, and human commitment throughout the system and the organizational design surrounding the algorithm. The courts have not yet updated their understanding of the ontology of this automated administrative state. But this understanding will have to evolve in order to scrutinize the new relationships and the new kinds of expertise that are needed to give effect to legislative intent and to protect individual rights.