Over the course of the past two decades or more, forensic science in the United States and many other countries has stood accused of failing to live up to its promise to enhance the truth-producing capacity of the criminal law. Instead, forensic science has been described as possessing serious “flaws.”
In Autopsy of a Crime Lab, Brandon Garrett has produced the best overview for a general audience to date of the legal-scientific problems at the heart of this controversy. Readers unfamiliar with the controversy could do no better for a big picture view.
Garrett produces this overview by skillfully blending anecdotal case studies of deplorable injustices to which forensic science contributed, brief discussions of how various techniques work, and the presentation of research results relevant to these techniques.
In Part I, the book begins with perhaps the weakest and most vulnerable of forensic techniques, bite mark analysis, a subspeciality of a legitimate technique, forensic odontology, that got trapped into making extraordinarily “aggressive conclusions, based on their subjective opinions, in life and death cases” in which they “would claim 100 percent certainty in their conclusions” (pp. 21–22), contributing to some notorious wrongful convictions. Part I then goes on to broadly describe what it calls “the crisis in forensics.”
The structure of Parts II and III serves as a broad outline of the supposed “flaws” in forensic science. In Chapter 3, the book takes on perhaps the paradigmatic forensic technique: fingerprint analysis. The chapter is titled “False ID,” and it places those two letters—the word “identification”—at the heart of the problem with fingerprinting. It argues that the claim of “identification” historically associated with fingerprinting and other forensic techniques “obscures … that any comparison of fingerprint evidence, or any other pattern evidence for that matter, involves some degree of uncertainty” (p. 42). Instead, “Forensic experts should reach conclusions using frequencies based on data” (p. 56).
Chapter 4 addresses the issue of error rates. It shows that forensic analysts made unfounded claims about error rates, including claims of 100% certainty, that they resisted participating in error rate studies, and that when studies were done they sought to obscure or discredit the results. Garrett argues that courts should not permit forensic evidence to be used without error rate data, that techniques with unacceptably high error rates should also be disallowed, and that, if the evidence is used, the error rates should be disclosed to the fact-finder.
In Chapter 5, using microscopic hair comparison analysis as an example, the book discusses the problem of systematic overstatement of the meaning of forensic results. The hair analysis discipline was relatively modest about what could be claimed from the results of a hair comparison, but analysts at the FBI and other crime laboratories routinely told juries that the strength of the evidence was greater than even the discipline itself would claim, resulting in numerous wrongful convictions.
While the above problems are systemic in forensic disciplines, Chapter 6 notes that some forensic problems are attributable not the disciplines but to unqualified individual practitioners. Garrett points to a “fundamental need for rigorous proficiency testing, so that we know how good a forensic analyst actually is” (p. 95). However, although “all accredited crime laboratories are required to have annual proficiency tests, … it is widely agreed that those tests are extremely elementary” (p. 99).
Chapter 7 addresses how the psychological phenomenon of cognitive bias can apply to forensic science. Garrett argues that “Cognitive bias is particularly concerning because forensic analysts usually work for law enforcement, use subjective standards, and do not have to document their work” (p. 109). He argues in favor of adopting “blinding” procedures, such as those used in clinical medical trials and many other scientific activities, to mitigate bias.
What incentive does forensic science have to address the above problems? The most significant incentive lies in the courts: they are the primary consumers of forensic evidence, and judicial restriction on the admissibility of forensic evidence would exert significant leverage. Chapter 8 is titled “Gatekeepers,” after the “gatekeeping” responsibility for regulating expert evidence that the U.S. Supreme Court assigned to trial judges in the famous Daubert v. Merrell Dow case. Garrett documents that judges have consistently failed to exercise this power forcefully and explores the “judicial psychology” behind this failure.
Even if judicial gatekeeping is ineffective, there are other ways of regulating forensic science. Chapter 9 describes the broad range of procedures falling under the general rubric called “quality control” that are intended to ensure the quality of forensic evidence. It discusses notorious failures of those procedures, principally a series of large-scale scandals in the drug analysis units for the Massachusetts State Police.
Lawyers and scholars tend to focus on the analysis of forensic evidence, but forensic scientists are fond of reminding us that many problems originate at the crime scene itself, in the collection, contamination, alteration or forensic evidence—or even the failure to preserve it at all. Chapter 10 surveys these issues.
After discussing all of the above flaws, Part IV turns to reforms. Chapter 11 focusses on a few model crime laboratories, which are more independent from law enforcement and have embraced, rather than resisted, proposed reforms, principally the well-regarded Houston Forensic Science Center. Chapter 12 discusses the potential for “big data” to improve forensic science through the use of statistical techniques to quantify uncertainty and algorithms that can avoid the bias issues described above (while generating new concerns about biases “written in” to the algorithms). Ultimately, Garrett argues, “Machines will not solve all of the problems in forensics; they have already created new ones” (p. 194).
In the final chapter, Garrett presents a list of eight sensible proposed reforms for forensic science. He recounts the failed 2009 call by the National Research Council to create a National Institute of Forensic Science and the creation, in 2013, of a “short-lived” (p. 200) National Commission on Forensic Science, which was closed by the Department of Justice in 2017 when Jeff Sessions became Attorney General of the United States. But Garrett still retains hope that “a rebirth of forensics is finally underway” (p. 206).
