Part IV - Courts, Data, and Civil Justice

13 The Disruption We Needed COVID-19, Court Technology, and Access to Justice

Bridget Mary McCormack

I think I bring good news. After decades of earnest efforts to address our growing access-to-justice crisis, we are weathering a storm that just might usher in the disruptive change we need. The COVID-19 pandemic has forced courts and lawyers to work differently, and we have learned that we can work differently. Innovation leapt into our courtrooms and our practices. Lawyers, judges, and court administrators were given the chance to rethink what they do and rebuild how they do it, and they seized the moment.

The pandemic has shown what many of us already knew, or at least suspected: There are many legal issues that can be effectively resolved without gathering in a physical space. In 2020 and 2021, “going to court” took on a whole new meaning. And as we look to 2022 and beyond, an opportunity remains for lasting, transformational change to our nation’s civil justice system.

“Access to justice” can be abstract, but in busy state courts it isn’t. For many Americans, and especially for those of limited financial means, our nation’s courts have not been able to deliver. A 2017 report from the Legal Services Corporations found that seven out of ten low-income households experienced at least one civil legal problem each year, and most families received inadequate or no legal help to address them.¹ These numbers are now so familiar that we have grown used to them.

Many individuals simply never try to navigate our justice system; they leave home when they get an eviction notice, they accept a default judgment, or they go without disability accommodations or public benefits. Some try on their own. All too often, their experiences in the legal system leave them discouraged and frustrated.

Those impressions are often earned. By and large, today’s courts are not structured to address the volume of modern legal problems. When COVID hit in February 2020, most courts looked a lot like they did during the 1918 flu pandemic. Legal proceedings played out in buildings constructed generations ago, governed by rules written by and for lawyers, at a time when dockets were small and most litigants were represented by lawyers.

Our profession has been focused on this problem for some time. Two decades ago, the Conference of Chief Justices resolved to support the promotion of equal justice by pledging to remove impediments to access, support public funding for civil legal aid, and expand the types of assistance available to self-represented litigants.² But the growth of self-represented and underserved individuals hasn’t slowed. At the same time, legal technologists have urged us to rethink our courts as service-oriented institutions and to adopt tools that will expand access. Though state bars, court administrators, academics, public interest groups and other stakeholders have supported these calls to action, change has occurred only around the edges.

The COVID-19 pandemic can change all of that. It has required state courts to radically rethink almost everything we do. In the past year we have seen rapid adoption of videoconferencing technology.³ State court leaders have been collaborating in new ways.⁴ And while the pandemic has shown that courts have the ability to replicate many in-person systems and processes on web-based platforms, reforming those systems and processes is a further step yet.

I think the pandemic has proven the technologists right: For state courts, expanding access to justice will require courts to implement technology solutions that bring courts “to” the people they serve. Aside from the technical details, this presents a variety of administrative challenges for state courts: questions about what platforms to use, who will implement and maintain them, and how to measure outcomes to ensure that the constitutional promises of equal protection and due process aren’t sacrificed in the name of efficiency. In this chapter, I will describe some of the remaining access-to-justice barriers as I see them, using my home state of Michigan, where I currently and proudly serve as Chief Justice, as an example.

But the challenges we face in Michigan are not unique. Rural communities without access to high-speed internet; large numbers of self-represented litigants; individuals who cannot afford or obtain adequate representation; disunified governance and funding; outdated and often decentralized court management systems with a patchwork of technology vendors and conflicting data standards – these are all obstacles that most state court systems face in one form or another.⁵ A growing literature has begun to lay bare these challenges as a central barrier to meaningfully expanding access to justice. This chapter will make these challenges concrete by describing the past, present, and future of A2J in Michigan.

I will begin by describing Michigan’s state court structure, dockets, processes, and culture as of February 2020. I will include here the excellent work under way before the COVID-19 pandemic to address the civil justice gap and to accommodate the large number of people who had to access courts without lawyers. I hope to provide enough detail to illustrate clearly the significant barriers to disruptive change. I will then describe the transition that courts made beginning in March 2020, including the specific solutions courts in Michigan have adopted to keep justice accessible while also keeping people safe. Here I will also describe the lessons we (judges, lawyers, and administrators) are learning from this rapid forced transition. Finally, I will address the potential I see for change despite the challenges that remain.

13.1 Michigan’s Judiciary – and the Challenge of Decentralized Governance

Judicial administration in the United States is famously decentralized. A 2011 report from the Office of Justice Programs of the US Department of Justice indicated that, in many jurisdictions, the state court’s central administrative office had only partial responsibility for providing information technology and technical assistance to trial courts.⁶ The OJP report also indicated that many trial courts receive a sizable percentage of their funding from sources other than state government. These disjointed arrangements between state and local governance inevitably raise issues of administrative control and the discretion that goes with it, complicating change management.

This challenge features prominently in Michigan, which for many years has struggled to achieve a more unified court system. Our existing court architecture is a product of our most recent state constitution, which dates to 1963 and divides the state’s courts into the Michigan Supreme Court, the Michigan Court of Appeals (an intermediate appellate court that hears both civil and criminal appeals), and over 200 trial courts, most of which are circuit, district, or probate courts.⁷

The Michigan constitution vests the Michigan Supreme Court with administrative oversight authority over all the courts of the state, which it exercises through the State Court Administrative Office (SCAO).⁸ Among other administrative functions, the SCAO develops statewide standards and guidelines, collects data, and provides technology solutions, assistance, and training for judges and court staff. The day-to-day operation of our trial courts, however, is managed by local administrators acting under the supervision of a local chief judge. Trial judges are selected in local, nonpartisan elections, and the chief judges are appointed by the Michigan Supreme Court. By law, the local county clerk (also an elected position) is designated as clerk of the circuit court but does not work for the court system. That means the clerk performs functions for the judicial branch and is subject to the circuit court’s direction in such matters, but the office is not directly subject to the court’s supervisory authority. (Seriously.)

The operations of our Supreme Court and Court of Appeals are funded by the state government. The funding of trial court operations is much more complex. Almost half of that funding comes directly from local governmental units. Court-generated revenue (including costs assessed on litigants and collected by the courts) represents about a quarter of all trial court revenue, as does funding from the state government. Federal funding represents less than 10 percent.⁹

This disunified trial court funding system creates problems that are well understood by the judges and court administrators. The problems are also familiar to lawmakers. In 2017, the Michigan Legislature tasked a commission to review trial-court funding. The commission found that, of the total operating budget for trial courts statewide, approximately 26 percent of that funding was generated by local courts in the form of assessments on criminal defendants. This excessive dependence on local funding led the commission to recommend a new approach: the creation of a statewide trial court fund that would receive all revenue generated by trial court assessments (as well as state funding), and then redistribute that money to trial courts based on specific operational requirements.¹⁰

One motivating concern was the conflict of interest that results when the assessing court keeps fees and costs to fund their own operations. But the commission also found that Michigan’s reliance on local funding was a significant barrier to a unified information-technology system. In fact, the commission found that Michigan’s trial courts used up to twenty different case management systems and an even greater number of computer systems. While historically localized funding may have allowed for “laboratories of innovation,” this technological diversity has led to significant duplicated efforts and inefficiency. The commission recommended that the state provide for the technology needs of the courts (hardware, software, infrastructure, training, and ongoing technology support), which would then be operationalized by local administrators and court staff. Legislators were taking the first steps toward this system in early 2020, before COVID-19 upended the state legislative agenda (and Michigan’s economy).

What does this mean for expanding access to justice? A unified approach, the commission thought, could save resources and improve the user experience. This view isn’t exactly groundbreaking: standardization of local court rules and procedures was one of the driving concerns of the court unification movement of the 1970s and 1980s.¹¹ A similar challenge exists as state courts integrate technological advances into their courtrooms.

Another benefit of unified courts and technology systems is a centralized data architecture that allows for real-time review of both management and outcome-related case data. In 2012, President Obama issued a directive entitled “Building a 21st Century Digital Government,”¹² which called for federal agencies to establish specific, measurable goals for delivering better digital services, including the goal of transforming unstructured content into structured data. Our state courts face a similar challenge. Many state court systems struggle to collect data in a usable, digital format, which in turns limits their ability to leverage technology. Where data exists, it often exists in an unstructured format (like paper or PDFs). And without usable data, technological solutions have a limited ability to help automate court processes, assist self-represented litigants, or measure judicial outcomes across categories to track equity. Despite decades of efforts to unify courts and track their work, many states struggle to collect and publish statistics beyond filing numbers, case types, and clearance rates.

To address our current piecemeal system, the SCAO created the Judicial Data Warehouse (JDW) in 2006 to provide a single repository for court data.¹³ Given the complexity of the court system, creating the JDW involved monumental effort, highlighted by scores of data-sharing agreements that coaxed local trial courts to share data while not giving up local control. Moreover, each time a new initiative requires additional data, courts need to retool their data collection systems and the JDW needs reprogramming to receive the new information. It’s clunky and inefficient.¹⁴

While JDW offers a workaround for nonunified systems, it is an example of how our decentralized funding structure requires the Supreme Court to exercise its constitutional administrative oversight duties by a mix of sticks and carrots. The sticks come in the form of rules and orders. The carrots come in the form of technology support and solutions, training, expertise, and some funding. Statewide access-to-justice improvements are thus extremely complicated; states with unified court systems have an enormous advantage in change management.

13.2 Access-to-Justice Efforts in Michigan in the Before Times

Each year, about 1.5 million people enter Michigan courts seeking resolution of some civil legal issue, and many do so without representation. In 2019, our trial courts handled about 105,000 domestic-relation cases, 35,000 adult personal-protective orders, 53,000 small-claims cases (up to $6,500), 209,000 landlord-tenant summary proceedings (including evictions), 6,000 housing and real estate matters (including foreclosures), 18,000 adult conservatorships and guardianships, and over a million civil infractions.¹⁵ When we add in criminal cases, the number of cases trial courts are adjudicating annually is between 3 and 4 million.

While our trial courts do not measure the number of self-represented litigants, surveys and studies both in Michigan and nationwide show that self-representation is very common in all of these case types. For example, a survey of the 42,000 divorce cases filed in Michigan in 2013 found that 48 percent of complaints were filed by self-represented people, 68 percent of cases had one or more self-represented parties, and 42 percent of cases had no attorneys involved at all, on either side.¹⁶ In other case types, self-representation is asymmetrical. A recent analysis of eviction cases in Michigan from 2014 to 2018 found that only 4.8 percent of tenants were represented by an attorney. Landlords, on the other hand, were represented in 83.2 percent of cases.¹⁷

Before the pandemic there were significant ongoing efforts to address this civil justice gap, by the Bar, the Court, and traditional legal aid funders. The State Bar of Michigan helps connect attorneys with pro bono opportunities for persons with limited financial means,¹⁸ and the Bar has several standing committees that focus on access-to-justice issues specifically.¹⁹ A separate institution, the Michigan State Bar Foundation (MSBF), provides grants to legal aid programs around the state.²⁰ These include five regional legal programs that take on various cases, as well as several statewide programs that focus on serving specific populations such as migrant farmworkers, Native Americans, and older adults. And Michigan’s five law schools have over forty-six clinical programs, many of which focus on serving clients who cannot afford an attorney.²¹ Along with direct representation, these organizations and institutions have a robust record of promoting access to justice through policy advocacy, amicus briefs, and service as advisers to the courts and policy makers.

There have also been less-traditional efforts to better address the civil justice gap, and some of these efforts leverage technology.

13.2.1 Michigan Legal Help

In 2012 the Court partnered with the State Bar to launch “Michigan Legal Help,” a website designed to provide practical, easy-to-follow information for people with civil legal problems who lack the resources to obtain an attorney (or who simply prefer to self-represent).²² The foundational idea for Michigan Legal Help is that promoting access to justice means helping those with legal problems get the information they need to successfully navigate their problem, even if they don’t need or cannot obtain an attorney. So many legal processes require a litigant to have a specific document, filed at the right time and in the proper place. Making sure people know what they need to do and giving them the tools to do it can go a long way.

By design, Michigan Legal Help does not provide legal “advice.” Instead, it provides topical information, forms, and dozens of “do-it-yourself” tool kits for specific legal problems. For example, there are tool kits for obtaining a divorce if you have minor children, another tool kit if there are no children, and a tool kit on how to respond to a complaint for divorce. The materials are constantly updated to reflect changes in the law, court rules, and real-world trends. There is an article on “Zoom hearings” that includes information on what to expect and tips on how to connect. A legal triage system – the “Guide to Legal Help” – is built into the website and directs users to specific resources based on their responses to a series of short, easy-to-understand questions. On weekdays, a “live help” chat feature staffed by volunteer law students provides personal assistance to help people find what they need; on weekends and after hours, users can leave messages that an agent will respond to. Michigan Legal Help staff members train the student volunteers and review the chat conversations to ensure quality control for the hundred-plus live chat sessions and a similar number of overnight emails that the system sees each day. Michigan Legal Help is also affiliated with (physical) legal self-help centers located around the state, where users can get assistance navigating the site and completing and filing documents.²³

Michigan Legal Help has enormous, unique benefits as one tool in a civil justice response system. In addition to the obvious, Michigan Legal Help (and similar web-based programs) generate data that provides a window into current trends and needs, which can then be used to inform other access-to-justice efforts. It is agile and iterative; the tool kits, forms, and processes are updated and improved based on user experience. And, of course, it is infinitely scalable. Indeed, Michigan Legal Help is maintained by a staff of only eight people, including four attorneys. In 2020, it had nearly 3.5 million sessions, 2.7 million users, and 6.6 million page views. Nearly 153,000 people used its resources to complete legal forms. Even accounting for the “pandemic spike” of users in spring of 2020, most of whom accessed information on unemployment benefits, the website has seen year-to-year increases since its rollout in 2012.

13.2.2 ADR/ODR

Michigan has promoted court-supported alternative dispute resolution as another solution for some legal problems and leveraged technology to support it. Beginning in 2014, a first-in-the-nation pilot program introduced online ticket resolution in two district courts. Using a system pioneered by University of Michigan Professor (and fellow volume contributor) J.J. Prescott, the program empowers residents to contest traffic tickets online. It is now offered in more than thirty district courts around the state.²⁴ Traditionally, ticketed drivers wanting to seek a dismissal or negotiate a reduction in penalties had to show up to court in-person; the only “remote” option was to accept responsibility and pay the ticket as issued. Through the online system, users can request a review of their citation, submit a statement, and upload photos or other information. According to a study of outcomes in six Michigan courts, this online ticket-review system reduces court staff time by 80 percent, and cases are resolved in an average of two weeks instead of seven. Most importantly, 39 percent of users reported that they would not have been able to come to the court in person.²⁵ Courts nationwide are now using this platform, and the service has been a critical tool in helping courts respond to the pandemic.²⁶

In 2019, the Michigan Supreme Court used this same platform to launch “MI-Resolve” in seventeen Michigan counties.²⁷ MI-Resolve is a fully online, asynchronous, dispute resolution program designed to resolve minor disputes, such as small claims, debt collection, noneviction landlord-tenant matters (like the return of a security deposit), and neighborhood disputes. Unlike traditional court-referred or party-selected mediation, which is usually done in-person and involves a mediator whose services are paid for by the parties, MI-Resolve is free for the participants and can be done with or without a mediator. The program is confidential and can be used regardless of whether there is active litigation. If the participants agree to a particular resolution, the system will produce a binding agreement and also generate any necessary forms for filing in court.

Michigan has led the way in online dispute resolution (ODR) programs like MI-Resolve, but experiences in other jurisdictions also show that ODR can provide an efficient and affordable way of resolving cases that do not involve complicated legal questions and where the amount at issue is modest. In 2020, the Joint Technology Committee of the Conference of State Court Administrators (COSCA), the National Association for Court Management (NACM), and the National Center for State Courts (NCSC) published case studies from ODR programs across the nation.²⁸ In Franklin County, Ohio, use of ODR for city income tax cases led to fewer default judgments and a more even distribution of positive case dispositions regardless of socioeconomic factors and race.²⁹ Utah used ODR as part of a total restructuring of its court system’s process for resolving small claims. The program has led to fewer hearings and faster dispositions in cases that do end up “in court.”³⁰ In Connecticut, ODR has been used to create an online ticket review program for traffic court. Most ticket recipients chose to “opt into” the program, which has shortened the number of days from citation to adjudication.³¹

13.2.3 SCAO’s IT Solutions for Local Courts and Filers Depend on Agile Culture

There have also been significant efforts in Michigan to transform the user experience of “traditional” court with technology solutions. Even without the benefit of a unified court system, the SCAO’s technology division has partnered with private sector firms to speed up development and implementation of IT solutions that connect Michigan’s courts with each other and with users, while improving the experience for both. If we could successfully provide excellent IT products and service to trial courts, we could make progress toward voluntary technological unification – carrots. But it means a fundamental transformation of our approach to IT. We are all familiar with government IT projects that promise to revolutionize process but fall short of expectations. Instead, we consulted private sector firms to bring agile culture to our public sector operation.

SCAO adopted Menlo Innovations’ “high-tech anthropology.”³² Just as smartphone apps undergo continuous incremental updates that don’t involve any inconvenience or loss of service, so too our court technology solutions improve as users provide feedback. We visit local courts and watch clerks, judges, and court users to see how they used our products and solicit their ideas to improve efficiency and service to the public. Agility is unusual in government IT and contrasts with the previous approach of taking down systems over a holiday weekend and launching a new platform on Monday morning. It limits risk, engages users, and steadily provides value-add, allowing for constant updates and changes with each iteration.

The agile process has been critical as we expand our case management product (MiCourt) to judges and court administrators who can be resistant to change. By engaging users for their input throughout the process, our developers benefit from on-the-ground knowledge and experience, and buy-in eases change. As a result, we have provided new features users asked for, including automatic calendaring, electronic docket boards, public dockets, case search, and text-message reminders of upcoming court dates and payments due. We are also using the agile approach to launch a new public website this spring that emphasizes mobile device use, which provides better and equal access to information for all users, as well as security, speed, and improved information architecture.³³

The success of MiCOURT can be measured both by the satisfaction of court users, but also by the expansion of the case management system to courts throughout the state as they learn about the unique, value-added features not provided by their current systems. And, as the list of MiCOURT courts grows, system-wide change becomes easier and more efficient.³⁴ A single case management system – our ultimate goal – creates enormous potential for statewide improvement. It will allow for real-time access to comprehensive court data, statewide process updates, and a simpler system for court users. That’s the holy grail.

To be sure, the picture is not all rosy. Michigan does not yet have statewide e-filing, and it will be another few years before that project is finished. Locally funded courts and locally selected IT systems mean that a statewide, online, case-filing system that aims to provide a consistent user experience for all filers in the state, regardless of jurisdiction and case type, is a heavy lift. Several counties had homegrown e-filing systems and electronic document management systems, while others continue to be limited to paper files. In other words, we are getting there, but it’s a slog.

13.2.4 Justice for All Taskforce

The State Bar of Michigan (MSBF), the Bar’s access-to-justice arm, and other access-to-justice oriented entities (like legal aid and law school clinics) traditionally collaborated through informal networks, not a formal structure. As a result, there had been no comprehensive assessment of the civil justice gaps and needs across the state, nor a measure of how the various programs and innovations were meeting the need.

In May 2019, the Michigan Supreme Court formed a task force to change that by conducting a full assessment of the current state of our civil justice system. Partnering with the MSBF and Michigan Legal Help, and with funding from the National Center for State Courts (NCSC), the task force set out to do a comprehensive statewide inventory of resources for civil legal problems, as well as identify the gaps in those resources. The task force was charged with developing a strategic plan to address the legal needs of every Michigander.

***

Heading into 2020, the task force felt that access to justice was a priority in Michigan and that the community of lawyers, judges, administrators, and technologists working on it were looking for innovative solutions to civil justice problems. Despite structural barriers, progress was being made. We would not have declared the problem solved, but we were optimistic about what we might accomplish. We were working diligently on pockets of innovation, as well as an overall strategy. And yet, transformational change was not really on our radar.³⁵

13.3 COVID-19: A Once-in-a-Lifetime Disruption

Then came March 2020. All of a sudden, courts had to pivot on a dime to do things differently. And (maybe to our surprise) we did.

In Michigan we were fortunate to have a running start. In 2010 the SCAO began implementing a videoconferencing project to equip at least one circuit, probate, and district courtroom in every jurisdiction. A year before the pandemic, we had distributed Zoom licenses to every trial court judge. This was not because we were clairvoyant, but because we had identified the potential for increased accessibility and efficiency.

As a result, in March 2020 every Michigan trial judge was equipped to quickly move their proceedings to the Zoom platform. We followed up by acquiring licenses for magistrates and quasi-judicial officers. We partnered with clerks and county commissions to develop training material and best practices for judges and court staff.³⁶ We suspended rules and processes that were barriers to remote proceedings.

To maintain public access, virtual hearings were made available on YouTube, either live-streamed or uploaded right after. In May 2020 the Court built and launched an online Virtual Courtroom Directory.³⁷ Using the agile approach, a project that typically would have taken at least six months went from concept to launch in three weeks. At first, the functionality was a map with green dots showing where courts were livestreaming. The directory grew into a fully functional application that allows users to click on a county, search for a court or judge, and find out if live proceedings are ongoing. Using the directory, users can search by judge or by location (county). Search results include a link to the court’s YouTube stream and show whether the court is “in session.” If the “live” button is lit up, users can click and watch. Anyone with an internet connection can easily find and watch the proceedings in any trial court.

Very quickly, judges, court administrators, and litigants discovered what worked online and what didn’t. The SCAO also organized more than fifty stakeholders in a work group to participate in a remote mock jury trial that generated detailed advice and guidance for courts.³⁸ Zoom proceedings work extremely well for some types of proceedings (motion hearings, status conferences, scheduling, civil trials, and so on) and less well for others (criminal jury trials). This aligned with our experience during the pandemic. For example, participants in guardianship and conservatorship hearings reported increased satisfaction with access and the ability to participate. There is preliminary evidence that online jury proceedings can increase jury participation and representation because those who do not have transportation or have to care for children or parents have more flexibility in serving.³⁹

The results of this technology-driven push to keep courts open were dramatic. By the end of 2021, our trial courts had conducted nearly 5 million hours of online hearings. Michigan residents have used the Virtual Courtroom Directory to find a hearing on YouTube more than 450,000 times. Collectively, trial court YouTube channels have nearly 185,000 subscribers.

We also significantly sped up our efforts to move MI-Resolve to the remaining sixty-six counties in the state. By July it was available in every county, making Michigan the first to have a statewide ODR. To date, over 1,000 cases have been filed in the system, and nearly 100 are pending. Increasing numbers of judges are now including in their orders that mediation can be conducted on MI-Resolve.⁴⁰ Local mediation centers are promoting the service on kiosks, billboards, and by highlighting the MI-Resolve web address on community partners’ websites.⁴¹ Promotional materials have also been distributed to local housing authorities, emergency service providers, and police agencies, so that they can provide the information to parties needing help resolving disputes.

Not only were we able to keep our courts operating; we could showcase the benefits that technology can provide to court users. Virtual hearings reduced or eliminated transportation and parking problems, as well as childcare and work conflicts. Recording the proceedings was seamless, and the number of people who could witness any hearing was endless; more than 7,300 people watched live a court hearing on a challenge to the governor’s emergency powers.⁴² And, frankly, the experience promoted digital literacy among judges and court staff.

Pandemic innovations were not only technology-based. For one example, we worked swiftly with the other government branches to establish a statewide eviction diversion program, funded by the American Rescue Plan Act and Michigan’s COVID Emergency Rental Assistance program, which is anticipated to help over 50,000 families remain in their homes.⁴³ We even exported the model to other states: Texas and Indiana followed our model.

13.4 Promise of Significant Reform

The Justice for All Taskforce continued its work through the pandemic, adjusting to remote platforms. The task force held in-person and virtual meetings and focus groups with judges, court staff, attorneys, and community organization staff members. It participated in town hall–style gatherings in Michigan’s two largest cities. It surveyed, interviewed, and focus-grouped members of the public about their experiences within the civil justice system.

The Taskforce delivered its report in December 2020.⁴⁴ Following guidance materials developed by the NCSC,⁴⁵ the Taskforce prepared a comprehensive assessment of fifteen interrelated civil justice system components, such as the integration of Michigan courts within the broader community of services, provision of legal assistance without involving an attorney, education of private attorneys about fee-reducing business models like limited-scope representation, and training of judges and court staff on how to engage with self-represented litigants effectively. The assessment found that while Michigan had made substantial progress on some components (especially highlighting the extensive efforts of Michigan Legal Help across a wide range of topics), not one of the fifteen was more than partially accomplished. Several key areas showed minimal progress, including little progress on emerging practices and innovations and a lack of navigator services.

For each component, the Taskforce identified gaps and barriers in our existing system and made specific recommendations for improvement. These steps range from the easy, obvious, and nontechnical, such as simplifying the language in hundreds of court forms or transforming courthouses as service-oriented institutions by having multilingual signage and a “courthouse concierge” role distinct from the clerk’s office. Many more of the recommendations, also highly achievable, rely on technology in one form or another: integrating self-help programs (physical and virtual) with a statewide e-filing program that is available to self-represented litigants and not just attorneys, revising court rules and procedures to maintain virtual hearings even after the pandemic, and providing publicly usable computers and printers in every courtroom (like the public library). Still others, both nontech and tech, are more difficult and potentially politically treacherous, like regulatory reform – that is, a relaxation of some of the rules that govern law practice – to give non-attorney “legal navigators” the ability to help clients handle simple legal problems.⁴⁶ Regulatory reform could extend to software-based service delivery models, too.

The Taskforce set an ambitious target of achieving 100 percent access to the civil justice system. Our goal isn’t to provide a lawyer for each and every person; rather, we aim to ensure that, wherever possible, the civil justice system provides information, tools, and resources for those who self-represent.

Finally, the Taskforce identified that a key stumbling block to comprehensive change is the lack of a permanent infrastructure between and among access-to-justice stakeholders. A month later, in January 2021, the Michigan Supreme Court created the Michigan Justice for All Commission, a thirty-member commission of judges, court administrators, legislators, state executive officials, legal aid attorneys, and community and business leaders.

* * *

The changes that have taken place in courtrooms and across the state judiciary and access-to-justice community during the pandemic are more significant than any that had taken place over many decades. We learned that sometimes different is better. And all this has taken place while our hair was on fire. It has us convinced that a lot more is possible.

13.4 Challenges That Remain

And yet significant barriers remain to growing access and making these changes permanent.

Some of these are cultural, maybe for judges most of all. While our judges and court staff responded admirably to the immediate challenge of the pandemic, I hear regularly from judges who “can’t wait to get back to how we used to do it.” I understand that sentiment: After all, judicial reasoning often depends on respecting decisions that have been made in the past, and many of our rituals are rooted in history and tradition. Being slow to change is a feature, not a bug.

Take one recent specific example. In 2019, the Michigan Supreme Court published for comment a proposed court rule that would allow non-attorneys to possess portable electronic devices (laptops, tablets, smartphones) inside a courtroom and use them to take notes, access the Internet, and make electronic copies of public records.⁴⁷ Previously, rules governing use of these devices varied widely based on locality. For self-represented litigants, restricting them to pen and paper was a barrier to access – one that was not imposed on attorneys (and therefore not on litigants who could afford attorneys). The push for change came from the public and the access-to-justice community. The resistance came from the bench. Some judges feared that allowing these devices into our buildings would lead to disrupted proceedings and diminish the “sanctity of the courtroom.” Others argued that such decision-making should occur at the local level – that a one-size-fits-all approach was inappropriate. Despite this pushback, we adopted the rule change in January 2020 – a happy ending. It all seems unremarkable in hindsight. And it shows that state high courts can compel change in their administrative oversight role.

But the episode also suggests that more comprehensive reform will require trial court leaders – local judges and administrators – who see as their roles to drive change from the ground up. And not all judges view engaging with the community, gauging its needs and concerns, and proactively responding to them as part of the court’s primary mission. While specialized dockets that emphasize problem-solving (like drug courts) have enjoyed support from local communities, many judges still take a hands-off approach to community engagement. In my experience, this judicial reluctance is driven, at least in part, by the idea that a disengaged court is also impartial, or at the very least appears so. There is also a concern that judicial leadership on issues of local concern will run into charges of “judicial activism.” But courts aren’t insulated from the realities of life, and judges who engage their communities are better able to understand the ways in which a justice system can respond to community needs.⁴⁸

There are other barriers, too. Funding remains a significant hurdle to reform, especially as state and local governments continue to struggle with the financial consequences of the pandemic.⁴⁹ Persuading legislatures who control state judicial budgets means convincing skeptical policy makers that expanding access to justice can return economic benefits, as well as social ones. Sometimes the pitch is an easy one: New technologies such as virtual hearings promise efficiencies, for both courts and litigants alike. Other changes and programs require more nuance, and the economic benefits can be harder to quantify. Court leaders should consider engaging nontraditional stakeholders. For example, employers understand the benefits of an employee able to join a court hearing from the workplace, instead of needing a day off; the business community is a logical ally for court modernization.

New access issues unique to remote hearings and online processes will also need attention. Rural communities are already underserved when it comes to legal aid.⁵⁰ These same communities often have limited or unreliable internet access. The pandemic has made this plain. Implementing change will require unique efforts for the technologically challenged.

Finally, until state justice systems collect good data, comprehensive reform will be elusive. Measuring rates of self-represented litigants is challenging, as is defining and tracking the type of data outputs that are necessary for evaluating the performance of programs attempting to expand access to justice.⁵¹ What is “fair” or “effective” can be highly contextual and is often (partially) subjective, and there is seldom adequate historical data to compare against. For states with disunified court systems, the challenge of developing data collection standards and mechanisms is even greater. Data collected through programs like Michigan Legal Help can help fill that void, but that data is only as good as the number of people who know about the service. Similarly, data and resource-sharing between and among access-to-justice stakeholders can be limited. Each may not have the time, resources, or expertise to collect and analyze data, even when they recognize the many ways that better data can improve their services. Courts and state Bar leaders can play a role by creating or facilitating an infrastructure through which separate players can collaborate with one another.⁵² Better data is critical to effective change.

13.5 So Now What?

Despite the significant barriers that remain, there is reason to hope that we are at an inflection point for access to justice and that lasting, disruptive change is possible.

First, feedback to pandemic-driven changes has been overwhelmingly positive. Litigants and attorneys like remote proceedings not because they are technophiles, but because of the advantages that remote proceedings can provide. This new normal has beaten back inertia. Usually the cat doesn’t get back in the bag; there will be no backtracking to the old way of doing business now that courts, lawyers, and the public have seen the advantages technology has brought to access and transparency. Many courts and legislatures are speaking openly about making permanent changes that were implemented as temporary, emergency measures.

Second, and perhaps more importantly, the pandemic forced us – judges, administrators, and lawyers – to think about our work differently. The pandemic has brought innovation to how we work. That skill, too, won’t be unlearned. As COVID-19 subsides, the ability to use remote platforms will allow courts to prioritize in-person time to things that don’t work remotely. This will help courts tackle backlogs: retired judges can preside in Zoom courtrooms from their homes, while local judges conduct jury trials in real courtrooms. The evidence that an entrepreneurial approach has taken root in the business of justice can be seen in many examples around the state. Consider these two.

In Washtenaw County, Michigan, a “mobile arraignment” program allowed a local magistrate to bring her courtroom to local community members who are homeless.⁵³ Partnering with a police officer with a smartphone, people in a local homeless shelter and even a local park were invited to clear up warrants with the officer’s iPad tuning into the magistrate’s courtroom. Legal documents were provided using a mobile printer. The roaming, tech-aided court operates weekly, serving members of the community most likely to have trouble navigating their justice system.

In East Lansing, Michigan, a local district court administrator who noticed how members of the public appreciated remote proceedings set up a virtual helpdesk. On the court’s website, users can choose the virtual helpdesk and then enter into a Zoom room where a clerk is waiting to answer questions, look up documents, and give instructions. People can get information previously available only by visiting the courthouse during work hours.⁵⁴ The response has been overwhelmingly positive.

Third, there is recent agreement about the importance of data to reform, and resistance to sharing data has receded tremendously. In 2020, the Conference of State Court Administrators and the National Center for State Courts published the National Open Court Data Standards to support the creation, sharing, and integration of court data.⁵⁵ And, the National Center has convened a group of state court leaders and academics to share information and data to see what lessons we can learn from all of the change. It is hard to overstate how different this approach is. There is both better and more data, a groundswell of interest, and agreement about how important data is to better our systems.

Fourth, as much as infrastructure barriers remain, there is potential for significant funding for overhaul, as a result of policy makers’ responses to the pandemic. Just as state and federal lawmakers quickly devoted funding to address the pandemic-caused eviction crisis, concerns about backlogs and realization that change is possible can spur funding to move state court systems beyond the clunky, data-poor, and ad hoc systems that have prevented large-scale reform.

Fifth, many states are exploring regulatory reforms that will allow testing of some of the bigger-picture solutions to our access-to-justice problem. Two major changes occurred just last year. Acting on recommendations from a task force on the delivery of legal services, the Arizona Supreme Court approved the creation of a new type of non-lawyer licensee that can provide (limited) legal services.⁵⁶ In neighboring Utah, the state supreme court established a new regulatory body (under the authority and oversight of the court) to license and oversee new types of legal providers and services through a so-called regulatory sandbox structure.⁵⁷ As noted previously, Michigan’s Justice for All Taskforce report recommended the creation of nonlawyer “legal navigators,” and the Commission has a working regulatory reform group. These experiments are perfectly timed, as we look for new ways of structuring the service of justice by welcoming new providers into the system, whether non-lawyer humans of the “legal navigator” sort, software-based systems that have begun to show promise as a way to serve self-represented litigants, or new models yet to come.

Finally, there is a national conversation afoot about all of this that includes academics and practitioners, technologists, and designers, activists, and advocates. That much is plain from the collection of voices in this volume. We have been shaken loose in a way to think creatively, and we like it. We are working collaboratively. And while many voices out in the states calling for change can be powerful, a “champion” in Washington, DC, to advocate for continued transformation that builds trust in our nation’s busiest courts is another important part of a national solution. The Department of Justice’s renewed Access to Justice Office could well serve that role.⁵⁸

As the pandemic (hopefully) recedes into something manageably endemic, the takeaway for our courts cannot be “the old way, but with Zoom.” Technology is an important tool in building a civil justice system with access for all. But we have to think about what kind of civil justice system we would build if we could start over. The moment to do that is right now.

13.6 Postscript

The COVID-19 vaccine rollout was still nascent when I drafted this essay in early 2021. Since then we have witnessed the vaccination of hundreds of millions of Americans, along with new coronavirus variants that have caused continued case surges and updated guidance from public health officials. Courthouses have begun the careful process of reopening their (physical) doors, even if sometimes temporarily closing them again. The roller-coaster continues, but there is unquestionably good news after an exhausting year.

During that time, I invited fellow state high court justices and their court administrators to participate in an informal survey. My goal was to capture their thoughts about technology changes spurred by the pandemic. The responses – from twenty-six of my colleagues from courts around the nation – have been illuminating.⁵⁹

When I asked what they perceived as the main barriers to adoption of court technology prior to the pandemic, almost all respondents cited funding (no surprise there), with reluctance among judges and court administrators a close second (about 75 percent). Those rates dropped in half when I asked the same question, but this time since the pandemic began. As one court administrator put it, “oddly enough, our barriers weren’t really barriers when we were forced with the need to do it.” Hopefully, an instance where perception is reality.

I asked questions about the adoption of specific technological solutions. Self-help resources and ODR were widely used before the pandemic, with a slightly higher percentage (increases of ~10 percent) answering that their court systems put these technologies to use since the pandemic began. The biggest increase was in the use of automated text-message systems that remind litigants about their court dates. About half of respondents indicated that such services were used before the pandemic, while 70 percent of all respondents said that these services were used as of spring 2021. About two-thirds of respondents whose states had implemented this technology credited it as improving appearance rates among self-represented litigants. For jurisdictions that have yet to adopt text reminders, the off-the-shelf commercial availability makes it low-hanging fruit.

Unsurprisingly, many respondents indicated that little-to-no court business was conducted online prior to the pandemic (excluding purely voice communications, such as VoIP), and where videoconferencing was used, it was highly limited (for incarcerated litigants, involuntary civil commitment, etc.). Survey-takers reported higher participation with the change to remote access necessitated by the pandemic.

Very few respondents (about 10 percent) collected data from litigants about the litigants’ personal satisfaction with court access and the perceived fairness of the proceedings. I note this in particular because the pandemic required many courts to either create or build out already-existing virtual platforms. I don’t mean to suggest that “customer satisfaction” is the only measure by which courts should be graded, especially in an adversarial legal system. But for self-represented litigants especially, satisfaction with the process can be just as important as the outcome.⁶⁰ Questionnaires can help identify potential areas for improvement without compromising ex parte communication, and digital platforms adopted during the pandemic offer a seamless means of delivering them to litigants.

The survey was done on a short timeline, but I see reason for cautious optimism in the answers I received. As in-person proceedings resume, I’ll end on this note for judges, court administrators, and lawyers: We can’t allow the lessons learned and the skills developed over the pandemic to go to waste.

14 Free PACER

Jonah B. Gelbach

Data can fuel innovation. The next generation of legal scholars will stand not just on the shoulders of their forbears, but – hopefully – on the structural foundation created by access to the electronic public record. When they “surf to the courthouse door,” they should find it open.¹

This is the worst of the best of times for federal court data. Docket reports are compiled electronically, and the overwhelming number of documents are filed electronically using the CM/ECF system. Litigants, attorneys, and members of the public can obtain case information using searches of the Judiciary’s PACER system, which stands for Public Access to Court Electronic Records. Not so long ago, none of these things was possible; people had to travel to the courthouse and conduct or request laborious searches for case information. But access is so much less than it could be, and this sorry state of affairs is the result of deliberate policy choices. We should free PACER’s data, making federal litigation data free to the same public whose judicial system creates and stores those data.

Much of my case is based on a general argument about the value of providing free public access, especially to the scholars and journalists whose analysis can help monitor and improve our justice system. But there’s a tech-specific case, too. Freeing PACER would broaden access to data, possibly allowing the development of accessible, affordable legal technology for the “have-nots” as well as those without the budgets necessary to collect or purchase their own data.²

Still, my focus in this chapter is on two other aspects of access to justice. First, the public’s access to the way the justice system operates: access for those who measure, monitor, and evaluate the justice system as a whole. The second aspect is society’s access to data-driven improvements in the justice system. Without access to large amounts of litigation data, society will forgo the gains from having scholars and journalists assess our judicial system’s performance; anecdotes and surmise will rule the roost. I take as given that scholarship and journalism can help us evaluate and improve the justice we mete out. Thus, better data access gives us access to better justice by allowing us to better understand, better design, and better operate our justice system. For these reasons, this essay will focus on the case for freeing access to court data, especially for scholars.³

My argument is mercifully simple: The various policies and practices that have limited access to PACER for the past three decades have, as the Supreme Court once put it in an unrelated context, earned their retirement.⁴ PACER began as an experimental dial-up system in 1988 before taking flight on the World Wide Web ten years later. It’s been a long flight, actually. Though PACER has radically expanded public access compared to the status quo ante a third of a century ago, PACER searches cost too much for users and deliver too little. Its search functionality is roughly the same as it was back when. You can’t search the text of docket entries, for example, much less the contents of filed documents. With some useful exceptions, PACER keeps our judicial system’s public records behind a kludgy and unjustifiably high paywall.

PACER costs so much because in 1988, when Congress allowed the Judiciary to provide electronic public access, it directed the Judiciary to fund the initiative through user fees.⁵ Happily, Congress could fix that problem with simple statutory text, and the fee revenues are so small relative to federal spending⁶ that Congress needn’t give a thought to whether it would break a sweat thinking about the question. Congress should mandate free PACER and cut the Judiciary a check large enough to replace the lost revenue. Problem solved.

* * *

Section 14.1 of this chapter provides a brief discussion of PACER’s history and present. In Section 14.2 I argue that opening PACER up to the public, with a focus on scholarly access, would lead to a greater quantity of research on the federal courts while also allowing scholars to carry out higher-quality studies.⁷ It is time to unlock the doors to the benefits of open research on our justice system. Section 14.3 offers a vision for the future, centered on free access to PACER and direct funding of Judiciary activities to replace lost fee revenues, as well as fostering an attitude of openness in the Judiciary.

It’s time to take down the paywall that divides judicial records and the public.

14.1 Diagnosing Scholarship’s PACER Predicament

Litigation in most federal courts is managed through the Case Management/Electronic Case Files (CM/ECF) system operated by the Administrative Office of the US Courts (AO).⁸ Parties file pleadings, motions, and other case documents through CM/ECF; the Court also uploads documents to this system, and textual information about case status, scheduling, and so on, exists as docket entry text. PACER is an online system that allows members of the public to get case information, including both docket reports that list case activity and case documents.⁹

14.1.1 PACER Basics

PACER¹⁰ began as a dial-up-based experiment in 1988 by the Judicial Conference,¹¹ at a time when the public had to go to physical courthouses for case information.¹² Because PACER isn’t free to operate, and Congress doesn’t appropriate funds for it,¹³ the Judicial Conference has long charged fees for PACER use. Since 1998, PACER has been accessible through World Wide Web connections; the per-page fee for downloaded information is now $0.10.¹⁴

Such fees make large case-info downloads costly. By policy choice, the Judicial Conference limits fees in some ways; today, charges are capped at $3.00 per document.¹⁵ In addition, PACER waives fees for any quarter in which a user incurs under $30 in charges.¹⁶ Parties and their attorneys receive a free copy of any electronically filed document “if receipt is required by law or directed by the filer.”¹⁷ Judicial opinions are supposedly free for anyone’s download,¹⁸ although that may not always be true in practice.¹⁹

Unfortunately, these policies do little for scholars who must download many documents for their work. The Judicial Conference does allow courts to exempt some persons from fees, including “individual researchers associated with educational institutions,”²⁰ and exemptions have facilitated important research, but much less than broader scholarly access would allow.

14.1.2 PACER Serves Up Case Records in a Clunky Way

What PACER allows users to access remains far too limited. Critically, as the Free Law Project notes: “There is no document-level search.”²¹ And although PACER users can search for litigants’ names or retrieve a case using a known docket number, it is impossible to search, say, for some version of “all cases that have at least one docket entry with the string ‘FAILURE TO STATE A CLAIM’ or ‘QUALIFIED IMMUNITY.’” The Free Law Project observes: “This makes the system utterly unusable for general purpose research.”²² Anyone seeking to obtain data from many cases of a type – those with a Rule 12(b)(6) motion, or involving qualified immunity – must pay for enormous numbers of documents. This drains research budgets for no good reason.

14.1.3 Financial Facts for PACER

The Judiciary receives no funding from Congress for PACER, and the CM/ECF backbone is supported entirely by PACER fees. Each year these fees bring in around $150 million.²³ To put it mildly, that greatly exceeds the costs to the Judiciary of operating PACER, which one source recently put at $3 million.²⁴

Who uses PACER? The Judiciary emphasizes PACER’s enormous user base – more than 3 million user accounts, by 2020.²⁵ As of 2015, 75 percent of PACER usage was unpaid due to fee waivers (perhaps principally the $30/quarter free-usage allotment), even as 75 percent of PACER revenues came from fees paid by just 1 percent of account holders.²⁶ Although I am aware of no precise data, it’s easy to guess that the biggest paying users include the big three electronic legal data vendors – Westlaw, LexisNexis, and BLAW – and bankruptcy data providers that service claimants’ data needs. As of 2015, 63 percent of PACER usage was undertaken by law firms, 12 percent by litigants themselves, 10 percent by commercial users, and only 3 percent by academics.²⁷ That academics use PACER so little is likely a testament to the fact that the large data pulls we need to do serious quantitative research would cost so much that most scholars either never start or rapidly abandon plans to study federal litigation using PACER data; see discussion in Sections 14.1.4 and 14.2.2 for more.

These facts paint a simple picture. PACER’s paywall brings some money to the Judiciary, allowing Congress to fund it a bit less. The wall doesn’t stop the biggest legal data vendors from downloading a lot of data, which they make available to subscribers who pay for platform access. The wall likely doesn’t stop litigants in most individual cases, at least not with respect to their own cases. Nor does it likely do much to keep interested small-hobbyists out.²⁸ But the PACER paywall sure does screen out scholars.

14.1.4 PACER Is Prohibitively Expensive for Scholars

To study litigation, the judicial system, and how they interact with social phenomena, scholars need large amounts of data. Because of PACER’s high per-page download fees, even projects involving relatively modest data can be very expensive.²⁹ To wit, a recent PACER fee waiver request estimates that to download only the docket sheets for just two years’ worth of FOIA cases would cost nearly $4,000.³⁰ And consider a commentary recently published in Science,³¹ which presents information on thousands of cases in which a party sought in forma pauperis status and shows considerable variation within districts in the likelihood such a waiver would be granted. The authors downloaded every docket report for all cases filed in 2016, at a cost they describe as “more than $100,000.”³² That won’t become a common research strategy.

Although some research is conducted with second-best data sources (see section14.2.2, infra), I’ve had numerous conversations over the years with scholars who abandoned research ideas they thought promising because PACER’s prohibitive costs made it too costly to push forward.

14.2 The Case for Access to Court Data

As a general matter, PACER’s paywall is unfortunate and out of place in a democracy, but it’s especially irrational as applied to scholars. Under the status quo, scholars contribute very little in the way of fee revenues, so there is close to nothing to lose from granting us access. On the other hand, we’d contribute a lot of free research if we were allowed access. And privacy risks seem limited given that all documents in question already may be downloaded by anyone who knows to look for them. In sum, right now the Judiciary and the public are getting nothing for something.

14.2.1 On Principle, Court Operations Should Be Presumptively Open to the Public

Publicity is the very soul of justice. It is to publicity, more than to everything else put together, that the English system of procedure owes its being the least bad system as yet extant, instead of being the worst.

– Jeremy Bentham³³

It’s a basic principle of a democratic society that government’s operations should be open to public scrutiny. Otherwise, the citizenry cannot tell whether government is operating justly, speedily, or inexpensively.³⁴ Even appropriate scrutiny of one governmental branch by another can be limited without sufficient openness.

Constitutional and federal common law bases for public rights to observe judicial proceedings are most developed in the context of criminal trials,³⁵ but these are of course only one spot in the landscape of judicial proceedings. Of the nearly 80,000 federal criminal cases filed in 2018, only about 2 percent went to trial,³⁶ a number that isn’t much different for civil cases. With so few trials, pretrial proceedings delineate the fairness and future incentive effects of federal litigation. Accordingly, whatever interests of publicness inhere in open trial proceedings also extend to pretrial ones, too.

Professor Peter W. Martin has described five purposes for public access to litigation-related documents: (1) assuring fairness – both actual and perceived – of proceedings, (2) ensuring that the public’s ability to assess and criticize judicial performance can be done on an informed basis, (3) educating the public about how litigation works, and – hopefully – inspiring confidence as a result, (4) enabling the “check[ing]” of the judicial system by the public, and (5) “[p]roviding an outlet for community ‘concern, hostility, and emotion’ in cases that are in the public eye”.³⁷ At least the first four of these are implicated in the Judiciary’s refusal to allow affordable public access to PACER for scholarly or journalistic research.³⁸

In sum, as House Judiciary Committee Chairman Jerrold Nadler recently stated: “It is a disservice that in today’s digital age, the public’s access to public records in public proceedings is so resource-intensive and burdensome …. It is indefensible that the public must pay fees – and unjustifiably high fees, at that – to know what is happening in their own courts.”³⁹

14.2.2 Important Research Could Be Done with Freer Access, and for Free to the Judicial System

This section provides a brief, ad hoc review of quantitative empirical work that has been done using federal court data.⁴⁰

The Limits of Non-PACER Data Sources. Even before PACER, there was much quantitative empirical research published using federal court litigation data. A chief alternate data source is the Federal Judicial Center’s Integrated Database (IDB), “by far the most prominent” of data sources used for statistical research by legal scholars.⁴¹ Scholarly work using the IDB has spanned the gamut of topic areas for years.⁴² But the IDB has its flaws.⁴³

So, when FJC researchers set out to study summary judgment practice at the request of the Advisory Committee on Civil Rules in the wake of the Supreme Court’s 1986 summary judgment trilogy, these researchers analyzed docket sheets randomly selected in three district courts.⁴⁴ Subsequent FJC studies of summary judgment practice also used docket information.⁴⁵ This is particularly notable because summary judgment dispositions are the sort of issue of interest not just to scholars or elected officials, but also to the Judiciary. In fact, the sort of data the FJC, and Professor Burbank,⁴⁶ has used to study summary judgment is the primary source of information for judges doing rule-making. Yet as intensive as this work was, it typically involved studying few districts, or even just one at a time.

The CM/ECF system now contains all data necessary to enable nationwide studies, and by 2008 (if not sooner), the FJC was doing so in response to Advisory Committee requests.⁴⁷ Due to PACER’s paywall, though, outside researchers cannot generally do likewise. That means the Judiciary, and society more widely, are partly subject to the limits of FJC’s resources, however impressive its ingenuity, in addressing matters of interest to rule-making.

Another area of great interest in which the IDB is limited is the pleading standard, whose alteration in Bell Atlantic Corp. v. Twombly⁴⁸ and Ashcroft v. Iqbal⁴⁹ was arguably the top focus of procedure scholars for a period of several years. As Professor Burbank wrote even before those cases, the IDB’s limitations make it difficult, at best, to determine anything specific about Rule 12(b)(6) and 12(c) motions decided under the Rule 8(a)(2) pleading standard, including even “the number of such motions filed or decided.”⁵⁰

There are still other data sources for scholars studying the federal legal system, all with key limitations. For example, the US Sentencing Commission posts defendant-level data on sentences in federal criminal cases.⁵¹ Interesting research has been conducted using these data, for example, on racial disparities and the federal sentencing guidelines.⁵² But they obviously concern only criminal sentences. Judicial opinion databases are another source of data; in some cases these can be useful,⁵³ though it is well known that for many others this approach often brings bias given that only some opinions are published.⁵⁴ Another alternative is the Court Listener RECAP archive,⁵⁵ hosted by the Free Law Project, which contains “millions of PACER documents and dockets”⁵⁶ and grows over time as more users use RECAP, even as users are able to avoid paying for documents that RECAP already has.⁵⁷ As great a resource as RECAP is for documents that have been stored, there is no general way to measure how representative RECAP’s sample of documents and docket reports is.⁵⁸

PACER’s Unrealized Potential. This brings me to work using all of PACER, or representative samples. Bankruptcy has been a prolific substantive area for empirical study, due in part to the enterprising efforts of Professors Teresa Sullivan, Elizabeth Warren, and Jay Westbrook. The challenges surmounted in this work are described engagingly by Professor Lynn LoPucki. In brief, the professors flew themselves and some photocopiers around the country, copied court files in the clerks’ offices, and sent the copies to Austin, Texas, where they coded the information they would use – even though copy machines in the clerks’ offices were unused after hours.⁵⁹ In 2001, after PACER’s advent, the same researchers were able to secure judicial permission to hire “moonlighting assistant clerks to photocopy the files and ship the copies,” because that allowed them to avoid PACER’s per-page download fees.⁶⁰ As Professor LoPucki writes, “With the political constraints removed by judges, the 1981 technology – photocopying in the clerks’ offices – was cheaper than the 2001 technology – internet PACER – with its one-page-at-a-time political restriction.”⁶¹ In sum, at enormous and duplicative cost, these researchers were able to obtain substantial data on bankruptcy cases. They used these data to publish numerous studies about an issue of great public policy concern.⁶² Subsequent research on consumer bankruptcy reform was able to use both PACER fee waivers⁶³ – obviously a welcome development – and help from AACER, the Automated Access to Court Electronic Records, a service that collects bankruptcy court data on a daily basis and repackages it commercially.⁶⁴ AACER must itself pay the Judiciary to download data that are already in the PACER system, but the facts of the bankruptcy system evidently make it possible for market forces to sustain AACER’s more rational approach to data distribution. There is little prospect for such a market solution to PACER’s paywall for nonbankruptcy cases (and would-be users must still pay AACER).

Other scholarly work has been published using the fruits of PACER fee waivers. Examples include work by Professor David Engstrom,⁶⁵ Professor Gillian Hadfield,⁶⁶ and Professor Alexander Reinert.⁶⁷ But PACER fee waivers are at most a partial solution to the PACER paywall, and a highly imperfect one at that. Even though there is now a centralized process for requesting a fee waiver,⁶⁸ the decision over whether to grant requested waivers is up to individual district courts.⁶⁹ This can extend the time and uncertainty related to project feasibility. It results from the fact that the CM/ECF system underlying PACER is really an amalgamation of individual systems, each run by an individual court as it sees fit. It is hard to think of good reasons why one court would deny fee waivers for the same scholarly research another approves, yet that happens.⁷⁰ Further, my conversations with other scholars indicate the possibility of inconsistent treatment. For example, although Professor Lawless and his coauthors received fee waivers from all but the Southern District of Texas,⁷¹ one scholar at a workshop on federal court data access that I organized in 2015⁷² reported being unable to receive fee waivers from more than thirty-five of the ninety-four district courts.⁷³ Study of the Supreme Court’s alteration of the pleading standard in Twombly and Iqbal also shows PACER’s potential, and the limitations of its alternatives.⁷⁴

Another anecdote involves a July 2020 request by Plymouth State University Professors Chantalle Forgues and Daniel Lee for PACER fee waivers so they could download docket reports, complaints, and petitions for all New England district courts.⁷⁵ The scholars planned to use computer programs to target a subset of cases as classified using Nature of Suit codes.⁷⁶ All district courts petitioned granted the request except for the District of Massachusetts.⁷⁷ Events over several months culminated in a court order denying the professors’ request,⁷⁸ because of the Chief Judge’s belief that the researchers planned to use software that, “according to our IT security personnel, would present an unacceptable degree of risk to our system.”⁷⁹ Professors Forgues and Lee explained that “the script that petitioners propose is used everyday, including on weekends, to download files from PACER. The script at issue is used by big law firms and larger universities on a regular basis, and has been used on the Court’s system consistently for over seven (7) years.”⁸⁰ A motion seeking reconsideration remains pending.⁸¹

The upshot of all of this is two-fold. First, bulk downloads are available, without limit, to users who can afford to pay PACER fees, even as federal courts create situations in which researchers who cannot afford PACER fees are out of luck. More generally, the PACER system, and judicial policies and practices around it, stymie what would otherwise be PACER’s staggering research potential. PACER contains over a billion documents,⁸² and that number grows daily. Many other federal agencies have made their data available, with privacy restrictions, allowing the public to benefit from enormous amounts of professional-quality research at no cost to the federal government.⁸³ The many examples of research noted above – from enterprising bankruptcy scholarship to illuminating work on summary judgment and pleading to work on trials – hint at the possibilities presented by piercing PACER’s paywall. This research, much of it using inferior or limited sources of data, is a drop in the bucket compared to what we could have with free PACER.

14.2.3 What about Privacy?

A natural basis for concern about free PACER involves privacy of both litigants and witnesses. This essay is not the place to fully develop either the bases for such concerns, which are real, or the responses to them. But engaging these topics can allay certain aspects of these concerns while informing further inquiry as to others.

A first concern involves risks in the criminal justice system. Stephen Schultze chronicles one example:

In 2006, the Department of Justice (DOJ) observed a disturbing new phenomenon: anonymous individuals had begun to use PACER to cull and republish witness information. DOJ stated in a letter to the Judicial Conference, “we are witnessing the rise of a new cottage industry engaged in republishing court filings about cooperators on websites such as www.whosarat.com for the clear purpose of witness intimidation, retaliation, and harassment.” DOJ urged the Judiciary to suppress public access to all plea agreements.⁸⁴

What DOJ wanted to preserve is what is sometimes called “practical obscurity” – “print-era barriers to access to sensitive criminal information.”⁸⁵ After describing how federal judges engaged these questions, including at the level of the Judicial Conference’s Court Administration and Case Management Committee, Schultze observes that “by the time whosarat.com appeared in newspaper headlines, the Judicial Conference had codified its approach to sensitive information in electronic court records,” with the result that the Justice Department “could protect sensitive plea agreements by sealing or redacting the documents, but it could not rely on the Judiciary to recreate anachronistic approximations of practical obscurity.”⁸⁶

If this balance between privacy concerns and public access is good enough for those who might be at risk of physical violence, it is difficult to see why it should not also be good enough for others in the justice system. This is especially true in the civil context, where concerns related to personal embarrassment or trade secret interests can be handled on a case-by-case basis. Just as the Freedom of Information Act contains exceptions for trade secrets and certain other delicate information,⁸⁷ so, too, may federal courts allow litigants to file documents under seal in appropriate circumstances.

The privacy case for limiting access to PACER records en masse seems especially weak given that the very same records are already available electronically. To the extent that practical obscurity exists now, it is thus limited only by PACER fees. The paywall does not protect anyone from PACER’s large commercial buyers, including, for example, bankruptcy-related services such as AACER. But these fees are high enough to keep out scholars, most of whom have little to no interest in individual information – the point of much scholarly research being to generalize rather than investigate details of unknown persons. It also includes journalists, which places PACER in tension with the First Amendment values that animate the constitutional basis for open court records that the Supreme Court found in Richmond Newspapers, Inc. v. Virginia.⁸⁸

14.3 A Vision for the Future

I‘m hardly the first one to advocate dismantling PACER’s paywall. More than a decade ago, Professor LoPucki argued that eliminating the paywall “would open a real-time window on court-system operation superior to any previously possible,” with the happy result that “[n]ot only researchers, but parties, lawyers, the government, and the public could see every important aspect of how the courts operate.”⁸⁹ The Free Law Project has long argued that there is a “PACER problem.”⁹⁰ Other scholars are on the case, in both academic journals and popular media.⁹¹ Litigation that could reduce the magnitude of PACER fees – shorten the paywall – has had some success and is ongoing.⁹² And as I discuss momentarily, there is a real chance that Congress will finally act to free PACER.

But what should a freed PACER regime look like? My purpose in this Section is to provide some suggestions. If anything like the Open Courts Act makes it into law, then some of these suggestions will have been either taken or eclipsed. Others would still be up for consideration if, as seems likely, Congress left the AO with substantial discretion in choosing how to make court records publicly available. Some of the possibilities I raise are concrete. The last section, though, is a meditation on the need for the Judiciary to adopt a new institutional attitude favoring openness. That could go a long way.

14.3.1 Free PACER and Replace Its Fees with General Revenue

Encouragingly, in December 2021 the Open Courts Act of 2021 was reported out of the Senate Judiciary Committee with bipartisan support.⁹³ This bill would require all court documents to be presumptively freely accessible to the public within three years after enactment.⁹⁴ The bill also calls for modernization of PACER’s feature set,⁹⁵ and a similar bill is pending in the US House as of this writing.⁹⁶ Although the costs of modernization might be offset partly by congressional appropriations,⁹⁷ future funding would need to come from fees paid by federal agencies for their PACER use, based on inflation-adjusted fees agencies paid in 2018,⁹⁸ and possibly also increases in filing fees on parties.⁹⁹

I have doubts about this approach to funding. First, I have not been able to find a breakdown of PACER fees by source, but logic indicates that fees paid by federal agencies are only part of the revenue from PACER. Thus, without increasing filing fees, the Judiciary would absorb at least some revenue loss. My goal is to free PACER, not cut courts. Second, if the Judiciary exercised the authority the Open Courts Act provides to raise filing fees, that burden would fall on litigants. My goal is to free PACER, not raise additional financial barriers to courts.

The better course is for Congress to simply (1) eliminate the Judiciary’s discretion to charge PACER fees and (2) increase general revenue funds for the Judiciary. Congress could hold the Judiciary harmless, replacing all PACER paywall revenues, for just $150 million.¹⁰⁰

Although $150 million would be a lot of money to have for yourself, it’s a teeny-tiny little droplet in congressional terms. By way of comparison, Americans spend roughly twenty-five times as much on wild bird food as PACER fees brought in.¹⁰¹ And overall congressional funding to the Judiciary in 2020 was fifty times what the PACER paywall brought in.¹⁰²

A situation in which we have hamstrung public inquiry into the functioning of the federal litigation system over a literal rounding error may not be funny, but it really is silly. Let’s stop it already. Congress should just cut an annual check to the Judiciary, allowing the public to better understand – and better inform Congress – how one of the three branches of our government functions.¹⁰³ It wouldn’t even make a dent in the monthly auction of Treasury debt, and it would allow the Judiciary to free PACER and lose nothing.¹⁰⁴

14.3.2 Infrastructure Improvements

As I discuss above, one of the real problems with PACER is that it provides very limited search capacity. The House bill introduced in the last Congress had modernization features directed at this problem,¹⁰⁵ and the Senate bill spoke generally of improvements in “data accessibility … and performance.”¹⁰⁶ Not knowing anything about PACER’s back-end design and capabilities, I don’t know whether its lack of a docket-entry or document text search function is a bug or is deliberate. But there are surely many nonprofit organizations that would and could host the data. As noted, the Internet Archive has already offered to do so for free, and to do so with considerably better functionality than PACER currently has.

There are other infrastructure improvements that the Judiciary could make. Some require more burden than others. Less demanding would be more refined docketing, for example, allowing docket coding for motions to dismiss according to the basis for the motion. On the high end would be redesigned forms such as the civil cover sheet, requiring more and/or better information from litigants.

A group of professors recently discussed many of the pathologies I have already described, in Science magazine.¹⁰⁷ They advocate a “collaborative research agenda to empower the public to access and analyze court records.”¹⁰⁸ This agenda includes better organization of the data because of the fact that court records tend to include unstructured text. The professors advocate developing

applications that not only support scholars and researchers who may want to analyze the data but also enable members of the judiciary, entrepreneurs, journalists, potential litigants, and concerned citizens to learn more about the functioning of the courts. To support inquiries made by the public, we should develop applications that can process natural language queries such as “What are the most recent data privacy cases?” or “How often do police officers invoke qualified immunity?”¹⁰⁹

So there are surely a range of other infrastructure improvements worth considering. My purpose here is only to suggest that more about PACER than just the paywall can be improved.

14.3.3 The Judiciary Should Adopt an Attitude of Openness

Certainly there are many judges who embrace openness. Judges have approved many, perhaps even most, PACER fee waivers that scholars have actually submitted to the courts. An amicus brief by retired judges in the ongoing PACER class-action litigation indicates laudable support for public access to court data.¹¹⁰ Over the years, I have had numerous conversations with professionals at the FJC and AO that indicate a range of support between partial and complete for providing more data access.

At the same time, it is hard not to read a message of opposition from some of the Judiciary’s choices, including setting PACER fees at an unnecessarily high level, opposition to legislation that would increase PACER access (including by circulating boilerplate “talking points” for federal judges to use in lobbying members of Congress),¹¹¹ and anecdotal evidence that some judges may reject openness as a categorical matter.¹¹²

My own experience is at least partially consistent with that picture, and it suggests that this rejection exists not only among particular judges, but also as an institutional fact. In 2015 I attended two workshops about access to federal court data. The first was one I organized at Penn Law, which was funded by the National Science Foundation,¹¹³ and the second was the Federal Courts Civil Data Project Roundtable, which was held in Washington, DC, and hosted by the ABA Standing Committee on the American Judicial System and the Duke Law Center for Judicial Studies. Participants at one or both of these workshops included federal judges; academics who work with federal data; staff researchers and/or attorneys from federal agencies including the Federal Judicial Center, the AO, the Administrative Conference of the United States; a representative from the National Center for State Courts; and Michael Lissner, executive director of the Free Law Project. I have limited space to describe my impressions of the views of the Judiciary’s representatives at these events. There were a variety of views, but it seems clear that AO representatives harbor skepticism about the economic, bureaucratic, and technological feasibility of opening up PACER (though the multi-court fee waiver exemption process that now exists occurred not too long after these events).¹¹⁴

In addition, at the ABA event, one Article III judge spoke forcefully against additional PACER access. This judge, whom I know from prior communications to be a devoted public servant, pointed to the example of a news story about a colleague that the judge thought was both unfair and professionally embarrassing. The judge at the conference expressed sincere and profound concern that the judiciary would be unduly pressured in this and other ways by the public availability of additional data that could be searched and filtered easily.

There is some evidence that the Civil Justice Reform Act distorts how judges approach their dockets, perhaps to avoid embarrassment.¹¹⁵ I’m an economist who believes that people generally respond to incentives, so I’m not surprised. Even so, Article III judges have substantial constitutional protections precisely to allow them to buck public pressure and embarrassment in the service of judicial independence. Most obviously, they have life tenure, and their salaries cannot be reduced.¹¹⁶ Their jobs carry considerable prestige, and no small amount of power; Chief Justice Stone once remarked, “the only protection against unwise decisions, and even judicial usurpation, is careful scrutiny of their action and fearless comment upon it.”¹¹⁷ I oppose unfair treatment of judges as much as anyone else, but it should go without saying that unfair treatment is not the same as informed public scrutiny. Citizens should have confidence that the federal judiciary will not be deterred from doing its job with integrity by the availability of such scrutiny, and for this reason the Judiciary should welcome rather than avoid it.¹¹⁸

Judges play an important and powerful role in our democracy, and citizens have a right to the fruits of studying how the courts do their, and thus our, business. Any defensible vision for the future of public access to court records should include a decision by the Judiciary to switch gears and foster an institutional attitude of openness toward the public.

14.4 Conclusion

There’s a lot of federal court data. It sits on servers in district courts around the country, with a single access point through PACER. It’s often inconvenient or impossible to use it to study the federal litigation system. Even where PACER use is technically feasible, it’s often prohibitively costly in financial terms to acquire the magnitude of data that would allow scholars to do research that would then be available to the courts and the public for free. PACER has severely limited search functionality, and individual courts decide whether even that will be open to researchers without their paying ruinous fees to download documents that are owned by the public. All the while, the Internet Archive and who knows how many other nonprofits would gladly crawl, index, and host PACER’s data for free public download.

This situation is an intolerable disgrace. It is unacceptable for a democracy to have but withhold such valuable information about the functioning of its courts. With only an imperceptible impact on the public fisc, Congress could both mandate free PACER and preserve the Judiciary’s budget.

There do not seem to be large hurdles related to privacy or pressure on judges, because the data in question are already freely available for public download – just at a deliberately high price that renders useful research infeasible. Nor is there any real technical feasibility problem, as the offer by the Internet Archive to host the data for free forever shows. This is a problem of political and attitudinal will. Congress and the Judiciary should solve it, so we can use court data to increase access to better justice.

15 Technological Challenges Facing the Judiciary

Albert H. Yoon

James Branch Cabell wrote, “The optimist proclaims that we live in the best of all possible worlds, and the pessimist fears this is true.” This witticism also applies to our present-day legal profession. Technological advances, having previously transformed labor markets that relied on physical labor, have in recent years influenced labor markets that rely on human thinking and judgment. During the 1980s, the legal profession underwent a dramatic shift from physical to electronic texts. Notwithstanding this change, the profession remained largely static for the next quarter century, with lawyers still required to process the texts, a time-consuming and costly process.

The advent of legal technology over the past decade has transformed the legal market. Companies – including many start-ups – have developed tools to help lawyers process information faster, assisting them in contract formation, discovery, and compliance, to name but a few.¹ More recent technologies use artificial intelligence (AI) to enable litigants to better understand their own case in light of existing legal precedent, predicting how a court would decide their case.

The optimist observes these developments and sees a world of potential. As in the past, new technologies will benefit consumers. The cost of legal services will decline, solving the paradoxical and seemingly intractable problem of a surfeit of both lawyers and litigants. The business model of the legal profession, long reliant on margin over volume, could reverse itself as legal technology enables lawyers to help more clients in less time.

The pessimist, observing the same developments, fears worsening inequalities that already hinder the legal profession. Thus far, legal technology has primarily catered to wealthier litigants, beyond what most lawyers – for example, solo or small-firm practitioners – can afford. These technologies will, as Marc Galanter prophesized nearly half a century ago, allow the “haves” to yet again come out ahead of the “have-nots.”²

This chapter identifies with both the optimist and the pessimist. The path toward a more efficient, democratized legal profession is within our reach. More to the point, the courts can help with this transition. Legal technology is expensive in part because judicial data is a barrier to entry. Legal technology centers around access to court documents – decisions and dockets – that are the foundation for data analytics. These documents are public, but often inaccessible and costly to access.³ Courts could eliminate these barriers by making these documents freely and directly available to the public. While this data may subject them to heightened scrutiny, courts should embrace this trade-off. This scrutiny already occurs, albeit from the smaller subset of the lawyers who represent well-resourced clients. Second, this release of data will have a progressive effect on access to legal analytics, benefitting a wider range of litigants.

This chapter proceeds as follows. Section 15.1 discusses the evolution of legal practice, which until recently has remained unchanged for much of its history. The predicament facing the courts is set forth in Section 15.2: some financial, but much of it institutional and cultural. Section 15.3 describes a path forward, partially initiated by the COVID-19 pandemic, but also the change in the composition of judges over time. Section 15.4 concludes.

15.1 The Evolution of Legal Practice

Historically, the legal profession has undergone little change. Clients retain lawyers to solve their problems. Lawyers gather information from their clients and situate the client’s case within the current law. Until recently, lawyers’ task in effectively representing their clients has been a labor-intensive undertaking.⁴ Lawyers are trouble-shooters, tasked with mapping their clients’ problems into one or more legal issues. To do so, they must first determine the relevant legal authority, and then how this authority relates to the clients’ situation.

15.1.1 The Stubbornness of High Legal Fees

The effort required to represent clients explains in large part why legal fees are expensive. Understanding their clients’ circumstances requires effort and time; so too does understanding the relevant legal authority. And the economies of scale are limited, as each client is unique. As lawyers gain experience, they are able to work more efficiently, accomplishing more over the same unit of time than less-experienced lawyers. To reflect this experience, however, experienced lawyers often are more expensive.⁵

In The Cost Disease, William Baumol and William Bowen explain how certain jobs rise in compensation, even in the absence of an increase in labor productivity.⁶ The authors characterize two types of jobs: (1) workers who become more productive over time, in large part due to technology and innovation and (2) workers largely immune to these advances. Auto workers provide an example of the former: because technology allows each worker to contribute more efficiently to the production of a car, her hourly wages have plateaued, even declined, as machines have come to perform more of the assembly process. For the latter, a string quartet: musicians devote the same amount of time to perform Beethoven today as did their predecessors. Compensation for workers in these technology-resistant fields rises if demand for these services remains constant and employers have to compete for these workers.

Until recently, lawyers were more akin to classical musicians than auto workers. The effort required to represent clients over time has remained stubbornly consistent. While digital libraries have enabled lawyers to identify relevant legal materials more efficiently, lawyers still have to process them.⁷ Lawyers historically used leverage to contain legal fees: partners delegating the straightforward and labor-intensive work to less-expensive associates.⁸ But the rate-limiting factor remains human effort, which – when unaffected by gains in productivity – becomes increasingly expensive.

The legal profession licensure requirements also create barriers to entry. The practice of law requires attending law school and passing the bar, reducing the supply of lawyers.⁹ Over time, non-lawyers have engaged in certain legal work, under the authority of lawyers, and in some instances entirely on their own.¹⁰ But even as non-lawyers reduce legal fees, human labor remains the rate-limiting factor.

15.1.2 Three Stages of Lawyering

For all these reasons, the legal profession has enjoyed an extended period operating as a well-paying guild.¹¹ Until recently, the fundamental nature of legal practice remained unchanged, even as the profession adopted new technologies. During that time, the profession has evolved across three stages: from analog to digital to computational.¹² It is only the third and latest transition that seems poised to force the legal profession to fundamentally change its approach, both conceptual and financial.¹³ I briefly summarize these stages.

Analog. This stage represents the era where lawyers relied on hard copies of legal references – for example, case reporters, treatises, regulations, statutes, restatements – in their course of their legal work. This era’s defining feature is that the lawyers themselves were the sole mechanism to process these references and to apply them to their clients’ situation. Humans were the rate-limiting factor in the speed and efficacy of legal representation. Although varying in ability, lawyers performed tasks primarily as a linear function of the hours worked. Accordingly, larger law firms were able to take on more complex work because they could deploy larger numbers of lawyers onto a legal problem.

Digital. This stage coincided with the emergence of two technological developments during the 1980s: the personal computer and the creation of electronic legal libraries. The personal computer enabled greater efficiency. Lawyers who previously used typewriters found that an electronic format allowed them to write, edit, and share documents more easily. The advent of electronic legal libraries – notably Westlaw and LexisNexis – enabled lawyers to conduct their legal research through a dedicated terminal and, soon thereafter, on their personal computer. At a minimum, lawyers could access any decision, statute, or regulation by looking up its citation. Even more helpfully, lawyers could use keywords to identify the universe of relevant materials. The advent of electronic legal libraries untethered lawyers from physical reference materials, while simultaneously allowing them to digitally memorialize their work. These advances improved the rate of production, but, as with the analog stage, lawyer productivity remained a function of hours worked.

Computational. This latest stage coincided with the emergence of AI and its application to law. Early developments during this stage are really extensions of the analog era, made possible because of the increased digitization of legal records. Discovery is one such example. During this pre-trial process, lawyers ask opposing counsel for materials – usually in the form of documents – that they believe could be legally relevant. The process historically required lawyers to read each document. Digitization expedites the process by allowing lawyers to electronically search for relevant words or phrases. A plethora of e-discovery start-ups exists to assist in this process.¹⁴ What once required a small army of lawyers now requires a single lawyer and a few lines of code.

As its name suggests, the computational stage refers to the ability of machines (computers) to go beyond identifying relevant legal references, drawing upon these references, in conjunction with specific information about a client case, to generate legal analysis. It uses two methods of AI: machine learning (ML) and natural language processing (NLP), described briefly below.

ML statistically analyzes structured data to predict outcomes.¹⁵ This process draws upon regression analysis, commonly used in both STEM and social sciences, as a means to test hypotheses by looking at data generated by observable phenomena. The difference lies in the objective: Rather than use regressions to explain outcomes, ML seeks to predict them. ML in legal technology involves digitizing legal information. Information from legal references – notably cases – can be expressed as pieces of data. These variables can be binomial in nature (is something X or Y); multinomial (is something A, B, C, D, etc.); or continuous (e.g., earnings, damages).

To illustrate: Whether a worker is an employee or independent contractor¹⁶ frequently arises in tax and employment law, determining the rights and obligations of the worker. In most jurisdictions, courts weigh several factors – for example, the amount of training, who sets the hours, the right to delegate work – in making this determination. It is possible to code published decisions into a dataset that captures how courts weigh these factors in reaching their decisions. ML can predict, given a specific set of facts, how a court would decide this matter, given existing precedent. Transforming written text to bytes of data can be labor-intensive, particularly when done manually. Once transformed, however, scaling this data is trivially small: A million people can draw upon this data just as easily as a single person.

NLP, which sits at the intersection of computer science and linguistics, is designed to understand and interpret human language, whether written or spoken. Whereas ML analyzes structured data – that is, that which could be captured in a spreadsheet – NLP is capable of working with unstructured data,¹⁷ such as the text of a memo or an opinion. Both NLP and ML can identify patterns or relationships in data faster and more comprehensively than humans. The relationship can involve individual words, phrases, or even the proximity of words in relation to other words. NLP allows for interpretation beyond words qua words, extending to words in relation to other words, even when those words are not juxtaposed.

NLP already has applications in both the consumer and legal context. For example, firms can use NLP to more effectively evaluate customer feedback across multiple formats – for example, email, surveys, conversations from call centers – to identify recurring or common problems, even as customers express themselves in a myriad of ways.¹⁸ Within law, firms can use NLP to evaluate and identify relevant provisions of a contract and their meaning.¹⁹ This technology is helpful, given the human time, expense, and error that often results when done manually. Even with the promise of comprehending unstructured data, NLP performs better the more structured the text.

15.1.3 Legal Ethics

As the number of applications grows, the rate-limiting principle will not be technological, but ethical. It is helpful to distinguish between the ethical and the legal ethical considerations of AI. The two overlap but remain distinct. The general ethical debate speaks to the ethical dilemmas that technology poses, which can inform whether the technology should even be built, and if built, how to do it in accordance with ethical considerations. Legal ethics is arguably more restrictive, looking to how the profession uses technology in accordance with its own fiduciary and ethical obligations.

Legal technology raises several ethical concerns.²⁰ One is privacy, the blurring of boundaries of attorney-client privilege. Many of these technologies rest in the cloud, and therefore potentially allow third parties illegal access to privileged information. The second is unlawful discrimination when algorithmic tools are used in criminal²¹ and employment contexts. The third is the unauthorized practice of law, particularly as technologies advance and can perform tasks that draw closer to legal advice.²² Finally, the evolution of legal technology raises issues of access to justice, because much of the technology is geared toward the elite private bar, not lawyers who serve regular people.²³ Most practicing lawyers lack these resources.²⁴

15.2 Challenges for the Courts

Courts face numerous obstacles with respect to technological change. Two obstacles are largely beyond the judiciary’s control: caseloads and funding. The remaining three are within the judiciary’s scope: judge Luddism, cybersecurity risks, and judicial aversion to public scrutiny. I discuss each in turn.

15.2.1 Caseloads

In a sense, courts perform a reactive role. They adjudicate disputes coming before them, but – save select courts, such as the US Supreme Court – do not control their docket. Nevertheless, the demand on courts continues to grow. Among US district courts, the caseload has grown over the years by 165 percent (from 127,280 cases in 1970 to 337,537 cases in 2017). The caseload demands are arguably even greater than the numbers suggest: Much of this growth involved civil disputes between private parties, which on average take more time to resolve than criminal cases (10.8 months versus 7.0 months in September 2019 in US District Courts, and there has been a relatively consistent discrepancy of increased length for civil cases since at least 2000).²⁵

The growth in caseload necessarily increases demands on the judiciary. The first is on the judges themselves. Among US district courts, the number of authorized judgeships from 1970 to 2017 grew by only 68 percent (from 394 judges to 663 judges).²⁶ Accordingly, judges are tasked with greater responsibilities, particularly after Iqbal²⁷ and Twombly,²⁸ whose heightened pleading requirements require judges to adjudicate matters earlier in the litigation process. Another is the litigants. Longer periods for a trial date, combined with costs of litigation, increase the likelihood that parties resolve their disputes through trial diminishes. It is a well-known and often repeated maxim that the vast majority of cases resolve prior to trial.²⁹

15.2.2 Funding

As a branch of government, the judiciary is subject to the financial constraints that all public institutions face. In nominal dollars, judiciaries are often spared the budgetary cuts that befall other public services.³⁰ For example, Article III of the US Constitution authorizes Congress to create federal judgeships, but not diminish their salaries. Congress wields the authority to reduce the number of federal judges; it has yet to do so.

In most jurisdictions, salaries constitute the single largest expense of the judiciary budget. Drawing again from the US federal example, salaries (and benefits) from judges, clerks, and staff totalled 64 percent of the FY 2021 total budget request.³¹ Given a total appropriation of nearly $6 billion for this year, the federal commitment to salaries is sizable.

At the same time, it is important to consider judicial salaries in a broader context. US district court judges, the most common form of Article III judge, in 2020 earn $216,400,³² much higher than the US median income,³³ but much less than the elite private bar.³⁴ These pay differentials may deter high-skilled lawyers from joining the bench or, in some instances, encourage departure.³⁵ Moreover, this gap has only grown over time as judicial salaries lag behind the rate of inflation.³⁶ Concurrently, legislatures resist expanding the number of judgeships, resulting in the number of judgeships failing to keep pace with caseload increases.

15.2.3 Judicial Reluctance toward Technology

It is an understatement to say that courts and technology are not the best of friends. Judges, on average, constitute the higher end of the age distribution. Judges typically have distinguished themselves earlier in their career, whether it be in private practice, government, or academia. This process takes time, so it is perhaps no surprise that judges are often in their fifties by the time they join the bench. For example, the average age of an Article III judge at the time of their commission is fifty-one years old. However, given lifetime tenure, most judges remain on the bench well past the traditional retirement age of sixty-five. In 2020, the average age of a district judge was sixty-seven years old and sixty-nine years old for a court of appeals judge.³⁷

Judges, like many older adults,³⁸ may be reluctant to adopt new technologies. Surveys show that among the general population, over 60 percent of Americans aged fifty to sixty-four and nearly 75 percent of those above age sixty-five identify as needing someone else to set up their new electronic devices.³⁹ Other academic studies show that older adults often possess a phobia about technology, particularly when it involves a technology that they did not develop familiarity with during their younger years.⁴⁰

Judges likely share these experiences of the general public. Many judges are years removed from the type of legal work now reliant upon this technology. Either the nature of their work at the time did not demand it or they had junior lawyers who could perform these tasks. On the bench, technology has changed little since the 1980s inception of digital libraries such as Westlaw or LexisNexis. Newer technologies, however, differ categorically from traditional digital search technology by going beyond search to providing answers.

These technologies are in potential tension with an idealization of courts, and judges in particular. Judges, by institutional design, are deemed impartial arbiters of disputes. They are equal to the legislative and executive branches, and are protected against removal without cause. Judges confer with their clerks, or even other judges, but ultimately issue their own decisions, free from any outside influence. Newer technologies challenge the independent role of judges, allowing actors outside the judiciary to influence how judges make decisions. Under this view, judges may cede autonomy. It is one thing for judges to draw upon resources – from clerks, judges, the common law – to inform their view. It is something altogether different for judges to draw upon resources advising them what to do.

15.2.4 Security of Judicial Documents

Another concern for the courts is security of judicial documents. Increasingly, legal technology utilizes cloud computing, hosting applications on remote servers on the web rather than personal computers or local servers. These servers are vulnerable to access by unauthorized third parties. This risk is particularly concerning for the legal profession, where lawyers are ethically bound to maintain strict client confidentiality.⁴¹ These ethics also apply to judges, who similarly must take affirmative steps to avoid disclosing case information outside of the judiciary.⁴²

Client security is a long-standing issue for the legal profession, at the same time co-existent with technology. Before email, lawyers corresponded with clients through traditional mail or fax. Both had the virtue of being secure, given their person-to-person formats. Email offered the lawyers the promise of instantaneous communication with their clients, while they risked the perils of breaches of client confidentiality, which in many instances could neither be easily remedied nor even traced. The legal profession initially responded by discouraging its use for client communication, but eventually recognized its advantages for facilitating communication between lawyers and clients.⁴³ Today, lawyers widely use email, cognizant of the inherent risks of electronic communication.⁴⁴ Contracts are now routinely finalized on the cloud.⁴⁵

15.2.5 Judicial Aversion to Public Scrutiny

Judges, in contrast to legislators, are public servants who operate predominantly outside of public view.⁴⁶ Whereas trials and certain pre-trial proceedings are open to the public, much of what judges do occurs in private. In contrast to a legislature, judges do not reveal their deliberations, only their final decision in the form of a motion or an opinion. Adding to this opacity, while the disposition judges’ decisions are of public record, they are not necessarily readily available. Moreover, judges elect whether to categorize their opinions as published or unpublished, which further limits their availability.⁴⁷

The public’s ability to access federal judicial opinions has historically required lawyers to have access to a physical volume of the federal reporter, or, alternatively, a subscription to an online legal database. In recent years, new media platforms (e.g., Justia) have made federal decisions and statutes publicly available and free of charge. Other public judicial documents, however, remain less accessible.⁴⁸

In addition, courts have been loath to link the identities of their judges to their decisions. For example, the Federal Judicial Center makes publicly available data on individual civil and criminal defendant filings and terminations filed in the federal district and appellate courts.⁴⁹ This data, however, omits the judges’ identities for each record.⁵⁰ This omission prevents those analyzing the data to observe how differences in case outcomes vary by individual judge.⁵¹

From an academic and policy perspective, knowing which judges decide cases is essential for rigorous statistical analyses. For example, in a recent influential study on the bail decisions, researchers found that state court judges make two types of errors: (1) release defendants on bail who subsequently commit a crime while released and (2) deny bail to defendants who, based on observable characteristics, would have complied with bail conditions if released.⁵² Given the random assignment of cases, researchers could identify when different judges were treating similarly situated defendants differently, and more broadly, the extent to which judges vary from one another.

Other countries worry about public scrutiny of judges. France has outlawed the publication of judge analytics, a crime punishable by up to five years of prison.⁵³ In accordance with the Act, any publication that is likely to undermine the security of any parties to a judicial decision, including judges, must be obscured.⁵⁴ In my own research, I sought to evaluate bail decisions of courts within a Canadian province. The province’s judicial council – comprising judges – was willing to release individual-level data on bail proceedings but declined to reveal the identity of judges for each case, on the grounds that it could draw undue criticism of the provincial judges.

Arguments in support of judge anonymity range from distortion of the litigation process to the safety of judges. In 2019, the Constitutional Court of France upheld the constitutionality of the prohibition on revealing judges’ identities, determining that court analytics based on individual judges could promote forum shopping, to the detriment of the administration of justice.⁵⁵ In 2020, an individual injured and killed family members of a US district judge, after having appeared before the judge in prior litigation and criticizing her handling of the case on social media.⁵⁶ This followed earlier instances – in 2005,⁵⁷ 1989,⁵⁸ 1988,⁵⁹ and 1979⁶⁰ – where federal judges were killed, all connected to litigants appearing before these judges. The Secretary of the Judicial Conference of the United States testified that identified threats and inappropriate communications directed at federal judges grew from under 1,000 in 2015 to nearly 4,500 in 2019.⁶¹

15.2.6 Litigation as Arms Race

Litigation has long favored those with resources. Litigants with the most resources tend to attract the highest-ability lawyers. At the same time, even sophisticated litigants prefer to trim unnecessary legal expenses. Many legal technology tools allow lawyers to perform legal tasks at a fraction of the time and cost that it took even a decade or two earlier. Thus, there is a segment of the practicing bar that embraces legal technology to improve both the efficiency and quality of their practice. On the other end of the spectrum, many lawyers represent litigants of more modest means. Given the nature of their practice and their own limited resources, these lawyers are slower to incorporate these technologies.

Judges are not exempt from this competition. This asymmetry of legal sophistication threatens our civil and criminal justice system. This imbalance might be more palatable if litigation involved only parties of comparable means: that is, affluent litigants filing suit against affluent litigants; poor litigants filing suit against poor litigants. While this phenomenon occurs in certain situations – for example, intellectual property and corporate disputes typically involve well-financed litigants⁶² – many instances reflect a more David-versus-Goliath disparity, such as personal injury, immigration, or civil rights. This imbalance may distort legal outcomes at trial, if lawyers financed by well-resourced litigants can generate outcomes that maximize chances of prevailing or, if that is not possible, ameliorating the terms of an unfavorable outcome. These decisions, in turn, affect both the manner and terms under which subsequent litigants resolve their disputes, irrespective of whether they proceed to trial.⁶³

The courts’ lack of technological sophistication also poses risks to our civil and criminal justice system. In terms of resources, courts fall somewhere between well-financed and modestly financed litigants. They certainly possess more resources – technological as well as labor – than the typical lawyer, but are increasingly falling behind the elite bar, typically those practicing at the largest law firms. Legal technology is transforming how the elite bar practices. In contrast, legal technology has had a modest effect on the judiciary. While existing developments are welcome – for example, judges increasingly allow litigants to electronically file their court documents – the courts’ earlier adaptation to technology suggests a slow transition.

15.3 A Path Forward

Legal technology presents both opportunities and challenges for the legal profession. It transforms the way that lawyers practice. For much of the profession’s existence, all legal tasks, however mundane, were the exclusive domain of lawyers. Even as paralegals have taken on tasks, legal work remained the domain of humans. New technologies enable machines to assist lawyers in increasingly more of their legal tasks. As a result, for example, discovery can take a few hours of one lawyer’s time rather than weeks requiring several lawyers. And finding answers to discrete legal questions can now take a few minutes rather than a few hours.

As with many new technologies, adoption of legal technology lags behind availability. Adoption among lawyers depends in large part on cost and value. Technologies that enable lawyers to improve both the quality and rate at which they work encourage broader adoption. Society is replete with examples of wide-scale technological adoption, both with respect to hardware (e.g., personal computers, smartphones) and software (Facebook; Twitter). Even when the costs of adoption are considerable, the costs of eschewing these technologies are arguably higher.⁶⁴

While the legal profession is still in its early stages of adoption of legal technology, the profession recognizes the need for lawyers to develop and maintain a level of technical competence. The American Bar Association now includes technological competence in its model rules.⁶⁵ Courts have been slower than the profession to adopt technological change, for the aforementioned reasons. The path forward involves two parts, which I discuss in turn.

15.3.1 Greater Technological Competence in the Judiciary

At first blush, this competence may appear unnecessary. Very few cases result in a trial. Among civil cases filed in district court in the period 1970 through 2019, fewer than 3 percent of filed civil cases ended in a verdict.⁶⁶ Given this relatively low percentage, one can argue that judges have ample time to properly adjudicate the cases before them, without additional technology. Looking only at trial outcomes, however, provides an incomplete picture of the integral role that judges have in helping parties resolve their disputes. Based on civil filings in federal district court, parties resolved disputes on their own in roughly half the cases, without court adjudication. But in nearly a quarter of all cases, judges adjudicated the dispute, prior to trial, on procedural – for example, lack of jurisdiction – or substantive – for example, summary judgment, affirming findings of an arbitrator or magistrate judge – grounds.⁶⁷ This data provides evidence that, at a minimum, judges play an active role in helping many parties resolve their disputes. Even when parties do not proceed to trial, they take guidance from judges during the pre-trial phase.

The judiciary’s response to the COVID-19 pandemic provides optimism that judges may be more open to technology than originally perceived. In the United States, every state has initiated online hearings in response to the pandemic.⁶⁸ In addition, a majority of states allow litigants to electronically file court documents.⁶⁹ In many respects, these transformations reflect the judiciary’s adopting well-established processes within the practicing bar.

These changes also provide second-order changes that are just as beneficial, namely democratizing the litigation process by increasing its access. Online hearings and electronic filing allow litigants to advance their claims without having to miss time off from work or find childcare. Even as the pandemic brought about curfews and shelter-in-place ordinances, courts reported a dramatic drop in failure-to-appear rates for civil and criminal proceedings.⁷⁰

It will be interesting to see how courts respond when governments – at all levels – lift restrictions as COVID-19 moves from a pandemic to an endemic state. Will courts revert to their traditional practices that rely heavily on in-person interaction, or will they make a permanent transition to using technology to reduce the transaction costs of litigating? While judicial adaption of technological has its bumps,⁷¹ its benefits, on balance, outweigh the growing pains.

Judges can and should make greater use of legal technology. While some obstacles to adoption may be budgetary, judicial norms likely play a larger role. As with most matters, judges exercise considerable discretion as to how they use technology in their courtroom. As more judges become comfortable with technology, and advocate for its use, judicial norms will evolve toward using legal technology as yet another resource at their disposal. A New York Supreme Court justice in support of using technology recently commented, “Judges should have some knowledge of the technology …. If [judges] have the knowledge … it makes the case more efficient.”⁷²

15.3.2 Wide Availability of Judicial Data

Using technology to increase access to justice is an important step, but only a partial solution. As discussed throughout the chapter, the available legal technologies go far beyond access and search. The next frontier of legal technology will be tools that assist lawyers in their tasks that involve discretion and judgment. Imagine a system where lawyers can, early on, have a comprehensive understanding of the client’s legal issues, the factors most likely to determine success, and a probabilistic assessment of the litigation outcome. Of course, a good lawyer is capable of doing all of this, but it involves a fair amount of time. The emergence of legal technology allows the lawyer to achieve these objectives more efficiently, more accurately, and at less expense to their clients.

Development of this legal technology requires greater access to judicial materials. Most judicial documents – for example, opinions, orders, motions - are deemed public documents but are often difficult and costly to access. Courts can make these materials more readily available, most easily through their own webpages. Just as importantly, courts can encourage great use of these documents by the public, by making them available in electronic format (rather than as an image or pdf). Doing so saves third parties the time and expense of digitizing the documents and focuses on developing legal analytic tools.

Litigants, in turn, could use these analytic tools to make more informed choices about their legal disputes. Many litigants already do this: Sophisticated litigants – often through their lawyers – effectively evaluate the merits of their dispute, and choose the appropriate path. Less sophisticated litigants, however, often lack the resources to make this same determination. Analytic tools can help litigants more accurately evaluate their claim: for example, whether they have a legal claim, whether their legal claim is supported by statute or precedent, and even how a court might rule.

Litigants, by better understanding the legal dimensions of their disputes, could collectively change the distribution of case filings. In this reformulation, parties could resolve disputes where precedent points to a clear resolution, allowing courts to focus on disputes where either

1. (1) parties agree on the facts and for which the existing precedent fails to produce a clear result or

2. (2) parties disagree on the facts, such that they lead to divergent outcomes as a matter of law.

Academia and private sector researchers analyzing judicial data can have a symbiotic effect on one another to better understand the litigation, the judiciary, the legal profession, and the common law itself. Some research may be of interest primarily to academics or to the private sector. In many instances, however, research will generate interest among both groups, and push each other to develop tools that satisfy the rigor of their respective markets.

In addition to making judicial opinions widely accessible, courts can also release information about other judicial documents, such as motions and orders. As with opinions, orders are public in nature, but often difficult for the public to access, depending on the jurisdiction. This data is useful as a means of evaluating the courts’ performance in both civil and criminal matters, both procedurally and potentially substantively. For example, judicial research has revealed that judges are most likely to deliver a favorable ruling to plaintiffs shortly after they had eaten.⁷³ A large-scale study revealed that judges routinely committed both Type 1 and Type 2 errors with respect to pretrial bail determinations.⁷⁴ These academic studies were made possible only because courts shared their data.⁷⁵

While this chapter has focused on the promise of legal technology of judges, there are reasons to exercise caution. Judges, more than litigants, must be sensitive to constitutional considerations of technology as it relates to privacy, discrimination, and access to the courts. Not every technology will be applicable to the courts, and even those that are may prove problematic. In addition, logistical issues abound, in part because of the decentralized nature of courts, both across and within jurisdictions, which impedes data availability.⁷⁶ Lastly, judicial governance of technology is highly complex, for which scholars provide conceptual frameworks.⁷⁷ Ultimately, courts should neither delegate their authority to legal technology nor eschew it altogether. Rather, like much of the elite practicing bar today, courts can use this technology as a tool to inform their own independent decisions.⁷⁸

15.4. Conclusion

This chapter embraces the potential virtue of legal technology to improve systemic access-to-justice challenges. The idea of using data to better understand the law is itself well established. Back in 1897, Oliver Wendell Holmes famously wrote, “For the rational study of the law the black-letter man may be the man of the present, but the man of the future is the man of statistics and the master of economics.”⁷⁹ In this respect, the recent developments in legal technology are simply fulfilling Holmes’ prophesy. The judiciary can and should play an active role in this progression.

16 The Civil Justice Data Gap

Tanina Rostain and Amy O’Hara

Every year, millions of people living on the margins cycle through a civil justice system that exacts an enormous toll on their housing security and financial well-being. Eviction rates reached crisis levels before the coronavirus pandemic and are expected to continue to climb as emergency moratoria have been lifted.¹ Debt-collection cases have not received the same national attention as evictions, but may represent a quarter to a third of state civil cases according to recent statistics.² The pandemic has contributed to a rise in consumer debt, creating a looming threat of increased collection cases.³ Civil courts have become eviction and collection mills, whose apparent function is issuing judgments to help landlords and debt collectors grind down tenants and creditors.⁴ According to available evidence, the civil justice system is failing in its promise to protect the rights of poor people against powerful private interests and entrenching them further in poverty.

But “available evidence” is sparse. We know little about how courts dispense justice or how court entanglements affect peoples’ lives. Our knowledge of how civil justice institutions function, despite the system’s societal centrality, is piecemeal and relies on extrapolation from site-specific or out-of-date studies. In a system where all parties are meant to be treated equally, we don’t know whether the race, ethnic origin, or gender of litigants affects case outcomes. We can’t determine whether there are biases in the system or where they are concentrated. We don’t know what events precipitate civil justice involvement, nor the consequences that flow from it. We don’t know how unrepresented individuals fare against businesses with representation. We don’t even have an exact number of how many unrepresented litigants there are or in which types of cases they are concentrated⁵ or whether the scarce resources that are available are being allocated effectively and equitably.

As a recent report by the American Academy of Arts and Sciences emphasizes, many institutions and individuals have an interest in answering questions like these about the functioning of the justice system.⁶ State, county, and municipal courts are a central feature in the lives of low-income people and, increasingly, the middle class. Courts have an obligation to the people who pass through their doors to administer justice equitably. Litigants are entitled to know whether courts are living up to this ideal. Courts and legal services providers have particular interests in gaining operational insights about how well they serve the people before them and how effectively they allocate their resources. And the public, which funds the civil justice system, has a stake in knowing how well the civil justice system functions and whether its outcomes are just.⁷ Erosion of public trust puts the legitimacy of this system at risk.⁸

In other public spheres, large datasets are producing new and actionable knowledge about the performance of public and private institutions. Health care and health insurance datasets have been linked to show that more years of childhood Medicaid eligibility correlate with fewer adult hospitalizations for Black Americans.⁹ In Cuyahoga County, Ohio, the Center on Urban Poverty and Community Development has used data from more than thirty-five different administrative programs to create two integrated data systems for research.¹⁰ Linkages in these comprehensive data systems have shown increased shelter use by evicted families and identified downstream consequences such as childhood lead poisoning.¹¹

As these projects show, big data are particularly powerful in the study of social institutions by delivering variety, volume, and velocity. Real-time data are captured for nearly all of our everyday whereabouts, purchases, searches, and interactions with private individuals and public institutions. Everyday interactions with other people and social institutions are chronicled by the terabyte through active or passive engagement with digital technologies. Data are created and available in near-real time.¹² These features produce broader knowledge than was possible with limited size and site-specific case studies, permitting both wide generalizations and highly granular views into the functioning of institutions and their effects on people’s lives.

What explains the lack of data in civil justice? For one, not every court collects all case level information in electronic form. Some simply don’t have the resources; others don’t see value in doing so. Even where data exist, rules often restrict access. Courts are unwilling to share data, or uncomfortable doing so, for fear that vulnerable litigants will be harmed or that the data will be used to cast a critical eye on courts themselves.¹³ Courts lack resources and incentives to improve their data, resulting in civil justice data that are incomplete, inaccurate, dirty, and disorganized. Despite the uniformity of needs among court administrators and a small handful of vendors providing case management systems, there are no widely used standards for data collection and harmonization. The absence of common standards makes it impossible to compare jurisdictions or even to study individual jurisdictions across different case types and subject matters.

The lack of high-quality, accessible data is a major deterrent to producing knowledge about civil justice. To obtain data, researchers must engage in a hunt-and-peck exercise that may or may not yield useful results. As a first step, they need to locate datasets that may be useful to them. Then, they determine whether there are legal limitations to obtaining the data. If the data are not publicly available but a court has discretion to share them, researchers must engage in protracted negotiations to obtain access. Once researchers have access, they must spend weeks cleaning the data. The process of finding and obtaining data and making them usable deters researchers from studying the civil justice system and impedes the development of evidence-based policies in the civil justice arena. Although data science is producing new insights about many social institutions, few data scientists are working in the civil justice field.

This chapter considers how improving data collection, sharing data, and ease of data analysis can make civil justice institutions more accountable to government institutions, their constituents, and the public at large, and create opportunities for civil justice reform.

Section 16.1 starts by identifying key questions that analysts are well positioned to address. These include problems of structural racism and bias against people of color, the advantages of being represented, and the antecedents and consequences of entanglement in the civil justice system. Section 16.2 considers the obstacles to increasing knowledge about civil justice problems. These obstacles include the lack of good (or any) data, legal barriers to obtaining data, and real and perceived institutional risks to sharing data.

Section 16.3 then describes a civil justice data commons as one promising avenue that the authors are prototyping to address current access-to-justice challenges¹⁴ and free up the empirical opportunities offered by civil justice data.¹⁵ This Civil Justice Data Commons (CJDC) functions as a trusted intermediary among data sharers and data users. The CJDC’s technical infrastructure is automated to satisfy conditions for the supply of data and the demands of users. To facilitate access for researchers – speeding up the evidence-building process – the CJDC hosts documented, harmonized data, cleans and links files as needed, and provides tools and analytic software for research. The data commons allows for faster access to data for academic and policy research, as well as operational insights for courts and civil justice institutions to improve equity and service.

Several different tech trends examined in this volume – from ODR to online courts to e-filing to the growing menu of legal tech tools – will transform civil justice in the years to come. Each will increase both the amount of and focus on data. A key question is whether the civil justice system – and the judges, lawyers, litigants, and justice partners who inhabit it – can harness data to promote the just, equitable, and efficient administration of justice. A civil justice data commons can help answer that question.

16.1 Important Questions about the Fairness of the Civil Justice System

16.1.1 Structural Racism and Bias

In the criminal justice context, abundant high-quality policing and court data have allowed researchers to document the scope and depth of structural racism and other biases. This research underscores what smaller-scale observational and quantitative studies had previously shown: Black Americans are systematically treated worse than white Americans at every juncture of the criminal justice system. Policing data have shown that Black people are more likely to be subjected to roadside stops and are more likely to be the victims of lethal force.¹⁶ Court datasets have allowed researchers to show that Black drivers are likely to pay higher traffic fines than white drivers¹⁷ and that judicial mechanics and the peremptory challenge system adversely affect Black defendants.¹⁸ If you are Black, you are more likely to receive a longer sentence than a white person in a plea deal.¹⁹ These studies confirm the systemic pervasiveness of racism in criminal justice. Data access permits studies with sufficient granularity to pinpoint specific areas where race might have the strongest magnifying effect, which may suggest priorities for formulating and testing reforms.

Large-scale studies that investigate the impact of race, ethnic origin, and gender in the civil justice system are sparse but show promise.²⁰ One major foray comes from the Princeton Eviction Lab, which published the first study documenting the demographics of evicted renters on a national scale. Using a dataset containing 4.1 million records from public, private, and commercial sources, the study showed Black renters experienced the highest rate of eviction filings and judgments, and Black and Latin female renters faced higher eviction rates than their male counterparts.²¹ Data-driven studies like those by the Eviction Lab can demonstrate the magnitude of inequity in the civil justice system. Their study is a significant step in understanding which demographic groups in the United States are suffering the brunt of the eviction crisis and raises new questions about how the civil legal system plays into this crisis. Do differing eviction rates among racial and gender groups mirror structural biases in society at large? Or does the civil justice system magnify these problems? Accessible civil justice data are needed not only to understand the system’s role in replicating or worsening structural biases in eviction cases, but also its role in consumer debt and other areas that disproportionately affect poor people, people of color, and women. Population-level datasets contain large numbers of observations and variables to help pinpoint court contacts and processes that have the greatest impact.

16.1.2 When Do Lawyers Make a Difference (and Why?)

Civil justice data can also shed light on whether lawyers make a difference in outcomes and, if so, in what types of cases and why.²² To date, studies comparing represented and non-represented parties have been site-specific and relied on observational data or randomized controlled trials. Consequently, their findings are not easily generalized to other subject matters and other settings. Large-scale court datasets will allow comparisons across courts, subject matter areas, types of representation, and jurisdictions, producing more nuanced knowledge with broader impact.

The few studies investigating whether being represented by a lawyer affects the outcome of a case tend to show that representation makes a difference, but how much difference it makes depends in large part on the type of service provided, case type, and procedural complexity of the case.²³ Some research suggests that, in select settings, being represented by a non-lawyer advocate confers similar advantages to being represented by a lawyer. These studies raise important questions about how courts should deal with cases where one or both parties are not represented by a lawyer and how legal services providers should prioritize the provision of their resources.

Data-driven research confirming and extending these findings offers insights into how to “level the playing field” between represented and unrepresented parties. Some jurisdictions have changed their rules to allow judges a more active role in explaining the law or eliciting facts to a pro se party.²⁴ Courts might also focus on simplifying procedures or prohibiting lawyers in certain matters or fora altogether.²⁵

Research showing whether and when being represented affects case outcomes can also be important for legal services providers allocating resources. In legal aid settings, eligible applicants may receive representation depending on the availability of a lawyer specializing in their type of case. Research on the differential effects of representation might persuade legal services providers, which turn away almost half of eligible applicants,²⁶ to shift their resource priorities. Some providers report that they have already begun targeting their resources to clients with income levels just below eligibility cut-offs after data-based research showed representation did not affect outcomes at lower income levels.²⁷ Large-scale data studies about the impact of lawyers would also provide evidence for or against various approaches being promoted to increase access to the civil justice system. With the eviction crisis worsening, the movement to fund lawyers to represent tenants at risk of eviction is gaining ground.²⁸

Such studies might also be helpful in answering the question of which types of services make a difference. In the last ten years, limited scope representation has been promoted as an important means of increasing access to justice by allowing fewer lawyers to represent more people, an approach that many legal services providers are pursuing on the assumption that they can help more clients.²⁹ Despite the growth of brief advice services,³⁰ there is little evidence that these services have a positive effect on outcomes. One study found that while brief advice assisted clients in surmounting procedural barriers, it did not have significant effects on substantive outcomes.³¹ However, this study, which focused on the evictions caseload of a legal aid provider in San Mateo County during the summer of 2009, may not be generalizable to other settings.

Improving short-term outcomes – helping a client win a specific case, stay in their home, or avoid wage garnishment – is one measure of effective representation. Others include helping clients understand the law, supporting them through the process, and empowering them to advocate for themselves.³² An understanding of whether and when representation makes a difference is important knowledge to have in allocating scarce resources among eligible applicants who live in poverty and seek legal assistance.

16.1.3 Civil Justice Involvement and Poverty

The questions we have identified so far focus on whether civil justice institutions treat people fairly, allocate resources equitably, and achieve just results. Other research avenues focus on the people involved, not the court encounters themselves.

Getting a Fuller Picture from Big Data. More accurate and consistent data provide opportunities for deeper understanding about individuals’ interactions with the civil justice system: who enters the system (and why), how they navigate the system, and what impacts this interaction has on them in the long term. Specifically, linked population-level datasets that consider the antecedents and consequences of court involvement can widen the lens of research, leading to inferences and interventions that can scale across sites. By linking information from before and after court involvement, it may be possible to understand mechanisms that help avoid entanglement in the justice system or to mitigate its negative effects. This research shifts the emphasis away from court processes and legal resources to understanding and addressing the conditions and events that lead up to court involvement to begin with. In doing so, it may help people avoid being evicted by landlords or sued by debt collectors altogether.³³

This type of analysis is already being done at other intersections of poverty and policy. For example, researchers in Sonoma County, California, integrated four years of county and state records from health, mental health, substance abuse, housing, criminal justice, and human services systems.³⁴ This cross-domain dataset showed that 1 percent of the population accounted for one-quarter of jail time and behavioral health services and half of overnight homeless housing.³⁵ Another study linked administrative records from the Child Care and Development Fund, a federal subsidy program, with American Community Survey data to reveal that women who worked full-time but had less social support were more likely to receive the child care subsidy.³⁶ The study considered characteristics of the mother, such as education level, attainment of labor skills, and community ties, and found that relative to other low-income women, the women who received the child care subsidy tended to have higher educational attainment (high school diploma or higher).³⁷

Understanding the long-term consequences of a government intervention has also been made possible with linked datasets. One recent study created more than 150 million parent-child links to analyze the downstream intergenerational effects of the 1970 Clean Air Act Amendment and found that regulation-induced air quality improvements during pregnancy correlated with parental investment and higher college attendance in later generations.³⁸ This analysis was achieved through linking decennial census data, administrative records, and survey data and utilizing Protected Identification Keys (PIKs) in order to anonymize personal identifiers.³⁹ In another study, researchers linked national administrative data on the labor market and earnings with records on public housing to find that children who resided in public housing that was demolished later earned more than children who remained in non-demolished public housing, suggesting that job accessibility improved for children forced to relocate.⁴⁰

The Interaction between Poverty and Entanglement with the Justice System. Just as these studies used linked data to understand the role of government policies in the lives of people living in poverty, research linking court datasets to datasets that are indicators of health, financial circumstances, and other measures of well-being can yield knowledge about the effect of court entanglement over the course of a person’s life. Research is already under way. The Institute for Research on Poverty at the University of Wisconsin–Madison found women with an incarcerated family member have a higher risk of poor health, and Black women are, in turn, disproportionately likely to have an incarcerated family member.⁴¹ Researchers looking at Connecticut evictions found persistent, largely untreated, mental health problems and continuing housing instability.⁴² Other research, using mixed methods and blending administrative data with court data, shows correlations between eviction and criminal justice involvement,⁴³ how neighborhood disadvantage can exacerbate racial health disparities,⁴⁴ and how eviction in early childhood is associated with neighborhood poverty, food insecurity, and obesity in later childhood and adolescence.⁴⁵

Studies that investigate correlations between involvement with the civil justice system and other public institutions are beginning to emerge. For example, a study linking bankruptcy filings to administrative tax and foreclosure data found that filers who received Chapter 13 bankruptcy protections experienced increased annual earnings and decreased foreclosure rates.⁴⁶ Several studies have also appeared that focus on the short- and long-term consequences of being evicted for poor families. Using New York City housing court records, researchers found that evictions correlated with an increased risk of homelessness, long-term residential instability, and emergency room use but did not significantly worsen longer-term financial circumstances.⁴⁷ A longitudinal comparison of Cook County court records to credit bureau and payday loans data illustrated how the effects of successful evictions were small relative to the financial strain experienced by both evicted and non-evicted tenants in the time leading up to eviction actions.⁴⁸ Taken together, these studies raise the possibility that the prevailing view (which assumes that preventing eviction will have long-term benefits on alleviating poverty) may be myopic and that the locus of policy should shift from court processes to poverty-reduction programs.

More and greatly accelerated research using large, linked datasets is needed to determine the consequences of court involvement for the life cycle of poverty across sites and population subgroups. Providing lawyers and legal assistance to poor litigants assumes that the best locus of intervention to alleviate the effects of poverty must be the civil justice system – and that more lawyers and more legal assistance is the best approach. More studies may tell us, however, that assistance and opportunities upstream may be the most effective way of addressing housing and financial insecurity to prevent eviction and consumer debt actions from being initiated in the first place. Studies may also show that the impact of representation varies across race, ethnic enclaves, age groups, and population density.

Easily accessible civil justice datasets are needed to undertake such studies. In Section 16.2, we describe current challenges to obtaining such data and explain how the development of a civil justice data commons may begin to overcome these barriers.

16.2 Barriers to Accessing Data

An overarching challenge with civil justice data is the disaggregation of authority over civil dockets among states, counties, and municipal jurisdictions. In some states, managerial authority over the court system, and therefore control over data collection and sharing practices, is centralized in the supreme court or chief court administrative office. This makes it possible for the court to dictate that the same data be collected under the same standards throughout the state’s civil courts. As former Chief Justice Bridget McCormack notes in her chapter in this volume, Michigan has adopted a uniform system that is spreading to all its counties. Connecticut has also implemented a similar approach.⁴⁹ Other states have decentralized systems where each court has the power to dictate what data are collected. In some states, courts in different jurisdictions throughout the state have the option of maintaining their official records in either paper or electronic form.⁵⁰ This results in data that are siloed in multiple institutions and starkly unstandardized. Accessibility is governed by a huge and bewildering hodgepodge of laws and policies, rendering their availability for research limited. These barriers must be addressed if data-driven approaches are to produce knowledge about civil justice institutions.⁵¹ This section describes five of the largest barriers to access.

16.2.1 Data Disarray

Although courts are moving to electronic case management systems, many are still managing case files the old-fashioned way: paper. Others are using case management systems, but case pleadings are handwritten hard copies scanned into the digital system and are not machine-readable.

The absence of machine-readable pleadings, which contain valuable information about the case and the litigants, is a major technical impediment to granular research on the civil justice system. To overcome this barrier, some researchers have resorted to hand-coding pleadings, an expensive and time-consuming process.⁵²

Over time, the adoption of e-filing that is integrated with case management systems may alleviate this problem (though many mandatory e-filing systems exempt unrepresented parties).⁵³ For now, it is necessary to hand-transcribe the information or develop optical character-recognition tools that can translate pleadings that are marked with stamps and often contain handwriting into machine-readable form.

Administrative case level data scraped from online court case search websites – which pull from case management system data originally entered by hand by a court clerk – can be messy and contain incorrect or inconsistent values, making them difficult to interpret. Addressing these issues often requires a manual comparison with the corresponding docket by a specialist who can align concepts properly. That is tedious when the dockets are available online and onerous when it is necessary to contact the clerk of the court to understand which dispositions numeric codes refer to. Improving data capture is critical to improving analyses and the function of courts.

16.2.2 Barriers to a Common Taxonomy

Another data challenge is the absence of a common taxonomy among courts. The lack of shared substantive and procedural classification systems results from the diverse sites of authority among state, county, and lower-level courts. Different definitions across courts, even for concepts as fundamental as indigency, hamper coding and comparisons. Court procedures with different terms for various pleadings, requirements, and judgments also complicate matters. A court may lump evictions and contract cases together under “contracts,” or it may separate evictions and contracts but not distinguish among consumer debt cases (with some number of these classified separately under “small claims”). A rare few might use more granular categories that distinguish medical debt, consumer debt, and other debts, as an example. Coding these types of categories is necessary to study correlations between court involvement and life-course events. For instance, being able to identify medical debt is necessary to understand how much healthcare debt collection has grown since the pandemic.

In an attempt to tame some of this chaos, the National Center for State Courts (NCSC) developed the National Open Data System (NODS), intended to guide data collection in courts throughout the United States.⁵⁴ NCSC’s exhaustive effort yielded standards for basic elements such as case type, representation by counsel, case filing, disposition, and case closure date. NODS is an ambitious effort to transform the collection and classification of court data. Its adoption will be dictated by the cost of implementing or reconfiguring court case management systems and the extent to which courts believe that its implementation is consistent with the court’s priorities.

While NODs is a forward-looking strategy, there may be other strategies to rationalize court records from different jurisdictions that have already been collected. One approach might be to use natural language processing to identify subcategories in court documents to generate classifications. These generated classifications could then be matched to NODS categories. For instance, an algorithm might be trained to recognize medical debt from other types of debt based on the language in pleadings. Whether such an approach could capture subcategories of case types or definitional and procedural variation – likely more challenging – remains to be seen.

16.2.3 The Special Challenge of Assigning Demographic Characteristics

A particular challenge involves obtaining demographic information for civil court participants – a prerequisite for understanding equity concerns within the civil justice system. Under NODS, civil courts are supposed to collect race, ethnicity, and other demographic data about litigants. Most courts do not currently capture these data, but even if they attempted to, they would face a fundamental difficulty: Many, if not most, defendants in certain types of civil actions do not appear in court. One recent study of debt-collection cases in state courts found that, for courts where there was available data, 70 percent of debt cases ended in default judgments.⁵⁵ With such high rates of default, it is impossible to collect information about the race, ethnic origin, or gender of the tenant or debtor using only court data.

One promising strategy to append demographic characteristics to court data is through data linkage to census or commercial data. Linking at the per-person level requires the court records to have complete, accurate name data or other unique identifiers to make a match. Previous linkages to census data have had match rates between 40 and 90 percent, depending on the completeness of the names and addresses available, with the higher match rates possible when additional fields such as age or date of birth are available. The sparse identifiers present on civil court records have led other researchers to incorporate additional data sources. For example, Case Western Reserve University’s Center on Urban Poverty and Community Development matched Cleveland Housing Court data to Ohio Public Assistance records to obtain race information.⁵⁶ While this linkage allowed researchers to add otherwise absent demographic characteristics to eviction files, they were able to do so only for defendants who applied for and obtained public assistance.

16.2.4 Legal Barriers

While individual case files are available either online or in the clerk’s office in every jurisdiction, legal rules may limit access to bulk electronic case records. Although all fifty states and the District of Columbia have enacted public records laws, more than a third exempt the judiciary from coverage.⁵⁷ Even when public record laws apply to the judiciary, it is unclear whether they entitle people to obtain electronic case files in bulk form.

Approximately thirty-two states have rules governing the availability in bulk form of electronic public case records. Some states like Arizona make case file records generally available for bulk download.⁵⁸ Even in states that do not bar access to case data, county courts may have rules that limit their accessibility.⁵⁹ Some states prohibit the dissemination of bulk court records in electronic form, except where explicitly provided by a court rule or order.⁶⁰ At least one state disseminates bulk court data only to commercial purchasers. Other states, in contrast, bar the bulk download of court files for commercial gain.⁶¹ Finally, there are states like Hawaii, which will grant some requests for bulk data as long as the entity making the request is willing to pay for fees associated with the cost of providing the data.⁶² Many states also charge a fee or have terms of service that prohibit the scraping of court websites or CAPTCHA intended to prevent it.⁶³ On top of this confusion, states may have different privacy policies for court records with varying levels of restrictions.⁶⁴

16.2.5 Institutional Barriers

Institutional barriers are arguably the most important to address to facilitate data sharing. In preliminary planning interviews for the CJDC, we found a wide spectrum of views among judges and court administrators about the value of collecting and sharing data. Some judges did not view the data as valuable for either research or operational purposes; others strongly believed that research on aspects of the civil justice system was important for determining how well the courts served their communities.

Courts also fear scope creep resulting in data misuse, where a project approved to study one topic veers to another and publishes a study that the data may not support. Safeguards and reviews of approved uses must keep pace with growing volumes of data. Courts may worry that errors in their data will come to light. The errors may result from hurried or flawed collection methods, or through poor data management, statistical analysis, logic, or communication. When errors arise, it is important to examine their origins and impacts.⁶⁵

Even when court rules provide for public access to data, court personnel face administrative burdens to sharing them. They receive numerous requests for data and are pressed to develop internal custom-made approval processes on the fly. Most courts lack a data governance committee. Providing requested data is a time-consuming and mostly manual process fraught with risks of data misuse, errors in court data, and embarrassing findings.

16.3 What a Civil Justice Data Commons Offers

A data commons provides institutions and policy makers a one-stop shop for both high-level and detailed information about civil justice problems, by giving institutions simple, secure methods to share their data with researchers, policy makers, and other civil justice stakeholders and researchers fast, frictionless, and facilitated data access. By centralizing data sharing and research functions, the commons can serve as a meeting place for analysts, and a space where participants can share subject matter expertise, technical tools, and research methods.

16.3.1 What Is the CJDC?

The civil justice data commons (CJDC) is built according to principles that underlie data commons in the health sciences, created to accelerate research on medicine and disease using large databases.⁶⁶ Like genomic and other health data commons, the CJDC is controlled by a governance regime negotiated between data sharers, data users, and the directors of the commons.⁶⁷ The terms of service reflect the data security needs and project scoping requirements of data contributors with access demands of researchers.⁶⁸ The governance model is “hardwired” into the commons’ permissions infrastructure, ensuring that approved uses and users gain access efficiently.⁶⁹ Transparency is key: The commons makes clear what is the path to access, what requests have been granted, and what has been learned. Housed in an academic research center, the commons is able to act as an independent “trusted intermediary” for all parties.

The CJDC draws on the “Five Safes” framework to provide data access, balancing the terms and conditions of data providers and the needs of researchers.⁷⁰ Under this framework, the CJDC approves only “safe projects” intended to increase knowledge of the civil justice system. Researchers cannot use data for commercial purposes or for administrative purposes that have direct implications for affected individuals in the data. CJDC users are “safe people,” who are affiliated with universities, courts, government agencies or non-profit institutions, such as policy institutes and community organizations. The CJDC is a “safe setting” for the data, which can be accessed only on the CJDC’s platform. The data are “safe data”: Data sharers certify that the data they provide are legally and ethically sourced, and records in the commons are documented so the chain of their provenance is clear. Last, the commons follows the principle of “safe outputs.” CJDC users must submit their analytic outputs to the CJDC team, who will ensure that the data were used for the study proposed and that statistical disclosure limitation methods have been applied to prevent re-identification of data subjects.

In essence, the data commons provides a regulated marketplace for data owners and data users. For data owners, the commons facilitates data provisioning and quality control and oversees researcher use of the data. For researchers, the commons provides an interface where researchers can conduct statistical analyses on requested data. The CJDC can enable data discovery across multiple sites and jurisdictions, allowing potential users to browse metadata documentation and propose projects. Project requests are reviewed and approved in accordance with terms of use dictated by the data contributors that govern such questions as who is allowed to use the data, the scope of the research, and the purpose for which the research is undertaken. When projects are approved, CJDC provisions data into a secure workspace with statistical applications that researchers need. By making this information more readily available, knowledge about the prevalence, incidence, and consequences of civil justice problems can be produced, reducing reliance on anecdotal, outdated, or qualitative data alone. CJDC provides a simple, responsive interface that permits researchers to determine what data are available and what research has already been done with various datasets. Users can receive updates about new or refreshed data sources, recommendations on new data topics, and findings from the community.

In its mature form, the CJDC will have a secure infrastructure to automate data provisioning and an interface for data contributors to facilitate data sharing. In addition, it will have separate interfaces for authorized researchers and civil justice institutions so that they have ready access to data for academic and policy research and operational insights. The CJDC will host only machine-readable and documented data, which will be encrypted and held according to industry standards for securely stored data. The CJDC will offer technical assistance or services to help with cleaning, harmonizing, data documentation, and linkages. Funding for the operations and expansion of the Commons might come from federal and state government and foundation grants. The prototype data commons integrates extracts of data from data contributors. A future version of the data commons may have a federated structure as well, enabling the controlled access of data from a data controller’s servers. This may alleviate concerns about storage and control.⁷¹

16.3.2 Addressing the Barriers

Institutional Interests and Concerns. As noted above, the greatest barriers to making civil justice data more easily accessible are institutional. Courts are hesitant to share data out of concerns about data security, misuse, and errors, and findings that are embarrassing or ignite unwanted political controversy. The CJDC addresses these concerns by intermediating data sharing and access processes. Court administrators state their terms and conditions for data use, and CJDC screens requests and users, approving only requests that meet these conditions for data access. CJDC also takes over the monitoring role for approved data uses, ensuring scope and data security compliance. By using the well-established Five Safes framework for data access control, the CJDC ensures balance between the concerns of data providers and the needs of researchers. Institutions may also be sensitive to the optics of using these data. The CJDC provides greater privacy protection for data subjects than the open data portals already in place.⁷² Finally, the CJDC helps courts avoid surprises by permitting output review. It is critical to have independent research and transparency but equally important for courts to be able to confirm that studies have interpreted local laws or policies soundly. Giving the courts this “heads up” can allow them to frame the findings as opportunities to address issues rather than being surprised by negative headlines.

The CJDC also provides valuable services for data sharers through the production of knowledge to increase their own transparency. We expect that CJDC documentation and data standardization will enable application developers to create tools that work across sites. CJDC data can inform institutions on where things are working well and where they are not, providing evidence to support resource reallocation.

Controlling the Data Disarray. Working with the commons is not an all-or-nothing proposal. Sites can contribute what they have and increase their data sharing over time. For example, a court may share their machine-readable docket information, knowing that key data elements are not yet digitized. Commons users may help the court address this gap, such as through character recognition software or clerical keying. This creates a win-win situation where both the court and researchers gain more usable data. Other courts may share data they know is discontinuous, perhaps because of a change in records management vendors, updates to laws or policies, or transitions from paper to electronic records. The CJDC staff will document the data, and CJDC users will enhance detail about the datasets and likely create code that bridges the discontinuity. By fostering a community of users, this knowledge will be shared back to the court and subsequent researchers.

A necessary step in building out the CJDC is the harmonization of court data across jurisdictions, with their disparate data collection policies and case management systems. The CJDC seeks alignment with and adoption of NODS. Important first steps are determining what court data are machine-readable across sites and documenting challenges to NODS standardization within and across datasets. CJDC also standardizes data elements across contributed files, for example, creating a common vocabulary for eviction and debt cases and geocoding location data to facilitate area-level analyses. This standardization can include both manual matching and algorithmic entity resolution, allowing an accurate picture of the actual cases despite typos or name changes in the data.

Much of this initial work will continue to be labor- and time-intensive until we understand paths to automate the process of aligning new sites and keeping information up-to-date from courts we have been working with. A key feature of the CJDC is user-friendly metadata (data about the data) noting the sources, time periods available, present or absent fields, and incompleteness rates. As this metadata conforms to NODS, both research and court users will benefit. This metadata goes hand-in-hand with tools to visualize the available data. This data cleaning and sorting is a significant benefit to data contributors, who likely do not have the resources or expertise to perform this work on their own, and provides a provides a clean foundation for research.

Appending Demographic Characteristics. Because court files cannot capture the demographic characteristics of litigants who do not appear in court, linkages to external data sets are necessary to produce this knowledge. The CJDC is pursuing linkages among civil justice, census, and administrative datasets to enable analyses of civil justice eviction and debt-collection cases to develop demographic information about court litigants. These data will be critically important for understanding systemic biases in the aftermath of the pandemic.

16.4 Conclusion

Data gaps are not filled overnight, nor by individuals or single institutions. The CJDC is as much a movement as a project. To succeed, it requires participation by courts, practitioners, researchers, policy makers, philanthropists, private industry, and the public. By making civil justice data findable, accessible, interoperable, reusable, and linked to other data, the CJDC will increase knowledge about biases that distort civil justice processes and outcomes. It can also reveal promising strategies to increase access to civil courts and deepen understanding of how civil justice involvement impacts people living in poverty. If successful, the CJDC can create the basis for evidence-based approaches to civil justice reform.