Represented but unequal: The contingent effect of legal representation in removal proceedings

Abstract

Substantial research and policymaking have focused on the importance of lawyers in ensuring access to civil justice. But do lawyers matter more in cases decided by certain types of judges than others? Do lawyers matter more in certain political, legal, and organizational contexts than others? We explore these questions by investigating removal proceedings in the United States—a court process in which immigration judges decide whether to admit noncitizens into the United States or deport them. Drawing on over 1.9 million removal proceedings decided between 1998 and 2020, we examine whether the representation effect (the increased probability of a favorable outcome associated with legal representation) depends on judge characteristics and contextual factors. We find that the representation effect is larger among female (than male) judges and among more experienced judges. In addition, the representation effect is larger during Democratic presidential administrations, in immigration courts located in the Ninth Circuit, and in times of increasing caseload. These findings suggest that the representation effect depends on who the judge is and their decisional environment, and that increasing noncitzens' access to counsel—even of high quality—might be insufficient under current circumstances to ensure fair and consistent outcomes in immigration courts.

INTRODUCTION

Substantial research and policymaking have focused on the importance of lawyers and legal service providers in ensuring access to civil justice (see, e.g., Albiston et al., 2017; Cornwell et al., 2017; Rhode & Cummings, 2017; Sandefur, 2008). The same is true of emerging research on access to justice in immigration courts (see, e.g., Eagly & Shafer, 2015; Markowitz et al., 2011; Srikantah et al., 2015). But outcomes can vary notably even among represented cases (see Miller et al., 2015a). Under what conditions does legal representation matter more or less for case outcomes? This study addresses this question of broad theoretical and policy importance in the context of removal proceedings—a high-stakes adversarial adjudication process that has garnered a great deal of public attention and controversy in recent years.¹

Removal proceedings are civil proceedings in which an immigration judge (IJ) must decide whether a noncitizen is admissible to the United States, or whether a noncitizen already in the United States should be granted relief from deportation. Removal proceedings mark a life-defining moment for many noncitizens and their families. In the words of the US Supreme Court, removal of a long-term US resident from the United States can be tantamount to banishment from their home and family, “a punishment of the most drastic kind” (Lehmann v. Carson, 1957). For noncitizens fleeing from violence and persecution in their origin countries, removal can be a death sentence (Marks, 2014). Hundreds of thousands of noncitizens find themselves in these proceedings every year. In fiscal year 2019 alone, the US government initiated over 690,000 new removal proceedings (TRAC, 2020a), which represent more than a seven-fold increase since 1992. Currently, over 1.1 million removal proceedings are pending in immigration courts due to the growing case backlog (TRAC, 2020b).

Certain features of removal proceedings have made legal representation a focal point of mounting public concern and scholarly inquiry. Removal proceedings take place in severely under-resourced courts where judges face an enormous caseload (ABA, 2019). In addition, immigration law is highly complex, and the hearings are adversarial, with the federal government always represented by a trial attorney. Yet, basic procedural protections available to criminal defendants, such as the right to court-appointed counsel, do not apply in removal proceedings. Further, many noncitizens must navigate their cases while they are detained because they are not eligible for a bond hearing or because they cannot afford to post bail (Ryo, 2016). Given this context, it is unsurprising that immigrants, advocates, and policymakers believe—and a growing body of empirical research generally confirms—that legal representation is associated with more favorable case outcomes in immigration courts. Thus, activists, philanthropic organizations, and local governments have increasingly invested in programs aimed at providing noncitizens in removal proceedings—especially those in immigration detention—greater access to counsel (Nash, 2019).

This sharpened focus on access to counsel, however, raises an important question: Is the effect of legal representation contingent? More specifically, does the effect of legal representation on case outcomes vary by the type of judges assigned to the cases? Does the effect of legal representation on case outcomes vary by the decisional environment in which the judges are making their decisions? By decisional environment, we mean the broader political, legal, and organizational context in which judges work. Addressing these questions has been difficult, in part because of the lack of reliable data amenable to quantitative analysis. We overcome this challenge by extensively cleaning the federal government's data on removal proceedings and combining them with original data that we manually compiled and coded on individual IJs and their backgrounds. Drawing on these unique data, this study analyzes over 1.9 million removal proceedings decided by more than 670 IJs between January 1998 and February 2020.

Our regression analysis shows that the representation effect (the increased probability of a favorable outcome associated with legal representation) differs by a number of important judge characteristics and contextual factors. We find that the representation effect is larger among female (than male) judges and among more experienced judges. In addition, the representation effect is larger during Democratic presidential administrations, in immigration courts located in the Ninth Circuit, and in times of increasing caseload. Notably, these results are due to substantial disparities in the probability of removal that exist among represented, rather than unrepresented cases, across various judge characteristics and contextual factors. Given the sheer number of removal proceedings decided annually, even relatively small disparities in probabilities of removal orders associated with these contingent effects of legal representation can mean the difference of tens of thousands of annual deportations.²

This study contributes to the growing body of research on the role of lawyers in the civil justice system. Access to justice often has been equated with access to counsel in this field of research (Zimerman & Tyler, 2010). Yet our findings suggest that increasing noncitzens' access to counsel—even of high quality—might be insufficient under current circumstances to ensure fair and consistent outcomes in immigration courts. Our findings also advance the longstanding body of research on judicial decision-making. Models of judicial behavior typically focus on direct effects of judges' personal attributes and institutional dynamics (Epstein & Lindquist, 2017) without considering how those attributes and dynamics might moderate the effect of legal representation on judicial decisions. Our findings show that a fuller understanding of judicial decision-making requires analyzing the interaction effects between lawyers and judges and their decisional environments.

BACKGROUND

Immigration courts and removal proceedings

Immigration courts are not part of the judicial branch but rather, part of the executive branch. IJs are attorneys whom the Attorney General appoints as his or her “delegates” (8 C.F.R. § 1003.10[a]) within an agency called the Executive Office for Immigration Review (EOIR). The EOIR is a component of the US Department of Justice (8 USC § 1101[b][4]), the executive branch that prosecutes immigrants in federal court. Unlike administrative law judges hired by some of the other federal agencies, IJs are subject to performance reviews and lack special protection from removal from the bench afforded by the Administrative Procedures Act (see Jain, 2019; Taylor, 2007). This means that IJs can be removed from the bench or reassigned to another position by the Attorney General. Thus, IJs may be “particularly vulnerable to political pressure and interference” (ABA et al., 2019). Over the years, these concerns have led to repeated—and thus far unsuccessful—calls by the American Bar Association, the National Association of Immigration Judges, and other prominent legal organizations, for the establishment of an independent immigration court system under Article I of the US Constitution (Gilbert & Cohen, 2017; Slavin & Marks, 2011).

Removal proceedings constitute the most common type of proceedings in immigration courts (for other types of proceedings that IJs conduct, see ABA, 2019). At the start of removal proceedings, the IJ must decide whether to sustain the government's charges. If the charging document known as the Notice to Appear does not state a valid ground for removal, the case will be “terminated.” If the charges are sustained, a number of outcomes are possible in the subsequent stage of the removal proceeding. For example, the noncitizen may request (at the beginning or the end of the proceeding), and the IJ may grant, a “voluntary departure,” which means that the noncitizen takes on the financial burden of returning to his or her country of origin and avoids triggering some of the legal bars to future lawful admission associated with removal orders. If the noncitizen does not file an application for relief from removal (e.g., asylum, cancelation of removal), the IJ will order them removed.³ If the noncitizen pursues legal relief from removal and is granted such relief, then they will be allowed to remain in the United States.

As noted earlier, government trial attorneys represent the Department of Homeland Security (DHS) in removal proceedings.⁴ In contrast, noncitizens in removal proceedings may hire an attorney at their own expense but they lack the right to a court-appointed defense counsel (8 U.S.C. § 1229a[b][4][A]). This means that a noncitizen seeking legal representation typically must find either pro bono representation or pay to retain a private attorney. Non-attorneys working for nonprofit organizations who obtain accreditation through EOIR's Recognition and Accreditation Program may also represent noncitizens in removal proceedings (for a list of accredited representatives by state, see EOIR, 2020).

Access to counsel for noncitizens

According to the Transactional Records Access Clearinghouse (TRAC), only about 44% of cases filed between fiscal years 2001 and 2020 involving noncitizens who were never detained or released from detention had legal representation (TRAC, 2020c). Among detained noncitizens in removal proceedings, the representation rate was even lower: about 17%. These low rates of representation reflect the reality that many noncitizens lack financial means to hire a private attorney. Some noncitizens may attempt to save up funds over time to hire a private attorney, but judges may deny their request for continuances, effectively foreclosing this option (Hausman & Srikantiah, 2016). In addition, studies have documented a shortage of qualified immigration lawyers who can provide deportation defense (Katzmann, 2014). Unsurprisingly, these challenges are magnified for noncitizens who are held in immigration detention, as prolonged detention invariably leads to a job loss and social isolation from the outside world (Ryo, 2017).

These access-to-counsel issues have drawn a great deal of public attention because legal representation—particularly representation of high quality—is associated with greater engagement with the legal process for noncitizens and higher chances of successfully fighting removal (Eagly & Shafer, 2015; GAO, 2016; Hausman, 2016; Markowitz et al., 2011; Miller et al., 2015a; Ryo, 2016, 2018). But largely overlooked in public discourse and scholarly research on access to counsel for immigrants are important questions about whether and to what extent the impact of legal representation might be contingent.

THEORETICAL FRAMEWORK

Which judge characteristics and contextual factors might moderate the effect of legal representation on case outcomes in removal proceedings? Why might the effect of legal representation depend on these characteristics and factors? To develop theoretically grounded expectations that will guide our empirical investigation of these questions; we turn to research on the role of lawyers and research on judicial decision-making.

Research on the role of lawyers

Given that lawyers play such a prominent role in the American legal system, the question of whether lawyers matter for case outcomes has been the focus of a large body of scholarship on the legal profession and access to counsel (for a review, see Taylor Poppe & Rachlinski, 2016). In addition, scholars have also examined how lawyers might matter for case outcomes. On that question, Sandefur's meta-analysis (Sandefur, 2015) offers important insights. Sandefur argues that there are two key types of professional expertise through which lawyers might make a difference in civil proceedings: substantive expertise and relational expertise.

Substantive expertise refers to lawyers' knowledge of the law and legal process. An essential aspect of lawyers' substantive expertise that can lead to positive case outcomes for their clients is the lawyers' understanding of the relevant legal rules (statutes, doctrines, case law, etc.), whether they are substantive or procedural. Substantive expertise also encompasses lawyers' knowledge about how to construct and present a set of facts in legally relevant terms (Sandefur, 2015, p. 3). One reason why substantive expertise is so important in court proceedings is because judges have substantial informational needs. Scholars of Supreme Court decision-making, for example, have pointed out: “In making decisions, the justices have certain information needs; they require a clear and faithful focus on the issues presented in a case, an understanding of the relationship of those issues to existing law, a clarification of uncertainties, and a view of the implications of a decision for public policy, tempered with candor” (McGuire, 1995).

Relational expertise, by contrast, “reflects skill at negotiating the interpersonal environments in which professional work takes place” (Sandefur, 2015, p. 16). Relational expertise can be real or perceived. On the latter type of relational expertise, Sandefur points out that “lawyer representation may act as an endorsement of lower-status parties that affects how judges and other court staff treat them and evaluate their claims” (Sandefur, 2015, pp. 16–17). Focusing on the flipside of this positive signaling effect of lawyers, Quintanilla et al. (2017) have found evidence of negative signaling effects of pro se litigants. Their experimental study involving a realistic case file in a Title VII sex discrimination case showed that law-trained study subjects viewed unrepresented claimants as less competent than represented claimants (despite their identical case facts), and that these stereotypes explained why the study subjects awarded lower settlement awards to unrepresented claimants. Moreover, Quintanilla et al. (2017) found this signaling effect among law-trained study subjects (law students and lawyers) but not among members of the lay public, suggesting that this type of a signaling effect may be a function of socialization within the legal profession.

In the immigration adjudication context, there is a growing body of research on the prevalence and the legal consequences of lack of access to counsel, but much less are known about how lawyers make a difference. In a study of asylum cases, Miller et al. (2015a) highlight the importance of dyadic relationships between judges and lawyers. Miller et al. argue, “a large component of attorney success is found in being able to tailor an argument to a specific IJ, perhaps knowing and playing on their proclivities” (Miller et al., 2015a, p. 230). Ryo (2018) extends this literature by examining procedural and substantive differences between represented and unrepresented immigration bond hearings using matched samples. This study shows that represented detainees engaged in greater levels of court advocacy (such as submitting documents and making affirmative arguments for release) but finds no evidence that such activities explain the positive effect of representation on hearing outcomes.

Taken together, these studies suggest that a fuller understanding of the role of lawyers requires us to broaden the inquiry beyond an exclusive focus on lawyers to consider how the effect of their professional expertise may be moderated by judges and their decisional environments. As to which judge characteristics and contextual factors might play this moderating role—and why—findings from research on judicial decision-making offer important guidance.

Research on judicial decision-making

The formalistic legal model of judicial decision-making posits that judges make decisions based on the applicable law (see Cross, 2003). Yet decades of empirical research across various areas of law show that judicial decisions are often better predicted by personal attributes of judges such as their race, gender, ideology, and length of tenure (see e.g., Epstein et al., 2013; Harris & Sen, 2019; Rachlinski & Wistrich, 2017), and contextual factors such as the legal, political, and institutional environment in which the judges are making their decisions (see, e.g., Brace & Hall, 1997; Engel & Weinshall, 2020; George & Lee, 1992). This approach to studying judicial decision-making has been influential in shaping the more nascent body of research on judicial decision-making in immigration courts, which has been motivated by concerns over large disparities in grant/denial rates across IJs and immigration courts (see, e.g., Hausman, 2016; Kim & Semet, 2020; Ryo, 2016).

In seeking to explain these disparities, studies of immigration adjudication have focused on a number of personal characteristics of IJs. For example, Ramji-Nogales et al. (2007, p. 342) found that respondents appearing before female IJs (as opposed to male IJs) had a higher chance of prevailing in asylum claims. Miller et al. (2015b) showed that increasing liberalism of IJs, as measured by their prior work experiences, increased the likelihood of asylum grant rates. The Government Accountability Office (2016, p. 32) found that IJs with more experience on the bench were less likely to grant asylum. Studies of immigration adjudication have also shown that certain contextual factors are significant predictors of IJ decision-making. For example, Kim and Semet (2020) found that the political party of the US president at the time of the proceeding completion is a significant predictor of IJ removal decisions. This finding is generally consistent with Rottman et al.'s argument (Rottman et al., 2009, p. 9) that IJs are “political decision-makers,” embedded in a milieu that “defines opportunities that are open to (them) and affects the probabilities that they will choose particular options over others.”

The foregoing review of research on judicial decision-making suggests that certain IJ characteristics and decisional environments are significant predictors of their decision-making. We argue that these same factors likely also shape IJs' predisposition toward and treatment of lawyers and their legal advocacy. Specifically, we focus on these judge characteristics that are most frequently examined in research on judicial decision-making: race, gender, political ideology, and tenure length. Existing research also points to the importance of examining the following dimensions of the IJs' political, legal, and organizational environment: the political party of the presidential administration in control at the time of the proceeding's completion, the federal judicial circuit in which the proceeding was adjudicated, and the caseload of IJs. Why might these judge characteristics and contextual factors amplify or dampen the potential impact of defense counsels' substantive expertise and/or relational expertise on case outcomes in removal proceedings? We draw on insights from studies of judicial decision-making to consider of number of possible explanations.

Moderating role of judge characteristics and contextual factors

Research that examines the impact of judges' race and gender on case outcomes have theorized that female judges and judges who are members of racial minority groups may vote differently than their white male counterparts, because the former set of judges use different judging styles or because they bring a different knowledge base that reflects the experiences of women or racial minorities (see Harris & Sen, 2019, p. 247). Likewise, Ramji-Nogales et al. (2007, p. 344) have argued that female IJs may have a greater inclination toward nonadversarial proceedings in their courtrooms and therefore more likely to “solicit a coherent and complete story” from the respondents (see Ramji-Nogales et al., 2007, p. 344). If so, the same disposition may also make female IJs more open to fully considering the evidence and arguments presented by defense counsel appearing before them. The same type of dynamic may also apply to nonwhite IJs to suggest that IJs' race will moderate the effect of legal representation on case outcomes. In short, particular judging styles or personal backgrounds of IJs that vary along racial and gender lines may facilitate or hinder the exercise of substantive and relational expertise of defense counsel in removal proceedings.

Research on judicial decision-making has also emphasized the importance of understanding cognitive processes that impact judicial behavior. We highlight one such cognitive process here that may be directly relevant to this study. Scholars of the attitudinal model of judicial decision-making have argued that law serves primarily as a post hoc justification for decisions that reflect judges' political preferences (Segal & Spaeth, 2002). Motivated reasoning—defined as “a biased decision process where decision makers are predisposed to find authority consistent with their attitudes more convincing than cited authority that goes against desired outcomes”—plays a central role in understanding this dynamic (see also Braman & Nelson, 2007; Bybee, 2012, p. 74). This insight suggests that IJs' ideology might moderate the effect of legal representation on case outcomes in the following way: IJs who are less favorably disposed to noncitizens and their claims may be more likely to discount the arguments made by defense counsel and more resistant to the positive signaling effects of their presence in their courtrooms.

Another important judge characteristic that we examine is IJs' tenure. On the one hand, more experience on the bench may mean greater substantive legal expertise among judges, which in turn might reduce their informational reliance on defense counsel (see Szmer & Ginn, 2014). On the other hand, the specific working environment of IJs suggests that the representation effect may become larger as tenure length increases. According to a survey of IJs by Lustig et al. (2008a), IJs experience levels of stress and burnout that are higher than many other professional, including physicians in busy hospitals and prison wardens. Furthermore, Lustig et al. (2008b) has found that some components of this burnout are significantly associated with increased number of years on the bench. This makes sense given that the risk of stress and secondary trauma exposure resulting from adjudicating cases involving extremely vulnerable populations is likely cumulative (Aschenbrenner, 2013).⁵ Relatedly, longer years on the bench may also breed indifference or apathy. Under such circumstances, we might expect defense counsel to play a more substantial role for long-tenured IJs than for newly appointed IJs in terms of helping to focus judges' attention on salient legal and factual issues.

Next, we turn to contextual factors that we expect will play an important role in moderating the effect of legal representation on case outcomes. Here too, insights from research on judicial decision-making are instructive. Studies show that judges attend to the preferences of other political actors and are sensitive to public opinion in rendering certain types of decisions (Brace & Hall, 1997; Gibson, 1980; Rogol et al., 2018). Similarly, Kim and Semet (2020) suggest that IJs take cues from and align their decision-making in accordance with real and perceived policy directives and preferences of the political leaders in power. In contemporary American politics, Democratic presidential administrations typically embrace a more pro-immigrant stance than Republican presidential administrations. Taken together, these studies suggest that all else being equal, IJs may provide defense counsel greater opportunities to exercise their substantive and relational expertise during Democratic administrations than during Republican administrations. Moreover, this dynamic may be even more pronounced in a legal environment that is protective of immigrant rights such as the Ninth Circuit (see, e.g., Nanos, 2021). Lawyers—unlike most pro se respondents—are uniquely positioned to invoke substantive legal arguments and relevant legal rules, given their training and expected role in the courtroom. But their ability to do so is limited by the applicable substantive and procedural laws in effect.

Finally, research on judicial decision-making also suggests that the impact of defense counsels' substantive and relational expertise may be stronger when judges are under time pressure to make decisions. IJs generally face a large docket and rushed deliberation is common in many immigration proceedings. The following statement by an IJ is illustrative of this problem: “In those cases where I would like more time to consider all the facts and weigh what I have heard I rarely have much time to do so simply because of the pressure to complete cases. What is required to meet the case completion is quantity over quality.” (Slavin & Marks, 2011, p. 1787). Research suggests that the pressure of unmanageable caseloads may induce judges to rely more heavily on instincts, heuristics, and stereotypes rather than reasoned legal analysis and the facts of each case (see Gupta, 2016; Marouf, 2011; Rachlinski & Wistrich, 2017). Under these circumstances, defense counsel's substantive and procedural expertise—or perhaps even just their mere presence in the courtroom that might provide a positive signal that bolsters the legitimacy of the case at hand—may operate as an important check against the activation of cognitive shortcuts and implicit biases among IJs.

In sum, IJs differ in their style of interacting (or not) with lawyers, and in their approaches to processing legal arguments and evidence presented by lawyers under different conditions. These varying levels and modes of engagement with defense counsel can either nullify or amplify the efficacy of the lawyers' substantive and relational expertise. To be clear, we do not and cannot formally test causal mechanisms through which the various IJ characteristics and contextual factors may moderate the effect of legal representation on case outcomes. However, testing these interactive relationships represents an important first step toward understanding both the potential and limits of legal representation in immigration courtrooms.

DATA AND METHODS

We draw on two key datasets in our analyses. The first is the dataset on removal proceedings that we constructed from a variety of data files released by the EOIR relating to cases in immigration courts (EOIR Dataset). The second is an original dataset that we collected and coded on IJs and their biographical information (Judge Dataset). After implementing the cleaning steps described below and merging the two datasets, our analytical sample contains 1,906,832 removal proceedings that were decided by 676 IJs between January 1998 and February 2020.

EOIR Dataset

Constructing the data on removal proceedings for our analysis required multiple steps. First, we combined separate data files released by the EOIR pertaining to cases, proceedings, hearings, judges, and attorney appearances. It is important to note that a given case may have multiple proceedings, and a given proceeding within that case may have multiple hearings. Next, we identified the relevant removal proceedings for inclusion in our analysis. The IIRIRA, which went into effect on April 1, 1997, substantially changed immigration law. To avoid the transition period post IIRIRA, we excluded proceedings that were completed before January 1, 1998. We also excluded proceedings that were completed after February 2020, as they were likely impacted by a substantial change in day-to-day administration and function of the immigration courts resulting from the COVID-19 pandemic. In short, our study period is between January 1, 1998 and February 29, 2020.

We subset the data to include only those proceedings that reached substantive merits decisions during our study period. We excluded stipulated removal proceedings, in absentia proceedings, and rider cases (for a description of these proceedings and cases, see EOIR, 2010; Koh, 2017). Given that some cases can have multiple proceedings, we kept only the first proceeding within a given case that reached a merits decision. EOIR sometimes may update information on the assigned IJ in the proceedings data after the completion of a proceeding (this might happen, for example, if a motion to reopen is filed after a proceeding's completion and a different IJ is assigned to handle the motion). Therefore, we excluded any proceedings where the proceeding completion date came before the IJ's appointment date.⁶ We also excluded any proceedings where the proceeding completion date preceded the assigned IJ's earliest hearing date in the hearings data file. Finally, we restricted our analysis to IJs who decided 100 or more proceedings during our study period. We describe in greater detail each of these steps and why they were necessary in the Data and Methods Appendix.

The EOIR Dataset contains a variety of information on noncitizens in removal proceedings and their case characteristics, such as the noncitizen's nationality, primary language, custody status (e.g., detained or not), and whether an attorney filed a notice of appearance. The EOIR Dataset also contains unique judge codes for presiding IJs, which allowed us to merge the EOIR Dataset with the Judge Dataset described below.

Judge Dataset

Although in more recent years, the EOIR has made public announcements of newly appointed IJs with their background information, EOIR has not released a comprehensive list of IJs who has ever served, along with their biographical information. Thus, we set out to develop such a database. Although our Judge Dataset does not contain every IJ ever appointed, it contains information on IJs who collectively account for 99% of all removal proceedings that reached a final merits decision between January 1998 and February 2020.

We based our sampling frame of IJs on two lists disclosed by the EOIR. The first is a list of IJ names and corresponding judge codes that the EOIR disclosed to TRAC pursuant to a Freedom of Information Act request (TRAC List). The TRAC List contains 625 unique judge name-judge code combinations pertaining to IJs who were active during or prior to fiscal year 2016. The second EOIR disclosure on which we rely consists of a table of IJs found in a broader set of case data files that the EOIR posted on its website (Judge Lookup Table). The Judge Lookup Table contains 785 unique judge name-judge code combinations. We paired the Judge Lookup Table with the TRAC List to create the master list containing 756 unique judge name-judge code combinations.⁷

Finally, using a variety of sources, we coded each IJ's biographical information, such as his or her gender, race, work experience, and appointment year. These sources include official announcements of newly appointed judges that the EOIR makes available online,⁸ judge biography files that we obtained from TRAC, and our own independent internet searches.

Measures

Table A1 contains summary descriptions of all of the measures that we discuss below.

Outcome variable

Removed is a binary variable, in which 1 means an IJ ordered a noncitizen removed, and 0 means an IJ granted relief from removal.⁹ Although for ease of reference we refer to the “probability of removal” when discussing this outcome variable, it is important to recognize that not all cases in which an IJ orders removal is the noncitizen physically removed from the United States. For example, some noncitizens may appeal the order of removal and win on appeal.

Attorney variable

Attorney is a binary variable that indicates whether a proceeding had legal representation, either by an attorney or by an accredited representative. The EOIR Dataset tells us when an attorney or an accredited representative filed an E-28, an attorney appearance form. But there is no information in the EOIR Dataset about when, if at all, legal representation might have ended during any given proceeding. This means that Attorney tells us only whether a proceeding ever had legal representation at some point during the proceeding.

The EOIR Dataset does not distinguish between attorneys and accredited representatives, but it is reasonable to assume that attorney representation predominates given that the accredited representative program remains relatively small and most accredited representatives are only partially accredited, which renders them unable to assist in deportation defense (Marouf & Herrera, 2020). If a proceeding had multiple hearings, it is possible for one hearing to have had legal representation while the other hearing lacked legal representation.

Judge variables

Judge characteristics are captured by the following variables. Female is a binary variable that indicates whether a given IJ is female. We relied on pronouns used in IJ biographies to assign values to Female. For a handful of IJs without formal biographies containing such pronouns, we used the NamSor classification tool developed by Carsenat (2013). NameSor uses proprietary machine learning algorithms trained on birth records and statistics from countries around the world that take into account cultural and sociolinguistic features of first and last names, to provide the likelihood of a given name being male, female, or unknown. White, another binary variable, indicates whether a given IJ is non-Hispanic white. Because IJ biographies do not contain information about the IJ's race, we also coded this variable using NamSor's classification tool.

Second, consistent with the common practice of using appointing president's party as a proxy for judicial ideology (see Bonica & Sen, 2021), we created a Democratic Appointee variable that reflects whether an IJ was appointed during a Democratic presidential administration. Following Miller et al.'s approach (Miller et al., 2015b) of transforming IJs' work experience indicators into an index measure of their “asylum liberalism,” we combined Work Experience and Democratic Appointee variables using factor analysis on a polychoric matrix. Table A2 contains the resulting factor loadings. Given the directions of the loadings, we interpret the measure as follows: The higher the value, the more liberal (or less restrictionist) the IJ's views on immigration and more nonadversarial the IJ's stance toward immigrants. Finally, Years on the Bench is a continuous variable that measures the number of years a given IJ has served at the time of the proceeding completion date. To calculate Years on the Bench, we subtracted the IJ's appointment year from the proceeding completion year.

Contextual variables

The first contextual variable is Democratic Administration, which indicates the political party of the presidential administration at the time of a given proceeding's completion. The second contextual variable is Ninth Circuit, which indicates whether a given proceeding was adjudicated in an immigration court located in the Ninth Circuit, which has jurisdiction over nine western states and two Pacific Islands. Immigration cases are governed by the law of the federal judicial circuit in which the immigration court is located (Markowitz & Nash, 2015), and the Ninth Circuit is widely recognized as generally more protective of immigrants' rights than other circuits. Current Caseload captures the median daily number of individual merits hearings over which a given IJ presided in the 2 weeks immediately prior to a given proceeding's completion. We did not exclude weekends or holidays in calculating the median.

Control variables

We also control for a number of case characteristics. Nationality is a categorical variable that captures the noncitizen's nationality and has the following three categories: Mexico, Northern Triangle, and Other/Unknown. Language captures the noncitizens' primary language and has the following three categories: English, Spanish, and Other/Unknown. Custody Status indicates whether or not the noncitizen was detained during his or her removal proceedings and has the following three categories: Never Detained, Released, and Detained. Had Hearing is a binary variable that captures whether a given proceeding had any scheduled hearings as recorded in the EOIR's hearings data file.

Analytical strategy

To test for the existence of contingent effects of legal representation in removal proceedings, we estimated a series of logistic regression models using maximum likelihood. For each of the variables previously described in the Judge Variables and Contextual Variables sections, we estimated a baseline logistic regression model that included a term for the focal variable, a term for legal representation, and a term for the interaction between the focal variable and legal representation.¹⁰ For the statistically significant interactions, we then estimated a series of models that included a term for the focal variable, a term for legal representation, a term for the interaction between the focal variable and legal representation, all control variables, and circuit fixed effects.¹¹ We refer to this latter set of models as full models, which take the form:

$\text{logit}\left(\pi \right)=\alpha +{\beta }_{1}X+{\beta }_{2}Z+{\beta }_3\mathrm{XZ}+{\beta }_{4}Q,$

where logit(π) represents the log odds of Removed = 1 versus Removed = 0. α and βs are parameters to be estimated. X represents Attorney. Z represents the Judge or Contextual variable of interest. XZ is the product term of Attorney and the Judge or Contextual variable of interest. Q represents the covariates as discussed earlier.

When discussing the estimates from our models, we do not discuss the coefficients, as the coefficients in nonlinear models do not necessarily provide accurate information about the interaction effects of interest (see Ai & Norton, 2003; Mize, 2019). Rather, we discuss the results in terms of predicted probabilities, first differences, and second differences. First difference refers to the average marginal effect (AME) of switching from 0 to 1 for Attorney for any given subpopulation of interest (e.g., female IJs). Second difference refers to the difference in AMEs for Attorney between any given pair of subpopulations (e.g., female IJs vs. male IJs). We use Wald tests to assess whether the second differences are statistically distinct from 0. We performed a series of robustness checks using these full models to test the sensitivity of our results to different data management and modeling assumptions. Across these various checks, we found that results were consistent. Thus, we present only the results from our main analysis to conserve space.¹²

A note on selection bias is in order. Noncitizens who have legal representation may differ systematically from those who lack representation. Observational studies that seek to examine the effect of legal representation on case outcomes are concerned with such selection issues, because any observed differences in case outcomes between represented and unrepresented cases may be due to this selection effect. This problem is less salient in the current study, as we do not focus on the main effect of legal representation on case outcomes but rather, on the interactional effect between legal representation and judge characteristics, and between legal representation and contextual factors, respectively. More specifically, we have no reasons to believe that the potential selection effects we have described above systematically vary by judge characteristics or decisional environments, given that IJs and lawyers have no control over which individual cases get assigned to which judge.¹³ Nonetheless, we re-estimated our regression models using a matched sample of cases that are comparable on various case characteristics but for their represented/unrepresented status. The results of this analysis showed relatively smaller first differences than those we found using the unmatched sample. But our key findings relating to the second differences—that is, the contingent effects of representation—remained substantially the same. The matching procedure that we used and the results of the regression models using the matched sample are described in the Data and Methods Appendix.

RESULTS

Descriptive analyses

Table 1 reports IJ-level descriptive statistics for IJs included in our analytic sample. About 39% of the IJs in the sample are women. Nearly 68% are non-Hispanic white. The median value on IJ Liberalism is −0.325; the highest value (0.446) is about 4/5th of a point higher than the median, and the lowest value (−1.369) is over a whole point lower than the median. The median year of appointment for the IJs in the sample is 2010, with one IJ having been appointed as early as 1966 and dozens of IJs appointed in 2019. Government, INS/DHS, and private practice are the three most common types of work experiences among IJs, with about 62%, 56%, and 46% ever having worked in those settings, respectively, before being appointed to the bench. Experiences in the military, NGO, and academia are less common, with about 14%, 13%, and 3% ever having worked in those settings, respectively, before being appointed to the bench. Finally, about 46% of the IJs were appointed during a Democratic administration.¹⁴

Table 1. Descriptive statistics on judges

Note: N = 676 judges.

Abbreviations: DHS, Department of Homeland Security; IJ, immigration judge; INS, Immigration and Naturalization Service.

Table 2 provides proceedings-level descriptive statistics for all variables used in our analysis. Nearly 80% of proceedings in our sample resulted in removal orders. About 40% of the proceedings had legal representation. The rest of our discussion focuses on the judge variables and the contextual variables. About 28% of the proceedings were decided by female IJs, and over 70% of the proceedings were decided by white IJs. The mean value on IJ Liberalism was −0.333, and the average number of years on the bench at the time of case completion was over 9 years. About 46% of proceedings were completed while a Democratic president was in office. Nearly one-third of the hearings took place in the Ninth Circuit Court of Appeals. Finally, the median caseload during the 2-week period immediately preceding the completion of a given proceeding was 1.7 individual hearings per day.

Table 2. Descriptive statistics on variables used in the analyses

Note: N = 1,906,832 proceedings.

Regression analyses

We now turn to our regression analysis results. To conserve space, our discussion focuses only on the full models that have statistically significant interactions, as evidenced by second differences that are statistically distinct from zero (p < 0.05).

Judge characteristics

Figure 1 shows predicted probabilities of removal from the full model by IJ gender and whether the proceeding had legal representation. Table 3 displays these same probabilities in the first column (Pr[removal]). The second column shows the size of the decrease in probability of removal associated with adding legal representation for male IJs and female IJs, respectively (first difference). The last column shows the difference in the first differences between male IJs and female IJs (second difference). The results show that, controlling for all other variables in the model, proceedings with legal representation have a considerably lower probability of removal among both male IJs (Δ = 0.433; p < 0.001) and female IJs (Δ = 0.471; p < 0.001). The reduction in probability of removal associated with legal representation, however, is larger for female IJs than it is for male IJs—by nearly 4 percentage points (Δ = −0.038; p < 0.01).

Figure 1. Probability of removal by IJ gender and attorney (full model)

Table 3. Probability of removal by IJ gender and attorney (full model)

Note: Covariates include all judge variables except Female IJ, all contextual variables, and all control variables. This model includes circuit fixed effects. Standard errors are clustered by IJs in the model. First and second differences are calculated using seven significant figures.

^*p < 0.05, ^**p < 0.01, ^***p < 0.001 (two tailed tests).

Figure 2 uses estimates from the full model to plot the differences in predicted probability of removal for proceedings with legal representation and proceedings without legal representation, across number of years on the bench. Figure 2 shows that, controlling for all other variables in the model, the reduction in probability of removal associated with legal representation becomes larger as the number of years on the bench increases. The first column in Table 4 (Pr[removal]) provides predicted probabilities of removal from the full model for IJs with the median value for Years on Bench (median = 9 years) and a one standard deviation (SD) increase from median value for Years on Bench (median + 1 SD = 15.955 years), by whether a proceeding had legal representation. The second column (first difference) shows the reduction in probability of removal associated with having legal representation for these two values of Years on Bench, respectively. The last column (second difference) shows the difference in the size of this reduction across the two values of Years on Bench.

Figure 2. Average marginal effect of attorney by years on the bench (full model)

Table 4. Probability of removal by years on bench and attorney (full model)

Note: Covariates include all judge variables except Years on Bench, all contextual variables, and all control variables. This model includes circuit fixed effects. Standard errors are clustered by IJs in the model. First and second differences are calculated using seven significant figures.

^*p < 0.05, ^**p < 0.01, ^***p < 0.001 (two tailed tests).

Table 4 shows that at both values for Years on Bench, proceedings with legal representation have significantly lower probabilities of removal (Δ for median = 0.448, Δ for median + 1 SD = 0.467; for both, p < 0.001). Finally, Table 4 indicates that a one SD increase from the median value in Years on Bench contributes to an additional 2 percentage point reduction in the probability of removal for those with legal representation (Δ = −0.020; p < 0.001).

We did not find statistically significant interaction effects (at p < 0.05) for IJs' race (White) and ideology (IJ Liberalism).

Contextual factors

So far, we have discussed how the effect of legal representation on removal proceeding outcomes varies by certain IJ characteristics. Now we turn to our analysis of how the effect of legal representation might vary by key contextual factors. Figure 3 shows predicted probabilities of removal from the full model by political party of the presidential administration at the time of the proceeding's completion, and whether a given proceeding had legal representation. Table 5 displays these same probabilities in the first column (Pr[removal]). The second column shows the size of the decrease in probability of removal associated with adding legal representation for Republican and Democratic administrations, respectively (first difference). The last column shows the difference in the first differences between Republican and Democratic administrations (second difference).

Figure 3. Probability of removal by Democratic administration and attorney (full model)

Table 5. Probability of removal by Democratic administration and attorney (full model)

Note: Covariates include all Judge Variables, all contextual variables except Democratic Administration, and all control variables. This model includes circuit fixed effects. Standard errors are clustered by IJs in the model. First and second differences are calculated using seven significant figures.

^*p < 0.05, ^**p < 0.01, ^***p < 0.001 (two tailed tests).

The results show that, controlling for all other variables in the model, proceedings with legal representation have a considerably lower probability of removal in both Republican (Δ = 0.420; p < 0.001) and Democratic administrations (Δ = 0.491; p < 0.001). Nonetheless, the reduction in probability of removal associated with legal representation is nearly 7 percentage points larger during Democratic administrations than during Republican administrations (Δ = −0.071; p < 0.001).

Figure 4 shows predicted probabilities of removal from the full model by whether a given proceeding was adjudicated in an immigration court located in the Ninth Circuit Court of Appeals, and whether the proceeding had legal representation. Table 6 shows these same probabilities in the first column (Pr[removal]). The second column shows the size of the decrease in probability of removal associated with adding legal representation for all federal judicial circuits other than the Ninth Circuit Court of Appeals and the Ninth Circuit Court of Appeals, respectively (first difference). The last column shows the difference in the first differences between all federal judicial circuits other than the Ninth Circuit Court of Appeals and the Ninth Circuit Court of Appeals (second difference).

Figure 4. Probability of removal by ninth circuit and attorney (full model)

Table 6. Probability of removal by ninth circuit and attorney (full model)

Note: Covariates include all Judge Variables, all contextual variables except Ninth Circuit, and all control variables. Standard errors are clustered by IJs in the model. First and second differences are calculated using seven significant figures.

^*p < 0.05, ^**p < 0.01, ^***p < 0.001 (two tailed tests).

The results show that, controlling for all other variables in the model, proceedings with legal representation have a substantially lower probability of removal across all circuits (Ninth Circuit: Δ = 0.527, other circuits: Δ = 0.423; for both, p < 0.001). However, legal representation reduces the probability of removal by an additional 10 percentage points in the Ninth Circuit Court of Appeals compared to all other federal judicial circuits (Δ = −0.104; p < 0.001).

Finally, Figure 5 uses estimates from the full model to plot the differences in predicted probability of removal for proceedings with legal representation and proceedings without legal representation, across IJs' current caseload. Figure 5 shows that controlling for all other variables in the model, the reduction in probability of removal associated with legal representation becomes larger as the current caseload increases. The first column in Table 7 (Pr[removal]) provides predicted probabilities of removal from the full model for IJs with the median value for Current Caseload (median = 1 individual hearing per day) and a one SD increase from median value for Current Caseload (median + 1 SD = 3.024 individual hearings per day), by whether a proceeding has legal representation. The second column (first difference) shows the reduction in probability of removal associated with having legal representation for the two values of Current Caseload, respectively. The last column (second difference) shows the difference in the size of this reduction across the two values of Current Caseload.

Figure 5. Average marginal effect of attorney by current caseload (full model)

Table 7. Probability of removal by current caseload and attorney (full model)

Note: Covariates include all judge variables, all contextual variables except Current Caseload, and all control variables. This model includes circuit fixed effects. Standard errors are clustered by IJs in the model. First and second differences are calculated using seven significant figures.

^*p < 0.05, ^**p < 0.01, ^***p < 0.001 (two tailed tests).

Table 7 shows that at both values for Current Caseload, proceedings with legal representation have significantly lower probabilities of removal (Δ for median = 0.443, Δ for median + 1 SD = 0.458; for both, p < 0.001). Finally, Table 7 shows that a one SD increase from the median value in Current Caseload contributes to an additional 1.5 percentage point reduction in the probability of removal for those with legal representation (Δ = −0.015, p < 0.001).

It is worth noting that among unrepresented cases, the differences in the probability of removal across various judge characteristics and contextual factors, respectively, are relatively small compared to the same set of differences among represented cases. Specifically, for each of the judge characteristics and contextual factors, we examined the first difference (e.g., the probability of removal among male versus female judges) among unrepresented cases, and we did the same among represented cases. We found that the magnitude of these first differences are consistently and significantly larger among represented cases than unrepresented cases, which suggests that representation exacerbates outcome disparities attributable to varying judge characteristics and contextual factors.

It is also important to note that although some of our findings indicate a relatively small effect size for any given judge characteristic or contextual factor, those findings can translate into a large substantive impact when considered in light of the sheer number of individuals facing removal every year. For example, recall that our analysis showed that the reduction in probability of removal associated with legal representation is 3.8 percentage points larger for female IJs than it is for male IJs. As we noted earlier, the US government initiated over 690,000 removal proceedings in fiscal year 2019. A difference of 3.8 percentage points for this population alone could mean a difference of 26,224 fewer removal orders.

DISCUSSION AND CONCLUSION

Despite the longstanding scholarship on, and mounting public concern over, access to civil justice, we know little about whether and how the effect of legal representation varies by judges assigned to cases, and by the broader political, legal, and organizational contexts in which the judges are making their decisions. Drawing on insights from research on the role of lawyers and research on judicial decision-making, this study examines for the first time contingent effects of legal representation on case outcomes in removal proceedings. Consistent with prior studies of access to counsel in immigration courts, we find that noncitizens in removal proceedings with legal representation are significantly less likely to be ordered removed compared to those without legal representation. But our analysis provides new insights on the impact of legal representation by showing that the reduction in probability of receiving a removal order associated with legal representation varies by a number of key IJ characteristics (gender and tenure) and contextual factors (Democratic administration, Ninth Circuit, and caseload).

One might wonder whether these results are driven by a nonrandom distribution of lawyers to judges. For example, if more qualified lawyers systematically appear before female IJs than male IJs, and before long-tenured IJs than newly appointed IJs, then the effect of legal representation on the probability of removal order may be larger for proceedings adjudicated by female IJs and long-tenured IJs, respectively. Because the EOIR no longer makes available information on the identity of attorneys involved in immigration proceedings, we cannot empirically assess this possibility with the current EOIR data. However, this hypothesis is inconsistent with what we know about immigration adjudication. IJs do not choose the cases over which they preside, let alone the quality of lawyers who might end up before them.¹⁵

Likewise, one might wonder whether our results are driven by a nonrandom distribution of lawyers across different types of decisional environments. However, there is no reasonable basis to assume that high-quality lawyers are disproportionately present only during Democratic presidential administrations, in the Ninth Circuit Court of Appeals, and in high-caseload dockets. Of course, it may be that the effect of attorney quality on case outcomes is amplified or dampened by various IJ characteristics and decisional environments. That is an important question that warrants systematic investigation in future research given Miller et al.'s (2015a) study that documents the significant role that attorney quality plays in explaining asylum outcomes.

What are the broader implications of our empirical findings? To address this question, we highlight four key contributions of this study and identify a number of important lines of inquiry for future research that can further deepen our understanding of the role that lawyers play, how judges make decisions, and the nature of inequalities present in the adjudication process. Although our discussion is grounded in this study's immediate empirical context of immigration courts, our findings and their implications may generalize to other adversarial civil court settings where judges are engaged in nonroutinized decision-making such as social security disability hearings (on comparisons between social security and immigration adjudication processes, see Hausman, 2016).

First, this study's findings advance a more complex understanding of the effect of legal representation in the civil justice system. Scholars and advocates of access to counsel commonly assume that legal representation will help to ensure fair and just outcomes because lawyers can lessen the power imbalance between repeat players such as the government and individuals lacking in resources caught up in the civil justice system. This is also why many noncitizens in removal proceedings and their family members are willing to make extraordinary financial sacrifices to retain counsel. Yet evidence on contingent effects of legal representation that we have found suggest that the field is not level even for respondents who retain representation. The following counterfactual analysis that we performed illustrates—at the aggregate level—the limits of even universal representation in light of the contingent effects of legal representation.¹⁶ Our analysis predicts that under conditions of universal representation and assuming all judges were male, the removal order rate would be 71.10% compared to the currently observed removal order rate of 78.89%. This nearly 8 percentage point difference, when applied to our analytic sample of 1,906,832 proceedings, would translate to about 148,645 fewer total removal orders.¹⁷ While this reduction is notable, our analysis also predicts that under conditions of universal representation and assuming all judges were female, the overall removal rate would be 67.80%. The roughly 11 percentage point difference, applied to our analytic sample, would translate to about 211,559 fewer removal orders. These differences in expected outcomes for male and female IJs, even with universal representation, underscore the importance of considering the contingent effects of legal representation.

Second, this study's findings have important implications for theory and policymaking relating to judicial decision-making. Specifically, examining why the representation effect might vary by certain judge characteristics and contextual factors can help to isolate the type of judicial temperament, predisposition, background, and judging practices that might foster a more informed and consistent application of the law. For example, based on research on judicial decision-making, we suggested that certain judging styles or personal experiences that might be associated with female judges may facilitate the exercise of substantive and relational expertise of defense counsel in removal proceedings. The EOIR Dataset does not allow us to empirically explore this possible mechanism. However, testing these and related (as well as alternative) possibilities can inform hiring practices as well as the training of judges that will allow immigration courts to more fully leverage the expertise that lawyers bring to removal proceedings. While this study is agnostic as to what level of representation effect might be normatively desirable from a system standpoint, we argue that optimizing the conditions under which judges can take full advantage of legal counsels' expertise will promote both procedural and distributive justice in an adversarial system.

Third, a deeper understanding of contingent effects of legal representation can advance growing efforts by scholars and policymakers to better understand the source and nature of wide disparities in case outcomes between represented and unrepresented respondents. Take for example the moderating effect of IJs' tenure length on the relationship between legal representation and probability of removal orders. Insofar as the larger representation effect for long-tenured judges stems from their greater reliance on lawyers to counter their burnout and stress, judges should be trained to recognize this dynamic and provided resources that substitute for this role of counsel in adjudicating unrepresented cases. The implications are similar for our finding that the representation effect on case outcomes increases as caseload increases. We suggested that lawyers might matter more in times of increasing caseload because lawyers might serve as an important check against IJs' use of cognitive shortcuts and implicit biases that come to dominate decision-making when judges are under time pressure. To the extent that future research can confirm the operation of this possible mechanism underlying the moderating role of judges' tenure length, appropriate safeguards can be developed to protect pro se hearings from disadvantages arising from this kind of a dynamic as judges' caseload increases.

Finally, this study has important implications for conceptualizing access to justice and strategies to improve access to justice for individuals facing civil proceedings. Consider our findings relating to the moderating role of Democratic Administration and Ninth Circuit. One possible explanation is that judges may allow lawyers to more easily and effectively advocate for their clients during Democratic administrations than during Republican administrations. For example, lawyers appearing before IJs during Democratic administrations may be able to obtain more time from an IJ to prepare for their cases and to present their cases during merits hearings. Similarly, lawyers are more likely to have opportunities to influence IJs who are making decisions in judicial circuits with caselaw that is more protective of immigrants' rights. In short, political and legal opportunity structures are central to the workings of lawyers, as their advocacy is ultimately limited by the applicable procedural and substantive policies and laws in place. The foregoing discussion suggests that a solution to the crisis of access to justice cannot simply be greater access to lawyers—but rather, access to just policies and laws that support and protect zealous legal advocacy.

ACKNOWLEDGMENT

The authors gratefully acknowledge the support of the ABF/JPB Access to Justice Scholars Program and Andrew Carnegie Fellows Program. The statements made and views expressed are solely the responsibility of the authors. We thank the participants of the workshops at Vanderbilt Law School, Texas Law School, Annual Meeting of the American Sociological Association, UCLA Public Policy and Applied Social Sciences Seminar, USC Center for Law & Social Science, Northwestern Crime, Law, and Society Workshop, and Access to Justice Symposium at UC Irvine Law School. We also thank Alejandra Chaisson, Danielle Flores, Paul Moorman, and Karen Skinner for their excellent research support, and Nancy Foner, Dan Klerman, and Roger Waldinger for their valuable feedback.