Judicial Specialization and Deference in Asylum Cases on the U.S. Courts of Appeals

Abstract

Many look to the federal courts as an avenue of control of the growing administrative state. Some advocate the creation of specialized federal courts of appeals in areas such as immigration and social security. Yet, little is known about whether repeat exposure to specific types of cases enables federal judges to overcome doctrines of deference and whether such an effect would be policy-neutral. Gathering a sample of over 4000 cases decided by the U.S. Courts of Appeals between 2002 and 2017, we demonstrate that exposure to asylum cases over time emboldens federal judges to challenge administrative asylum decisions, asserting their personal policy preferences. The effect is particularly strong when the legal issue should prompt deference based on bureaucratic expertise. These findings not only address important questions raised by bureaucracy and court scholars but also inform a salient public debate concerning the proper treatment of those seeking refuge within our borders.

INTRODUCTION

As the administrative state has grown in size and complexity, scholars and officials have wrestled with questions of bureaucratic accountability and control (e.g., Balla 1998; Lowande 2018; see also West Virginia v. EPA 2022). The federal court system offers one avenue of control (e.g., McCubbins, Noll, and Weingast 1987), yet the complexity of the administrative state has the potential to attenuate the effectiveness of this oversight. Some say generalist judges are unwilling to scrutinize agency decisions and lack the needed specialization to do so (Hamburger 2014; 2016; Posner and Vermeule 2010, 29; Postell 2017), while others (Baum 2010; 2011; Cheng 2008) assert that they can acquire degrees of specialization.

We build upon Baum (2010; 2011), arguing that repeated exposure to decisions by a specific agency enhances judicial specialization ¹ in that area of case law, enabling judges to develop the knowledge and ability they need to take a hard look at agencies’ decisions and provide opportunities for greater oversight in the future. We focus on the U.S. Courts of Appeals’ review of the “hidden judiciary” (Guthrie, Rachlinski and Wistrich 2009), controversial federal administrative adjudicators with a heavy caseload (Barnett 2016), which functions as the final check on administrative abuse of discretion (Humphries and Songer 1999; Songer 1991). ² We contend that specialization gives judges the tools to engage in more effective bureaucratic oversight, but that this oversight may not be policy-neutral (Miller and Curry 2015). Indeed, judicial policy preferences play a central role in our model. We demonstrate the ways in which repeated exposure to BIA appeals and the case context condition the role of judicial policy preferences in determining the level of court deference to agency decisions. Deference—the degree of esteem or scrutiny given to an administrative decision—does not necessarily equate with an affirmance, although it may make one more likely. Less deference means greater scrutiny or more effective bureaucratic oversight. We argue that specialization, legal considerations, and ideological preferences are important determinants of the ultimate outcome.

We analyze this theory in a salient area of the law—asylum cases. We focus on appeals involving asylum claims because they are complex cases handled with a considerable amount of variability among circuits, ³ making this a fertile testing ground for our theory. Asylum applicants claim that they have or will suffer persecution in their country of origin and ask U.S. courts to grant them refugee status. They must demonstrate a well-founded, or reasonable, fear of suffering persecution in their home country based on their race, religion, nationality, political opinion, or particular social group. ⁴ Applicants denied at the administrative level can appeal to the U.S. Courts of Appeals, which are, for all practical purposes, an asylum seeker’s final opportunity in a federal court because few have the resources to appeal to the U.S. Supreme Court and the high court rarely grants certiorari in this area of law (Law 2010). Courts of Appeals can correct erroneous denials by remanding a case and sending it back to the administrative court with instructions to reevaluate.

We develop a measure of specialization based on the number of Board of Immigration Appeals’ (BIA) asylum cases heard by the U.S. Courts of Appeals judges over their time on the bench. Our measure is replicable and generalizable to other contexts, including review of agency decisions by the Social Security Administration, the Securities and Exchange Commission, and the National Labor Relations Board. Although prior research failed to find an impact of judicial experience, measured simply as the number of years on the court of appeals (Miller and Curry 2009; 2013; 2015; 2017), we argue that our more refined measure of experience will yield different results. Following Miller and Curry (2009; 2013), we expect that the effect of experience will vary by judicial ideology. We also theorize that the impact will depend on the legal issues in the case, as the law directs judges to apply varying levels of scrutiny to different questions based on the agency’s comparative expertise advantage.

This research is important for several reasons. First, it sheds new light on the relationship between agencies and courts, laying the foundation for future research concerning the extent to which judges can rein in the administrative state. As specialization is a growing phenomenon in the courts, understanding its impact is crucial. The potential for specialization, gained through repeated exposure to specific types of cases, to make the courts of appeals more effective in managing and overseeing the bureaucracy has important implications for the separation of powers, and the checks and balances believed to be so vital for American democracy. In addition, we create novel measures of the issues in the case to increase our understanding of specialization’s interaction with legal standards, an understudied area. Our theory and methodology can be applied to review of agency decisions in other subject matter areas such as social security and labor relation cases. Second, this research provides insight into the influence of judicial experience on decision-making and its relationship with the impact of ideology. The latter has significant meaning for contemporary questions concerning consistency in judicial decisions (Miller and Curry 2009). Furthermore, knowledge of the extent to which specialized judges will substitute their preferences for those of agencies not only contributes to academic literature but also informs policy debates. Finally, our analysis has specific implications for an important policy question examined by both legal scholars and political scientists. Because of a dramatic increase in immigration cases on the U.S. Courts of Appeals, scholars and policymakers have considered creating a specialized federal immigration court of appeals (Baum 2010; 2011). The consequences of creating such a court would be substantial and uncertain (Baum 2011). Similar debates surround the creation of specialized appeals courts in other areas of law, such as social security disability determinations (Verkuil and Lubbers 2003). We address this uncertainty, speaking to the probability that judges sitting on such a court would become more likely to rein in the administrative state.

This study proceeds as follows. First, we explain our theoretical approach, the principal–agent perspective, and explore the effect of specialization on this dynamic. Second, we apply this framework to the review of immigration cases on the U.S. Courts of Appeals, highlighting the aspects of asylum claims that make them a particularly useful avenue for exploring the effects of judicial specialization on the likelihood of deference to agency decisions. Then, we describe our dataset of over 4,000 courts of appeals cases decided between 2002 and 2017 and explain our methodology. Finally, we present our results and draw conclusions.

Specialization and the Shifting Dynamic of the Principal–Agent Relationship

We employ principal–agent theory to understand the dynamics of the relationship between the U.S. Courts of Appeals as principal and administrative adjudicators as agents. Principal–agent theory, with roots in economics (e.g., Spence and Zeckhauser 1971), has been applied by political scientists (and many others) to understand a variety of institutional arrangements (e.g., Miller 1992; 2005), including one with multiple principals, as in this case (Clinton, Lewis, and Selin 2014). ⁵ Briefly, the principal must delegate to the agent in order to accomplish some objective, but the agent has “an informational advantage over the principal” (Miller 2005, 204), and so may not act in accordance with the principal’s wishes. The principal is motivated to exercise effective control of agents to ensure faithful implementation, yet the ability to scrutinize the agent’s decisions is undermined by informational asymmetry. The capacity of the courts to accomplish this objective is a subject of considerable scholarly debate (e.g., Baum 2010; 2011; Cheng 2008; Posner and Vermeule 2010). This information asymmetry should be most apparent in complex cases. A substantial and diverse literature explores conceptualizations of case complexity, understood as encompassing characteristics such as “density, technicality, differentiation, and indeterminacy,” and defines a complex legal environment as one that “taxes cognition” (Goelzhauser, Kassow, and Rice 2021, 93). ⁶ Time constraints on the U.S. Courts of Appeals may serve as another reason for informational asymmetry (Baum 2010). We add to this robust discussion concerning the courts’ ability to effectively control agents in complex cases.

Modern government is specialized. This feature is presumed to enable better quality decisions and more efficiency and consistency, while creating a narrow perspective that limits actors’ subject matter understanding and opens them to external control by interested groups (Baum 2011). Generalist judges on the U.S. Courts of Appeals are supposed to be an exception to the trend, as court rules and norms require the random assignment of cases to ensure judges’ exposure to case types (Cheng 2008). Yet, evidence suggests that the idea of a generalist judge on the federal courts of appeals is a myth (Cheng 2008). U.S. Court of Appeals judges have specialized in particular subject areas through the process of opinion assignment (Cheng 2008, 526). Federal court of appeals judges can also specialize in an area of legal policy, like immigration law, as a result of a high concentration of such cases on their dockets stemming from geographical patterns of litigation (Baum 2010; 2011). For example, the high percentage of petitions to review BIA decisions in the Ninth and Second Circuits provides these judges with an opportunity to develop “a degree of specialization in immigration” (Baum 2010, 1504). As the vast majority of these cases involve asylum claims, judges are particularly likely to develop knowledge and ability in this area (Baum 2010). ⁷ Baum (2011) refers to this as the case concentration dimension of specialization.

The case concentration dimension of specialization can be likened to the construct of experience developed by Miller and Curry (2009; 2013). Although prior studies conflated expertise and experience, Miller and Curry (2009) highlight the importance of the two types of specialization. The authors define expertise as specialization possessed by a judge before ascension to the bench and experience as specialization accumulated over time by a judge after appointment (an acclimation effect) (Hettinger, Lindquist, and Martinek 2006) and provide evidence that these two constructs are distinct. A Federal Circuit Court judge’s expertise in patent law, measured as possession of background technical skills (an undergraduate or graduate degree) and previous membership in the patent bar, makes it significantly more likely that the appellate judge will overturn the decision of the agency it reviews, the BPAI (now PTAB), in obviousness cases (in short, whether the invention is so obvious as to not warrant a patent). Experience, measured by the number of years the judge sat on the Federal Circuit at the time the case is decided, was not a significant predictor of overturning the agency.

Drawing on Baum (2010; 2011) and Miller and Curry (2009), we expect that the effect of our more refined measure of experience, based on exposure to specific agency decisions, will mitigate the “informational monopoly” held by agencies that constrains the courts’ (as principals) options (Moe 1984, 769). Experience reduces the need to rely on agency characterization of the case. ⁸ As Baum (2009; 2010) suggests, judges can use cognitive shortcuts as they gain experience and narrow their focus only to the case’s critical elements. As noted above, following Baum (2010), we expect that for the effects of specialization to be seen, cases must reach a certain level of difficulty, or at least not be so easy that a specialist has little informational advantage over a generalist. At the same time, the difficulties should not be so pervasive that experience does little to overcome them.

Numerous areas of law could be considered factually complex, and certainly many are the purview of administrative agencies, such as environmental, social security, bankruptcy, securities, and tax cases (Miller and Curry 2015). In general, legal principles suggesting judicial deference to agency decisions focus on applications of statutes to complex factual situations (Humphries and Songer 1999). Miller and Curry (2015) found that prior expertise and opinion specialization in antitrust law, a complex area of law, had an impact on federal court review of these cases (including Federal Trade Commission Decisions). Social Security Administration (SSA) disability determinations also meet this description, as they involve “the application of intricate substantive standards through an elaborate, multi-layered procedural framework” employing regulations (such as those listing impairments and providing grids) that are “highly technical and complex” (Levy 1990, 465 and 467). SSA disability determinations have been, like immigration law, a source of dramatic increases in the federal courts’ caseload (Levy 1990).

Review of immigration agency decisions is certainly complex, with statutes described as labyrinths second only to the tax code in complexity (Law 2010). In asylum cases, the ambiguity of the relevant facts, the challenge of applying vague standards such as “well-founded fear of future persecution,” and the difficulty of discerning conditions in applicants’ home countries and assessing credibility make the judge’s job particularly difficult (Baum 2010). Each of these complexity factors taxes cognition in different ways. Factual complexity refers to the need for careful review of the factual record, which is often incomplete, provided in a foreign language, and communicated through the lens of a different culture and by a myriad of accounts. Interpretations of vaguely written statutes/regulatory provisions and discretionary decisions are based on a scientific expertise possessed by agencies but not generally by the courts. We address each of these types of complexity in our legal issue variables discussed below.

Prior research supports the expectation that repeat exposure to certain types of cases can address information asymmetries. Courts are less likely to affirm the decisions of agencies appearing before them frequently (Zaring 2010) and that handle familiar subject matters (Eskridge and Bauer 2008). Similarly, the D.C. Circuit, which reviews more agency decisions than other circuits, affirms fewer such cases (Miles and Sunstein 2008; Schuck and Elliot 1990). Administrative law experts attribute this to judges’ increased understanding of the area of law gained with exposure to it, reducing the agency’s comparative informational advantage (Pierce 2011). Studies of district court review of the Social Security Administration’s disability decisions, a significant portion of their caseloads, are also consistent with this theory, as the non-affirmance rate (or remand) for these cases greatly exceeds the percentage expected given the standard of review (substantial evidence review, discussed below) and is much higher than that of the Supreme Court (Verkuil 2002). ⁹

It is possible that judges must reach a certain level of exposure to a particular area of law before experience has an effect. Baum (2010) hinted that caseload numbers must be large enough to provide judges with sufficient familiarity. At the same time, once a judge reaches a threshold of experience, additional exposure may no longer impact decision-making, as judges may think they “know” how to discern which cases should be remanded after hearing a particular number. We consider this possibility in our analysis. We also recognize that dynamics may be distinct in the Second and Ninth circuits, which hear significantly more of these cases than other jurisdictions. The Ninth Circuit has also instituted screening procedures through which routine cases, or those identified as meritless or hopeless, are handled by central staff rather than receiving “full-scale judicial involvement” (Law 2011, 668). Both differences may mean that these circuits’ judges have substantially more experience and, at least in the Ninth Circuit, increased experience with nonroutine cases. ¹⁰ We control for these factors below. In addition, we consider alternative hypotheses, specifically, whether increased exposure to administrative adjudications will lead to a judge becoming “captured” by agency officials and more deferential to them with increased exposure (e.g., Anderson 2018; see also Baum 2011). Agencies may be “repeat players” who gain credibility with judges from multiple interactions and advance intelligence of judges’ preferences they can use to shape arguments (Galanter 1974). The effect is likely greater if the judge is predisposed to agree with the agency’s policy position (in asylum, a conservative’s preference for affirming a denial).

Indeed, we expect specialization’s effect to be contingent upon ideology. Familiarity with issues may breed greater ideological divisions (Bartels 2011), and specialization may make judges “more assertive than generalists in their policy making” (Baum 2011, 35). Although Miller and Curry (2009; 2013) argue that specialists’ more sophisticated understanding of the subject matter enables them to better apply ideological schema to cases, we expect that judges would not need experience to do so in asylum cases because applicants are clearly the underdogs and rights claimants, and liberal judges are more likely to vote in their favor (Westerland 2009; Williams and Law 2010). ¹¹ The low probability of Supreme Court review and congressional attention allows the pursuit of policy preferences (Baum 2010), and heightened time pressure in circuits with dramatic increases in the number of immigration may increase reliance upon them (Baum 2011; Westerland 2009).

We theorize that these well-defined judicial policy preferences in immigration will amplify the experience’s effect because they reduce the constraint imposed on the principal (the courts) by the agencies’ informational advantage, and judges who feel they have “no choice” but to defer to a decision with which they disagree (Moe 1984, 770) will with greater experience be freer to reject it. As noted above, experienced judges can invest less time and cognitive effort in considering all the claims and facts, instead using heuristics and selectively focusing on dispositive aspects of the case. The effect of this reduced constraint here should be greater for more liberal judges because they are predisposed to disagree with the agency’s consistently conservative characterization of the issue (against the asylum applicant). Thus, experience reduces deference, but the result of this increased scrutiny, in terms of the final vote, will depend on the judge’s personal policy preferences. The impact of experience on more liberal judges should manifest as a greater propensity to vote to remand an asylum denial, but be different for less liberal judges, who are predisposed to the agency’s policy position. The mental shortcuts the latter develop may involve a “hardening” over time to certain aspects of the case, as they shift scrutiny to what they view as critical issues. Thus, for less liberal judges, specialization’s impact should manifest as an increased likelihood of a vote to affirm the agency’s asylum denial. We test these expectations in our analysis below. First, we lay the foundation by providing basic knowledge of asylum law and the manner in which the expected level of judicial deference to agency decisions varies according to the characteristics of the claim.

The Expected Level of Deference in Immigration Cases

We argue that repeated exposure to an agency’s decisions provides federal appellate court judges, as principals, with the experience needed to check their agents’ expertise. Therefore, to the extent that the law matters, the magnitude of the effect of experience and its interaction with ideology should be conditional on the degree to which agency expertise justifies deference. The law dictates that federal appellate judges generally defer to administrative decisions because agencies hold a comparative expertise advantage. The extent of that deference varies, however, as standards of review direct court of appeals judges to employ different levels of scrutiny depending on the issue (fact, law, or discretion) (Childress and Davis 2010). Courts apply the most intense level of scrutiny with de novo review, meaning courts decide the case as if the issue had not been decided before, affording no deference (Garner 2019). Progressively less scrutiny is applied to the other main standards of review: substantial evidence, clearly erroneous, and abuse of discretion (Peters 2009). In short, it is extremely difficult to overcome the abuse of discretion standard, as the amount of respect due to the administrative decision is at its zenith, while it is slightly easier to overcome the substantial evidence standard (Peters 2009, 245-6). The specific application of these expected levels of deference varies by area of law. Thus, to develop our hypotheses, we first briefly detail the asylum process and applicable law.

By the time an asylum claim reaches the U.S. Courts of Appeals, ¹² an asylum applicant may have presented the claim before numerous administrative decision-makers, including a border patrol officer in the Department of Homeland Security, Immigration and Customs Enforcement (ICE); an asylum officer in the Department of Homeland Security, United States Citizenship and Immigration Services (USCIS); and an immigration judge and the Board of Immigration Appeals (Department of Justice, Executive Office of Immigration Review). Immigration judges are the trial court, hearing witnesses and examining evidence firsthand, while the BIA is the highest administrative tribunal applying immigration laws nationally (Miller, Keith, and Holmes 2015). In 2002, the Attorney General streamlined BIA processes, allowing single-member review and affirmances without BIA opinions (Westerland 2009). An applicant denied asylum at the administrative level can appeal to the U.S. Courts of Appeals, which can remand the case to the administrative court for reevaluation in accordance with the circuit court’s opinion. All asylum cases before the federal court of appeals have been heard by an immigration judge and the BIA, and in the vast majority of these cases, the applicant lost at the administrative level, as the government rarely appeals (Miller, Keith, and Holmes 2015). ¹³ These appeals are often the applicant’s last chance, as the Supreme Court rarely reviews immigration cases (Law 2010).

When reviewing a factual issue decided by the BIA and/or the Immigration Judge, courts of appeals generally defer to agency fact-finding, upholding a ruling that the alien is not eligible for asylum if “supported by reasonable, substantial, and probative evidence on the record considered as a whole” and reversing if no reasonable fact-finder would fail to find the requisite level of persecution (INS v. Elias-Zacarias 1992, 481). ¹⁴ This substantial evidence standard of review is based on the need to defer to agency expertise and experience (Knight 2006). Therefore, we expect that the impact of judicial specialization is likely to be contingent on whether the case includes factual issues underlying the asylum eligibility determination. Facts are difficult to discern in asylum cases; therefore, judges with limited experience likely have a strong inclination to defer to administrative decision-makers. Judges with repeat exposure to these cases, however, may believe they know how to determine the facts. Evidence suggests that judges vary in their level of deference on factual issues. Some courts are willing to expand the rule into a “hyper-deferential approach” (Knight 2006, 135), while others employ reasonableness criteria to justify giving closer scrutiny to BIA decisions (Hamlin 2014). Court of appeals judges express alarm at their colleague’s willingness to question agency determinations concerning whether the requisite fear of persecution exists in an asylum case, characterizing it as denying the BIA’s role as the “master” on this issue (Jahed v. INS 2009, Kozinski, J., dissenting, 1002), and asserting that colleagues recite the substantial evidence standard of review but “effectively replace[d] it with a much lower standard in violation of Supreme Court precedent and our nation’s immigration law” (Bringas Rodriguez v. Sessions 2017, Bea, J. dissenting, 1090).

When the agency decision involves the interpretation of a statute or implementing regulation, both issues of law and expectations concerning the effect of judicial experience are mixed. This is the realm of Chevron deference, where generally courts are supposed to defer to agency interpretations of silent or ambiguous provisions that Congress intended to leave to agency discretion and affirm permissible agency constructions (Chevron v. Natural Res. Def. Council, Inc. 1984, 842-3). This deference is justified by the agencies’ superior subject matter expertise as compared to the courts, better equipping agencies to make policy choices accommodating “manifestly competing interests” within a “technical and complex” regulatory regime (Chevron, 865). A judge experienced in asylum cases could believe that he or she has the expertise needed to understand the complexity of the area and make a better choice than the agency. For example, in Valdiviezo-Galdamez v. Attorney General (2011), the Third Circuit agreed with the applicant’s argument that the BIA’s requirement that a social group is characterized by “particularity” and “social visibility” was “contrary to, and inconsistent with, the text of the INA” (592).

Although Chevron deference is indisputably applicable in some areas of immigration law, disagreement about whether it applies in other areas exists across and within circuits (Rubenstein 2007; Slocum 2003). For example, some courts argue that Chevron deference only applies when the BIA, based on its expertise, has analyzed a statute it has been given to enforce, and it intends to bind other IJs or itself (Miranda Alvarado v. Gonzales 2006, 922, citing Lagandaon v. Ashcroft 2004). ¹⁵ Empirical studies document the unpredictable and erratic application of Chevron deference in immigration cases (Caballero 2020). Courts may give a lesser form of deference based on the extent to which the agency’s interpretation possesses the “power to persuade, if lacking the power to control,” employing a list of factors from the U.S. Supreme Court’s decision Skidmore v. Swift (1994) (Miranda Alvarado at 917, citing Skidmore, 140).¹⁶ If the extent to which judges should defer due to agency expertise is unclear, the effect of experience is difficult to predict. Other factors relating to statutory interpretation may become more influential, such as the degree of detail, which can condition ideology’s effect (Randazzo 2008; Randazzo, Waterman, and Fine 2006). Therefore, we expect that the impact of judicial specialization is likely to be contingent on whether the case involves statutory interpretation, but due to this confusion, it may not.

Third, U.S. Court of Appeals judges review administrative agencies’ interpretations of the U.S. Constitution in asylum cases de novo, without deference (Anker 2016, 33; see, e.g., Peters 2009, 246; Singh v. Lynch 2015), because they possess more expertise in interpreting it (Marbury v. Madison 1803). For example, in our dataset, federal courts reviewed administrative denial of a due process claim, such as the argument that an immigration judge’s one-day continuance of the final hearing denies applicants’ due process (Zhang v. Gonzalez 2005). We do not expect a judge’s level of specialization in immigration to influence the likelihood of a vote to remand in these types of cases.

Finally, although agencies’ expert knowledge may be used to legitimize deference to the BIA and IJ on discretionary matters (Kanstroom 1997), we do not expect specialization to significantly impact judicial behavior on this issue because, as noted above, the applicable abuse of discretion standard is very difficult to overcome (Peters 2009). This standard applies to the agency’s decision, after an applicant is found statutorily eligible for asylum, regarding whether an applicant should be granted asylum in the agency’s discretion (Aschenbrenner 2012). To overturn this decision, the appellate court must find that it was “manifestly contrary to law and an abuse of discretion” (Collopy 2015, 992: citing INA section 242(b)(4)(D)). The standard also applies to the review of procedural matters, including a motion to reopen (Peters 2009), the most common context for the application of this standard in our dataset. Contrasting de novo review with abuse of discretion, scholars say that under the former agency decisions are protected by “gossamer film,” but under the latter they are covered by “a Kevlar shield” (Peters 2009, 246). We do not expect judicial specialization in asylum to pierce this armor.

Although we argue that the expected level of deference will condition the impact of specialization, we recognize that standards of review can be manipulated (Peters 2009). Evidence indicates that judges’ policy preferences influence whether a federal circuit court applies Chevron deference in reviewing agency decisions (Barnett, Boyd, and Walker 2018). We therefore do not focus on the level of deference cited by judges. Rather, we code the expected level of deference based on the clear distinction between whether the court interpreted a statute, a regulation, or the constitution; reviewed a factual finding; or considered a denial of a motion to reopen or of asylum to a statutorily eligible applicant. In this manner, we avoid navigating the murky distinction between questions of law and questions of fact. Our coding protocol directs us to examine the specific issues at hand and is not based on whether the court recites words such as “substantial evidence” or “Chevron deference.” Also, as a general rule, we removed any cases in which we could not confidently code the issue.

In sum, as discussed in the section above, we expect that the impact of an increase in exposure to asylum cases is contingent on a judge’s policy preferences, framed as the degree of liberalism.

Hypothesis 1: As a judge becomes more (less) liberal, the probability of a vote to remand will increase (decrease) as the judge’s level of experience in asylum cases increases.

In addition, we expect that the effects of experience’s interaction with ideology will depend on the legal issues in the case. Based on the law discussed above, specialization’s effect should be greatest when a case involves an agency finding of fact, where agency expertise justifies deference, but can be countered by a critique of the decision’s reasonableness. Specialization may also have an impact when a case involves an agency’s statutory interpretation. If only constitutional or abuse of discretion issues are at play, experience is not likely to have a significant impact. As we explain further below, to test these expectations, we split our sample between “liberal” and “conservative” judges. Therefore, we write the hypotheses with this dichotomy in mind.

Hypothesis 2a: For liberal judges, the increase in the probability of a vote to remand will be significant on questions involving agency findings of fact and statutory interpretation.

Hypothesis 2b: For conservative judges, the decrease in the probability of a vote to remand will be significant for questions involving agency findings of fact and statutory interpretation.

As noted above, we recognize that the impact of specialization may be particularly pronounced for conservative judges hoping to influence precedent in their preferred direction (given that the BIA’s position is essentially “conservative”). ¹⁷ In other words, experienced conservative judges may use cognitive shortcuts and focus on questions of statutory interpretation because they have a greater precedential effect, and therefore, specialization’s impact may only be significant when such a question arises. We consider this possibility below.

Data and Variables

To create our database, ¹⁸ we employed the Westlaw Key Number System, which organizes cases by legal issues and topic. ¹⁹ A topic and key number combination represents a unique point of law. We narrowed the cases by collecting a random sample of 10 percent of the total cases under each Key Number and topic within the category of Aliens, Immigration, and Citizenship and Asylum, Refugees, and Withholding of Removal (24k490-k649), dropping multiples of cases listed under more than one Key Number to avoid double counting. We collected a total of 4,343 U.S. Courts of Appeals immigration cases that involve asylum claims decided by the First Circuit through the Eleventh Circuit from 2002 to 2017. Cases from the D.C. Circuit are not included because it reviews so few BIA appeals each year. The majority of cases were heard after the reform of the BIA. We include both published and unpublished decisions, as prior research indicates that unpublished decisions should not be ignored in the immigration context (Westerland 2009, citing Law 2005). ²⁰

The Specialization variable measures the number of asylum claims appealed from the BIA that each judge has heard before the year the case was decided. ²¹ Thus, it is a cumulative score based on each individual judge that incorporates the judge’s years on the bench. ²² To obtain this information, we searched for each judge by name and pulled all the cases he/she had heard from the time he/she was appointed to the federal court of appeals until 2017. We narrowed the search to cases previously heard by the Board of Immigration Appeals and then to the subset including asylum claims. ²³ We recorded the number of asylum claims heard each year and created the cumulative score by adding the raw yearly number from the time of appointment to the year before the case was decided. Each judge’s specialization measure is based on the circuit in which he or she sits, not necessarily the circuit hearing the case. This method addresses the occurrence of visiting judges (sitting in another circuit by designation). ²⁴ Thus, although we build on Miller and Curry’s (2009) work, we employ a more nuanced measure of experience, as their variable was simply the number of years a judge had been on the federal court when the case was decided (2009, 2013, 2015, and 2017). To further specify our results, we include a measure of the total number of BIA appeals heard by each circuit per year from the United States Courts database, Circuit Yearly BIA Cases. This variable controls for the judges’ general exposure to immigration cases, narrowing the impact of our specialization measure to experience with asylum cases over time. The two measures are not highly correlated (0.3).

To test the interaction of specialization with ideology, we employ Giles, Hettinger, and Peppers’ (2001) widely accepted approach to measuring judicial policy preferences. When a vacancy occurs in a state in which a senator is from the president’s party, the president will generally defer to that senator. Therefore, Giles, Hettinger, and Peppers (2001) use the senator’s Poole–Rosenthal score to measure the judge’s policy preferences. If two senators in the judge’s home state are from the president’s party, the variable is coded as the average of their scores. When neither of the senators from the home state are members of the president’s party, the president’s Poole–Rosenthal score is employed because the president likely has the final word. The GHP scale ranges from negative scores (more liberal) to positive scores (less liberal). The variable Judicial Ideology measures the influence of the individual judge’s policy preferences. We use this measure to create the interaction variable Specialization*Judicial Ideology. Prior research indicates that liberal judges are more likely to vote for the alien in immigration cases (Westerland 2009). Hypothesis 1 predicts that as a more liberal judge’s level of exposure to immigration cases increases, the probability of a vote to remand will increase. The opposite should be true of less liberal judges. In addition, recognizing that scholars have questioned the subfield’s reliance on GHP scores, we run robustness checks employing other measures of judicial ideology developed by Bonica and Sen (2017) and Howard and Hughes (2022) and provide the results, which are consistent with those in our main analysis, in the Supplementary material.

In coding the expected level of deference, we examined the legal issues in the case, considering only those meriting a KeyCite headnote. We determined the issues based on our reading of the opinion first and then used the West KeyCite as a check on coding. In doing so, we acknowledge that this is a measure of issues the opinion author thought merited discussion, and many issues raised by litigants are not addressed. Westlaw editors read the case and identify the important issues, write a short description for each one called a headnote, and assign each headnote a Key Number, connecting it to a subtopic. These headnotes and Key Numbers are generally found at the beginning of each opinion. This approach allowed us to include a Key Number for each issue, providing another method of checking the consistency of our coding.

Dummy variables account for whether the applicant is challenging on the grounds of administrative fact-finding (Finding of Fact), a statutory or regulatory question (Statutory Question), discretionary decisions (Abuse of Discretion), or constitutional questions (Constitutional Question). Finding of Fact captures whether a question arose regarding the factual account below, such as whether sufficient evidence supported the asylum claim. Statutory Question measures whether the opinion addressed an issue of statutory or regulatory interpretation—for example, whether an imputed political opinion can be a protected ground. Abuse of Discretion captures whether the applicant questioned an exercise of discretion, an agency choice between several legally permissible courses of action or inaction (Kanstroom 1997). Constitutional Question measures whether the applicant claimed his or her constitutional rights, such as the right to due process, were violated. The issue variables are not mutually exclusive. This approach recognizes that a court may uphold an agency on one matter but rule against it on another (Kerr 1998). We follow prior studies and include variables accounting for each legal issue, distinguishing, in particular, factual issues from statutory interpretation (Humphries and Songer 1999). In other words, if a case has both factual and statutory interpretation issues, we would code the Finding of Fact dummy variable as 1 and the Statutory Question variable as 1. Further details are provided in the Supplementary material.

We include additional control variables based on prior research—including the level of human rights abuses in the applicant’s home country. We describe them in the Supplementary material. To address concerns about case numbers and selection effects in the Second and Ninth Courts of Appeals, we include dummy variables for both (Second and Ninth). We provide descriptive statistics for each of our variables concerning the average number of asylum cases and remands by circuit in the Supplementary material. We use logistic regression with yearly fixed effects (to account for swings in caseloads in some years) ²⁵ and employ standard errors clustered around the circuit to account for expected nonindependence in the data (Zorn 2006) and conduct parallel analysis with standard errors clustered around the case and the judge to address that source of nonindependence. ²⁶ The results reported below are those for the circuit clustered models. The interactions clustered between the judge and the case are included in the Supplementary material.

Analysis

We begin with the findings concerning specialization and its interaction with ideology, addressing Hypothesis 1. Although specialization alone just misses statistical significance (p=0.056), the interaction of specialization and judicial ideology is significant. This finding indicates that, as specialization increases and judicial ideology becomes less liberal, a remand is less likely. In other words, consistent with our expectations, the impact of an increase in case exposure is contingent on a judge’s policy preferences. Table 1 provides the results for our main variables of interest when we cluster on the circuit. ²⁷ We provide the full results in the Supplementary material. The findings are consistent with those when we cluster on the judge and the case.

Table 1. Effect of Specialization Contingent upon Ideology

Note: See Supplementary Table 6A in the Supplementary material for full results. Yearly FEs included but not shown. Standard errors clustered on the circuit.

*p<0.05; ** p<0.01; ***p<0.001.

Figure 1 displays the predicted probabilities of a vote to remand as case exposure increases and the judge is less liberal.

Figure 1. Marginal Effect of Judicial Ideology by Level of Specialization

Note: Figure corresponds with the results in Supplementary Table 6A in the Supplementary material.

Specialization, which is measured as cumulative exposure to asylum cases over the judge’s time on the bench before the year in which the asylum case in question was heard, begins to exert a statistically significant effect at 78 cases, just above the median level of specialization (77 cases). At this point, a standard deviation increase in a judge’s ideology score (reflecting a less liberal judge) corresponds with a very slight 0.8% decrease in the probability of a vote to remand. Moving to the 75th percentile of specialization (154 cases), an identical standard deviation increase in a judge’s ideology score is associated with a 2.03% decrease in the probability of a remand. At the 90th percentile (289 cases), this same increase reduces the probability of a vote to remand by 4.24%; at the 99th percentile (597 cases) the probability is reduced by 9.48%; and at the maximum level of specialization (1,004 cases), the probability is reduced by roughly 16.04%. An additional means of visualizing this effect is presented in Figure 2, which shows predicted probabilities of remand for a more liberal judge (a judge with an ideology score one standard deviation below the mean of Judicial Ideology) and a less liberal judge (a judge with an ideology score one standard deviation above the mean of Judicial Ideology).

Figure 2. Effect of Ideology on the Probability of a Vote to Remand

Note: Figure corresponds with the results in Supplementary Table 6A in the Supplementary material. More liberal judges are measured as those one standard deviation below the mean of Judicial Ideology, and less liberal judges are measured as those one standard deviation above the mean of Judicial Ideology.

While our results are modest, they are significant, given that the likelihood of remand is typically low. ²⁸ We also note that the probability of a remand for a less liberal judge is always comparatively low, and while specialization reduces it, the rate of reduction is less than the rate at which specialization increases the probability of a remand for a more liberal judge. The probability of a remand for a less liberal judge at the median level of specialization is approximately 21.90%, while at the 90th percentile this probability has declined to just 20.66%. For a more liberal judge, the probability of remand at the median level of specialization is about 23.53%, while at the 90th percentile this probability has increased to 29.18%. These findings certainly provide support for our principal hypothesis, which argues that ideology conditions the effect of specialization on the probability of a remand. The impact is greater for more liberal judges because remands are a fundamentally “liberal” outcome and an affirmance is fundamentally “conservative.” ²⁹ We argue that, as experience increases and the agency’s comparative informational advantage decreases, both more and less liberal judges rely less on the agency’s characterization of the case and develop a greater willingness to act on their ideological priors. It stands to reason that, at low levels of specialization, both sets of judges would be more deferential to the BIA. As they gain knowledge, more liberal judges will have more “room” in which to move. That is, both sets of judges begin with a relatively low likelihood that they will vote to remand, and because a remand is a more liberal outcome, it is more liberal judges who will see a more dramatic shift in their probability of casting a vote to remand.

To test Hypotheses 2a and 2b, we conduct additional analyses that interact our specialization measure with the legal issues in the case—findings of fact, statutory questions, abuse of discretion questions, and constitutional questions. Because we expect these interactions to vary by ideology, we split the sample between conservative and liberal judges, ³⁰ as three-way interactions can be difficult to interpret. Judges with a Judicial Ideology score below zero are categorized as liberal, and judges with a Judicial Ideology score above zero are categorized as conservative. The full results are included in the Supplementary material. The effects do vary quite a bit between judges of different ideologies. For liberal judges, specialization has significant positive effects on the likelihood of a vote to remand in cases dealing with findings of fact, abuse of discretion questions, and constitutional questions. For conservative judges, there is a significant negative effect associated with specialization and the likelihood of a vote to remand only on statutory questions. These results are illustrated in Figure 3.

Figure 3. Probability of a Remand by Case Issue

Note: Figure corresponds with the results in Supplementary Tables 7A and 8A in the Supplementary material.

To provide more context for the results, we discuss the predicted probabilities. Beginning with the finding of fact questions, for liberal judges at the minimum level of specialization, the likelihood of a vote to remand on such a question is 24.28%. Moving to the median level of specialization, this probability rises slightly to 26.40%. It climbs to 28.63% when specialization is at the 75th percentile, 32.78% at the 90th percentile, and 43.19% at the 99th percentile, and at the maximum level of specialization, the probability of a vote to remand reaches 57.87%. On abuse of discretion questions, the probability of a vote to remand at the minimum level of specialization is 17.59% for a liberal judge. This rises to 20.25% at the median level, 23.16% at the 75th percentile, 28.89% at the 90th percentile, and 44.41% at the 99th percentile, and at the maximum level of specialization, the probability of a vote to remand reaches 66.54%. Finally, for liberal judges, the probability of a vote to remand on a constitutional question is 11.46% at the minimum level of specialization, 14.85% at the median level, 18.94% at the 75th percentile, 27.87% at the 90th percentile, 54.71% at the 99th percentile, and 85.95% at the maximum level of specialization.

Again, only in cases dealing with statutory questions do we observe a link between specialization and the likelihood of a vote to remand for conservative judges, and the association is negative. At the minimum level of specialization, the likelihood that a more conservative judge will vote to remand a case dealing with a statutory question is 34.23%. At the median level of specialization, this probability drops slightly to 31.01%. At the 75th percentile, the probability drops to 27.95% and then to 23.03% at the 90th percentile and 14.05% at the 99th percentile. Finally, at the maximum level of specialization, the probability that a more conservative judge will vote to remand a case dealing with a statutory question is only 6.66%.

We therefore find some support for our expectations regarding Hypothesis 2a for the finding of fact questions for liberal judges and Hypothesis 2b for statutory interpretation for conservative judges. This latter result is consistent with the expectation that conservative judges shift focus to cases with greater precedential impact. However, we did not expect specialization to have much of an effect when constitutional questions or abuse of discretion questions were at issue. This is only true for conservative judges; for liberal judges, we observe significant positive effects exerted by specialization on the likelihood of a vote to remand in both types of cases. These results provide, in some respects, greater support for Hypothesis 1, as we see that ideology plays a significant moderating role in the relationship between judicial specialization and votes to remand. We further illustrate these effects in Table 2, which shows the effect of a standard deviation increase in specialization on the likelihood of a vote to remand for liberal and conservative judges.

Table 2. Effect of Standard Deviation Increase in Specialization by Legal Issue and Judicial Ideology (Circuit Clusters)

Note: Results correspond with Supplementary Tables 7A and 8A in the Supplementary material.

We explore this finding further in the discussion section. The results for the parallel analyses are largely consistent. ³¹ Taken as a whole, our results provide robust support for the significance of Specialization across model specifications.

Discussion and Conclusion

Can judges exercise effective oversight of the vast and sprawling administrative state, or are they by design generalists who must frequently defer to the judgment of expert bureaucrats? How does repeated exposure to certain types of cases influence the decisions that judges make with regard to the bureaucracy? We have endeavored in this study to provide some insight into these crucial questions. The findings have a number of important implications beyond asylum cases.

First, there is indeed support for the proposition that repeated exposure to complex cases significantly influences the probability that a given judge will second guess the decisions of more expert bureaucrats. This finding is an important addition to the specialization literature, as Miller and Curry (2009; 2013; 2015; 2017) did not find experience significant. Like expertise gained before appointment to the circuit, experience with complex cases enhances the effect of judges’ policy preferences on decision-making. Thus, our results indicate that the impact of this type of specialization, which is broadly applicable to federal courts of appeals judges across circuits, is not policy-neutral. Increases in experience tend to make more liberal judges more likely to vote in favor of the individual challenging agency action. For less liberal judges, the effect is the opposite: More cases are associated with a greater likelihood of siding with the agency against the individual. In other words, judges with more experience are more likely to act according to their ideological preferences. We argue that judges’ experience with complex cases decreases the informational asymmetries between courts and agencies, leading them to rely less on the agency’s characterization of the case. How they act upon this increased freedom depends upon their ideological predisposition toward agency holdings, but the impact is substantively significant. We therefore provide further evidence supporting the conceptualization of information asymmetry in principal–agent relations as dynamic, not static (Waterman and Meier 1998). Expanding the approach to examine the relationship between the courts and administrative adjudicators, we find that repeat exposure through appellate review enables judges as principals to acquire offsetting information they can use to further their goals. This repeat exposure has the added benefit for a judge of being a function of their normal duties; while acquiring information has attendant costs for principals, those costs may be less for judges because of the nature of their work. They need not seek out the information as a legislator or president might, but wait instead for it to come to them naturally.

In future research, we will delve into the influence of specialization in other contexts. We will consider whether specialization makes judges more likely to author opinions or write dissents, to seek out particular types of cases, and the extent to which judges with higher levels of experience influence their peers on a panel, as women and minorities have been shown to do in cases where gender and race are salient (Moyer and Haire 2015). We also plan to investigate how specialization interacts with background characteristics, including prior expertise. Exposure to complex cases may also affect other actors within the judicial system who influence judges, such as clerks. The potential for experience in complex cases to impact judicial behavior is significant and has wide-ranging implications.

Indeed, specialization has a greater influence than we expected. For liberal judges, the effect of specialization may be strong enough to overcome even the highest expected level of deference, in abuse of discretion cases. It also has an effect when courts review a case de novo. This result provides some support to those who question the practical importance of standards of review. Yet, the impact does vary according to the legal issues in the case, which suggests that the law does matter. Indeed, we show how legal rules can matter alongside other key factors influencing judicial decision-making—specialization and ideology. Exposure to immigration cases appears to drive liberal U.S. Court of Appeals judges’ decisions to overturn the experts when the case involves a finding of fact, often the finding that the applicant did not demonstrate the requisite fear of persecution. Judges have expressed alarm at their colleagues’ willingness to usurp the BIA’s power in this area. As courts of appeals are not generally in the fact-finding business, this result is particularly interesting. We find mixed results with regard to specialization’s impact when the court must interpret a statute or a regulation. The finding has important implications for the ongoing debate concerning the impact of the Chevron doctrine. Our result concerning abuse of discretion cases may stem from the fact that so many of these cases involve motions to reopen, and a common basis for such a motion is a claim that conditions in the applicant’s home country have changed to the extent that he or she is likely to face persecution. Experience in asylum cases can certainly speak to this determination. With regard to constitutional cases, our finding may also relate to the types of claims raised in the asylum context. The majority of the constitutional questions in our dataset are due process challenges, such as claims the adjudicator failed to review relevant evidence or the IJ exhibited bias. Specialization may increase a liberal judge’s understanding of how matters should be handled by the BIA and IJ, thus making them less likely to rely on the agency’s assertion that it followed proper procedure and more likely to remand when such issues are presented. We plan to explore these findings in depth in the future. For example, does repeated exposure to complex cases in other areas of law lead judges to question the procedural choices of administrative agencies? As the due process in administrative adjudication is hotly debated, the answer to this question will be of interest to scholars across disciplines.

Finally, if expert judges are generally better positioned to evaluate administrative decision-making than their nonexpert colleagues (e.g., Miller and Curry 2013), this fact has considerable implications for bureaucratic control. That is, agencies not subject to expert review may have an inappropriate level of freedom. Future research will consider how agencies react to this freedom and, alternatively, to potential review by federal judges with experience in their particular area of law. In investigating these questions, we recognize that courts can only exercise meaningful control over the bureaucracy when Congress has afforded them that right to do so (Baron 2019; Johnson 2019). In this way, courts are also agents of the legislature. As we noted above, Congress eliminated judicial oversight in most immigration decisions other than asylum in 1996. Thus, while courts may be able to safeguard our democracy against an out-of-control bureaucracy, they are limited by the reality that they cannot act until authorized (Barak 2008) and asked to do so by an aggrieved party (Baird 2007; Cichowski 2007; Wofford 2018). ³²

Where they are authorized to act, courts can play a significant role. In particular, with regard to high-stakes areas of the law such as asylum cases, excessive bureaucratic freedom may mean the difference between life and death. Scholars argue that the immigration system is fundamentally broken, and part of that brokenness lies in the fact that the bureaucracy operates with essentially no oversight (Cohen 2020). Our evidence suggests that there may be ways to encourage that oversight, recognizing that judges’ response to such prompting will vary by their policy preferences. Scholars should examine experience’s impact in other potentially complex, salient areas where courts play a significant role in vulnerable individual’s lives, such as social security disability determinations and workers’ compensation cases. A limitation of our analysis may be a strength in this regard—the fact that the government usually argues the “conservative” policy position. Our findings serve as a useful comparison for an analysis of issue areas in which the government typically represents a “liberal” position, such as environmental policy. ³³ Future research would do well to take experience into account when studying how judges exert influence over the administrative state. Indeed, if judges lack the ability to gain specialization in other areas of the law, we must again return to the question of who in particular, if anyone, controls these agencies Questions of bureaucratic accountability and control continue to be central in political science, and future research, both our own and the work of other scholars, has much left to uncover.