Comparing Circuits: Are Some U.S. Courts of Appeals More Liberal or Conservative Than Others?

Abstract

This article investigates possible ideological differences between circuits of the U.S. Courts of Appeals. It looks at the distribution of three-judge panel ideologies on the circuits and at differences in decisionmaking patterns, testing several theoretical approaches to circuit differences: the attitudinalist approach, arguing that different judicial ideologies account for intercircuit differences; historical-institutionalist approaches that argue that circuit norms lead to differences in the proportion of conservative decisions and in the effects of judicial ideologies; and the rational-choice institutionalist argument that overall circuit preferences constrain three-judge panel decisions through the en banc process. Using a multilevel logit model, the study finds some support for the attitudinalist and historical-institutionalist accounts of circuit differences. It also finds that intercircuit ideological differences contribute comparatively little to the prediction of appeals court outcomes.

Popular commentators and scholars alike frequently state (and often simply assume) that there are ideological differences between different federal appeals circuits. Currently, the Ninth Circuit receives the most attention, mainly from commentators critical of its presumed liberal tendencies: Commentator Bill O'Reilly calls it the “wild bunch” (2002); Senator Orrin Hatch blames it for “judicial activism and overreaching” and claims that it is “out of the mainstream of both American law and culture” (2002); lawyer and legal commentator Bruce Fein, in the Washington Times, accuses the court of “manipulative judging at its worst” (2006, p. A16); and on it goes. On the other side of the spectrum, the Fourth Circuit is often singled out as particularly conservative: New York Times reporter Deborah Sontag calls it “the shrewdest, most aggressively conservative federal appeals court in the nation” (2003:40), while legal commentator John Dean goes one step further, calling it “the most conservative circuit court in modern American history” (2009). Other circuits have similar reputations, though their descriptions typically create less political heat. Sunstein et al. (2006:108) summarize the conventional wisdom as they see it: “In accordance with standard lore, the Third and Ninth Circuits are two of the most liberal, and the Seventh and Eighth Circuits are two of the most conservative.”

There are good theoretical reasons to expect the common wisdom on ideological circuit differences to be true. Different circuits “house” different judges with different political predispositions; circuits set their own precedent, only rarely overturned by the Supreme Court; different circuits can be expected to develop different political and jurisprudential cultures.¹ Considering this, it is surprising that, while several political scientists and empirical legal scholars have investigated intercircuit differences in decision patterns and court composition in passing, the actual ideological differences between circuits have rarely been placed at the center of analysis. There is little analysis of what it means that circuits are more liberal or more conservative and of which factors influence the differences between circuits. In fact, legal researchers do not even know whether ideological circuit differences can be shown with a high level of empirical confidence.

The question of intercircuit ideological differences is more important than the heightened rhetorics and political point-scoring of popular discourse suggest. From a practical perspective, if the decisionmaking in the U.S. Courts of Appeals follows ideologically identifiable intercircuit differences, then the uniformity of American law is likely to be undermined. This may not necessarily be a bad thing—intercircuit differences may reflect differences in the political cultures and socioeconomic conditions of different regions in the United States. But they may also increase legal uncertainty in a geographically mobile society. To evaluate these questions, it is important to understand which factors are associated with intercircuit differences. From a scholarly perspective, intercircuit differences and their causes can help researchers understand which influences are at work when federal appeals panels make their decisions.

This article compares attitudinalist, historical-institutionalist, and rational-choice institutionalist accounts of circuit differences. The first section following the introduction discusses how ideological circuit differences have been addressed in the scholarly literature, particularly focusing on what it means for a circuit to be more liberal or conservative than others. I discuss that question in more detail in the second section, which draws on attitudinalist, historical-institutionalist, and rational-choice accounts to determine how ideological intercircuit differences can be detected and explained. An empirical investigation follows, using Giles et al.'s Judicial Common Space (JCS) scores (Giles, Hettinger, et al. 2002; see also Epstein et al. 2007) together with data from Songer's Court of Appeals database (Songer 2005; see also Kuersten & Haire 2007) and employing a multilevel Bayesian model. I conclude with a discussion of the results and caveats.

Conceptualizing Ideological Differences Between Circuits

Quantitative studies of the U.S. Courts of Appeals have repeatedly looked at differences between circuits, but the question of whether some circuits are more liberal or conservative than others has rarely been the central focus. Richardson and Vines (1967), for example, look at reversals of lower court decisions in civil liberties cases in the Third, Fifth, and Eighth circuits and find differences in the percentages with which liberal and conservative decisions are overturned. Their main focus, however, is on the factors that influence behavior on the appellate courts as a whole, not differences between circuits. Similarly, Atkins and Green (1976) find that support for the government in criminal cases varies across circuits, but their main focus is on dissent levels, not on the directionality of outcomes. The same can be said about the series of seminal studies on the federal appellate courts that Songer has published on his own and with various coauthors. While several of these studies provide information on differences between circuits, the main focus is usually a different question. For example, Songer and Sheehan (1992) look at the success rates of different types of appellants and appellees and find along the way that appellants are more likely to win in the Seventh Circuit, compared to the Fourth and Eleventh circuits. Songer (1982) compares high- and low-dissent circuits and finds that in circuits whose dissent rates are below average, there is a significant relationship between panel composition and liberal/conservative directionality of decision outcomes; in circuits whose dissent rate is above average, Songer does not find such a relationship. While this result captures some of the differences between circuits, the focus of that study is not the differences per se but the question of whether or not unanimous decisions reflect consensus on the court.

Songer and colleagues (Songer, Sheehan, et al. 2000:119–28) directly address the question of regional and circuit differences in the proportion of liberal votes and find that there are substantive intercircuit differences in civil rights/liberties, criminal, and labor/economic regulation cases and that the intercircuit differences explain more variation than regional differences. These results address an important piece of the puzzle concerning possible ideological differences between circuits and raise a number of questions that further analysis should address. First, Songer et al. focus on a fairly long time period—1946 to 1988—during which the political/ideological profiles of circuits most probably changed. A study that specifically asks whether some circuits are more liberal than others might be better served by focusing on one or more shorter time periods. Considering that public charges of ideological decisionmaking on some circuits have arisen more recently, it will be useful to investigate more recent data (which Songer et al., understandably, had not yet collected by 2000). Second, because Songer et al. compare the differences between regions to the differences between circuits, they have to exclude several circuits of interest because they either straddle two regions or are identical with a particular region. Specifically, they do not look at data from the Sixth, Eighth, and Ninth circuits. Excluding these circuits makes it difficult to address some of the concerns raised by the current debate among political commentators who argue that the Ninth Circuit is particularly liberal.

The analysis presented by Sunstein et al. (2006) includes more recent cases: Comparing decisions in a variety of issue areas, Sunstein et al. find that there are differences in the proportion of liberal decisions between circuits, and that the Ninth Circuit—probably the circuit that has been the target of the strongest criticism—is among the most liberal circuits. Furthermore, Sunstein et al. show that the “party difference”—the difference in the percentage of liberal decisions between appointees of Democratic and Republican presidents—differs among circuits and is largest in the Ninth Circuit. These results indicate that there may be some truth to the popular view of a liberal Ninth Circuit, but that the differences between the circuits are not large. Several questions remain unanswered: For example, to what extent can circuit differences be explained by overall circuit tendencies for liberal or conservative decisions, and to what extent are they the function of the fact that different circuits have different proportions of liberal judges? And are appointees of Democratic (Republican) presidents more liberal (conservative) in some circuits than others, and does this account for the different decision patterns? While Sunstein et al. show generally that circuits with more Democratic appointees make more liberal decisions, they do not systematically compare the impact of the ideology of individual circuit panels with the overall impact of circuit differences.

Haire (2006) advances further toward comparing individual judicial ideology and circuit differences, with a particular focus on the Ninth Circuit. She shows that Democratic appointees on the Ninth Circuit are indeed more liberal than those on other circuits (at least in civil rights decisions) and that this increased liberalism can be largely attributed to President Jimmy Carter's appointments. Haire does not focus mainly on circuit differences, however. Her point is to show that judicial selection matters, not that the Ninth Circuit is generally more liberal than other circuits. The fact that Carter appointees are more liberal than other Democratic appointees suggests that more sophisticated measures of judicial predispositions than the party of the appointing president should be employed to investigate circuit differences.

Part of the problem of researching circuit differences is that “the most liberal (or conservative) court in the country” is an ambiguous statement. Does it simply mean that a circuit makes a larger proportion of liberal decisions than other circuits? Or does it mean that it contains larger numbers of liberal judges than other circuits, or that its judges are more liberal than those of other circuits? Or does it mean that its judges are more strongly influenced by their political views than judges of other circuits? Or does it mean that a circuit has developed a circuit-wide norm of liberal decisionmaking that causes even its conservative members to make liberal decisions?

Attitudinalist, Historical-Institutionalist, and Rational-Choice Institutionalist Accounts of Circuit Differences

From an attitudinalist perspective, the answer to these questions is easy: A liberal circuit is one that makes more liberal decisions than a conservative circuit, and it does so because its judges are more liberal. While the attitudinalist model has been most successfully employed in the study of the Supreme Court (Segal & Spaeth 2002), several studies have provided evidence that various measures of U.S. appeals court judges' political predispositions are associated with the directionality of their decisions (see, for example, Giles, Hettinger, et al. 2002; Goldman 1966; Songer, Sheehan, et al. 2000; Sunstein et al. 2006). To show that the attitudinalist model accounts for circuit differences, it is necessary to show, first, that the judges on the different circuits have different political predispositions and, second, that their political predispositions are associated with the directionality of their decisions.

While the attitudinalist model has the benefit of parsimony, institutionalists will point out that the institutional framework in which appeals court judges operate is sufficiently different from the Supreme Court to undermine the attitudinalist model (Scott 2006; Songer, Segal, et al. 1994). Most important, rational-choice institutionalists will point out that while Supreme Court decisions cannot be overturned by a higher court,² appeals court decisions can and are. While the Supreme Court reviews and overturns only a very small proportion of appeals court rulings, the circuit as a whole, sitting en banc, regularly reviews the decisions of three-judge panels (Giles, Walker, et al. 2006). It is at least reasonable to assume that circuit judges try to avoid being overturned in an en banc ruling (Hettinger et al. 2004; Kastellec 2007; Van Winkle 1997). First, en banc rulings take time, and circuit judges do not exactly have a light caseload. Second, being overturned en banc, particularly in cases that do not present issues of first impression, is associated with an expression of professional disapproval and may be associated with a (slight) loss of professional reputation (Baum 2006). So why not avoid being overturned if the outcome of the case remains the same?

Of course, there are enough circumstances in which circuit judges may decide to risk being overturned en banc—maybe they believe that their legal arguments can convince a majority of their peers, maybe they simply believe that the circuit majority is wrong, maybe there is not enough time to think about the preferences of other judges on the circuit. In fact, there are plenty of en banc decisions. Nevertheless, if the rational-choice institutionalist argument that I have sketched out is at least true for a number of cases, then the preferences of the circuit as a whole—typically measured with the preferences of a pivotal judge in en banc decisions—should influence the directionality of decisions in the circuit, thereby leading to differences between circuits as far as their pivotal judges have different political beliefs.

In contrast to rational-choice institutionalists, who focus on strategic behavior in institutional settings such as hierarchies, historical institutionalists focus (among other things) on the development of norms that contribute to the formation of actor preferences and identities (Gillman 1999; Thelen & Steinmo 1992). On the federal court of appeals, it is possible to identify (at least) two normative factors that may lead to circuit-specific behavioral patterns. First, while federal appeals court decisions set precedent in their own circuit (if they are not overturned either by the whole circuit en banc or by the Supreme Court), they have no precedential value in other circuits.³ As a result, over time, circuit-specific precedent can be expected to lead to circuit differences that are independent of the political predispositions of the deciding judges or the circuit as a whole.

Second, circuits can be expected also to develop more informal norms, as judges interact with each other in various capacities, and as new judges are socialized as they join a circuit. Cohen notes some elements of what he describes as the “organizational culture” of the circuits (Cohen 2002:169). Among other norms, Cohen points out that circuits have to strike a balance between formal justice—the use of formal decisionmaking rules such as precedent—and substantive justice—the decisions that individual justices believe are the just ones, even if they do not clearly follow doctrinal rules (2002:178–80). How a circuit strikes this normative balance should be reflected in the strength with which the political predispositions of judges correlate with decision outcomes. If a circuit emphasizes formal rules more strongly, then this correlation should be weaker; if a circuit emphasizes formal rules less strongly, then the correlation should be stronger.

Data

My analysis of decisionmaking patterns in the different federal appellate circuits relies on the U.S. Courts of Appeals Database (Songer 2005), with the Kuersten/Haire update for the years after 1996 (Kuersten & Haire 2007). While the circuit-based stratification of this data set is ideal for the comparison of decisionmaking in the different circuits, the nature of the data imposes a number of limitations on the analysis. First, the Songer/Kuersten/Haire data sets exclude cases that have not been published in the Federal Reporter. It is fair to assume that appellate court decisions that have not been “published”⁴ are less likely to be influenced by judges' policy preferences: They do not establish precedent and therefore do not typically have an impact beyond the individual case; many unpublished decisions serve an error-correction function and simply apply existing law; often, they are used to deal with frivolous appeals or questions of sufficiency of evidence in criminal convictions, or with cases involving complicated facts (Wasby 2004, 2005). As a result, the cases analyzed here tend to be those in which possible ideological differences between the circuits are more important. Second, I exclude cases in which the directionality of the decisional outcome cannot reasonably be coded as liberal or conservative, or in which the outcome is partly liberal and partly conservative. This restriction is necessary—if the directionality of a decision is not clear, it is not possible to identify possible systematic connections between judicial predispositions and outcome. Third, the analysis excludes en banc decisions. Part of my theoretical argument points to the threat of en banc review as a factor that influence the decisions of three-judge panels. Including en banc decisions in the analysis would make it more difficult to address this argument. The fact that the Songer/Kuersten/Haire data include only a small number of en banc decisions makes it impossible to analyze them separately. Fourth, to control the potential influence of at least some political factors external to the courts, the analysis is restricted to President Bill Clinton's years in office (1993–2000).

Because the Courts of Appeals data set is based on a nonproportionate stratified sample, Songer (2005) recommends the use of poststratification weights in most analyses. In the present analysis I use a different strategy to address the stratified nature of the sample. The circuit-based stratification is addressed by explicitly modeling circuit differences as part of the statistical analysis. Variation between years but within circuits is minor in the time period covered by the present study (compare the caseload data in Cohen 2002) and does not have to be corrected.

I use Giles and colleagues' (Giles, Hettinger, et al. 2002) JCS scores for appellate judges, in the Epstein et al. (2007) version (using Epstein's July 12, 2009 update), to measure judicial ideology. While this measure has been shown to have construct validity, one must keep in mind that it merely measures the ideology of those actors involved in the appointment of federal judges; its use as a judicial ideology measure requires the assumption that the appointment process is dominated by political/ideological considerations—an assumption that is not always justified. Nevertheless, Giles and colleagues' (Giles, Hettinger, et al. 2002) JCS scores are currently the best available systematic political preference measures for the lower federal courts. Due to missing JCS scores, the analysis excludes decisions of panels on which district judges are sitting on assignment. It is unlikely that the exclusion of visiting judges will systematically distort the overall analysis, as they participate only in a small proportion of cases and tend to defer to sitting circuit judges (Administrative Office of the United States Courts 2008:406; Cohen 2002:191–201). In addition, panels with visiting judges can be viewed as special cases; the present analysis tries to capture the “normal” differences between circuits. The analysis of special cases will be left for another study.

Ideological Differences in Circuit Composition?

One important difference between circuits is that they are composed of different judges. But does this difference translate into ideological differences—are judges of some circuits more liberal or conservative than those of others? And, more important for litigants, what is the likely composition of the three-judge panels that make decisions in different circuits? Here, I focus on the latter questions. In general, in the U.S. Courts of Appeals three-judge panels are formed by random selection from the pool of sitting circuit judges, and the cases they hear are randomly assigned to panels. There are minor modifications of this process in the different circuits—in the Ninth Circuit, for example, staff attorneys try to distribute the actual workload more or less evenly across panels (Cohen 2002:72).

The liberalism scores of appeals panels can be compared in two ways: First, one can investigate the distribution of median panel ideologies in the different circuits; second, one can determine the probabilities of getting more liberal or conservative panel medians in a different circuit.⁵ Instead of looking at the actual panels that are included in the Songer (2005) database, I simulate the panel assignment process in the 12 circuits. For each circuit and each year from 1993 to 2000, I take 1,000 random samples of three judges (i.e., 1,000 random panels) and calculate the median JCS score in each sample. The resulting numbers have to be seen as an approximation to the actual distribution of panels; as noted, the simulation excludes visiting or senior judges, for example. Still, this simulation process allows me to track personnel changes in the circuits across the Clinton years; the yearly random samples of 30 panels included in the Songer (2005) database do not permit this type of analysis with a reasonable degree of reliability.

In order to increase the comprehensiveness of the analysis, I focus here on six circuits that are representative of the different regions in the United States and that are generally identified with different ideological outlooks. The Fourth and Ninth circuits' ideologies have been highlighted in the popular literature, as the introduction shows. The Second Circuit, located in the Northeast, is typically supposed to be liberal, while the Fifth Circuit, in the Deep South, is presumed to be conservative. The Seventh Circuit, according to Sunstein et al. (2006), is identified as one of the most conservative circuits of the country. The Third Circuit, covering Pennsylvania, New Jersey, and Delaware, has been described as both liberal and conservative (Barrett 2005; Sunstein 2005).

Figure 1 uses histograms to depict the distributions of panel medians in the six circuits (combining the eight years from 1993 to 2000). The histograms confirm some of the common wisdom: The Fourth, Fifth, and Seventh circuits have higher frequencies of panels with median common space scores above zero (on the conservative side of the spectrum), while the Second and Ninth circuits have more panels below zero. The Third Circuit's modal panel score is located at zero, with more conservative than liberal panels. Undermining the familiar story of ideologically polarized circuits, most are comparatively balanced: The Ninth Circuit, the poster child for claims of judicial liberalism, has a fairly balanced distribution of panels, and all circuits except for the Second have panels located at both extremes of the spectrum.

Figure 1. Histograms of median common space scores, 1993–2000. For each circuit, 8,000 (1,000 per year over eight years) random samples of three judges were taken from the circuit's pool of appeals judges. The horizontal axes represent panel median JCS scores (Epstein et al. 2007; Giles, Hettinger, et al. 2002; higher JCS scores denote more conservative medians), while the bars represent the percentage of circuit panels that are expected to have the corresponding median JCS score.

The second approach to comparing the panel composition in different circuits looks at the probabilities of getting panels in other circuits whose medians are more liberal or more conservative than the median of a random panel in a given circuit. For the sake of comprehensiveness, I focus only on the Ninth and Fourth circuits. For each of the 1,000 randomly created Ninth (and Fourth) circuit panels, I have calculated the proportions of panels from other circuits that are more liberal and use the average of those proportions as an estimate of the probability that a Ninth (or Fourth) Circuit panel is more conservative than a panel from another circuit. The results are depicted in Figure 2.

Figure 2. Probabilities of panels being more liberal than a random Fourth/Ninth Circuit panel. Based on 8,000 (1,000 per year and circuit) random samples of three judges from each circuit's pool of appeals judges; panel ideologies are based on comparisons of median common space scores (Epstein et al. 2007; Giles, Hettinger, et al. 2002).

Note that the probability distributions of panel medians in the different circuits are not continuous (as is evident from Figure 1); as a result, the probability of getting a more liberal panel than a random Ninth or Fourth Circuit panel is not necessarily one minus the probability of getting a more conservative panel. Indeed, the probability that another Ninth (or Fourth) Circuit panel is more liberal than a random Ninth (or Fourth) Circuit panel is below 0.5 and varies by year.

The results from Figure 2 confirm the conclusions of Figure 1. While the rank-order of the circuits overall confirms common preconceptions of the ideological characteristics of the circuits, the results are weaker than expected: While both panels indicate that the Ninth Circuit is more liberal than the Fourth, the probability that a random panel on the Ninth is more liberal than a random panel on the Fourth varies between 0.7 and 0.8; correspondingly, the probability that a random panel on the Fourth is more conservative than a random panel on the Ninth varies between 0.2 and 0.3. If one uses 0.95 as the common standard of evidence in the social sciences, these results are less than impressive. Compared to the Fourth Circuit, only the Second Circuit's liberalism approaches conventional standards of evidence; compared to the Ninth, only the Fifth Circuit's conservatism, in 1993 and 2000, approaches such standards. While there are differences between the circuits, they are not overwhelming, and they vary over time.

One must keep in mind that JCS scores do not directly measure judicial ideologies; they measure the ideologies of those actors (presidents and possibly senators) involved in the appointment of judges. The results presented here therefore show that judges on some circuits have been appointed by more liberal (or conservative) presidents, in collaboration with more liberal (or conservative) U.S. senators than those on other circuits—not a surprising result, considering the different ideological positions of different presidents and senators. It is very well possible that the judges' ideologies do not perfectly reflect the ideologies of those who appointed them. As a result, it is important to see whether common space scores are associated with the actual decisions made by different panels in different circuits.

Circuit-Level Differences of Decisionmaking Patterns

A basic test of whether differences in circuit composition lead to differences in circuit decisions can be based on a (logit) regression analysis of circuit panel decisions, with the median common space score of each panel as the independent variable. If the parameter estimate (that is, in regression parlance, the β estimate for panel ideology) is positive, then more conservative panels are more likely to make a conservative decision than more liberal panels. This test does not account for the possibility that the threat of en banc review influences three-judge panel decisions as well, as rational-choice institutionalists would argue. To include this possibility in the analysis, one can add the median common space score for the whole circuit as an additional independent variable. If the parameter estimate for that variable is positive, then it is reasonable to conclude that the decision patterns of different circuits are also influenced by the control exerted through en banc processes—circuits with more conservative median judges lead to more conservative decisions, even if one controls for the ideology of the panel judges.

These analyses do not, however, address the historical-institutionalist arguments noted above. To test the question of whether some circuits have norms that suppress the influence of judicial ideologies, one has to compare the parameter estimates of panel common space scores across circuits. To test whether some circuits, due to circuitwide norms such as circuit precedent, on the whole tend to make more conservative or liberal decisions than their panel and circuit setups would predict, one has to compare the constant parameters of the circuit-level logit regressions. If a circuit has a larger constant than another circuit, for example, then this indicates that panels in the first circuit make more conservative decisions, compared to panels with the same ideological makeup in the second circuit, even if the median judges of the two circuits have the same ideology.

One solution to conducting these tests would be to estimate separate logit regressions for the different circuits. However, this would mean that all parameters in this analysis, including those for the overall circuit median, would have to differ across circuits. In addition, such an analysis would have to deal with comparatively small sample sizes (the number of observations ranges between 94, in the Eleventh Circuit, and 175, in the Seventh Circuit), and it would ignore potential correlations between circuit-level parameter estimates. To address these problems, methodologists recommend the use of multilevel models that combine the estimation of group-level (in this article, circuit-level) parameter estimates with estimates that are based on the whole sample (Gelman & Hill 2007). More details on the estimation of the multilevel model (using a Bayesian approach) can be found in the Appendix.

To summarize, this article tests the following statistical model:

$\begin{array}{c}p\left(panel\hspace{0.17em}decisio{n}_i\right)=f({\beta }_c^0+{\beta }_c^1\cdot \hspace{0.17em}panel\hspace{0.17em}media{n}_i+{\beta }^2\cdot \hspace{0.17em}circuit\hspace{0.17em}media{n}_i\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+{\beta }^j\cdot \hspace{0.17em}control{s}_i^j),\\ \end{array}$

where f is the logistic function. The unit of analysis is the panel decision, coded as conservative (‘1’) or liberal (‘0’); p (panel decision_i) denotes the probability of a conservative panel decision in case i. The independent variables of theoretical interest are the constant (which varies by circuit), the median ideology of the three-judge panel, and the median ideology of the entire circuit, both measured with JCS scores.⁶ The c subscript indicates that separate sets of parameters are estimated for each circuit.⁷

I include the following control variables. First, I control for the directionality of the lower court decision. The federal appeals courts affirm the lower court decision (or dismiss the appeal) most of the time (in about 67 percent of the cases included in the present study). There are probably multiple reasons: many appeals may have little merit—the lower court has made the correct decision in a legally clear-cut case; the appeals court may be reluctant to second-guess the lower court in factually complicated cases; or the appeals court simply may not have enough time to scrutinize every appeal, focusing more on cases that in one way or another stand out as important cases. As a result, without a control for lower court decisions, the estimated relationship between panel and median preferences and decision outcomes may simply reflect which cases have been appealed to the higher court. For the same reason, I also include controls for basic fact patterns—whether cases deal with civil rights, civil liberties, labor law, or economic regulation (criminal cases are part of the baseline cases).

As noted, the controls at least partly address the problem of case selection. The cases decided by the U.S. Courts of Appeals are not a random sample of lower court decisions; their selection is centrally based on the decision by the losing party in the lower court whether or not to bring an appeal. Lower court losers are more likely to become appellants if they believe they could win an appeal. Organizations that sponsor appeals may be able to engage in circuit shopping by choosing between cases dealing with similar issues in different circuits. Several authors have pointed to the possibility that strategic appellant decisions may influence the decision patterns one can observe at the appeals court level (Kastellec & Lax 2008; Sunstein et al. 2006). While this insight should serve as a caution in interpreting the results of the present study, a direct investigation of how case selection influences decision patterns is beyond this project. The controls for lower court decision and case types at least alleviate some of the problem.

Figure 3 summarizes the estimated parameter distributions (“posterior distributions,” in Bayesian terminology): the dots represent the medians, and the thick lines are the central 50 percent of the distributions around the medians (in Bayesian terminology, these are called “highest density intervals”). For the constants, the lower court decision, and the case type controls, the thin lines represent the central 95 percent of the parameter distributions. For the panel and circuit medians, the thin lines represent the central 90 percent distributions. Because the hypotheses for these parameters are directional—the expectation is that they are positive—90 percent intervals are appropriate. If such an interval is above 0, then the probability that the corresponding parameter is above 0 is at least 0.95. The hyper-parameter category includes the distributions of those parameters that essentially summarize the across-circuit commonalities of constants and panel-median parameters.

Figure 3. Posterior distributions of parameter estimates. Dependent variable: conservative decision dummy; modeled with a hierarchical Bayesian model with logistic data density and uninformative priors. Dots represent distribution medians, thick lines the 0.5 highest density intervals (HDIs), thin lines the 0.95 HDIs (0.9 HDIs for panel and circuit medians). N=1,558.

The results in Figure 3 indicate support for the attitudinalist hypothesis that circuit differences in decision outcomes are at least partly based on differences in circuit composition: The panel-medians for most circuits are larger than zero with probabilities above 0.95. Nevertheless, for the Second, Seventh, and Eleventh circuits, the probability of a positive parameter estimate is below 0.95. The rational-choice institutionalist hypothesis, focusing on hierarchical control in the circuit through the en banc process, does not receive support—the probability of a positive parameter for circuit medians is clearly below 0.95.

The historical-institutionalist hypotheses receive mixed support. While the circuit-level parameter distributions for the panel medians are not all the same, their differences are moderate. While the Third and Fourth circuits stand out as having slightly larger estimates than the other circuits, there is considerable overlap between distributions: All distribution medians are located within the central 90 percent intervals of all other circuits. The probabilities that the parameter in one circuit is larger than those of other circuits are all below 0.95. Figure 4 displays predicted probabilities of conservative decisions as a function of panel medians in the six circuits selected for closer comparison (the range of panel medians in the graphs roughly reflects the range of values in the sample, from −0.53 to 0.57). While the prediction lines for the Third and Fourth circuits are clearly steeper than those of the other circuits, the differences are not huge. Interestingly, the relationship between panel composition and conservative decisions in the Ninth Circuit is clearly not among the strongest, contradicting the common claim that the Ninth Circuit is among the most ideological ones.

Figure 4. Predicted probabilities of conservative decisions by panel median. The vertical axis represents the probability of a conservative decision; the horizontal axes represent the positions of appeals panel medians (roughly covering the range of JCS scores in the data). The predicted probabilities are based on the hierarchical Bayesian logit model summarized in the equation on page 183 and Figure 3. To estimate the probabilities, circuit medians were held constant at their circuit means, case dummies were set to zero, and the lower court decision was assumed to be liberal.

The historical-institutionalist argument that there are circuit-level influences—such as circuit-level precedent—that go beyond the preferences of panel and circuit judges receives some support from the circuit-level constants. The constants for the First, Second, Fifth, Seventh, and Eighth circuits indicate that these circuits are more likely to make conservative decisions than what one would expect if considering only the panel and circuit preferences. To identify which circuits have larger constants than other circuits with a probability of at least 0.95, one just has to check which circuits' parameter medians are outside the 0.95 intervals of other circuits. Based on this, it turns out that the Seventh Circuit, with at least a 0.95 probability, makes more conservative decisions than the Third, the Sixth, and the D.C. circuits, keeping panel and circuit medians (and case types) constant, and assuming a liberal lower court decision. By the same token, the First and the Eighth circuits are more conservative than the Third.

While these findings support the hypothesis that there are differences between the probabilities of conservative decisions in different circuits, these differences do not all confirm the common wisdom about which circuits are more conservative and which are more liberal. While the Fifth, the Seventh, and the Eighth are more conservative than their judges' preferences would predict, so are the First and (just) the Second, which are not commonly known as particularly conservative. Conversely, while the Third and the Ninth belong to the more liberal group of circuits, according to the constant parameters, so do the Eleventh and the Fourth, contrary to popular wisdom.

Table 1 compares the overall quality of the Bayesian multilevel model to two baseline models: first, a logit model that ignores circuit, case, and panel differences and models the probability of a conservative decision as a constant (“Constant”); and, second, a logit model that ignores circuit differences but otherwise includes all independent variables of the multilevel model (“Single-Level”).⁸ The Deviance Information Criterion (DIC) is the Bayesian statistic of choice to compare overall model quality; it is not very intuitive to interpret—it reflects an estimate of the model's remaining residual information (slightly but not completely incorrectly put, the model's “error”) with a penalty for model complexity (Spiegelhalter et al. 2002). It can be thought of as somewhat similar to an inverse (adjusted) R-square statistic. A smaller DIC indicates a better model, and the DIC for the multilevel model is considerably smaller than the DICs for the constant and single-level logit models. More intuitively, the probabilities with which the models correctly predict (PCP) observed court decisions also favor the multilevel model by a small margin. While the single-level logit model leads to a proportional reduction of (prediction) error (PRE) of only 3 percent, the multilevel model leads to a 10 percent PRE. The probability of correct predictions is an imperfect measure of model quality, however, as it does not reflect the probability with which one outcome or the other has been predicted. Therefore, I also present expected probabilities of correct predictions (ePCP) (Herron 1999). This measure confirms that including independent variables clearly improves the probability with which case outcomes are correctly predicted, though by only a small margin.

Table 1. Comparison of overall model quality

Notes: Multilevel model (see Figure 3), single-level logit model, and a logit model that includes only a constant parameter (equivalent to a model assuming all decisions are conservative). N=1,558. For further model details, see text. A smaller deviance information criterion (DIC) indicates the better model (Spiegelhalter et al. 2002). PCP: proportion of decisions correctly predicted; PRE: proportional reduction of prediction error, compared to the constant model; ePCP: expected probability of correct prediction (Herron 1999).

Conclusion

This article partly confirms common wisdom; it adds to it, but it also contradicts it. First, I show that there are in fact differences between federal appeals circuits that can be characterized as ideological, and some of the circuits commonly believed to be more liberal or conservative are in fact more liberal or conservative in a number of ways: The Second and Ninth circuits have more liberal than conservative three-judge panels, while the Fourth, Fifth, and Seventh circuits have more conservative panels. Several circuits, including the Fifth, the Seventh, and the Eighth, make more conservative decisions than one would expect from their median circuit and three-judge panel ideologies.

Second, I add to common wisdom by taking a closer look at what causes circuit differences. The data are clearly consistent with an attitudinalist explanation: Different circuits have different shares of liberal and conservative judges, and the median ideology of three-judge panels is clearly associated with the probability of a conservative decision. In addition, the data are consistent with at least some arguments that derive from a historical-institutionalist perspective. Several circuits are more likely to make conservative decisions than their ideological composition would predict, which indicates that circuit norms, such as precedent, favor conservative decisions in those circuits. In this respect, the Seventh Circuit stands out, as its decisionmaking is more conservative than that of several other circuits with a high probability. On the other hand, another historical-institutionalist argument, suggesting different circuit norms about the use of ideology in decisionmaking, does not find support. True, in several circuits the median three-judge panel ideology is not clearly associated with decision outcomes, while in other circuits it clearly is. Nevertheless, the probabilities are pretty small that the relationship between panel ideology and decision outcome is larger in some circuits than in others. Finally, the rational-choice institutionalist argument that three-judge panels have to follow the overall circuit ideology, due to the principal-agent relationship established by the institution of en banc review, is not consistent with the data. Median circuit ideology is not clearly associated with the probability of conservative decisions.

Third, the results presented here partly contradict common wisdom. The case of the Ninth Circuit stands out in particular. While that circuit has more liberal panels than other circuits, its distribution of panel ideologies is neither lopsided nor polarized—not surprising, considering the large number of judges on that circuit. Furthermore, the coefficient estimating the relationship between panel ideology and decision outcome is not among the largest—the coefficients for the Third and Fourth circuits are larger (though with comparatively modest probabilities). Finally, the Ninth Circuit is not among those circuits whose decisionmaking is more conservative or liberal than what would be expected based on panel ideologies. Another circuit that at least partly defies expectations is the Fourth Circuit. While it has more conservative than liberal panels, it does have a proportion of liberal panels. Furthermore, once panel ideologies have been taken into account, the Fourth Circuit's decisionmaking is not more conservative than that of other panels.

Overall, the results presented here suggest that the popular political discourse over ideological circuits is overblown: The aggregate statistics indicate that knowing panel and circuit ideologies, distinguishing between circuit-level effects, and controlling for lower court decision and case types increases the ability to predict case outcome only moderately over a model that simply posits that the appeals courts always make conservative decisions.

Above, I emphasize that the results presented here are consistent or inconsistent with different accounts of circuit differences—I do not claim that they unambiguously support or disprove these accounts. I have looked at patterns in the data, not at indicators of causal processes. While this helps researchers approach a better understanding of what happens in the federal appeals courts, the results have to be taken with a grain of salt. First, even though I use the term ideology in reference to panel and circuit composition, keep in mind that the ideology measures employed here are based on who was involved in appointing different judges, not on characteristics of the judges themselves. While intercircuit differences may be related to the degree to which circuit ideologies differ, they may also be partly due to the possibility that the ideology measures are more valid in some circuits than in others. In the analytical framework used here, though, it is impossible to distinguish between the validity of JCS scores and their impact on decisions. Second, while the coefficient for circuit medians does not indicate that these influence decision outcomes, this is no proof against a principal-agent account of appeals court decisions. The intracircuit variance of that variable is comparatively small, leading to a large standard error. Furthermore, more recent accounts of en banc decisions have pointed to specific conditions that lead to the influence of overall panel ideology (Bartels & Westerland 2009; Kastellec 2007); such a focus would go beyond this article. Third, this study includes only limited case-level controls. It is possible that different circuits deal with different sets of cases, brought by litigants with different goals, which in turn may lead to different decisionmaking patterns. This focus on judicial decisionmaking has been woefully understudied—at the appeals court and the Supreme Court level. Nevertheless, a detailed study of case selection and its influence on decision outcomes is beyond the confines of this article. Fourth, this study is confined to the Clinton years; possibly, ideological differences between circuits increased or decreased under President George W. Bush. That is something for future studies to decide. Fifth, I focus on panel decisions, not on the behavior of individual judges. I believe that this is the right focus to understand the differences between circuits that actually matter for litigants. Nevertheless, it may be the case that circuit differences are more pronounced when one looks at the behavior of individual judges.

Despite all caveats, however, this study shows two things very clearly: First, there are differences between circuits that one can characterize as ideological. Second, knowing who appointed the judges on a federal appeals panel, and which circuit the panel is located in, improves the ability to predict case outcomes only marginally. Political commentators should take a deep breath and relax.