Fear of Reversal as an Explanation of Lower Court Compliance

Abstract

Lower courts in the United States are generally responsive to specific precedents and trends in the decisionmaking of their judicial superiors. In this article, we ask why. We test one popular explanation—that compliance can be attributed to judges' fear of having their decisions reversed—through an analysis of search and seizure cases decided in the U.S. Courts of Appeals between 1961 and 1990. Since the Supreme Court cannot reverse a decision unless it agrees to review it, we ask whether circuit judges are more likely to decide as the Supreme Court would be expected to when they face cases that are otherwise more likely to be reviewed by the Court. Finding that they are not, we conclude that fear of reversal cannot account for widespread circuit court compliance in these cases, nor, presumably, more generally. More broadly, our findings point to the importance of factors apart from supervisors and the threat of sanctions in determining subordinates' compliance.

Like others who sit atop government hierarchies, U.S. Supreme Court justices would seem to have a problem: they must rely on subordinates to see that their policies take effect, but they have only a limited set of tools with which to induce compliance. Not surprisingly, scholars have found abundant evidence of evasive or even defiant behavior by their subordinates, lower court judges (e.g., Peltason 1961; Romans 1974). Yet there is considerably more evidence of judges' acting, in Songer, Segal, and Cameron's (1994) language, as faithful agents of their higher court principals. For the most part, lower court judges tend to follow specific higher court precedents, and their decisions generally track ideological trends in the higher court (Romans 1974; Baum 1980; Gruhl 1980; Johnson 1987; Songer 1987; Songer & Sheehan 1990; Songer & Haire 1992; Songer, Segal, & Cameron 1994; Benesh 2002).

It is far from obvious why lower court judges act this way. In this study, we attempt to determine whether widespread compliance can be attributed to lower court judges' aversion to having their decisions reversed by a higher court. To be clear: We do not ask whether fear of reversal has any effect. Rather, we ask whether its effect is strong enough and pervasive enough to explain substantial amounts of compliance.¹

Compliance is a difficult concept to pin down theoretically or empirically, and scholars have attempted to measure it in various ways. We use the term somewhat loosely to refer to decisionmaking behavior by the lower court that furthers, or at least does not undermine, the higher court's efforts to determine legal policy in its jurisdiction. Taking a broad view, two different types of behavior fit this description. One is the faithful application of existing higher court precedents; the other is deciding cases as the higher court would be expected to. These behaviors can diverge where the current members of the higher court are inclined to repudiate its precedents. But far more often they will lead to the same result, for what the higher court has done before will typically be the best guide to what it is likely to do next. For this reason, it is reasonable to believe that findings for one type of behavior can be generalized to the other. In this article, we focus on the latter type of behavior, comparing the actual decisions of U.S. courts of appeals judges to the decisions that the U.S. Supreme Court would be expected to make in their place. Our findings suggest that fear of reversal does not play a major role in this behavior.

The Puzzle

Cases often offer judges chances to shape public policy. Even when they do not, judges' sympathies may lie with one party or the other. In the many cases where the attitudes of lower court judges match those of the higher court majority, compliance is no mystery. But there are also many cases where the attitudes do not match. In these cases, why might lower court judges choose to weight the views of their superiors more heavily than their own?

Perhaps because measuring compliance has presented such an interesting and important challenge, scholars have written much more about the extent of compliance than about its causes. Still, it is possible to identify a number of possible explanations in the literature. It may be easiest to think of them as falling into three categories.

The first category emphasizes the potential reactions of the higher court—more specifically, the threat of reversal. As we discuss shortly, reversal can impose significant costs. The second group of explanations concerns the effects of compliance or noncompliance on the attitudes or behavior of actors other than higher court judges. For instance, Canon and Johnson point out that noncompliance can impair a “judge's reputation among other judges and attorneys” and even contribute to “a breakdown in the judicial system” (1999:36). In the third category, compliance is seen as a good in itself. Included here are explanations of compliance as resulting from judges'

1. 1 role conceptions—following precedents and/or anticipating a higher court are regarded as essential elements of the job of a lower court judge (e.g., Howard 1981);

2. 2 respect for the authority of the higher court—the higher court possesses some characteristics that entitle it to obedience (Petrick 1968; Baum 1977; Pacelle & Baum 1992); or

3. 3 efforts to produce legally accurate decisions—precedents (and perhaps the higher court's current leanings) are seen as data to be taken into account in determining the legally best answer (e.g., Johnson 1987).²

We cannot hope to test every possible explanation for compliance in a single article. Instead, we focus our attention on the first category, asking how well the fear of reversal explains compliance. We have two key reasons for concentrating on this class of explanations. First, in recent years reversal-based theories have been the most clearly articulated and forcefully presented and have received the most attention. Second, the role of reversal in lower court compliance has substantial implications for our understanding of hierarchical dynamics in the judicial system—especially the extent to which a higher court's control over policy depends on its supervisory capacity—and the motivations and influences at work in judges' decisionmaking. We discuss these implications in the final section of the article.

Fear of Reversal

Theory

Why might judges wish to avoid being reversed? Caminker (1994) points to two important reasons involving judges' self interest: “(1) fear that their professional audience, including colleagues, practitioners, and scholars, will disrespect their legal judgments or abilities; and (2) fear that a high reversal rate might reduce opportunities for professional recognition and advancement (including promotion to a higher court or appointment to judicial or other commissions)” (1994:77–8, footnotes omitted). The proposition that the desire to protect reputations, promotion prospects, and even current positions encourages compliance by lower court judges has been incorporated into empirical work as well (Elder 1987; Hansen, Johnson, & Unah 1995).

Concern for their careers and reputations is not the only reason judges might dislike being reversed. Another result of reversal—one that has received more attention in recent years—is that it impedes judges' efforts to shape policy. At best, it results in a specific holding contrary to their wishes. At worst, a doctrine the lower court dislikes becomes binding policy throughout the higher court's jurisdiction (the entire United States, for the Supreme Court). Accordingly, fear of reversal plays the central role in lower court compliance in models where judges are viewed as acting rationally to further their policy preferences (Cameron 1993; Songer, Segal, & Cameron 1994; McNollgast 1995).

Thus, the “core assumption” in McNollgast (1995) is that all relevant actors, including lower court judges, “act rationally to bring policy as close as possible to their own preferred outcome” (1995:1636). Judges' “compliance with legal doctrine results not from a preference for stability or an adherence to a norm; rather, it results from the Supreme Court's threat to enforce the doctrine. When the Supreme Court punishes lower courts, the punishment meted out is not a loss of prestige or reputation; rather, the punishment consists of the loss in utility to the lower courts from the substitution of the lower court's decision with the Supreme Court's ideal point” (McNollgast 1995:1681).

Although the specifics of Cameron's (1993) model are different, his ultimate position is essentially the same: “[t]he prospect of review, remote as it is, is sufficient to motivate most judges” (1993:30), even if reversal imposes no costs other than deviation from judges' ideal policy points. There are dissenting voices. Benesh (2002) has argued that the prospect of reversal is too remote to produce substantial compliance. Cross (n.d.) offers a full-scale theoretical challenge to Cameron and McNollgast. Yet in recent years it has become increasingly common for scholars to treat judges as strategic pursuers of policy preferences. As this view continues to gain adherents, invocation of the fear of reversal as an explanation for compliance is sure to become even more common. Before this explanation gains the status of accepted truth, it is essential that it be subjected to empirical tests.

Empirical Evidence

As of now, we have very little evidence on the extent to which judges' decisions are influenced by the desire to avoid reversal.³ For one thing, it is unclear how much judges really mind it. Some informed observers—on or off the bench—have claimed that judges dislike being reversed (Schick 1970:143–7; Howard 1981:140, footnote j; Satter 1990:227–35; Posner 1996:224). But the court of appeals judges surveyed by Howard (1981) claimed not to care much. Furthermore, they ranked the “anticipated response of the Supreme Court” well down in a list of influences on their decisions. The district judges interviewed by Kitchin (1978) responded similarly when asked about the anticipated responses of their courts of appeals.

Turning from judges' attitudes to their behavior, the story is much the same. Songer, Segal, and Cameron (1994) did find that circuit court decisions deviating from the Supreme Court's preferences were more likely than others to be appealed, “a necessary condition for monitoring [by the Supreme Court] to be effective” (1994:681). But as the authors recognized, rational behavior by litigants only allows the Supreme Court to monitor effectively; it does not ensure that it will. More fundamentally, their finding provides no grounds for concluding that circuit judges react to or even pay attention to the behavior of either the litigants or the Supreme Court.

Cross and Tiller (1998) and Van Winkle (1997) approached the issue a bit more directly. Cross and Tiller examined circuit court cases reviewing agency interpretations of statutes in the wake of the Supreme Court's Chevron decision, which is generally viewed as directing lower courts to accord substantial deference to agency interpretations (Chevron v. Natural Resources Defense Council 1984). They uncovered a “whistle-blower” effect in these cases: judges were more likely to defer to interpretations that ran contrary to their own ideological leanings if there was an ideological split on the panel. At first glance, this finding seems attributable to fear of reversal. Judges in the majority appear to realize that if they were to follow their own preferences, the other judge might dissent, signaling a possible violation to the Supreme Court. Nevertheless, the authors concluded that this interpretation is untenable. The cases they studied were decided between 1991 and 1995, a period when the Supreme Court was conservative in its leanings. Since liberal decisions had a smaller chance of surviving Supreme Court review, liberal judges should have been more constrained than conservative judges by the presence of a potential whistle-blower. But the authors found the opposite.

Van Winkle (1997) asked whether circuit judges sitting on panels seek to avoid being reversed by their own circuit en banc. He found that the likelihood that members of a circuit's ideological minority would vote consistently with their leanings increased with the number of ideological allies on the panel with them and the narrowness of the ideological divide on their circuit and that these judges were less likely than members of the circuit's ideological majority to dissent when they were outnumbered on a panel. Although these findings might be explained in other ways, it seems reasonable to interpret them as evidence that circuit judges try to keep from being reversed, in part by avoiding unwanted attention. However, it is not clear how strong the motivation or its effects are or whether they carry over to interactions with the Supreme Court.

Approach and Logic

In short, we simply do not know whether the fear of reversal is powerful enough to produce substantial compliance. Our purpose in this study is to test whether it is. To construct our test, we rely on four well-known facts and two assumptions that we believe are uncontroversial.

The facts are these: (1) the Supreme Court cannot reverse a decision unless it reviews it; (2) it does not review all circuit court decisions (in fact, it reviews very few); (3) its choices of what cases to review are not made randomly—certain kinds of cases are more likely than others to be reviewed; and (4) whether the Supreme Court disagrees with the lower court decision is only one of several factors that enter into its decision whether to review it. Disagreement is neither a necessary nor sufficient condition for review.

We assume that circuit judges are aware of the facts listed above and have a reasonably accurate sense of which types of cases are more likely to be reviewed. The first assumption is, we think, indisputable. It would be ludicrous to suggest that circuit judges were ignorant of such basic facts about the judicial system. The second seems nearly as obvious, unless it is interpreted to claim more than we mean. We are not suggesting that circuit judges read the empirical literature or that they take note of precisely the same indicators that social scientists do. We simply mean that they recognize certain characteristics that make a case a better candidate for review. For example, as we discuss later, the Supreme Court is more likely to take cases involving issues that are important or that have engendered intercircuit conflict. Is it possible that circuit judges are unaware of this? Surely not. Similarly, we are sure that they realize that a circuit court case's chance of being reviewed when it is one of 20,000 decided in the same year is considerably smaller than if it were one of 5,000. Thus, we do not assume that judges can measure precisely the probability that a case will be reviewed, but we do assume that they can make rough estimates and that these estimates will tend to be fairly accurate.

These facts and assumptions lead to a simple proposition that forms the basis of our test. If the threat of reversal plays a major role in circuit judges' choices whether to decide a case as they think the Supreme Court would, then circuit judges should be most likely to decide in a way that is consistent with the Court's preferences when the possibility of review is most real. To put this a bit differently, if we were to find that circuit judges deferred just as often in cases where the likelihood of review was low as in cases where it was high, we would find it hard to believe that fear of reversal was driving their behavior.

Thus, the key to our strategy is to distinguish cases that would strike circuit judges as reasonable candidates for Supreme Court review from those that seem less likely to gain its attention. In doing so, we must view each case from the perspective of judges about to decide it. (Since we are trying to understand the judges' choices, we cannot analyze factors that did not come into existence until their choices were already made.)

We focus on four broad case characteristics that circuit judges could reasonably be expected to observe by the time they reach their decision in the case. These are case importance, the presence of an intercircuit conflict, the Supreme Court's interest in a particular circuit, and court caseloads (which establish a baseline probability that any one circuit court case will be reviewed by the Supreme Court).

There are two ways in which we could investigate the effects of these characteristics on circuit court decisions. Each method has advantages and limitations, so we employ both. The first begins by estimating a model of Supreme Court review using our measures of the characteristics just described. Doing so allows us to confirm that they do, in fact, affect the likelihood of review and to compute an estimated probability of review for each case. The estimated probability then becomes the key independent variable in our first analysis of compliance.

The second method is simply to estimate the effects of the individual variables on circuit court/Supreme Court consistency. In a sense, this is a less satisfactory test of the fear of reversal. We give each variable the same weight, regardless of how much—if at all—it affects the Supreme Court's certiorari decisions, even though reversal-fearing judges should pay more attention to the more important factors. However, the judges do not have access to our statistical results, so the importance they ascribe to different variables may not match their importance to the Supreme Court. Analyzing the variables independently allows us to explore this possibility. It also provides the benefit of more detailed information for analysis.

Because the measurement of the case characteristics and the construction of the first model present some complications, we delay the discussion of them until we have introduced the dependent variable and control variables to be included in both analyses.

Data and Measures

Our data come from a sample of search and seizure cases from 1961 through 1990 collected by Songer, Segal, & Cameron (1994). We selected fifteen cases at random for each year, for a total of 450. Since not all variables were available for each case, the sample size in the final analyses was 440.

Dependent Variable

To begin, we needed an educated guess as to whether the Supreme Court would uphold each challenged search if the case were before it. Fortunately, Songer, Segal, and Cameron have done most of the work for us. Drawing on Segal's earlier work (1984, 1986), they constructed a model of Supreme Court search and seizure decisions based on fact patterns, e.g., where the search took place, how extensive it was, whether the searchers possessed a warrant. The model also took into account the growing conservatism of the Court from the beginning of the Burger years. Applying the coefficients from their model to the facts—including the decision year—of each circuit court case, we imagined that the search was before the Supreme Court and calculated the estimated probability that the Court would rule for the government, deeming the search valid. If the probability was greater than 0.5, we predicted that the Supreme Court would rule for the government; if it was less than 0.5, we predicted that it would rule for the defendant, invalidating the search. (The probability is not exactly 0.5 for any case.) The dependent variable took the value 1 if this predicted decision of the Supreme Court matched the actual decision of the court of appeals, 0 otherwise.

As previous findings would lead us to expect, the level of lower-court/higher-court consistency in our sample of cases was quite high. The actual decision of the court of appeals matched the decision that would be expected from the Supreme Court in 354 of 450 cases (78.7%).

Control Variables

In addition to the key explanatory variables to be described shortly, we included three control variables. The first two take account of circuit judges' ideology, an important influence on their decisionmaking (Goldman 1975; Howard 1981; Johnson 1987; Songer & Haire 1992; Songer, Segal, & Cameron 1994). Here its influence should be manifested through decisions that are less deferential, the less ideologically comfortable the judges are with the decision the Supreme Court is thought to prefer. We began by creating ideology scores for each panel, following the procedure set out by Songer, Segal, & Cameron (1994).⁴ The panel scores ranged from about −0.3 to about 0.96, with higher scores indicating greater conservatism. We then created one ideological comfort variable for searches the Supreme Court would be expected to uphold and one for searches it would be expected to reject. The first, CA Ideology-Uphold, was equal to the panel ideology score if the Supreme Court would be expected to uphold the search and equal to zero if it would not. Because higher scores indicate greater conservatism—and presumably greater willingness to uphold a search—the effect of this variable should be positive. The second, CA Ideology-Reject, was equal to −1 times the original ideology score (so that now liberal judges score higher) where the Supreme Court would be expected to invalidate the search and equal to zero if it would be expected to approve it. Its effect, too, should be positive.⁵

If circuit judges attempt to decide their cases as the Supreme Court would, they should be more likely to succeed where the Supreme Court's position is easier to identify. Whether or not they attempt to do so, their attitudes should be more likely to match those of the Supreme Court where the facts of the case are particularly one-sided. These two considerations provided the rationale for our third control variable, SC Predictability. We first subtracted 0.5 from the estimated probability that the Supreme Court would uphold the challenged search, then took the absolute value of this difference. The range of possible scores was 0 to 0.5, with higher values indicating a greater probability that circuit judges would correctly predict the Supreme Court's position or that, because of the details of the case, they would view it in the same way as the higher court. We expected this variable to have a positive effect on circuit court/Supreme Court consistency.

Key Independent Variables

Importance

For the discussion of our key independent variables, we begin with the things that make a case important—the breadth of its effects, the potential for a decision to bring clarity to the law, and so on. It is well established that the Supreme Court prefers to reserve its limited resources for such cases (Provine 1980; Caldeira and Wright 1988; Perry 1991). Because importance is hard to measure directly, we took an indirect approach, looking for evidence that someone other than the parties to a case considered it important, even before it had been decided by the court of appeals.

The first of our two measures of importance is whether the district court judge who first decided the case elected to publish an opinion. Publication is costly, in that it occupies space in the Federal Supplement, increases the reading load for lawyers and other judges, and requires extra effort from the publishing judge. For these reasons, it is also uncommon. District judges are encouraged to reserve publication for cases “of general precedential value” (Songer 1988:206), and they would not seem to have much incentive to do otherwise. Judges may not always follow this suggestion, and they may sometimes be overly selective. Yet after a careful review of research on unpublished opinions, Rowland and Carp (1996) were able to reach this conclusion:

Although many decisions that should be published are not, and a few that should not be published are, it is still fair to say that published opinions generally do represent an atypical population dominated by nonroutine cases that require the exercise of judicial judgment. (1996:119)

Given their conclusion, we think that district court publication is a valid indicator of case importance, and we expect the Supreme Court to grant certiorari more often in cases with published district court opinions than in those without them. If circuit court compliance is due to the fear of reversal, we should find more compliance in cases with published district court opinions.

The second measure of importance is whether the circuit court case is heard by a three-judge panel (0) or en banc (1). An en banc hearing, in which all of the circuit's active judges (or a substantial portion, in the Ninth Circuit) participate, is costly to the judges. It disrupts the circuit's division of labor and forces judges who may be based many miles apart to meet together. Accordingly, circuits engage in the practice only sparingly, reserving it for cases where it is critical that the circuit speak with full authority—either because an issue has generated conflict or because it is particularly important. Howard's (1981) and George and Solimine's (2001) findings that en banc cases were considerably more likely than panel cases to be reviewed by the Supreme Court enhanced our confidence in the en banc hearing as an indicator of importance.⁶

Conflict

The second characteristic is the presence of a legal issue that has generated intercircuit conflict. When circuits disagree on an issue of federal law, confusion, inefficiency, and inequities can result (see Haire & Lindquist 1997). As head of the federal judicial system, the Supreme Court is expected to resolve such conflicts. The Court's rules and members state that conflict is an important determinant of review (Perry 1991), and empirical evidence supports their statements (Ulmer 1984; Caldeira & Wright 1988; Perry 1991).

Our coding was based purely on the judges' reports. The variable was scored 1 if any opinion in the case identified an existing conflict and the court took a position on the contested issue, 0 otherwise. We did not second-guess the judges' claims of conflict or require that the conflicts involve search and seizure issues. If judges considered possible conflicts important enough to identify, it is reasonable to assume that they took them into account in assessing the likelihood of review.⁷

Baseline Probability of Review

Each case had a baseline probability of review that had nothing to do with any of the case's special characteristics aside from the time at which it was decided. This probability is a function of the Supreme Court's appetite for cases and, to a much greater extent, the number of cases that are decided around the same time in the courts of appeals (and so compete for space on the high court's docket). For instance, in a year when the Supreme Court chooses to hear 120 out of the 6,000 cases decided in the circuits, a case has a much higher baseline probability of review than one decided in a year when the Supreme Court hears 70 out of 20,000 cases.

We measured each component of the baseline separately. For the circuit court caseload, this was simply the number of cases decided in the courts of appeals at about the same time. Specifically, the measure was the number of cases terminated on the merits in all courts of appeals in the one-year period ending June 30 of the year in which the case at issue was decided.⁸ (For instance, for a case decided any time in 1989, the value for this variable was the number of merits terminations from July 1, 1988, to June 30, 1989.)

Circuit judges wishing to estimate the Supreme Court's appetite for cases are most likely to look to its behavior in the recent past. Our initial measure was the number of circuit court cases decided by the Supreme Court in the October Term (OT) leading into the year when the circuit court case of interest was decided. (For instance, for a 1980 circuit court case, we used the number of circuit cases decided by the Supreme Court in OT 1979, which ran from October 1979 to June 1980.) Because circuit judges might look back more than one year, we also tested measures going back two and three years. The two-year measure predicted Supreme Court review most successfully, and it is included in the models discussed below.

Supreme Court Oversight of a Particular Circuit

As with any supervisor, we might expect the Supreme Court to keep a particularly close watch on subordinates who have shown a tendency to deviate from its preferences. Evidence gathered by Lindquist, Haire, and Songer (2000) is consistent with this expectation.

Measuring a particular circuit's vulnerability to review is complicated. The key element is the recent treatment of the circuit's decisions by the Supreme Court. But simply counting the number of times a circuit has been reversed would seem inadequate, since it does not take caseloads into account. A circuit that decides five times as many cases as another should not feel threatened by the fact that it has twice as many cases reversed as the other circuit. On the other hand, simply dividing the number of reversals by the number of cases decided by the circuit would not be correct, either. This is because all circuits' reversal rates must have declined dramatically over time as their caseloads have grown. That variation across time is already measured by our fourth independent variable.

To measure variation across circuits, our solution was to (1) divide the total number of cases reversed or vacated by the Supreme Court (not counting memorandum decisions such as grant-vacate-remands) in a particular period (the one, two, or three terms leading into the current year) by the number of reversals or vacates suffered by a given circuit in that period, yielding the circuit's proportional share of reversals and vacates for the period; (2) divide the total number of cases decided on the merits by all circuits in the appropriate period by the number of cases decided by that circuit,⁹ yielding the circuit's proportional share of all reviewable cases for the period; and (3) calculate the ratio of the circuit's share of reversals and vacates to its share of all reviewable cases by dividing the result from Step (2) by the result from Step (1). A ratio above 1.0 indicates that the circuit suffered a disproportionate number of reversals in the period just past.

Again, we examined the Supreme Court's activity in the term leading into the year of the circuit court case, the two terms leading into it, and the three terms leading into it. The third measure predicted Supreme Court review most successfully, and it is included in the models discussed below. Descriptive information for this and the other independent variables is presented in Table 1.

Table 1. Summary of variables (and what they measure) with descriptive statistics. Expected effect is positive for all independent variables except CA cases. DC=district court

Before turning to our models, we want to be sure that our assumptions are clear. We do not assume that judges pay attention to all of the specific measures just described. In some cases, we assume only that they are aware of the broader phenomena reflected in these measures. For instance, we doubt that circuit judges usually notice whether or not a district court opinion has been published, but we suppose that they tend to notice the same aspects of the case that told the district judge whether it was important enough to publish. Further, we are sure that both our estimates and judges' estimates of the threat of review contain error. For any one pair of cases, reversal-averse judges could anticipate the Supreme Court in the case we thought was less likely to be reviewed and not do so in the case we thought was more likely to be reviewed. This could happen either because our predictions overlooked a special element of a case that made it more or less cert-worthy or because the judges deciding it made a mistake in their calculations. But across several hundred cases, if fear of reversal is really driving decisions, broad tendencies should emerge. Cases with a higher likelihood of review should more often be decided consistently with the Supreme Court's preferences.

First Model—The Likelihood of Review

Testing the first model required that we have—for each case, viewed from the perspective of circuit judges who had not yet decided it—an estimate of the probability that the Supreme Court would eventually choose to review the case. Deriving an estimate entailed several steps. First, for each of the 450 cases, we coded whether it was eventually reviewed by the Supreme Court (1) or not (0). We thus created the dependent variable for the second step, in which we used logit to estimate the influences of the seven measures of cert-worthiness just discussed. One control variable was added—whether the circuit court decision was consistent with the predicted Supreme Court decision.

Results are presented in Table 2. The control variable and four of the key independent variables—published district court opinion, en banc at the circuit court, conflict, and circuit court case volume—had highly significant effects. The Supreme Court's appetite for cases and recent tendency to reverse a particular circuit had no discernible influence on its current review decisions.

Table 2. Logit model of Supreme Court review (1=case reviewed by Supreme Court; 0=case not reviewed, either because not appealed or because certiorari denied)

Next, we dropped the two insignificant variables from the logit and ran it again. Using the coefficients from this model, we calculated logit scores for each case based on the actual values of the four key independent variables, with the value for the consistency variable set at zero (inconsistent) for all cases. Our decision to include the consistency variable (our ultimate dependent variable) in this equation and then set its value at zero when calculating scores was necessary—it allowed us to obtain a predicted probability of review for each case based only on the direct effects of the key independent variables, with the effects of the lower court decision excluded. But since it may be the source of some confusion, we explain it more fully in the Appendix.

Once we had predicted logit scores for each case, the next step was to transform the scores into probabilities.¹⁰ The result for each case was an estimate of the likelihood that the Supreme Court would agree to hear such a case if it went contrary to the decision that would be expected from the Supreme Court. This is the independent variable of interest in the first model. If fear of reversal was driving lower court deference to the Supreme Court, this variable should have had a positive effect on the likelihood that the actual court of appeals decision matched the predicted decision of the Supreme Court.

Results

First Analysis—Effect of the Likelihood of Review

We turn to logit again to analyze this effect. The results, shown in Table 3, are quite dramatic. While all three control variables behaved just as expected, the coefficient for the review variable, expected to be positive, was in fact decidedly negative. Cases with characteristics making them more likely to be reviewed by the Supreme Court were less likely to be decided consistently with its apparent preferences.

Table 3. Logit model of effect of case's probability of being reviewed by Supreme Court on circuit court-Supreme Court consistency

Although the coefficient was very nearly statistically significant by the traditional standard, the effect uncovered was not large, as became apparent when we computed estimated probabilities of consistent decisions. For instance, if the other variables were held at their medians, moving from a 0.1 probability of review to a 0.5 probability decreased the probability of a consistent decision by only 0.06 (standard deviation=0.03).¹¹ For this reason, we did not feel compelled to seek an explanation for the negative sign. The key point is that we did not find the effect we were looking for.

We recognize that judges, like other human beings, have “cognitive limitations that inhibit their ability to respond directly to stimuli whose complexity exceeds their computational capacity” (Rowland and Carp 1996:156). Perhaps our test requires too much of judges. Assessing the likelihood of review may simply be too hard, and circuit judges, while attempting to identify the cases where they need to proceed cautiously, may make mistakes. We can investigate this possibility by looking more closely at the probability of review.

Table 4 displays ranges for the predicted probability of Supreme Court review, given a noncompliant circuit court decision, along with the actual frequency of compliance in each range. It is most instructive to examine the extreme categories. Judges may well have had difficulty spotting the subtle distinctions between cases with a 0.35 likelihood of being reviewed and those with a 0.4 likelihood, but surely even cognitive misers would recognize that the kinds of cases falling in the 0.8–0.9 or 0.9–1.0 probability ranges (the last two rows) pose a far greater danger of reversal than those in the 0–0.1 range (first row). Did the disparity in danger affect judges' decisions in the predicted way? Obviously not. In the range of cases with the smallest threat of reversal, circuit judges were most likely to decide as the Supreme Court would be expected to—fifty out of fifty-two times. In the nine cases where the threat was greatest, they made four inconsistent decisions.

Table 4. Frequency of circuit court decisions consistent with and inconsistent with predicted decision of Supreme Court, by estimated probability of Supreme Court review for inconsistent decision

For a more rigorous version of this less demanding test, we can return to the full model. Instead of the original measure of the likelihood of review, we included a dichotomous measure, scoring cases at 0 if they had a predicted probability of review of 0.2 or less and at 1 if their predicted probability of review was 0.5 or higher. We omitted cases with predicted probabilities between 0.2 and 0.5. In other words, we had three categories of cases: one where the probability of review was quite low (trivial threat), one where it was even or better (serious threat, assuming judges are risk-averse), and one, excluded, where the threat was harder to judge. When we tested this model, the result for the threat of review was again fairly strongly in the wrong direction: coefficient=−0.64; standard error=0.35, p=0.065, N=286. Analyses using different cutoff scores and exclusions produced essentially the same results.

The next analysis, in which we consider the measures of cert-worthiness separately, may shed some light on this strange result.

Second Analysis—Individual Variables

Table 5 shows logit estimates for this model. Table 6 lists the predicted change in the probability of a consistent decision, moving from the minimum to the maximum value of the listed variable (from 0 to 1 for the dummy variables), with all others held at their medians. Leaving aside the control variables, one coefficient was both statistically significant and in the expected direction. Circuits that had been reversed particularly often in the preceding two years were more likely than others to defer to the Supreme Court. As Table 6 shows, other things being equal, a circuit with the least favorable two-year Supreme Court reversal score in the entire period of this study would be about 17% less likely to reach a decision inconsistent with the Court's preferences than the circuit receiving the gentlest treatment. Of course, this comparison is quite unrealistic, being drawn from the extreme ends of more than two hundred scores. To take a more realistic but still sizable interval, the difference in predicted probabilities for a court at the 20th percentile on this variable and one at the 80th percentile was about 0.04 (standard deviation=0.02).

Table 5. Logit model of circuit court-Supreme Court consistency, all cert-related variables included

Table 6. Estimated change in probability of consistent decision, moving from minimum to maximum value of listed variable, all other variables held at their medians. Estimates were generated through the CLARIFY program (Tomz, Wittenberg, & King 1998), using STATA 6.0

This finding does count as evidence that compliance stems from a desire to avoid reversal, but it is fairly weak evidence and it stands alone. The presence of conflict has about as strong an effect on consistency, but it is negative. The negative effect of an en banc hearing is far stronger still. As for the other three variables, one—the number of circuit cases reviewed by the Supreme Court in recent years—is incorrectly signed, though insignificant; the other two have no effect at all.¹²

Discussion

We believe these results tell a rather straightforward story: the substantial congruence between circuit court decisions and Supreme Court preferences in the search and seizure cases analyzed here does not arise from circuit judges' fear of having their decisions reversed. Estimating a prior likelihood of Supreme Court review for each case and analyzing its effects, we obtained results directly contrary to what we should have found if fear of reversal were the prime driving force behind circuit judges' decisions. Instead of acting more cautiously in the cases that seemed to have a better chance of reaching the Supreme Court, the judges were actually less likely to decide these cases as the Supreme Court would be expected to.

When we analyzed the individual measures of cert-worthiness, the results told essentially the same story. Only one measure actually worked as it should, and the unexpected effects (for conflict and en banc) were considerably stronger. Most tellingly, the variable that performed well was one of only three that failed to predict Supreme Court review, while the two strongest predictors of Supreme Court review were the worst performers in the model of circuit court decisionmaking. We concede that circuit judges' assessments of certain factors' impact may not match our estimates perfectly, but it strains credulity to imagine that they give the least weight to the factors that are objectively most important and the most weight to those that are least important.

Interpretations

Because our findings run counter to a widely held view, it is important that we consider how they might be explained and what the explanations might suggest about the force of our conclusions. We can think of two possible ways to account for the weak or negative effects of en banc hearings, conflict, and district court publication. First, all three appear more often in cases involving unsettled issues of law than in routine cases. Judges may find it hard to predict how the Supreme Court would decide these cases. Second, district court publication, en bancs, and, to a lesser extent, conflicts, are more likely to be present in cases with substantial policy implications. In such cases, judges may be particularly reluctant to cede their policymaking power to the higher court.

Thus, it is logically possible that fear of reversal generally exerts a strong influence on lower court decisionmaking but is canceled out or even, in the case of conflict and en bancs, overwhelmed by judges' uncertainty about the Supreme Court's preferences or their desire to shape policy. On examination, though, this possibility does not appear credible.

We address the point about uncertainty first. One reason to doubt that uncertainty about the Supreme Court's preferences can adequately explain our findings is that many legal questions are unresolved only because they have not arisen previously; once they come up, it may be reasonably easy to predict how the Supreme Court would decide them. This can even be true of issues that engender conflict. The circuits may split because of ideological or other differences, even where the Supreme Court's position is discernible.

Second, even assuming that the Supreme Court's views on a particular doctrinal issue are difficult to read, we must avoid confounding uncertainty as to the Supreme Court's resolution of a particular legal issue and uncertainty as to its ruling on a search. As Segal (1984) wrote, in his original fact-pattern analysis of Supreme Court decisions.

The opinion delivered in any given decision more often than not relies upon a single aspect of the case, for example, was there probable cause, or is a search several hours after a lawful arrest reasonable? Yet the decision itself is dependent upon many factors—presumably those mentioned above—that go largely undiscussed in the main body of the opinion. (1984:900)

The fact that Segal's model, incorporating only a handful of facts and ignoring the precise legal issues in the cases, was able to account for three-quarters of the Supreme Court decisions he analyzed demonstrates that the Supreme Court's decision whether to uphold a search is typically not very hard to predict, even in the tough cases that it chooses to hear.

Finally, recognizing that the fact-pattern model will not always yield a clear prediction for the Supreme Court, we included a variable measuring the strength of the prediction. We thus controlled for uncertainty about the Supreme Court's stance statistically.

Similarly, it is difficult to accept the notion that policy goals completely dominated an otherwise powerful desire to avoid reversal in the most cert-worthy cases. Note that in cases where the law is unsettled, circuit judges risk triggering an unfavorable precedent binding the whole nation, whereas in cases where the law is settled, the worst that can happen is an undesirable result in that case. Thus, while there is more of a chance to make law in the first type of case, there is also a much greater incentive to avoid being reversed.

Furthermore, judges can attempt to evade reversal while moving policy closer to their preferred points by ruling on a particular legal issue as they prefer to but, on the basis of some other issue, deciding the case as they think the Supreme Court would. This is not always possible, but cases often present enough legal questions to allow it. For these reasons, it is far from clear that the circuits' decisions should deviate from the Supreme Court's more often in cases involving unsettled issues than in others.

Finally, lack of certainty and policy goals can only help account for the findings as to conflict, en bancs, and district court publication. It is especially hard to see why, if fear of reversal were an important influence, we would find no effect for changes in circuit court caseloads. The circuits together decided more than seven times as many cases in 1990 (19,322) as in 1961 (2,681). Circuit judges could hardly have failed to notice the growth in caseloads, and they must have recognized the greater opportunity this gave them in later years to deviate from the Supreme Court's preferences while escaping review. Indeed, a positive relationship between caseloads and noncompliance is an explicit prediction of McNollgast's (1995:1651) model. Yet our best estimate is that, in the average case, judges in 1990 were only 2% less likely than those in 1961 to anticipate the Supreme Court, and the standard error of that estimate is far too high to let us conclude confidently that the increasing caseloads affected judges at all.

In sum, while we believe that legal uncertainty and policy goals can help explain our findings, these explanations do not change our basic conclusion: fear of reversal was not a major influence on judges' decisions in the cases examined here. This is not to deny that the fear of reversal had any influence at all. Far less do we mean to suggest that it never affects judges' behavior. We strongly suspect that it does have an impact on some decisions and occasionally on other aspects of their behavior (such as the choice of language in their opinions). But the goal of this research was not to discover whether the desire to avoid reversal has any influence; rather, it was to determine how well that desire could account for the high rates of congruence between circuit court decisions and Supreme Court preferences. For search and seizure cases, at least, the answer seems clear: not well at all. Faithful agency on the part of circuit judges appears to flow from other sources.

Implications

There is nothing new about the proposition that fear of reversal plays a major, if not dominant role, in lower court decisions to defer to higher courts, but in recent years especially it has become common for scholars to invoke this proposition as an explicit premise (Songer, Segal, & Cameron 1994; McNollgast 1995; Cross & Tiller 1998; Brent 1999; Spitzer & Talley 2000; Brace & Langer 2001). Whether or not one accepts this proposition has crucial implications for one's understanding of judicial hierarchy. Because reversal is only a potential, not inevitable, consequence of a noncompliant decision, if the desire to avoid it were a major force behind compliance, then deference to a higher court would vary with the threat of reversal: other things remaining equal, compliance would be more likely where the probability that a noncompliant decision would be reversed was higher. In such a situation, a higher court's control over the development and application of the law in its jurisdiction would depend largely or entirely on its ability to maintain a credible threat of reversal.¹³

Although we do not claim that our evidence conclusively disproves this proposition, we feel that it presents a serious challenge to it. Most immediately, the evidence suggests that higher courts can plausibly hope for substantial—though far from perfect—compliance and responsiveness even where the threat of reversal is weak. Since we are very unlikely to see either sustained decreases in lower court caseloads or increases in the work capacity of higher courts in coming years, this finding should be comforting both to higher court judges and to anyone who thinks it desirable for those courts to maintain some control over legal doctrine in their jurisdictions.

Our findings also have implications for how we understand judges' motivations and decisions. Here, our discussion is necessarily more speculative, but speculation is justified by the importance of the issues.

We will return to the main story of this article shortly. But first, it is important to emphasize two points that we have largely overlooked to this point: As in most studies, while we observed widespread compliance, we also found a substantial number of noncompliant decisions in our data set; and as in most studies, the measure of circuit judges' ideology was significantly related to their voting behavior. Our focus on compliance should not obscure what we take to be an incontrovertible finding of the empirical literature—that policy preferences exert a powerful influence on circuit judges' decisions. It is true that circuit judges are not Supreme Court justices; they encounter a different mix of cases, can more reasonably hope for advancement to higher positions, and must answer to a higher court. But the differences and their consequences should not be exaggerated. Advancement to a higher court is possible, but chances are very slim. Most cases will involve issues clearly covered by higher court precedents, but many will not. There is always the threat of reversal, but the threat will often be quite remote and, as we have argued here, may have only a limited effect. In other words, circuit judges will often operate in conditions similar to those of Supreme Court justices, and it would be odd if a factor that so greatly influences Supreme Court decisions had no effect in the circuits.

Ideology—more precisely, shared policy preferences among judges—could even help explain the observed congruence between actual circuit court decisions and predicted Supreme Court decisions. Circuit judges might decide as the Supreme Court would because their ideological leanings are similar. Ideological agreement must surely account for some decisional congruence. Yet considering the ideological divisiveness of search and seizure issues, the wide variation in ideology evident among judges, and the high levels of congruence found here and in other studies, we do not believe ideological agreement can explain all of the congruence. This doubt is reinforced by the fact that the findings of decisional congruence and responsiveness hold even when researchers control—admittedly imperfectly—for judges' ideology. (For the cases examined here, see Songer, Segal, & Cameron 1994).

Thus, while we believe that our study demonstrates the influence of ideology, we think it provides equally strong evidence that something other than ideology is at play in judges' decisions. One might reasonably ask, “What else?” A number of answers are possible, but we would offer two.

The first will probably seem less familiar but still deserves to be taken seriously. Over time, as the caseloads of circuit courts have become increasingly burdensome, judges have adopted a number of formal and informal shortcuts to cope with them (Posner 1996; Richman & Reynolds 1996). Instead of investing the time needed to weigh every detail of each case before them and consider the legal merits of every issue anew, judges might limit their attention to key facts or patterns and simply ask themselves, based on what they have observed in the past, how the case would normally be decided in the federal courts. In fact, it would be very surprising if judges did not do this often; the ability to simplify a judge's work is one of the arguments in favor of stare decisis. Since what has been done in the past is surely a good predictor of what the Supreme Court would do in the future, the desire to save time may promote decisional congruence even if judges do not care about congruence itself.

Another possibility strikes us as especially likely—that congruence flows from lower court judges' attempts to reach legally sound decisions. This is almost certainly the explanation most judges would offer, and there is reason for scholars to take it seriously. Trained as lawyers and facing an audience trained in the same way, judges might place a high value on making decisions in accordance with accepted principles of reasoning, whether they derive satisfaction from doing so, think it ethically right for them to do so, or desire the good opinion of their colleagues and audience.¹⁴ Since faithful adherence to a higher court's precedents is a central element of legally sound decisionmaking, the desire to make sound decisions should tend to produce congruence in cases involving settled issues of law—that is, in typical cases. Less obvious, the desire to make legally sound decisions should tend to have the same effect even in less routine cases, at least where the Supreme Court does not deviate sharply from its own precedents. It will sometimes be reasonably clear how existing doctrine should be extended to new issues or circumstances. And even where existing precedents are not sure guides, if judges and justices make sincere attempts to weigh the legal strength of opposing positions, the fact that they received similar legal training and interpret the same sources of law should more often than not lead them to come down on the same side.

We repeat that the foregoing is speculation. Our data do not allow us to draw any firm inferences about the forces that do foster compliance. Nor is it clear to us how one could identify these forces.¹⁵

It is also important to be cautious in generalizing from our findings. They might not hold for other time periods or other areas of law. In the period studied here, search and seizure law underwent a good deal of doctrinal change. Circuit judges might have had trouble charting these changes or predicting outcomes. This possibility strikes us as unlikely, given Segal's (1984) finding of fact-pattern coherence across Supreme Court decisions and the nearly 80% congruence rate in the cases we examined. Further, it is easier to picture variation in rates of compliance across time and issues than in causes of compliance. Still, the possibility cannot be denied.

A more likely possibility is that results would be different for different courts. The threat of reversal may carry little weight with circuit judges only because it is so weak. Judges whose decisions are more often reviewed, such as appellate judges in some states and, especially, trial judges in virtually every court system, might take the threat more seriously.

These caveats aside, we believe that our results entitle us to draw two important conclusions: Lower court compliance can flow from sources other than the fear of reversal; consequently, higher courts may be able to enjoy substantially faithful, responsive behavior from their subordinates even without maintaining a credible threat of reversal.