The Chief Justice versus the iconoclast: Popular constitutionalism and support for using “sociological gobbledygook” in legal decisions

Abstract

Conventional wisdom assumes that the public wants judges that will simply interpret and apply the law as it is written. However, existing evidence shows a substantial portion of the American population supports the doctrine of popular constitutionalism. Using two experiments involving the use of social science in legal decisions, we show that popular constitutionalists evaluate the judiciary using a different set of criteria than legal traditionalists. For legal traditionalists, using social science in legal decisions is perceived as an undesirable nonlegal influence and reduces acceptance of a court decision. For popular constitutionalists, social science is perceived as objective evidence that can be used to understand the practical effects of a decision and increases acceptance. We conclude by discussing the need for more research on popular constitutionalists, as little is known about how this group evaluates the judiciary and interprets its actions.

INTRODUCTION

During oral arguments for the recent gerrymandering case Gill v Whitford, Chief Justice John Roberts expressed a concern that if the Court relied on “sociological gobbledygook” (Transcript of oral arguments, p. 40) to settle partisan gerrymandering cases it would “cause very serious harm to the status and integrity of the decisions of this court” (Transcript of oral arguments, pp. 37–38) by making the Court appear political. The public would not understand the social science data and formulas used to decide a case, and thus would assume the decision was driven by partisan politics. While the Chief Justice worries that using social science will harm the legitimacy of the Court, iconoclast jurist Richard Posner argues in favor of using evidence from social science in judicial decisions and, importantly, that the public will respond positively to its use. In his doctrine of legal pragmatism, judges base their decisions on “facts and consequences rather than on conceptualisms and generalities” (Posner, 1999, p. 227), and social science provides useful evidence concerning those consequences. In addition, Posner argues that the American people are an especially pragmatic people who desire judges concerned more with the consequences of their decisions than the legal theories justifying them, and “(i)f pragmatic reasonableness is what people want of the judiciary, why is it illegitimate for the judiciary to give it to them?” (Posner, 2003a, p. 350). While Chief Justice Roberts and Posner may be advancing their arguments in order to support their preferred case outcomes, they make competing and empirically testable claims regarding how judicial decisionmaking will affect the legitimacy of the courts.

Unfortunately, because of the lack of empirical research on the public's reaction to judges basing their decisions on social science, these two respected jurists must rely on intuition rather than empirical evidence to support their claims. The studies presented in this article examine whether it is the iconoclast or the Chief Justice who has a better sense of the American public. However, in the end, we cannot offer a simple answer because just as elites are divided on the question of what should influence judicial decisionmaking, so too are the American masses. Our two experiments will show that among legal traditionalists who desire a more formalist court that interprets the law as written using traditional legal theories like original intent, the Chief Justice is correct: the use of social science evidence undermines the acceptance of decisions. However, not all Americans hold the same normative preferences for how judges should make decisions (Bonneau et al., 2016; Gibson, 2012; Gibson & Caldeira, 2009; Parker & Woodson, 2020; Scheb & Lyons, 2001). Among those who reject traditional legalistic values and instead support the doctrine of popular constitutionalism, it is the iconoclast who is correct: the use of social science evidence increases acceptance of decisions.

While the doctrines of legal pragmatism and popular constitutionalism are not intertwined in legal scholarship, they are related in the minds of the American people, with attitudes toward both doctrines loading onto a single factor in Scheb and Lyons (2000). Our results will reinforce this by showing that the same iconoclasts who reject the legal orthodoxy in favor of popular constitutionalism also support legal pragmatism, perhaps reflecting that both doctrines argue for a more flexible interpretation of the Constitution. Importantly, it appears that this group of iconoclasts make up a substantial portion of the population. Scheb and Lyons (2001) found using a nationally representative survey that 68% of the American population wants majority public opinion to have at least some impact on the decisions of the U.S. Supreme Court.¹ Multiple experiments show a manipulation that varies whether the Court is influenced by public opinion or uses purely legalistic arguments has no aggregate effect on its public support (Bonneau et al., 2016; Farganis, 2012). Those two strands of evidence showing a widespread desire for a representative Court that follows public opinion are reinforced by the high levels of support for judicial reforms meant to bring the Court more in line with public attitudes. Golde (2019) notes that 72% of the population supports term limits for U.S. Supreme Court justices and 43% support the more extreme option of court-packing. When moving to the state courts, the vast majority of Americans support electing judges (Anderson, 2003; Geyh, 2003), with ~75% supporting elections according to nationally representative polls from 2001 to 2008 (American Justice Partnership, 2008; Justice at Stake, 2001). Taken as a whole, the evidence clearly shows the American people are hungry for a more representative judiciary. Gibson in his study on the effect of elections on state high courts came to a similar conclusion and stated that while high and low knowledge individuals may differ in their desired form of representation, “[a]ll agree … that in one form or another, judges should reflect the views of their constituents” (Gibson, 2012, p. 102).

Previous research on normative preferences has shown that a disjuncture between individuals' normative preferences and their actual perceptions of a court's decision-making process causes evaluations of the court to become more negative (Bonneau et al., 2016; Gibson & Caldeira, 2009; Scheb & Lyons, 2001). This research proves that normative preferences matter, but understates their impact. Supporting popular constitutionalism involves a more fundamental restructuring of a person's attitudes toward the Court than simply moderating whether being influenced by public opinion increases or decreases the Court's perceived legitimacy. Supporting popular constitutionalism affects the interpretation of virtually every action taken by the Court, even those like using social science in legal decisions that are not directly related to the core concepts of popular constitutionalism. This broad effect occurs because much of what appears as a spasm of politicization to a legal traditionalist morphs into a blossoming of democracy in the eyes of a popular constitutionalist. That shift in interpretation is why dissents decrease legitimacy perceptions for legal traditionalists while increasing them for popular constitutionalists, according to an experiment by Parker and Woodson (2020).

In the current paper, we expand upon the understanding of how this group of iconoclasts who reject the legal orthodoxy in favor of popular constitutionalism interprets the actions of the judiciary, and how those interpretations affect their evaluations of it. Because of the almost complete lack of research into the psychology of popular constitutionalists, virtually nothing is known about the criteria they use to evaluate the Court (but see Parker & Woodson, 2020). We will show—using an experiment involving the use of social science—that those criteria are different than the criteria used by legal traditionalists, and offer an initial glimpse into the previously-opaque minds of popular constitutionalists who make up a substantial and perhaps even a majority of the American population (Gibson & Caldeira, 2009; Scheb & Lyons, 2001).

SOCIAL SCIENCE AND THE LAW

The Supreme Court has long looked outside the law and turned to expertise in the various social sciences including economics, psychology and sociology to reach their conclusions. The Court has relied on sociological data in Muller v. Oregon (1908) to uphold labor protections for women and has turned to research in psychology in cases like Brown v. Board of Education (1954), Atkins v. Virginia (2002), and Roper v. Simmons (2005). In the area of criminal procedure the Court's reliance on social science has been increasing from 2001 to 2015, (Meitl et al., 2020), with judges appointed by Democrats and those appointed more recently using social science to a greater extent (Meitl, 2020).

This practice of reaching beyond the law to inform majority opinions is not without controversy. Textualist judges and scholars tend to reject the practice of departing from the text of the Constitution to achieve an outcome. Originalists argue that judges should not depart from the text of the Constitution, and should only engage in analyses that help determine what the words meant at the time they were written. For example, in his concurring opinion in Missouri v. Jenkins (1995) Justice Clarence Thomas argued that the Court erred in Brown v. Board of Education (1954) by basing its decision on psychological factors that are “irrelevant to the question whether state actors have engaged in intentional discrimination.” These criticisms suggest that it is inappropriate to use social science findings that are constantly in flux to evaluate laws that remain textually consistent.

However, a number of scholars have argued that there is an important place for social science in the legal system. Social science can be used to address questions of fact, or the objective reality surrounding a case (Faigman, 1989). This is important because legal precedents and remedies often rely on theories about how changes in the law might influence subsequent human behavior. For example, in Elkins v. United States (1960) and Mapp v. Ohio (1961) the Supreme Court expanded the application of the exclusionary rule under the assumption that if illegally seized evidence was deemed inadmissible in trials then there would be an incentive for police officers to respect the protections of the Fourth Amendment so that the criminals they catch do not go free.

In his legal pragmatism approach to the law, Posner proposes a doctrine in which the use of empirical evidence and social science is an integral and central component. While many people may associate Posner most closely with his economic approach to the law in which he applies behavioral economics and economic theory to legal rules (Posner, 2003b), his overall doctrine of legal pragmatism does not limit the utility of social science to economics alone, but includes any scientific evidence that can be used determine how legal rules and decisions will affect the future behavior of individuals and groups. He describes his vision of pragmatic adjudication as basing decisions on “facts and consequences rather than on conceptualisms and generalities” (Posner, 1999, p. 227). Legal pragmatism rejects the use of moral, political or abstract legal theories such as fairness, equality and original intent as a guide to judicial decisionmaking, arguing that the while judges may “claim to be engaged in the neutral scientific activity of matching facts to the law” they are actually engaging “in a basically political activity of formulating and applying public policy called law” (Posner, 2003a, p. 46). Instead, he calls for judges to aim “at the decision that is most reasonable, all things considered, where ‘all things’ includes both case specific and systematic consequences” (Posner, 2003a, p. 13).

Social science's role—whether through economic theory, psychology experiments, or sociological field studies—is to provide evidence about those consequences. Legal pragmatism does not mean simply deciding cases based on the what produces the best outcome, but it does mean issuing rulings based on contextual judgments of the law combined with empirical evidence regarding the efficacy of government actions (Dorf, 1999). Posner argues that this approach is preferable to deciding cases based on constitutional theory, because constitutional theory “lacks the agreement-coercing power of the best natural and social science.” (Posner, 1998, p. 3). According to this view, not only will it result in better decisions, there will also be greater public acceptance of judicial decisions that are guided by empirical data rather than abstract constitutional theories.

SOCIAL SCIENCE AND ACCEPTANCE OF COURT DECISIONS

Though legal scholars have devoted significant attention to the normative value of social science research to the law, there is very little research on the question of how the use of social science affects attitudes regarding the courts or their decisions. To our knowledge, Redding and Reppucci (1999) is the only study that examines whether people approve of the use of social science in legal decisions. In a sample of only judges and law school students, they found that those with a more liberal and expansionist view of judicial interpretation had a more positive reaction to using social science when interpreting the law. Because that analysis was only briefly mentioned in an article focusing on a different topic, it is unclear precisely what the authors meant by a more liberal and expansionist view of judicial interpretation or how it was measured, but their evidence suggests that a person's normative preferences for how judges should interpret and apply the law affects how people feel about the use of social science in the law.

In this study, we concentrate on the public's acceptance of court decisions—that is, whether people accept a court decision and move on, or whether they want to challenge it and get it overturned. The conventional wisdom—as stated by the Chief Justice—is that using nonlegal arguments based in social science as the basis of disagreeable court decisions will be perceived by the public as “a bunch of baloney…that is going to cause very serious harm to the status and integrity of the decisions of this Court in the eyes of the country” (Transcript of oral arguments for Gill v. Whitford, pp. 37–38), and the existing literature provides some evidence for his view. Legitimacy perceptions are the main determinant of acceptance judgments (Gibson, 1989, 1991; Gibson et al., 2005, 2014), and multiple studies show that other types of nonlegalistic influences on judicial decisionmaking can lead to a decrease in public support for the court (Baird & Gangl, 2006; Benesh, 2006; Scheb & Lyons, 2000; Tyler, 2006), which will subsequently lead to a decrease in acceptance.

However, this conventional wisdom does not apply uniformly to the entire public because not everyone desires a court that applies the law mechanically without any use of discretion (Bonneau et al., 2016; Gibson, 2012; Gibson & Caldeira, 2009, 2011; Scheb & Lyons, 2001). Some prefer that rather than just applying the law mechanically, judges should take into account public opinion (Parker & Woodson, 2020). According to the doctrine of popular constitutionalism, the courts are viewed as another political institution, and thus they should similarly reflect the public will (Friedman, 2003; Kramer, 2004; Kramer & Tribe, 2001; Pozen, 2010). This does not mean that the courts should not have the power of judicial review, or should always support the majority public opinion at a specific time. Rather, precedent and legal rules should evolve with society so that they are consistently updated in accordance with real and enduring changes in the popular will. Those who do not view the law as unchanging and immutable, or do not see the Constitution as a “dead” document, should not be expected to react negatively to nonlegal influences in court decisions. Indeed, using social science research to make sure that legal standards best meet the current needs of society could be seen as a feature, not a bug, in terms of evaluating institutional legitimacy.

For the iconoclasts like Richard Posner who support legal pragmatism, the use of social science is a desired feature in judicial decisionmaking because the empirical evidence provides a foundation on which to rationally base a legal decision, and provides the best evidence available to determine the consequences of that decision (Posner, 1998, 1999, 2003a). Posner does not argue for popular constitutionalism, but there is reason to believe that adherents to popular constitutionalism would be more accepting of legal pragmatism than would legal traditionalists. Both schools of thought argue for a more flexible interpretation of the Constitution than do legal traditionalists, and are more accepting of judges using nonlegal information (whether it is public opinion or social science) to guide their decisions. Posner also establishes a link between his doctrine of legal pragmatism and popular constitutionalism by recognizing the importance of acknowledging public preferences. He stated that “judges in a democratic society must accord considerable respect to the deeply held beliefs and preferences of the democratic majority when making new law” (Posner, 1999, p. 251). The two ideas are also connected in the minds of the public, as attitudes toward popular constitutionalism are associated with attitudes toward legal pragmatism. While using actual decision-making perceptions rather than normative preferences, Scheb and Lyons (2000) show in a factor analysis that an item measuring whether judges do “what is good for the public” loads onto the same factor as one that measures whether judges do “what the majority of the public favors.” Importantly, this factor was separate from one measuring the degree of ideological influence and another that measured legal influence.

Supporting popular constitutionalism relates to approval of the judiciary using social science, not necessarily because people believe it will lead to a more representative court, but because support for popular constitutionalism is associated with a fundamental restructuring of a person's attitudes and orientation toward the judiciary. That restructuring should affect not only views concerning the accountability of the judiciary to the public, but also attitudes seemingly unrelated to the versions of popular constitutionalism espoused by legal elites. Among those attitudes is support for the doctrine of legal pragmatism. As a result, those who support popular constitutionalism should also tend to support the use of social science in judicial decisions. The lack of connection between the two doctrines in most normative debates occurring within the confines of academia does not mean they are not associated within mass public opinion. How the doctrine of popular constitutionalism manifests itself in the minds of the public will be dramatically different than how it is articulated in the theories of legal elites, and it is this public manifestation of popular constitutionalism that we examine in this paper.

Our theory argues that normative preferences change the criteria used to evaluate the judiciary, and Gibson and Caldeira (2009) provide evidence supporting this claim. The study found that exposure to political advertisements concerning the Samuel Alito confirmation hearings decreases support for the Court, but only among those who had legalistic normative preferences. Two other studies have expanded upon that to show that specifically attitudes toward popular constitutionalism affect the criteria used to evaluate the judiciary. Parker and Woodson (2020) found that dissents had a negative effect on both legitimacy and acceptance judgments, but only for those who espoused traditional legalistic views. For supporters of popular constitutionalism, the presence of a dissent increased legitimacy perceptions and had no effect on acceptance judgments. Woodson (2018) found that a person's view about whether or not the Court was actually influenced by public opinion when making the NFIB v Sebelius (2012) decision changed the effect of displeasing decisions on legitimacy perceptions. For those who thought the Court was influenced by public opinion, a displeasing decision had no effect on the Court's legitimacy. While a displeasing decision increases legitimacy among those who thought the Court used a legalistic process and decreases it among those who thought it used an ideological process. These studies show that reactions to Supreme Court decisions are highly variable based on a person's preferences and perceptions regarding how the judiciary should decide cases. The results from our current study will build upon them to show once again that supporting popular constitutionalism causes “a change in a person's overall orientation toward an institution, which changes the criteria used to evaluate the institution” (Woodson, 2018, p. 77).

HYPOTHESES

Our overall hypothesis, which will be tested using two separate experiments, is that support for popular constitutionalism mediates the effect of using social science in judicial decisions on the public's level of acceptance for court decisions. For those who support traditional legalistic values, the use of social science violates the norm of judges doing nothing more than interpreting law to the best of their ability. For this group, evidence from social science is interpreted as an illegitimate nonlegal influence that decreases their level of acceptance for the decisions (H1). While popular constitutionalists are not necessarily adherents to Posner's legal pragmatism, they should prefer judges that incorporate a broader range of information into their decisions. For those who support popular constitutionalism, social science is an important piece of evidence that can be used to understand the effect of judicial decisions, and rather than undermining the decision, it provides both further evidence of its wisdom and objective unbiased evidence on which to base the decision, causing an increase in acceptance (H2). In addition, we hypothesize that support for legal pragmatism is more highly associated with support for popular constitutionalism than any other factor of normative preferences (H3).²

STUDY DESIGN

We use two separate studies to examine our overall hypothesis. Each study has a similar design. For both studies, we expose the subjects to either a decision using legalistic reasoning or a decision using a consequentialist analysis supported by social science research. The major difference between them is the type of social science evidence. For Study 1, we use empirical social science studies that examine the effect of a law—in this case “stop and frisk” policies—on criminal activity. In Study 2, we use the behavioral economics approach to the law that relies on theory rather than empirical data. The variable measuring the experimental manipulation is coded as a 1 for the decision using consequentialist reasoning involving social science, and 0 for the legalistic reasoning condition.

Both studies begin by asking the subjects for their position on a political issue. Next, the survey assesses a variety of attitudes including the subjects' knowledge of the court, normative decision-making preferences, evaluation of science, partisanship, and demographic variables. Following this, the subjects read the experimental manipulation, which included a description of the court's decision.³ Importantly, the Court's decision always went against the subject's stated preference from the beginning of the study. Research shows that people virtually always accept decisions with which they agree because acceptance is about lacking the desire to challenge and overturn a court decision. The same motivating reasoning processes that cause people to accept without question information that agrees with their prior viewpoint and strongly counter-argue against information that disagrees with their prior viewpoint (Lodge & Taber, 2013) apply just as equally to court decisions as it does to legislation passed by Congress. Ordinary Americans will rarely have the motivation to take actions to challenge a decision that they agree with (Gibson, 1989, 1991). Because almost everyone will accept any decision when they agree with the policy implications, “the most common research design involves presenting people with a policy with which they disagree and then trying to ascertain whether a court ruling increase the probability of acceptance …” (Gibson et al., 2005, p. 187). Of course, all research designs involve some tradeoffs, and asking people about the issue position beforehand is not ideal, but without doing so, we would be unable to separate out those who agree with the policy implications of a court decision from those who disagree, even though we would expect the determinants of acceptance for those two groups to be dramatically different. Woodson (2015) used three distinct study designs to examine the effect of politicization perceptions on acceptance—one that split people into groups based on ideology, one that asks directly about issue positions and then presents a disagreeable decision, and a third that asks about acceptance in general rather than about a specific decision—and found similar results across all three. This suggests that the type of study design used here results in similar conclusions as the other potential options.

Because the type of social science evidence differs between the two studies, the issue used must also differ because not all types of social science apply equally well to all legal issues. For Study 1 involving empirical studies, we ask the subjects about “stop and frisk” programs and whether police should be required to have probable cause or just reasonable suspicion to search someone they suspect of criminal activity.⁴ Our manipulation involved randomly assigning respondents to read a summary of either a decision that relied on purely legal arguments, or a decision that cited social science research to support the decision. Both decisions involving legal arguments used an interest balancing approach weighing the invasiveness of the search against the government's interest in public safety. The social-science based decision that ruled “stop and frisk” programs were unconstitutional mentions research finding that “stop and frisk” was an ineffective program and reduced public support for and cooperation with law enforcement. For the social-science-based decision upholding “stop and frisk” programs, the decision cites research highlighting the deterrence effect of these programs.⁵ The length of the descriptions ranged between 151 and 167 words.

For Study 2 involving behavioral economics, we ask the subjects about a hypothetical situation involving whether the current owner of an oil pipeline should be liable for damages based on environmental contamination caused by the pipeline prior to the current owner purchasing the pipeline. The manipulation involved randomly assigning respondents to read purely legal justifications involving when contract law requires new owners to pay past liabilities, or decisions that relied on an analysis of how economic development would be affected by the case. The length of the descriptions ranged from 99 to 104 words. We chose this scenario because we needed a decision that invoked behavioral economics analysis and, at the same time, be simple enough for the respondents to understand the issue and decision based on a single short paragraph. That did not seem possible with the more salient constitutional issues that normally receive substantial attention from the media and public. Using a statutory rather than constitutional issue has the advantage of showing our theory works in both constitutional and statutory cases. Virtually all existing research examining the acceptance of court decisions involves constitutional questions so examining acceptance across both the constitutional and statutory contexts is a noteworthy contribution.

While using liability for damages as the issue has advantages, it also has a disadvantage. A substantial number of people do not find that issue to be important, and this should affect how people make acceptance judgments. The decision to accept or challenge a court decision is not just about disagreement with the issue but also about the motivation to take efforts to overturn a decision. The motivation to challenge a decision is greatly reduced when people like the policy aspects of a decision, or when they do not care about a policy, causing people to make judgments concerning decisions they like and those they do not care about much differently than displeasing decisions they consider important. As a result, we should expect that people are highly likely to accept a decision if they do not consider it important (Parker & Woodson, 2020; Woodson, 2015, 2019). For most issues, the percentage of people who do not care about an issue is small enough that it does not affect the results, but for an issue involving the liability over oil pipelines a substantial part of the sample will find the issue entirely irrelevant to their lives. We can see this in the percentage of people in the two studies who said the issue was not important at all. In Study 1 using a decision on “stop and frisk,” 11% of the sample stated the issue was “not important at all.” That percentage rose to 26% in Study 2. In our regression analyses below, we exclude from the sample for both Study 1 and Study 2 those people who deemed the issue “not important at all.”

Sample

The sample for both of our studies comes from Mechanical Turk, an online platform that allows researchers to upload a survey to their website and have people complete that survey for a small amount of money. For Study 1, 504 subjects completed the survey. The median completion time was ~17 min, and each person was paid $1 for completing the survey. For Study 2, 619 respondents completed the survey. The median completion time was again ~17 min, and each respondent was paid $1.75.

The use of Mechanical Turk (MTurk) has become common in social psychology, political science, and the law and psychology field because of its ability to offer inexpensive experimental subjects, lowering the barriers needed to test novel questions that would have been left unaddressed otherwise. Multiple sets of authors from different backgrounds have examined MTurk samples finding that while they are not nationally representative, they are more diverse than other types of convenience samples, and authors were able to replicate the findings from previously published studies (Berinsky et al., 2012) while also finding that the data quality (Paolacci et al., 2010) and psychometric properties (Buhrmester et al., 2011) from MTurk samples was comparable to other common types of samples. Irvine et al. (2018, p. 344) examined experiments specifically involving law and psychology, again finding that MTurk samples can be used to replicate previous finding and concluding that “peer-review publications in our field [should] put aside some of the latent skepticism that they may have held for that platform.”

A newly emerging problem with MTurk samples involves subjects using VPNs to hide their true location, allowing those living in other countries to appear as if they are in America. As recommended by Kennedy et al. (2020), we used a software package called rIP to filter out any subjects from outside of America or those using a VPN network to mask their identity from taking our survey. In addition, in Study 1 we used a screener question to filter out those subjects who were not paying attention to the study. It began like a normal survey item, but at the end of the question's text, the subjects were told to answer in a specific way by clicking on two of the answer options. Twenty-three people failed that test and are excluded from the analysis. While Study 2 did filter potentially fraudulent subjects using the rIP package, it did not include an additional screener question to filter out inattentive subjects. In addition to these data cleaning efforts, we also excluded non-American citizens from our samples, which reduced the sample in Study 1 by 5 subjects and Study 2 by 10 subjects.

While MTurk samples are generally more representative than other methods of convenient sampling, they are still generally more educated, more liberal, less diverse, and younger than a nationally representative sample (Berinsky et al., 2012). Our samples generally match that pattern, with the largest deviation from the national average being the large percentage of college graduates (52% and 56% respectively) and a strong skew toward democratic and liberal identifiers. In Study 1, 47% identify as Democrats and 21% as Republicans. The Democratic skew in Study 2 is slightly smaller with 41% of the sample being Democrats and 28% Republican. See the Supporting Information for a more complete breakdown of the sample demographics.

This demographic skew in MTurk samples leads to generalizability concerns about the ability to make broad causal claims. In addition, the process through which MTurk respondents are recruited, and the self-selection of the MTurk workers into that group, leads to concerns that this type of sample could be different from other types of samples in unobservable ways. However, our study design should assuage some of those concerns because we are not trying to determine the overall effect of using social science on acceptance judgments. If that was the case, it would be possible that some unobserved difference between MTurk samples and the general population could moderate that effect and bias our results.

Rather, we are trying to determine whether popular constitutionalism moderates the effect of using social science, and thus we empirically identify one group—popular constitutionalists—and compare them to another group—legal traditionalists. For our results to be biased by the use of an MTurk sample requires the differences between legal traditionalists and popular constitutionalists in an MTurk sample to be systematically different from in a general population sample. While we cannot directly test whether that is the case, Clifford et al. (2015, p. 7) showed that MTurk samples can be used to draw inferences about the differences between liberals and conservatives as their “results suggest that the same values and personality traits that motivate ideological differences in the mass public also divide liberals and conservatives on MTurk.” Clifford et al. (2015) provides evidence that MTurk samples can be used to compare the reactions to an experimental manipulation between two different empirically identifiable groups, and that is what we propose to do with our studies, by comparing legal traditionalists and popular constitutionalists.

MEASUREMENT

The wording of the survey questions was similar for both studies. They start by presenting the subjects with a paragraph describing the basic legal arguments for both sides of the case, and then asking which side they agreed with. For the full text of all items used in the survey, see the Supporting Information. For Study 1, the subjects indicated whether they supported the ability of police to “stop and frisk” individuals they suspected may be about to commit a crime or whether police should be required to have probable cause before stopping an individual. For Study 2, the subjects were asked whether the current owner of the oil pipeline “should have to pay damages for the contamination resulting from the prior oil spill?” and were given the answer options of yes or no. In both studies, following these initial dichotomous items the subjects indicated how strongly they held that position and the importance of the issue on four-point scales.

Normative decision-making preferences

Much of the discussion from scholars—and especially pundits and politicians—around how the judiciary makes decisions involves the false dichotomous choice of the Court being either political or legalistic. However, Gibson and Caldeira (2011) revealed not everyone desires a purely legalistic court. Their analysis argues that while the public does not like strategic and insincere decisionmaking, the public seems amenable to ideological decisionmaking. In addition, both Parker and Woodson (2020) and Woodson (2018) show that not all types of decisionmaking that are traditionally deemed political are evaluated similarly. The studies show that attitudes toward popular constitutionalism are a separate and independent factor from ideological or politicized decisionmaking. In both studies, two factors of decision-making perceptions emerge that have separate effects on evaluations, with one factor ranging from legalistic decisionmaking at one end to ideological or politicized decisionmaking at the other. The second factor measures attitudes toward popular constitutionalism or the degree to which public opinion influences the Court. While using actual decision-making perceptions rather than normative preferences, Scheb and Lyons (2000) also show that perceptions about public opinion affecting the Court load onto a separate factor than both perceptions about ideological and legalistic influences.

The current paper uses a similar measurement strategy as Parker and Woodson (2020) to measure normative preferences for the Court's decision-making processes. The subjects are presented with a statement about the Court's decision-making process and asked their level of agreement with the statement on a 7-point scale. The items are designed to measure how people think the Court SHOULD make decisions not how it actually makes decisions. As a result, each statement begins with the phrase “In the ideal world, …” to emphasize the normative nature of the items. Each item then proceeds to describe a potential way in which the Supreme Court could make a decision. See Table 1 for a list of the normative preference items.

Table 1. Normative preference items

While we only expect two factors to emerge, we include four distinct subsets of normative preference items. The first subset measures the degree to which people support the doctrine of popular constitutionalism. These questions do not mention popular constitutionalism directly but rather ask in various ways about what type of influence the public and public opinion should have on the Court. Many, if not most, people who support the concept of popular constitutionalism would not be able to define it if asked. Rather, they have some vague notion that in a democracy the meaning of the Constitution should be influenced by the public and not solely determined by an unelected and unaccountable Court. Study 1 includes four items designed to measure the degree to which people believe the public should influence the Court, with Study 2 including the original four plus an additional five new items.

The three remaining subsets of items are all expected to load onto one factor that ranges from support for legalistic decisionmaking at one end to support for ideological or politicized decisionmaking at the other end. One subset of three items measures the degree to which people believe the Court should do little more than neutrally interpret and apply the law. The other two subsets of items are based on a distinction made by Gibson and Caldeira (2011) between ideological and politicized decisionmaking. They define the latter as being when judges act in a self-interested and strategic manner like an ordinary politician and the former as when judges are influenced by their personal political opinions or ideology. Because of the literature's previous distinction between these two concepts, we include three items that directly ask about personal political opinions or ideology affecting the Court's decisions and three items that directly ask if judges should act like strategic or self-interested politicians. While we separately measure legalistic, politicized and ideological decision-making preferences, based on prior published results (Parker & Woodson, 2020) and our own pre-tests, we expect all three subsets of items to load onto a single factor.

Support for legal pragmatism

In the second study, we included an additional six items measuring a person's support for the doctrine of legal pragmatism using the same format as the other normative preferences items, with the items focusing on whether the Court should take into account the consequences of their decisions. The items asked whether the Court should be concerned with how a decision affects people, follow precedent in the face of harmful consequences, interpret the law to provide the greatest benefit, take into account impartial evidence about the effect of the law, use scientific evidence, and avoid making decisions with harmful outcomes. All six items load onto a single factor, with a Cronbach's alpha of 0.72.

Other pre-manipulation variables

Prior to administering the experimental manipulation, we also measured the subjects' knowledge of the judiciary, their evaluation of science and a variety of background characteristics. We measured partisanship on a six-point scale using an initial item with three options of Democrat, Republican, or Independent, with a partisan strength or leaning partisanship follow-up. We measured education on a five-point scale that ranges from “Less than High School Diploma” to “Graduate Degree.” To assess judicial knowledge, the subjects answered six factual questions about the judiciary, and each individual's number of correct responses was used as the measure. The questions asked whether U.S. Supreme Court justices were appointed or elected, whether they served life-terms; the number of justices' votes required to hear a case; how many decisions the court makes per year; how a decision can be overturned; and how the number of justices on the Court is determined.

We used eight items to measure the subject's confidence and trust in the scientific community. Four of the items were taken from a previous survey that uses a four-point scale to ask how much trust the person has that scientists will “tell the truth,” “reporting their findings accurately,” “give impartial evidence on matters of public debate,” and “report findings even if they go against the sponsor of the research” (American Academy of Arts and Sciences, 2018). We added four more items that more directly ask about social science and a current politicized scientific debate. They were asked how much trust they have in scientists to “understand how American society, politics, and economy functions,” “know the causes of human behavior,” “predict the effect of government policies,” and “forecast the effect that pollution has on the environment and future climate.” For both studies, all eight items load onto a single factor, and so we use an additive index of the eight items as our measure. The Cronbach's alpha for an additive index of all eight items is 0.90 for Study 1 and 0.88 for Study 2. We will use this scale to examine whether a person's reaction to social science varies based on their evaluation of science.

Acceptance of the decision

Directly following the administration of the experimental manipulation, the subjects answered four questions that assess whether they accept the decision or want to challenge it and get it overturned. The first item asks about acceptance in general and the subjects indicate whether they “accept the decision and consider it the final word on the matter” or whether they “want to challenge the decision and get it changed.” They are then asked the strength of that opinion on a four-point scale. The initial item is then combined with the strength follow-up to form a single 8-point indicator. The other three items used in the acceptance scale all ask about the subject's support for various ways of challenging the decision. One asks whether the subject supports or opposes removing the judges who voted for the decision. Another asks whether the subject would sign a petition attempting to overturn the decision, and the third asks whether they support efforts to overturn the decision with a constitutional amendment. This strategy of measuring acceptance was first introduced in Gibson et al. (2014). To form the acceptance index, all four indicators of acceptance are recoded to range between 0 and 1. The acceptance scale is an additive index of those four recoded items. After adding the four indicators together to form an additive index, that index is then recoded again to range from 0 to 1, with the intermediate values ranging between those end-points. For both studies, all four items load onto a single factor, with a Cronbach's alpha of 0.77 in Study 1 and 0.80 in Study 2.

DATA ANALYSIS

Before examining the effect of the experimental manipulations, we first must establish that support for popular constitutionalism is a separate factor and empirically distinct from the spectrum of normative preferences that ranges from legalistic to ideological and/or political decisionmaking. Since the survey items and results for both studies are similar, we analyze the factor analysis from both prior to examining the effects of the experimental manipulations. Table 2 shows the results for both factor analyses. For both studies, only two factors emerged, with the eigenvalues for both the first and second factor larger than 1, and the eigenvalues for all other factors below 1. Because the two factors of normative decision-making preferences are expected to be somewhat correlated, an oblique factor rotation was used.

Table 2. Structure of normative preferences

Note: All factor loadings 0.30 or larger are displayed in bold text.

Table 2 shows that in Study 1 the items measuring support for ideological, political and legalistic decision-making load onto one factor with the items measuring support for popular constitutionalism loading onto a second factor. All nine items that measure support for ideological decisionmaking, politicized decisionmaking or legalistic decision-making load onto the first factor with loadings higher than 0.42, and those items all have low loadings below 0.17 on the second factor. The four items measuring support for popular constitutionalism all load onto the second factor with loadings at or higher than 0.63, with those items also having factor loadings lower than 0.22 on the first factor. These results clearly show that support for popular constitutionalism is a separate and distinct factor from the spectrum of normative preferences that ranges from politicized to legalistic decisionmaking. We call the factor measuring support for popular constitutionalism the public norm scale, and the factor that includes the political, ideological and legalistic items the political-legal norm scale.

We included additional items in the public norm scale for Study 2 to measure different aspects of popular constitutionalism and to provide a fuller view of how the public conceives of the doctrine. As in Study 1, the results clearly show that support for popular constitutionalism is a distinct and separate factor from the political-legal norm scale. All nine items measuring support for popular constitutionalism load onto the first factor with loading higher than 0.56. When examining the second factor, 8 of the 9 items measuring ideological, politicized, or legalistic decisionmaking have factor loadings higher than 0.34 on the second factor.

Two of the items have somewhat different factor loadings in Study 2 when compared to Study 1.The second legalistic norm item that asks whether justices should act as “neutral arbiters of the law” has a relatively low factor loading of 0.27 on the second factor measuring the political-legal norm scale. In addition, the second item measuring the public norm scale, which asks whether the public should have little to no influence on the court's constitutional interpretations and decisions, has a high loading on the factor measuring the public norm scale, but it also has a relatively high loading of 0.3 on the political-legal norm scale.⁶ To be consistent with the theoretical expectations and to keep the measurement of key concepts consistent across the two studies, the second legalistic norm item will still be included in the political-legal norm scale, and the second public norm item will be included in the public norm scale for Study 2.⁷

For both studies, the political-legal norm scale is created by summing the nine items that measure ideological, politicized, and legalistic normative preferences, with the appropriate items being reverse coded. For that scale, the low end of the spectrum represents support for legalistic decisionmaking while the high end represents support for politicized or ideological decisionmaking. The public norm scale is constructed by adding together all the items measuring support for popular constitutionalism (either 4 or 9 depending upon the study), with higher numbers indicating more support for popular constitutionalism. The two measures are significantly correlated, with r = 0.32 in Study 1 and r = 0.37 in Study 2. As with all other variables used in the following models, these variables are recoded with 0 as their minimum, 1 as their maximum, and the other values ranging in between.

Figure 1 shows the distribution of both variables, and the pattern matches that found in Parker and Woodson (2020). The political-legal norm scale is highly skewed toward people preferring a legalistic approach over a political one, with only a very small proportion preferring a political and/or ideological court. The distribution of the public norm scale shows a more consistent distribution with approximately half the sample above the neutral mid-point of the scale and half below it. This suggests that at least among this nonrepresentative sample support for popular constitutionalism is more widespread than support for a political or ideological court.

Figure 1. Distribution of normative preferences

Study 1—Empirical social science study

We now move to examining how the two normative preference scales described above moderate the effect of our experimental manipulations on acceptance. In the models of acceptance in Table 3, we test those moderating effects by including an interaction between the experimental manipulation and both normative preference scales. We also include an interaction between the manipulation and the respondent's knowledge about the Court to control for the possibility that people who are more knowledgeable about the Court may treat using social science evidence differently than those with less knowledge.

Table 3. Normative preferences and social science in legal decisions

Note: SEs in parentheses.

^*p < 0.05. ^**p < 0.1.

The model in Column 1 of Table 3 includes the model for Study 1 using the complete sample with all three interactions and a number of control variables.⁸ The coefficients for the interactions of the social science manipulation with both the political-legal norm scale and the court knowledge index are insignificant. The coefficient for the interaction between the social science manipulation and the public norm scale is marginally significant with p < 0.1, and as we predicted, the effect of the social science manipulation on acceptance becomes less negative and more positive as support for popular constitutionalism increases. While in the expected direction, we delay more fully describing it because it is only marginally significant, and we expect the effect of the social science manipulation to differ based on a person's evaluation of science.

The models in Columns 2 and 3 of Table 3 split the sample based on the respondent's evaluation of science. The sample for the model in Column 2 includes everyone at the 50th percentile and below (0.67), and the sample for the model in Column 3 includes everyone above the 50th percentile.⁹ As the results show, for those with low evaluations of science there is no interaction between the social science manipulation and any of the three moderating variables, including the public norm scale. When shifting to Column 3, which runs the same model on those with high evaluations of science, the interaction between the public norm scale and the social science manipulation is positive and significant with p < 0.05. The other two interactions are insignificant, indicating that only a person's support for popular constitutionalism moderates the effect of using social science in court decisions on acceptance.

The line labeled Study 1 in Figure 2 displays the marginal effect of the social science manipulation on acceptance for those with high evaluations of science across the range of the public norm scale. The line is solid when the marginal effects are significantly different from 0 at p < 0.05 and dashed when they are not. These marginal effects were calculated holding all other variables at their means. The marginal effect is negative and significant at p < 0.05 for everyone at 0.21 and below on the public norm scale, representing 16% of the sample used in that model. When the public norm scale is 0, the marginal effect of the social science manipulation is −0.19. In the legalistic condition, the average level of acceptance is 0.58, and that decreases to 0.39 in the social science condition. For those at the high end of the public norm scale the effects are in the opposite direction. The marginal effect of the social science manipulation is positive and significant at p < 0.05 for everyone at 0.63 and above on the public norm scale, representing 48 percent of the sample used in the model. At the highest point of the public norm scale (1), the marginal effect of the social science manipulation is 0.21, with the average level of acceptance in the legalistic condition being 0.23 and increasing to 0.45 in the social science condition.

Figure 2. Public norm scale and the marginal effect of the experimental manipulations. Two-tailed significance tests are used for the social science manipulation in Study 1, and one-tailed significance tests are used for the behavioral economics manipulation in Study 2

These results show that the public norm scale moderates the effect of the social science manipulation as expected by H1 and H2, but only among those with high evaluations of science. For those who support popular constitutionalism and approve of science, the social science manipulation increases acceptance, and for those who dislike popular constitutionalism and approve of science, the social science manipulation decreases acceptance. Among those who disapprove of science the manipulation had no statistically detectable effect on acceptance across the whole range of the public norm scale. We now shift to testing our theory in a different context using the behavioral economics approach toward the law as the social science alternative to a traditional legalistic argument.

Study 2—Behavioral economics analysis

Column 4 of Table 3 contains the model examining the effect of the behavioral economics manipulation. Because we have strong prior expectations based on our previous study that the effect should be negative among those low on this scale and positive among those high on this scale, we use one-tailed significance testing throughout this analysis. Replicating the results from Study 1, the only significant interaction with p < 0.05 is between the public norm scale and the behavioral economics manipulation. Neither court knowledge nor the political-legal norm scale moderates the effect of the manipulation. Figure 2 displays the marginal effect of that manipulation across the range of the public norm scale, using the same solid/dashed line convention as with Study 1. As the line labeled Study 2 shows, the negative effect is significant with p < 0.05 for everyone at 0.28 or below on the public norm scale, which represents 20% of the sample. At the lowest end of the public norm scale (0), the marginal effect of the behavioral economics manipulation is −0.10, with the level of acceptance being 0.77 in the legalistic condition and decreasing to 0.67 in the behavioral economics condition. For those high on the public norm scale, the marginal effect is significant at p < 0.05 for everyone at 0.96 and above on the public norm scale, representing 3% of the sample. At the highest end of the public norm scale (1), the marginal effect of the manipulation is 0.08, with the level of acceptance being 0.34 in the legalistic condition and increasing to 0.42 in the behavioral economics condition. The findings from Study 2 once again align with the predictions from H1 and H2.

Unlike with the manipulation from Study 1 where the interaction between the public norm scale and the manipulation only appeared for those who had high evaluations of science, the behavioral economics manipulation is unaffected by a person's evaluation of science. When using the same split-sample approach as Study 1, the magnitude of the interaction between the manipulation and the public norm scale is similar for both those with low evaluations of science and those with high evaluations of science. The lack of an effect for evaluations of science is likely due to the fact that in the first study the experimental manipulation specifically mentioned “social science studies,” whereas in the second study the experimental manipulation only referred to economic consequences without references to scientific studies, and thus did not prime respondents to think about science.

We would also like to note another difference between the two studies. Partisan identity has a statistically significant effect at p < 0.05 in Study 1 (Columns 1–3 of Table 2), but not in Study 2 (Column 4 of Table 3). Democrats are less likely than Republicans to accept a decision involving stop and frisk policies but are not less likely to accept a decision involving oil pipeline liability. That this effect only occurred in one study, and not both, suggests that the effect of partisanship on acceptance depends upon the political issue. This divide makes sense in retrospect as stop and frisk invokes racial issues and attitudes toward law enforcement, which are both commonly discussed in the news media and an issue that divides the two major parties. On issues that invoke partisanship to a lesser extent—like those involving whether the current or past owner of an oil pipeline should be held liable for damages—partisanship does not have an effect.

Legal pragmatism and popular constitutionalism

We hypothesized in H3 that support for legal pragmatism should be associated with support for popular constitutionalism, and that this relationship can help to explain our results. To test this, we use an additive index of six items that measure support for legal pragmatism and examine its relationship to the two normative preference factors. The results match our expectations. The correlation between the political-legal norm factor and support for legal pragmatism is 0.03, both insubstantial and insignificant. The correlation between the public norm factor and support for legal pragmatism is 0.50, which is both large and highly significant (p < 0.05). This shows that support for popular constitutionalism and legal pragmatism tend to trend together and are linked within minds of the mass public, as predicted by H3.

DISCUSSION

The Chief Justice and the Iconoclast made different predictions about the effect of using social science in decisions because they made different assumptions about the American public. The former assumed that the public sought judges to act as umpires that call balls and strikes as he stated in his confirmation hearing. The latter assumed the public desired practical judges that were concerned with the consequences of decisions. Our two experiments provide evidence supporting both of their predictions, but only among the portion of the public that shared their assumptions and normative preferences about the proper way for judges to make decisions. For the iconoclasts who support legal doctrines like popular constitutionalism and legal pragmatism, social science is seen as a legitimate piece of evidence concerning the wisdom of a decision and thus perceived as a positive feature of a court's decision-making process, which results in the level of acceptance increasing. For the legal traditionalists who support judges acting as umpires calling balls and strikes, the use of social science is an improper and illegitimate nonlegal influence on judicial decisionmaking, and as a result decreases acceptance.

These results do not allow a simple answer concerning whether using social science in decisions helps or hurts the Court's ability to induce compliance and acceptance of its decisions. Gauging the overall effect requires balancing the positive effect among the iconoclasts with the negative effect among the legal traditionalists. Since the magnitude of the positive and negative effects among the two groups was similar, the overall effect among the total population depends upon whether the number of people supporting traditional legalistic ideas is larger than the number of popular constitutionalism supporters. Based on a survey from 2005 about 1/3 of the American population thinks it is “very important” for the court to “represent the majority” (Gibson & Caldeira, 2009), and thus we can conclude that at least 1/3 of the population would be receptive to the Court using social science in their decisions. Not every remaining person will be a strict legal traditionalist turned off by social science in judicial decisions. Thus, most likely the overall effect of using social science is not to substantially decrease or increase the aggregate level of acceptance among the population, but to change the composition of the people who accept. Whenever the Court uses social science, it increases the number of people who accept the decision among supporters of popular constitutionalism, and decreases it by roughly the same number among legal traditionalists.

Structure of normative decision-making preferences

Because normative preferences have such a transformation effect on how people evaluate the judiciary, understanding the structure of those preferences is essential. Gibson and Caldeira (2011) moved scholars past the false dichotomous choice commonly imposed in political rhetoric between either a political or legal court and made many important points that are reinforced by the results of our own analysis. First, our factor analysis of normative preferences clearly shows that the public's views on the judiciary do not fit a simple one-dimensional structure ranging from politicized to legalistic decisionmaking. Instead, there are various types of nonlegalistic courts between which the public makes distinctions. In addition, our results confirm that some people desire an ideological court. However, where our data diverges from Gibson and Caldeira (2011) is over whether an ideological court and a political court can be empirically separated, and whether they are treated as conceptually separate by the public. While Gibson and Caldeira (2011) said that politicized and ideological decisionmaking are two separate concepts, in our two studies we separately measured ideological decisionmaking and politicized decisionmaking as defined in Gibson and Caldeira (2011), and both loaded onto a single factor. This suggests the public cannot differentiate between those two concepts, making little distinction between judges following their ideology and those acting like ordinary politicians. Rather, the public makes a distinction between a politicized court and a court following popular constitutionalism.

The American tradition of popular constitutionalism

Most of the scholarly literature concerning popular constitutionalism involves a normative debate among legal elites concerning its wisdom (Friedman, 2003; Kramer, 2004; Kramer & Tribe, 2001; Pozen, 2010), but virtually no research empirically examines the American public's thoughts on the doctrine. America's long tradition of both mass and elite support for popular constitutionalism makes this lacuna in research even more astounding. Kramer (2004, p. 963) states that in early American history “it is possible, and maybe even probable, that popular constitutionalism was the dominant public understanding…even as judicial supremacy was favored by and within the legal profession.” However, if there was a political dispute concerning judicial supremacy and popular constitutionalism, “what is certain is that popular constitutionalism was the clear victor each time matters came to a head…The pattern began to change only in the latter half of the twentieth century.” That mass public support for popular constitutionalism was matched at the elite level with many of America's most important Presidents like Jefferson, Jackson, Lincoln and both Roosevelts espousing support for the Court following the public will (Engel, 2011). In marked contrast to how the most judges would describe themselves today, the Framers' of the American Constitution predominately viewed judges “as representatives of the people and servants of the popular sovereignty equal to other branches and not necessarily as neutral arbiters held apart from politics” (Engel, 2011, p. 74).

That earlier dominant legal tradition of popular constitutionalism has been obscured by “judges and legal scholars engaged in a systematic enterprise of constructing jurisprudential history into a clear pattern of judicial supremacy…[that] involved a deliberate re-imagining of Marbury v Madison” (Engel, 2011, p. 10). This reimagining of history, emphasizing past claims of judicial supremacy while quieting the many contemporaneous voices raised in support of popular power, allowed the judiciary to “cloth this new position in a seemingly long tradition” (Engel, 2011, p. 84). That reimagination of America's legal tradition by elites did not seem to completely dampen the public's support for popular constitutionalism though. Polling data showing a substantial percentage and perhaps even a majority of the population wants the Court to represent the majority (Gibson & Caldeira, 2009; Scheb & Lyons, 2001), experiments showing the lack of a negative effect on the Court's legitimacy when it follows public opinion (Bonneau et al., 2016; Farganis, 2012), and the public's widespread support for legal reforms designed to increase accountability (Anderson, 2003; Geyh, 2003; Gibson, 2012; Golde, 2019) all indicate current, persistent, and widespread public support for this integral part of America's early legal traditions.

Psychology of popular constitutionalism

Perhaps, the same “systematic enterprise” that de-emphasized popular constitutionalism in the traditional historical narrative of the American judiciary has contributed to the lack of scholarly attention on popular constitutionalism supporters. As a result of that lack of attention, the discipline failed to adequately recognize the many existing signals referenced in the previous paragraph that a substantial portion of the population supports popular constitutionalism, and also provides little to no insight into the criteria that group uses to evaluate the judiciary. Combined, this means little is known about how a substantial portion of the American people think about the judiciary and interpret its actions. The only other article to our knowledge that focuses on empirically examining the mass public's support for popular constitutionalism reinforces our own findings by showing that support for popular constitutionalism alters the criteria the public uses to determine whether to accept or challenge a decision. As Parker and Woodson (2020) noted supporters of popular constitutionalism seem to treat the court as if it is a quasi-representative institution, and as a result, dissents increase legitimacy perceptions while having no effect on acceptance because they suggest the justices are using a deliberative decision-making process in which all sides are given representation and a voice. The current studies add to our understanding of popular constitutionalism supporters by showing they also reward courts whenever they use social science as evidence and consider the real-world consequences of their decisions during these deliberative processes. Together, these two papers suggest that supporters of popular constitutionalism desire a court that deliberates over issues, allowing dissent and disagreement to be discussed and publicized, while using all the available evidence about the consequences of those decisions, including evidence from social science.

Our second study, which had a manipulation based on Posner's doctrine of legal pragmatism and measured support for the doctrine, provides another insight by showing that support for legal pragmatism is associated with support for popular constitutionalism. It seems that whatever motivation drives people to support one iconoclast doctrine drives them to support the other as well, suggesting there may a collection of iconoclast ideas and concepts that go along with accepting the basic tenets of popular constitutionalism. Future research is needed to determine what other types of iconoclast ideas besides legal pragmatism coincide with a belief in popular constitutionalism.

As Woodson (2018) notes, attitudes toward popular constitutionalism change “a person's overall orientation toward an institution, which changes the criteria used to evaluate the institution” (Woodson, 2018, p. 77). Part of that change may include the endorsement of other iconoclast legal doctrines like legal pragmatism, but also a fundamental transformation in the basis of the institution's legitimacy. Whenever a Court begins to follow the doctrine of popular constitutionalism, “rather than being an institution guided by unaccountable judges, it becomes an institution representing the will of the people” (Woodson, 2018, p. 79). Perhaps, the effect of supporting popular constitutionalism is to reorient the basis of the Court's legitimacy away from the myth of legality (Scheb & Lyons, 2000) toward an alternative form of democratic legitimacy based on the Court following the public will. As a result of that fundamental transformation, factors like the presence of dissents or the use of nonlegal evidence that would undermine a form of legitimacy based on the myth of legality instead buttress a form of representation-based legitimacy derived through accountability. The literature on judicial elections shows courts are able to acquire legitimacy based on representation and accountability. As Gibson (2012, p. 141) notes in his study on how elections affect the legitimacy of courts, “for most constituents of courts, the predominant essence of judicial elections is not foul. Because it is not, holding judges accountable, with its messiness and fuss, still serves to make courts more legitimate.”

This alternative form of legitimacy based on accountability offers another route through which justices can build support for their decisions. Rather than concentrating on convincing the public their decisions were made using legalistic reasoning, our research suggests that if justices openly base their decisions on public opinion and the consequences of those decisions, while using social science to understand those consequences, a substantial portion of the public will be pleased and reward them for it.¹⁰ Existing evidence shows that a form of legitimacy based on accountability is stable and resists the negative effect of displeasing decisions. Woodson (2018, p. 79) found that for those who perceive the Court as actually following the doctrine of popular constitutionalism, “legitimacy perceptions become insulated from policy concerns, and neither pleasing nor displeasing decisions … affect legitimacy perceptions.” While more research is needed, this suggests that if the Court began to present itself as a representative institution that follows public opinion, it could still issue displeasing decisions without threatening its store of legitimacy.

This idea that displeasing decision would not affect the legitimacy of the Court seemingly contradicts recent research from Bartels and Johnston (2020) showing that policy attitudes toward the Court's decisions dramatically affect its level of support. However, they are not necessarily contradictory because Bartels and Johnston (2020) do not take into account normative preferences. In some ways, the most important point to take away from the current article has little to do with social science or legal pragmatism, but instead involves the necessity to take into account normative preferences whenever examining public opinion of courts. Just as using social science can both increase and decrease support for the Court depending upon an individual's normative preferences, many other things will both increase and decrease support for the Court. If someone applied normative preferences to the policy attitude-legitimacy debate, they may find that pattern. For some, a displeasing decision decreases legitimacy and for others it may have no affect or even increase legitimacy. That would provide a route through which both Positivity Theory (Gibson & Caldeira, 2009) and Bartels and Johnston (2020) are correct. Neither would make correct predictions for everyone, but both would be right for some, and which theory is correct depends upon the normative preferences of the individual.

Beyond informing future scholarship, these results are also relevant to contemporary debates about judicial reform. The highly contentious Supreme Court confirmation process of the past several years has led to renewed conversation about how to reform the Supreme Court to prevent political polarization from entering the Court and harming its legitimacy. Some have called for the creation of term limits for Supreme Court justices, so that, for example, a new justice would be appointed at consistent 2-year intervals (Kruzel, 2020), and multiple candidates for the Democratic presidential nomination in 2020 proposed a nonpartisan version of court-packing (Lederman, 2019). However, much of the conversation surrounding judicial reforms assumes that the most legitimate courts would be independent and decide cases purely based on the law. Our results suggest that this assumption is not accurate for a substantial proportion of the public. Judicial legitimacy may in fact be strengthened for much of the public by pursuing reforms that make the courts more accountable to the public will.

Regardless of the issue involved, our research highlights the necessity of incorporating normative preferences into any theory concerning public opinion of courts. Whether it is the effect of a judicial reform, the impact of using social science, or the link between policy attitudes and legitimacy perceptions, a relationship that is strong for people with one set of normative preferences may be nonexistent or in the opposite direction among people with different normative preferences. Rather than asking questions like whether something affects support for the judiciary, scholars should pose questions like for whom and how do these things affect support for courts. That is the only way to truly understand the complexity of public opinion toward the judiciary.