Failing to ignore the ignorant: Mistaking ignorance for error

Abstract

Expertise is a reliable cue for accuracy – experts are often correct in their judgments and opinions. However, the opposite is not necessarily the case – ignorant judges are not guaranteed to err. Specifically, in a question with a dichotomous response option, an ignorant responder has a 50% chance of being correct. In five studies, we show that people fail to understand this, and that they overgeneralize a sound heuristic (expertise signals accuracy) to cases where it does not apply (lack of expertise does not imply error). These studies show that people 1) tend to think that the responses of an ignorant person to dichotomous-response questions are more likely to be incorrect than correct, and 2) they tend to respond the opposite of what the ignorant person responded. This research also shows that this bias is at least partially intuitive in nature, as it manifests more clearly in quick gut responses than in slow careful responses. Still, it is not completely corrected upon careful deliberation. Implications are discussed for rationality and epistemic vigilance.

1 Introduction

Expertise is a powerful heuristic cue for persuasion and accuracy. People are often persuaded by experts into taking their advice and adopting their opinions. This work asks whether people overgeneralize this heuristic, such that they think that, just as expertise is a positive cue for accuracy and truth (i.e., people think the expert is probably right and thus are likely to choose the same response that the expert gave), ignorance might have a negative influence (i.e., people think the ignorant is wrong and thus choose the opposite response).

There is a great deal of work on the positive effect of expertise on persuasion and advice taking (DeBono & Harnish, 1988; Harvey & Fischer, 1997; Hovland & Weiss, 1951; Petty et al., 1981; Reimer et al., 2004). In comparison, and to our knowledge, much less is known about the influence of ignorance (which can be conceived of as negative expertise) on people’s judgments and decisions. The key questions in the present research are: 1) Just as expertise is a positive cue for credibility and accuracy, whereby people think the expert is right, is ignorance a negative cue, such that people think the ignorant is wrong? And 2) do people ignore the ignorant’s opinion, or, because they think that opinion is wrong, do they go against it?

From a normative perspective, people should realize that, in a dichotomous choice/judgment task, complete ignorance still gives the ignorant a 50% chance of success (likewise, in a judgment task with more than two choices, a random guess by the ignorant has a 100%/N of choices chance of being correct). The ignorant’s response should carry no weight (positive or negative) in one’s response. Not to ignore the ignorant is irrational – it is as if the ignorant person has negative knowledge (i.e., incorrect knowledge), such that choosing the opposite response is a correct strategy. In the worst case, the ignorant has at least a 50% chance of being correct, and even minimal knowledge would raise that chance level above 50%. Moreover, even if people have absolutely no knowledge about a specific fact, they might extrapolate from other facts they know. Thus, a case could be made that the baseline against which to judge the rationality of the perceived likelihood of the ignorant’s response being correct might be set higher than 50% (though in the present research we use the conservative 50% benchmark). In short, people should ignore the ignorant. However, we suspect that people might overgeneralize the expertise=correct heuristic, and expect that ignorance=error.

Our hypothesis is in line with research on heuristics and biases, showing that people sometimes overgeneralize the use of reliable cues and sound judgment strategies to domains where they are no longer valid or useful (Arkes & Ayton, 1999; Haws et al., 2017; Hsee et al., 2019; Hsee et al., 2015; Peysakhovich & Rand, 2016; Zhu et al., 2019). This hypothesis also follows from research in different domains of psychology showing that people have difficulty in ignoring information – for instance, research on the perseverance effect (McFarland et al., 2007; Ross et al., 1975), repetition effects (Unkelbach et al., 2007; Weaver et al., 2007), and redundancy (Alves & Mata, 2019). Here too we suggest that people should ignore the input coming from an ignorant person, but that they might have a hard time doing so.

Note that the effect of interest here is a form of influence – a negative influence – as the person’s response is influenced by the ignorant’s response, and so the responder ends up giving a different response than the one they might give had they not learned about the ignorant’s response. For example, if a person is given a movie recommendation by someone they perceive to be ignorant about movies, this might actually lower the chances that the person will choose to watch the movie. Previous research in social psychology has shown similar cases of negative social influence, where people do the opposite of others (e.g., Ariely & Levav, 2000; Berger & Heath, 2007; Chan et al., 2012). However, in that research, people choose differently from others in order to signal non-conformity. In our research, we suggest that people might do the same but for different reasons: they do so because they infer that the ignorant is wrong, generalizing the rational heuristic that associates expertise with accuracy to the less defensible case of thinking that ignorance implies inaccuracy.

Data and histograms for all studies can be accessed at: https://osf.io/fnbua/.

2 Study 1

Study 1 tests whether people fall prey to this ignorance = error heuristic by assessing how likely they are to choose the same responses as other people who are described as experts or ignorant about some domain. In line with research on the effects of expertise on advice-taking and persuasion, we predict that people will give the same response as experts. But in line with the hypothesis presented here, we predict that people are more likely to reject than to follow the ignorant’s response.

2.1 Method

2.1.1 Participants

One hundred participants were recruited through the Prolific online platform (101 ended up participating; 65 Female, 27 Male, 9 who chose not to disclose). Participants were aged 18-74 (M = 33.43, SD = 12.88).

2.1.2 Materials

Twenty 2-choice history questions were collected (see Appendix I) from several internet quizzes. Questions were selected with the criteria that they were difficult, so that the correct answers would not be known to most (non-expert) participants (this was assessed independently by the authors, and the final set of questions were those for which there was agreement).

2.1.3 Procedure

Participants were told they would be asked to answer 20 history questions and presented with the answer of a previous participant (target: Susan or David) before they answered. One of the targets was described as an expert in history (see Appendix II for detailed instructions), and the other as completely ignorant about history — which of them (Susan or David) was described as ignorant or expert was randomized per participant.

Half the questions were randomly attributed to the expert, and half to the ignorant. The questions were presented sequentially, but always in two blocks: participants always saw all 10 questions to which one of the targets gave their answer, before seeing the 10 questions to which the other target answered — whether participants saw the ignorant’s or the expert’s questions first was counterbalanced.

Additionally, the study was coded so that each of the targets answered correctly only half the time, though participants were presumably not aware of whether the answers were correct or not. Thus, each participant answered five questions for which they saw an incorrect answer and five questions for which they saw a correct answer, from a supposedly ignorant person, and five questions with an incorrect answer and five questions with a correct answer from a supposedly expert person.

Finally, participants were debriefed and provided with the correct answer to all questions, along with the answers they had given.

2.2 Results

For each participant, the number of times they gave the same answer as the other person was computed, for both the expert and the ignorant. Without surprise, a one-sample t-test revealed that participants followed the expert’s answer on most trials (M = 8.07 out of 10, SD = 1.75, t(100) = 17.62, p < .001, d = 1.75). More importantly, participants chose the same option as the ignorant person less than 50% of the time (M = 4.41, SD = 1.90, t(100) = –3.09, p = .003, d = 0.31).

3 Study 2

Study 2 seeks to illuminate the mechanism underlying this bias, whereby people mistake ignorance for error. First, it tests whether people think that an ignorant person has a chance of success below random levels, which would explain why participants reject their response. Moreover, Study 2 tests the intuitive vs. deliberative nature of this heuristic. Participants made two judgments about the likely accuracy of the target person’s response: an initial fast judgment, and a slower and more careful judgment (Bago & De Neys, 2019; Thompson et al., 2011; Vega et al., 2021). If this heuristic results from faulty intuition, then it should manifest more clearly in time-constrained responses, and then be corrected when participants are given the chance to think more carefully about the problems. If, however, it is an explicit belief, then the manipulation of cognitive resources should not affect its expression.

3.1 Method

3.1.1 Participants

Ninety-five undergraduate students (82 Female, 1 Nonbinary, and 1 who did not disclose) were recruited for participation in return for course credit. Participants were aged 19-65 (M = 21.89, SD = 7.36).

3.1.2 Materials

The same twenty 2-choice history questions from Study 1 were translated to Portuguese.

3.1.3 Procedure

The procedure was similar to that of Study 1 with two key changes. First, for each question, participants were provided with both options, as well as the alleged previous participant’s answer. However, rather than having to choose their own answer, participants were asked to judge how right or wrong the other person’s answer was, on a 9-point scale (from 1- Definitely wrong to 9- Definitely right).

Second, following the 2-response paradigm that is commonly used in the study of intuition versus deliberation (e.g., Bago & De Neys, 2019; Thompson et al., 2011; Vega et al., 2021), participants were asked to answer each question twice: First they were asked to answer as quickly as they could, to gauge their intuitive answers, and then to answer a second time where they could take as long as they needed, having the chance to revise their initial answer (see Appendix III for the full instructions).

As in Study 1, half the questions were presented with the answer of the ignorant person — 5 questions with the correct and 5 with the incorrect answer — and the other half with the answer of the expert person — again 5 questions with the correct and 5 with the incorrect answer. But unlike Study 1, all the answers were presented in random order, rather than in blocks of expert/ignorant.

3.2 Results

One-sample t-tests show that judgments differed significantly from the middle point of the scale, for both levels of knowledge and for both the fast and slow judgments. Specifically, the expert was judged as likely to be right at both time 1 and time 2, and the ignorant as likely to be wrong at both time 1 and time 2. To account for item variance, we also tested the judgments at the item level and obtained the same pattern of results (Table 1).

Table 1: Average responses for fast and slow judgments, per target, analyzed at subject- and item-level (t tests refer to differences from the midpoint of the scale; Study 2).

In order to test whether participants significantly revised their answers, we used a repeated measures ANOVA that enabled us to compare judgments across fast versus slow judgments. There was a main effect of time (F(1, 94) = 7.27, p = .008), with slow judgments being overall lower, as well as expertise (F(1,94) = 169.60, p < .001), with experts being judged as more likely to be right than ignorants. More importantly, there was an interaction effect of judgment X expertise (F(1,94) = 13.64, p < .001): Paired-samples t-tests show that, while judgments for experts decreased from the fast to the slow response (t(94) = 4.42, p < .001, d = 0.45), judgments for ignorants were, as expected, revised upwards (t(94) = 1.69, p = .095 (one-tailed: .047), d = 0.11). Once again, testing at the item-level yielded the same pattern of results: main effects of judgment (F(1, 19) = 16.41, p < .001) and expertise (F(1, 19) = 602.98, p < .001), as well as an interaction effect of judgment X expertise (F(1, 19) = 119.10, p < .001). Paired-samples t-tests show that judgments for experts decreased from the fast to the slow response (t(19) = 11.13, p < .001, d = 2.49) and judgments for ignorants increased (t(19) = 3.29, p = .004, d = 0.74).¹

4 Study 3

There are five goals to this study: 1) To generalize the effect to a new knowledge domain: arts and culture; 2) to test the effect with a new measure: a summary estimate of the number of correct responses provided by the other person; 3) to assess the explicit heuristic belief that ignorance implies error; 4) to test the relation between this belief and the estimated number of correct responses provided by the ignorant person; and 5) to test the effect in a between-subjects design. Whereas Studies 1-2 manipulated the expertise-vs.-ignorance cue within-subjects, a between-subjects design prevents explicit comparison between the expert and ignorant conditions. In the literature on judgment and decision making it is well-documented that within- vs. between-subjects designs can modulate effects, making them larger or smaller, or even reversing them (e.g., Erlandsson, 2021; González-Vallejo & Moran, 2001; Hsee, 1996; Krüger et al., 2014), and the same might apply to the present effect. It is not easy, however, to anticipate the direction of the effect. It might be that the effect emerges only in within-subjects designs, as they might encourage participants to overgeneralize the sound heuristic that expertise signals accuracy to the irrational belief that lack of expertise signals inaccuracy. Or, on the other hand, the juxtaposition of expertise and ignorance could make it easier to realize the validity of one cue and the invalidity of the other.

4.1 Method

4.1.1 Participants

One hundred fifty-seven undergraduate students (127 Female, 5 who did not disclose) participated voluntarily. Participants were aged 17-46 (M = 19.44, SD = 3.55).

4.1.2 Materials

Ten 2-choice questions about Arts & Culture were collected from a game of Trivial Pursuit (see Appendix IV). The questions were selected with the criterion that none of the authors knew the correct answer.

4.1.3 Procedure

The study was carried out in pen and paper format. The study started by having participants answer 10 2-choice Arts & Culture questions by circling the correct answer. In the following page, participants were asked to imagine a hypothetical person, about whom they read a short description, and then predict how many of the previous questions this person would answer correctly. Finally, participants rated their agreement with three statements meant to gauge their heuristic beliefs (e.g., “Just like an expert on a subject is probably correct when answering questions about that subject, a person who is ignorant about that subject is probably incorrect when answering questions about that subject”).

For half the participants, the hypothetical person was described as an expert on arts and culture, and for the other half as ignorant about arts and culture. Additionally, the gender of this target person was counterbalanced (see Appendix IV for full instructions).

4.2 Results

One-sample t-tests show that the estimated number of correct responses differed significantly from 5, both in the expert condition (t(76) = 15.07, p < .001, d = 1.71) and in the ignorant condition (t(74) = –10.53, p < .001, d = 1.22), with participants expecting mostly correct responses for experts (M = 8.00 (out of 10), SD = 1.75) and mostly errors for ignorant responders (M = 2.75 (out of 10), SD = 1.85).

The three items pertaining to the heuristic belief that ignorance implies error were aggregated (α = .72). Participants in general endorsed this heuristic belief (M = 6.84, SD = 1.45). Moreover, this belief correlated with estimates in the critical ignorant condition, such that the more participants believed in the heuristic, the more they estimated that the ignorant would make more mistakes (r = –.43, p < .001²; this correlation holds for each of the three items: r ≥ –.24, p ≤ .036).

Lending further support to the hypothesis that estimates that the ignorant person would give more incorrect than correct responses are based on a belief in the heuristic tested here, participants who estimated that the ignorant would give 5 correct responses (chance level) scored lower on the heuristic belief items (M = 6.56, SD = 1.34) than those who estimated that the ignorant would have less than 5 correct responses (M = 7.30, SD = 1.15; t(69) = 2.04, p = .045, d = 0.63). In a less conservative test that compared those who estimated that the ignorant would have less than 5 correct responses against those who estimated that they would have at least 5 or more,³ this difference is clearer still (M = 7.30, SD = 1.15 vs. M = 6.47, SD = 1.32; t(73) = 2.55, p = .013, d = 0.70).

5 Study 4a

In Study 4 we asked participants to estimate the performance of an ignorant person in a set of knowledge questions with dichotomous response options (as in Study 3), but we also asked them to estimate the performance of the same person in a set of questions where it should be clearer that success is at chance level: In Study 4A, this involved guessing whether the outcome of coin tosses is heads or tails; In Study 4B, it involved choosing blindly between two response options. The goal was to 1) replicate the results of the previous studies, and 2) perform a sanity check on our paradigm, to make sure that participants understand that random outcomes for dichotomous-response questions have a likelihood of 50%. Finally, 3) we manipulated the order in which those two estimates (knowledge and coin tosses) were requested, in order to test whether making estimates about obviously random events (i.e., coin tosses) first would then debias estimates about events that might otherwise be perceived as less random (i.e., knowledge).

5.1 Method

5.1.1 Participants

Two hundred participants were recruited through the Prolific online platform. Of these, 9 failed the attention check, and therefore the final sample size was 191 (113 Female, 59 Male, and 20 who chose another category or did not disclose their gender). Participants were aged 18–71 (M = 37.23, SD = 12.86).

5.1.2 Procedure

Participants were told to consider two games where someone (referred to as X) would be asked to answer 10 questions. Participants were further told that their task would be to estimate how many of those questions X would answer correctly. In one game, coin, they were told X would go through 10 coin tosses and, each time, try to guess which way the coin would land. In the other game, knowledge, they were told X would answer 10 two-choice Art History questions, and they were shown an example — “Who painted Woman Listening to Music?”: a) “Miró” or b) “Matisse”. Participants were informed that X knew nothing about neither the coin — and thus had no idea what the outcome of the coin tosses would be — nor Art History — and thus had no idea about what the correct answers were. After reading about each game, participants estimated how many, out of 10 coin tosses/questions, X would answer correctly. All participants made estimates for both games, in a counterbalanced order. Unlike Study 3, participants did not read all the questions. This was done so that participants’ inferences regarding the questions would not contaminate their judgments.

We included a captcha at the beginning of the study, and a multiple-choice attention check at the end: “In this study, you were asked to estimate how many questions of a certain topic a person would answer correctly. What was the topic of those questions?”. Participants who failed to select “Art History” were excluded from the analysis.

5.2 Results

One-sample t-tests against five show that people underestimated the number of correct answers for both the coin tosses (t(19) = 6.67, p < .001, d = 0.48) and the knowledge task (t(190) = 8.37, p < .001, d = 0.61). Still, a mixed model ANOVA with estimated number of correct responses for knowledge and guessing coin tosses as a within-subjects factor and order of presentation of those domains as a between-subjects factor revealed a main effect of domain (F(1, 189) = 22.54, p < .001), such that estimated performance was higher and closer to 50% for guessing coin tosses (M = 4.55, SD = 0.93) than for knowledge (M = 4.03, SD = 1.60).

There was also a main effect of order (F(1, 189) = 4.29, p = .040), such that estimated performance was higher in the condition where knowledge came before coin tosses than in the other order condition (M = 4.44, SD = 0.89 vs. M = 4.13, SD = 1.07). Admittedly, we expected this order effect to be either non-significant or to go in the opposite direction (if going through the coin tosses task would then improve estimates for the knowledge task). Still, the largest effect was the predicted difference in estimated performance across tasks, which did not differ across order conditions (for the interaction, F < 1).

In conclusion, participants discriminated between domains where randomness was clearer (guessing the outcome of coin tosses) versus less obvious (answering knowledge questions). However, having gone through the obviously random domain first did not improve estimates for the less obviously random domain.

6 Study 4b

6.1 Method

6.1.1 Participants

Two hundred participants were recruited through the Prolific online platform. Of these, 2 failed the attention check (the same check that was used in Study 4A), and therefore the final sample size was 198 (124 Female, 66 Male, and 8 who chose another category or did not disclose their gender). Participants were aged 19-80 (M = 38.03, SD = 13.43).

6.1.2 Procedure

Study 4B was similar to Study 4A, with some differences. First, rather than a coin toss, the knowledge condition was contrasted with a condition where the target also had to answer Art History questions, but did so blindly. Specifically, the target was described as having the answers to each question sealed inside closed envelopes and, thus, having to pick one of the two answers without knowing which option they were picking. In this way, we wanted to make it clear to participants that the answers were completely random.

Second, to avoid consistency or transfer effects across task conditions, we presented a different target person in each task (we referred to target X in the blind answer condition, and Y in the ignorant answer condition).

6.2 Results

One-sample t-tests against five showed that people once again underestimated the number of correct answers for both the blind responder (t(197) = 6.21, p < .001, d = 0.44) and the ignorant responder (t(197) = 6.88, p < .001, d = 0.49). But again, a mixed model ANOVA with estimated number of correct responses for knowledge and blind choice as a within-subjects factor and order of presentation of those domains as a between-subjects factor revealed a main effect of domain (F(1, 196) = 7.13, p = .008), such that estimated performance was higher and closer to 50% for blind choices (M = 4.57, SD = 0.97) than for knowledge (M = 4.30, SD = 1.42).

There was also a main effect of order (F(1, 196) = 11.53, p = .001), such that estimated performance was higher in the condition where blind choice came before knowledge than in the other order condition (M = 4.67, SD = 0.84 vs. M = 4.21, SD = 1.09). This effect went in the opposite direction to that observed in Study 4A, but it is more in line with the predicted effect (going through the blind choice task first should improve estimates for the knowledge task).

Finally, and as in Study 4A, the interaction effect was not significant (F < 1), which again leads to the conclusion that participants discriminated between domains where randomness was clearer (blind choices) versus less obvious (answering knowledge questions). However, having gone through the obviously random domain first did not improve estimates for the (apparently less obviously random) domain: ignorance.

7 General Discussion

The present research (using different samples from different countries, online and in the lab) revealed a bias whereby people mistake ignorance with error. Participants in the present studies were shown the responses of other people who were described as either knowledgeable or ignorant about a certain topic. Participants in general believed that the knowledgeable person was likely to be correct (Studies 2–3), and therefore they were likely to follow this person’s response (Study 1). This makes perfect normative sense, as expertise is naturally associated with accuracy. However, the results also showed that participants also relied on ignorance as a cue for accuracy – in this case, lack of accuracy. This is, we argue, normatively inappropriate, as the chance of a completely ignorant person being correct when answering to questions with dichotomous response options can never be lower than 50%. And, as we argued in the introduction, even the slightest bit of knowledge, or the ability to extrapolate from related knowledge, would improve those odds.

Thus, the present results reflect the overgeneralization of a sound heuristic (expertise = accuracy) to cases where the heuristic fails (the absence of expertise does not imply inaccuracy). Nevertheless, a more nuanced reading of these results is that participants seemed to be somewhat aware of this bias, as the positive effect of expertise was larger than the negative effect of ignorance. This fact, which was consistent throughout studies, enables a more generous interpretation of the present results.

Relatedly, one may ask whether this bias results from an intuitive heuristic, which one can correct upon further deliberation, or whether it results from an explicit belief. On the one hand, the results of Study 2 show that participants did revise their intuitive response, and they showed less bias when they could think more carefully about the problems. This suggests that this bias stems from faulty intuition, and that people can correct for it (yet another point in favor of a more generous reading of this effect).⁴ On the other hand, however, the bias manifested in both the intuitive and the deliberate responses, and in Study 3 participants explicitly expressed their belief in the heuristic that ignorance is synonymous with error, which suggests that either people realize it is a bias but insufficiently correct for it, or that people might not even realize it is a bias to begin with.

Study 4 shows that the bias is not a consequence of a basic failure to understand that random dichotomous events have a 50% chance. Indeed, participants’ estimates were better (i.e., closer to 50%) when guessing how the ignorant person would perform in tasks where the outcome was more obviously random (coin tosses, in Study 4A, or blindly choosing the response options, in Study 4B). Still, thinking about the obviously random task first did not debias estimates for the domain where randomness was less obvious (knowledge). We were unsure whether the order manipulation (i.e., going through the obviously random task before or after the ignorance task) would be effective in eliminating the bias, though the order manipulation was implemented as (what seems to us) a reasonable test of whether realizing that random = 50% chance of success would then prompt better judgment for the focal ignorance task. The fact that it didn’t suggests that this is a robust and powerful bias, and that beliefs about randomness might be domain-specific. Apparently, responding blindly is considered to have a better chance of success than choosing under ignorance.

One question one might ask is whether it might sometimes be rational to follow this ignorance=error heuristic and give the opposite response to the one given by an ignorant person. Particularly for tricky problems involving counter-intuitive reasoning and judgment (e.g., the typical tasks used in the heuristics-and-biases research program), it might be a good strategy to choose the opposite of what an ignorant/illogical person responded. For such tasks, correct responders are often aware that there is an intuitive but incorrect response option that is quite appealing (as they too might have thought of giving that intuitive response at first; Mata, Ferreira, et al., 2013; Mata, 2019ab, 2020). However, as they are aware of the trick, they do not become less confident but rather grow more confident in their less consensual (but more logical) response. Indeed, research shows that correct responders in these tricky tasks are often quite good at using cues about how other people respond (e.g., quickly or slowly) to infer their responses (i.e., the incorrect/intuitive response or the correct/deliberative response; Mata & Almeida, 2014). Future research might explore whether the perceived ignorance of the other person can also serve as a cue that can be validly used when deciding about one’s response and how confident one is about it. Consensus is a powerful cue for confidence judgments (i.e., the more people favor a certain opinion, the more correct it is perceived to be), but the perceived expertise versus ignorance of the opinion-givers might moderate this effect, mitigating or reversing it (i.e., confidently choosing the opposite of what an ignorant majority defends).

Another question concerns the size of the ignorance=error effect. This question might be answered by examining the effect size for aggregate results, or by analysis of individual-differences. Concerning the former, and considering as cut-off levels that a Cohen’s d of 0.2 corresponds to a small effect size, d ≥ 0.5 corresponds to a medium effect size, and d ≥ 0.8 is a large effect size, we observed a small effect in Studies 1, 2, and 4B (in Study 1 this is not surprising, because this study did not directly test the ignorance=error bias, but rather a consequence of that bias whereby people choose to respond differently from the ignorant person, presumably because they think that he is incorrect, which can be seen as a stronger test, or at least a less direct test of the hypothesis), a medium effect in Study 4A, and a large effect in Study 3.

In a further individual-differences analysis, we assessed whether the bias holds for most responders, or rather whether it is the result of a minority of responders manifesting the bias to extreme degrees (i.e., a large bias in a small subsample). This analysis compares the percentage of biased vs. non-biased responders. In Study 1, we considered those who responded the same as the ignorant person in < 50% of the trials vs. those who did it in 50% of the trials: 48.5% vs. 25.7%. In Study 2, because participants gave several ratings, we considered instead the percentage of slow/deliberate ratings for the ignorant target that fell below the midpoint of the scale vs. the ratings that were in the middle of the scale: 48.1% vs. 16.5%. In Studies 3 and 4, we considered responders who estimated that the ignorant person would have < 50% of correct responses vs. those who estimated 50% accuracy: 77.3% vs. 17.3% in Study 3; 41.5% vs. 53.5% in Study 4a; and 36% vs. 58% in Study 4b. Thus, one might say that the proportion of biased responders is at least non-negligible, or even the majority in some of the studies. In either case, we should not attribute the average effect to a small minority displaying a large bias; the effect seems more generalized. These two analyses seek to inform about the magnitude of the bias, both in terms of how large it is for the average responder and how pervasive it is across individuals. In either analysis, the magnitude of the bias is non-negligible. See Appendix VIII for histograms of responses in Study 4a and 4b, which illustrate the sorts of distributions found in all studies.

Yet another question is how impactful the bias is in everyday judgment and decision-making. Of course, the 50% odds mentioned earlier apply only to dichotomous response tasks. When confronted with scenarios that have more than two possible options, it can be rational to assume any given answer is more likely incorrect than correct. However, we do not propose that the bias refers simply to estimating a lower than 50% chance of being correct. Rather, we suspect that for tasks with more response options (e.g., 4 instead of 2), people might still underestimate the probability of the ignorant’s response being correct (i.e., estimating less than 25%). Future research might test whether the present hypothesis holds for questions with more than two response options.

Many heuristics and biases are explained as cases of attribute substitution (Kahneman & Frederick, 2002), whereby one takes a cue (e.g., fluency or ease of thought) as an indicator of the dimension that one is actually trying to estimate (e.g., frequency or probability), thus replacing a hard question with a much easier computation. In the case of the present bias, the problem is not so much one of relying on more or less valid cues to make inferences about a related attribute, but rather one of overgeneralizing a sound rule to cases where it no longer holds, due to a logical fallacy of reverse inference. Such fallacies have already been shown to underlie other cases of overgeneralization that give rise to heuristics and biases.⁵ The rule may even be appropriate in some situations, such as politics: If we are uncertain about whether to support a proposal, but know that members of the opposite party support it, that alone may give us good reason to oppose it (though it can also lead to biased judgments; Cohen, 2003).

That said, there are quite a lot of cases where real-life decisions are two-sided and where a bias may occur. Indeed, we would argue that this bias is not limited to trivia games, but that it applies to a wide range of domains. Several examples come to mind: whether to watch a certain movie or not; to go on vacation to a certain destination or not; to buy or sell a stock or currency; to choose one graduate program or another; to vote in favor or against a certain policy in a referendum; to take one route or means of transportation versus another to get to a certain destination; to take a safe bet versus a risky gamble; to hire a job candidate or not; to try out a certain health regime or physical exercise practice; choosing to undergo a certain medical procedure or not; or of greater relevance recently, to choose to get vaccinated or not. Or it might even be that the same bias holds when thinking about one’s own competence at a certain domain: For instance, true/false or multiple-choice exams are sometimes graded with a penalty for guessing, which forces people to choose whether to respond at all. If they think that they are more likely to be wrong than chance, they may choose not to guess even when they know enough to be better than chance. These are but a few of many domains where life asks us to choose between door A and door B, and the perceived expertise (or lack thereof) of the responder may sway us in one direction or another: either prompting one to choose in line with their suggestion, or against it, if one believes them to be ignorant about a certain topic (as in Study 1).

Indeed, these studies revealed a curious case of social influence where a person wishes to avoid the response of the other person, and ironically ends up being influenced by it. If the definition of social influence is the influence that other people exert on one’s thoughts and behavior, then negative influence (i.e., choosing the opposite of what others chose) is just as good a case of social influence as positive influence (i.e., choosing the same as others). Moreover, we hope that we were convincing in arguing that this is a bias. The argumentative theory of reasoning and epistemic vigilance (Mercier & Sperber, 2011; Sperber et al., 2010) argues that rationality is well-adapted to the social context in which people navigate, and that people are sensitive to the credibility of others, and the possibility that they are wronged by others (because others might be deceitful or incompetent sources). We fully agree that epistemic vigilance and sensitivity to social context are adaptive. However, the strategies that one uses upon perceiving low credibility in others might vary in their adaptiveness: they might be reasonable, as when people put further deliberation into scrutinizing the responses of others who they perceive to be biased (Janssen et al., 2021; Mata, Fiedler, et al., 2013; Trouche et al., 2016); but they might also be less reasonable, as when they uncritically give the opposite response, as in the present studies.