Epistemic justification and the folk conceptual gap

Abstract

Recent experimental epistemology has devoted increasing attention to folk attributions of epistemic justification. Empirical studies have tested whether lay people ascribe epistemic justification in specific lottery-style vignettes (Friedman and Turri 2014, Turri and Friedman 2015, Ebert et al. 2018) and also to more ordinary beliefs (Nolte et al. 2021). In this paper, I highlight three crucial yet hitherto uncritically accepted assumptions of these studies, and I argue that they are untenable. Central to my criticism is the observation that epistemic justification is a philosophical term of art mostly foreign to lay people: as such, it is not suitable for direct empirical testing without being previously introduced. This point reveals a folk conceptual gap between the subject matter of these experimental studies and the conceptual repertoire we can reasonably expect lay people to possess. I elaborate on this worry, and I end on a cautiously optimistic note: after suggesting better strategies to survey folk attributions of epistemic justification, I conclude that the challenge raised by the folk conceptual gap remains difficult but can in principle be addressed.

1. Introduction

After a long-standing focus on lay ascriptions of knowledge, experimental epistemologists have recently turned the attention to folk attributions of epistemic justification. ¹ In two extensive studies, Turri and Friedman (2014: 2015) investigated whether lay people ascribe epistemic justification to lottery propositions (i.e., that a particular ticket has lost a large enough lottery). The very same issue was further explored in another detailed study by Ebert et al. (2018). Notably, interest in folk attributions of epistemic justification is not circumscribed to lottery-style cases: with the aim of assessing the thesis that knowledge entails justified belief, Nolte et al. (2021) have surveyed folk attributions of knowledge and epistemic justification in more ordinary scenarios than lottery-style vignettes. Taken together, these studies underscore a shift in interest towards folk attributions of epistemic justification. Individual differences aside, experimental epistemologists share one common expectation: folk attributions of epistemic justification ‘can straightforwardly be tested’ (Ebert et al. 2018: 112) by simply asking to lay people whether they regard the belief under consideration to be justified.

In this paper, I cast serious doubt on this expectation. Folk attributions of epistemic justification cannot be straightforwardly tested nor should they be surveyed as they currently are. Unlike knowledge, epistemic justification is a technical concept that needs to be carefully introduced and explained. A folk conceptual gap has gone unnoticed: lay people can hardly be expected to possess and master the concept of epistemic justification familiar to epistemologists. One negative aim of this paper is to criticise the study design of these experimental studies in light of this folk conceptual gap. However, despite this criticism, I remain sympathetic to the general project of investigating folk attributions of epistemic justification, as these empirical data can benefit epistemological theorising. Accordingly, this paper also pursues a more positive aim: proposing a better way to survey folk attributions of epistemic justification.

My plan is as follows. In Section 2, I take a closer look at the design of these experimental studies, and in Section 3, I identify three crucial but uncritically accepted assumptions underlying the choice of their probe questions and vignettes. The significance and relevance of these studies essentially depend on the tenability of these assumptions. In Section 4, I reject each assumption in turn, and I articulate in more detail the difficult challenge raised by the folk conceptual gap. I end on a more cautiously optimistic note: after suggesting better strategies to survey folk attributions of epistemic justification, I conclude that the challenge raised by the folk conceptual gap remains difficult but can in principle be addressed. ²

2. A closer look: Vignettes and probe questions

The current studies surveying folk attributions of epistemic justification are very rich. Owing to limitations of space, I cannot do justice to their complexity here. Instead, I will focus directly on the details that matter for my purposes, namely the choice of vignettes and probe questions. ³

In their two experimental studies, Friedman and Turri are interested in the folk epistemology of lotteries: they aim to investigate whether lay people attribute knowledge and justification to lottery propositions (i.e., that particular ticket of a fair big enough lottery will lose). Epistemologists have often claimed that one cannot know that a particular ticket is a loser solely on the basis of the odds involved and that knowledge of lottery propositions requires testimonial evidence – for example, hearing on the evening news the announcement of the winning numbers (Pritchard 2005; Williamson 2000). These claims play an important role also for theorising about epistemic justification, as some (though not all) epistemologists have claimed that one cannot justifiably believe that a particular ticket is a loser solely on the basis of the odds involved (Nelkin 2000; Smith 2010). Friedman and Turri’s experimental studies aim to examine whether folk judgements about lottery beliefs lend support to these claims often made by epistemologists. To achieve such aim, they presented the participants with these vignettes:

Lotto. ⁴ Abigail is out shopping with her son. In a store, they see a man with a super Lotto ticket. Abigail’s son says: ‘I bet that ticket’s not a loser. It might win the jackpot!’. Abigail answers, ‘It is a losing ticket’. And Abigail is exactly right: the ticket is a loser. (Friedman & Turri 2015: 1065)

News. Ellen bought a ticket in this week’s Super Lotto. Her numbers are 49-20-3-15-37-29-8. Ellen just finished watching the evening news, and they reported that a completely different number won. It was the same newscaster that reports the winning number every week on the local channel that Ellen watches. On that basis, Ellen concludes that her ticket lost. And she’s right: her ticket lost. (Turri & Friedman 2014: 51)

The vignettes introduce the contrast of interest between statistical and testimonial evidence. To survey folk attributions of epistemic justification, the following were used as dichotomous test questions in Lotto and News: (1) Abigail is [justified/unjustified] in thinking that the ticket is a loser (Friedman & Turri 2015: 1066) and (2) Ellen is [justified/unjustified] in thinking that the ticket is a loser (Turri & Friedman 2014: 49). Notably, these probe questions did not draw the attention to the evidence available to the protagonist of the vignette nor did they include additional qualifications specifying the relevant sense of epistemic justification.

Ebert et al. (2018) are also interested in the folk epistemology of lotteries. To build the contrast between statistical and testimonial evidence, they presented participants with the following case-pairs:

Probabilistic Evidence. Harry owns a single ticket in a lottery. His numbers are 23-42-12-8-28-31. The only way for a ticket to win is to match all six numbers, and any other ticket is a losing ticket. Harry wasn’t able to watch the evening news at which the winning numbers were announced. He recalls from his statistics class that the chance of a single ticket winning this lottery is one in 14 million, i.e., there is a 99.999993% chance that his ticket has lost. (Ebert et al. 2018: 116–117)

Testimonial Evidence. Harry owns a single ticket in a lottery. His numbers are 23-42-12-8-28-31. The only way for a ticket to win is to match all six numbers, and any other ticket is a losing ticket. Harry has just watched the evening news at which the winning numbers were announced and none of his numbers match. (Ebert et al. 2018: 117)

It’s immediately worth noting that Ebert and colleagues’ study shows significant differences from the two studies on lottery beliefs run by Turri and Friedman. Firstly, these vignettes are more precise. The probabilities are made explicit, and it is clear that a winning ticket must match all the numbers. Moreover, since these vignettes do not specify that the ticket is in fact a loser, concerns about truth effects on belief evaluation and outcome bias are plausibly avoided (Gerken 2020). Secondly, and perhaps more importantly, Ebert and colleagues are more cautious. They explicitly refrain from drawing strong conclusions from their experimental surveys, and they do not reject any normative theory of epistemic justification on the basis of empirical data (Ebert et al. 2018: 134). ⁵

Yet, despite these improvements and differences, the probe questions asked to the participants remain the same in both Probabilistic Evidence and Testimonial Evidence: (3) Suppose Harry believes that the ticket has lost. Is Harry justified in believing that the ticket has lost? Yes/No (Ebert et al. 2018: 117). Participants could express confidence in their answer on a five-point Likert scale. ⁶ Once again, the probe question on (epistemic) justification did not include additional qualifications or mention the available evidence.

The study by Nolte, Rose and Turri (2021) has a different focus: they investigated whether folk attributions of knowledge and justification support the view that knowledge entails justified belief. Predictably, they presented a series of vignettes to the participants and ‘simply asked for both knowledge and justification’ (Nolte et al. 2021: 32). Here’s one vignette of interest:

Accusation. Andy’s daughter has been accused of a serious crime. She lacks an alibi and the police have gathered strong evidence against her. But, even after considering it all, Andy feels that she must be innocent. And in fact he is correct: his daughter is innocent. (Nolte et al. 2021: 34)

Two test statements followed, and participants could express agreement with each statement on a six-point Likert scale. The first statement concerned whether Andy knows that his daughter is innocent. The second statement was meant to survey attributions of epistemic justification: (4) Andy is justified in believing that his daughter is innocent (Nolte et al. 2021: ${\rm{\S}}$ 1.4). In this experimental study too, the justification question did not include any additional qualification. ⁷

Let’s take stock. Each of these experimental studies is supposed to target folk judgements about epistemic rather than pragmatic or moral justification, but surprisingly enough none of these studies actually qualifies or mentions such target explicitly. To fully appreciate this point, here’s a list of the probe questions for epistemic justification used in each study:

1. 1) Abigail is [justified/unjustified] in thinking that the ticket is a loser.

2. 2) Ellen is [justified/unjustified] in thinking that the ticket is a loser.

3. 3) Is Harry justified in believing that the ticket has lost? Yes/No.

4. 4) Andy is justified in believing that his daughter is innocent.

These unqualified occurrences of justification should give us pause, as well as reasons to raise the alarm about possible equivocations. Why should we think that philosophically untrained participants correctly understood what they were asked? The validity of these experimental studies essentially depends on a satisfactory answer to this question and on further substantial assumptions. I tackle this question and bring out these assumptions in the next section.

3. The assumptions outlined

The success of these experimental surveys of folk attributions of epistemic justification is conditional on some substantial assumptions: if we are to take these surveys at face value, then these assumptions have to be granted. While these assumptions have never been made explicit, it is important to bring them out clearly in order to best assess the validity and the relevance of these experimental studies. The assumptions are the following:

Folk Understanding . Lay people can plausibly be taken to understand that they are being asked to assess whether a subject’s belief is epistemically justified.

Immediate Testability . Folk attributions of epistemic justification can be tested immediately without first introducing and explaining the concept of epistemic justification.

Theoretical Neutrality . Folk attributions of epistemic justification can be surveyed independently of any substantial theory of epistemic justification explicitly assumed by the experimenters or implicitly attributed to the participants.

A few comments on each. Firstly, Folk Understanding must be granted in order for the data to have any significance. For example, an interesting result replicated by each experimental study on lottery propositions is that lay people generally tend to ascribe justification to these propositions on the basis of both statistical and testimonial evidence (Ebert et al. 2018: §5.1). This result can inform epistemological theorising on epistemic justification only if participants correctly understand the question and have a grasp of what it is for a belief to be epistemically rather than morally or prudentially justified. Without this assumption, lay people would be assessing a different type of justification, and the data would be about something else. ⁸

Secondly, and relatedly, Immediate Testability must also be granted; otherwise, the choice of vignettes and probe questions would be inapt: as highlighted in the previous section, neither the vignettes nor the probe questions make any attempt to introduce the concept of epistemic justification. As a result, it must be granted that folk attributions of epistemic justification can be tested without having to provide any preliminary explication of epistemic justification.

Finally, Theoretical Neutrality captures the ambitions of generality of these experimental studies. As Nolte et al. admit (2021: 46), these studies are not testing whether folk attributions of epistemic justification favour any particular theory of the nature or structure of justification (internalism or externalism, and coherentism or foundationalism). Instead, they are surveying folk attributions of epistemic justification in general and regardless of the implicit folk notion of epistemic justification that lay people operate with. Accordingly, we must also take for granted that folk attributions of epistemic justification can be surveyed neutrally, and without making any reference in the vignettes to substantive theories of epistemic justification.

Summing up, these are fairly minimal but ultimately necessary requirements for the validity and relevance of these experimental studies. First, that participants can plausibly be taken to understand and assess the epistemic justification of beliefs. Second, that the concept of epistemic justification needs no previous explication in the vignettes. Third, that lay people attribute epistemic justification independently of any substantial theory of epistemic justification.

Unfortunately, these assumptions will not hold up to scrutiny, and I will provide specific considerations against each of them. But before doing so, I want to be charitable and pause to make the following observation: there is a seemingly good reason why these assumptions were made in the first place. All these experimental studies were designed to survey attributions of both knowledge and epistemic justification, and these assumptions are innocent – perhaps even natural – for the purpose of surveying lay attributions of knowledge. Of course lay people understand the question when they are asked to attribute knowledge. Of course there is no need to previously explicate the concept of knowledge in order to survey knowledge attributions. And of course, lay people can attribute knowledge independently of any specific theory of knowledge: intuitively enough, knowledge attributions require no prior grasp of anti-Gettier conditions. ⁹

Yet, what is true of knowledge need not apply to epistemic justification, and in this case, it plausibly does not. As I show next, there’s an important asymmetry between lay attributions of knowledge and folk attributions of epistemic justification, and the assumptions made to investigate the folk epistemology of the former do not automatically carry over to the latter. The assumptions underlying these experimental studies neglect the following key point: lay people cannot be expected to already possess or immediately grasp and apply the epistemologist’s concept of epistemic justification. This reveals a folk conceptual gap between the subject matter of these experiments and the conceptual repertoire available to the participants. Rejecting each assumption in turn will bring out this gap more clearly; this is the task of the next section.

4. The assumptions rejected

The significance of the data concerning folk attributions of epistemic justification depends on whether the participants correctly understand the probe questions. However, none of the experimental studies explicitly qualify the kind of justification which is supposed to be surveyed. To evaluate the lottery vignettes, participants were asked whether the protagonist is ‘justified’ in thinking (or believing) that their ticket is a loser. Similarly, in Accusation, participants were asked if the protagonist is ‘justified’ in believing that their daughter is innocent.

The absence of a suitable qualifier is a problem for Folk Understanding . Participants may have reasonably picked up on the more ordinary sense of ‘justified’ and interpreted it as pragmatically or morally instead of epistemically justified. This diagnosis gains further support from empirical data recently gathered through corpus analysis. Broadly speaking, corpus linguistics aims to gain insight into ordinary language by surveying large collections of texts in plain English such as the Corpus of Contemporary American English (COCA), the British National Corpus (BNC), or the EnTenTen20 Corpus. Evidence from a recent corpus study on epistemic justification suggests that the expression ‘justified belief’ occurs almost exclusively in philosophical discourse: in the EnTenTen Corpus alone, 91% of the occurrences of justified belief belong to a philosophical context (Grindrod 2022: 19). And when ‘justified’ and ‘justify’ occur in non-philosophical contexts, they are used to evaluate actions or emotions, not beliefs (Grindrod 2022: 21). These results jointly indicate that the probe questions employ unusual terminology: in ordinary discourse, justification talk does not normally take belief as an object. This problem is compounded by the salient moral features in Accusation: the protagonist forms a belief about the guilt of his very own daughter. Participants may have reasonably attributed justification for reasons of partiality and thought that the protagonist is morally justified in believing that his daughter is innocent because he’s her father. ¹⁰ These considerations against Folk Understanding are however limited in the following important sense. They apply more clearly to the study run by Nolte and colleagues, but it remains unclear whether a similar problem affects the surveys of lottery beliefs. Despite the often noted connection between lottery beliefs and action (e.g., Hawthorne 2004: 29–30), it’s difficult to characterise with precision how exactly participants may interpret justification in a pragmatic sense when asked about a losing lottery ticket. Moreover, some sensitivity to epistemic cues may suggest that participants are picking up on the epistemic sense of justification. In support of this point, Ebert and colleagues found that folk attributions of epistemic justification are influenced by changes in probabilities due to the size of the lottery and also by further contextual information on the type of evidence in the vignettes (Ebert et al. 2018: 126; 133). Relatedly, it’s hard to maintain that participants are confused about different senses of justification given the high confidence rates replicated in each of the three studies on folk attributions of justification in lottery cases. For these reasons, the problems I raised with Folk Understanding are particularly pressing for Nolte and colleagues’ study, but they are less troublesome for Ebert and colleagues and Turri and Friedman’s surveys.

Despite this acknowledged limitation, the assumption remains at the very least questionable. Taken together, the facts that the ordinary use of justification tracks pragmatic or moral propriety and that the vignette emphasises moral features cast some doubt on Folk Understanding : overall, participants could not obviously assess epistemic justification.

Immediate Testability , the assumption that folk attributions of epistemic justification can be tested immediately without providing any explication of the concept at issue, is also untenable. Technical concepts seem especially unsuited to be surveyed directly without first being introduced or explained, and epistemic justification is often taken to be a technical concept denoted by a philosophical term of art. ¹¹ In fact, one point of agreement in the otherwise divided literature on epistemic justification is exactly that epistemic justification is a technical concept. In support of this, textual evidence abounds: as epistemologists have emphasised, epistemic justification is ‘a term from the philosophy classroom’ (Hawthorne & Stanley 2008: 578) an instance of ‘philosophical jargon’ (Reynolds 1998: 533), ‘a term of art in contemporary epistemology’ (Kvanvig 2010: 25) and ‘manifestly technical terminology’ (Williamson forthcoming: 3). For this reason, we should be sceptical of the possibility of surveying folk judgements about a distinctively technical notion that is not in any way introduced or explained. ¹²

Crucially, the problems with Immediate Testability will be more or less pressing depending on the exact dialectical role played by the empirical data. The more technical aspects of the concept of epistemic justification are particularly problematic when folk attributions of epistemic justification are taken to count against a normative theory of epistemic justification (Turri & Friedman 2014: 64), or to support the more substantive thesis that knowledge entails justification (Nolte et al. 2021: §5). These technical aspects may be less problematic when the empirical data are more modestly used to corroborate various armchair predictions on folk attributions of epistemic justification (Ebert et al. 2018 pursue this markedly different aim).

Bracketing such specific dialectical subtleties, the problems with Immediate Testability also affect Theoretical Neutrality . Since the concept of epistemic justification needs some introduction and explanation, folk attributions of epistemic justification can hardly be theory-neutral: these judgements will inevitably be constrained by the specific and substantive theory of epistemic justification which needs to be introduced. I return to this point in more detail below; now I just want to note that both generality and theoretical neutrality seem unfeasible given the theory-laden subject matter of these experimental studies.

Summing up, the design of all these experimental studies is problematic on the following fronts. Firstly, because it fails to specify the epistemic rather than the moral, pragmatic or prudential sense of justification, it employs ambiguous probe questions. Secondly, it purports to survey directly folk judgements on a technical notion which is neither introduced nor explained to the participants. Finally, it aims for a kind of theoretical neutrality which seems hard to achieve with the notion of epistemic justification.

With these observations in play, we can appreciate the more fundamental problem. These experimental studies neglect an important gap between their subject matter, folk attributions of epistemic justification, and the conceptual repertoire that we can reasonably expect lay people to possess. It is unreasonable to expect lay people to possess and master the concept of epistemic justification routinely employed by epistemologists, or to grasp it without previous explanations. It is equally unreasonable to survey folk judgements on justified belief directly using terms that are not ordinarily used to assess beliefs. Such is the folk conceptual gap, which crucially went unnoticed. Each experimental study trades on assumptions that overidealise the conceptual repertoire of the participants, and this overidealisation is reflected in the choice of the vignettes and probe questions.

It’s important to note that the folk conceptual gap does not necessarily point at a performance error nor does it predict that lay people make obvious mistakes in assessing the vignettes. The gap is in place because the probe questions employ ambiguous and technical terminology which led the participants to reasonably pick up on different aspects of the justification question. Participants are not being irrational because they should but nonetheless fail to appreciate the epistemic sense of justification: rather than highlighting the irrationality of the participants, the folk conceptual gap may underscore more subtle instances of dysrationalia, a term coined by Stanovich (2009) to denote the inability to think accurately despite adequate levels of intelligence and rational capacities. Dysrationalia is caused by mindware gaps (Stanovich 2009: 129–130; 150–151; West 2010: 102), broadly conceived as the lack of relevant concepts and the knowledge required for accurate responses to specific cognitive tasks. A strikingly similar analogy applies to folk attributions of epistemic justification: participants confidently assess the vignettes but nonetheless lack the relevant concepts, knowledge, and training to evaluate whether a belief is epistemically (as opposed to morally, or pragmatically) justified. Mindware gaps of this kind might constitute the cognitive bases of the folk conceptual gap. ¹³ Unless this gap can be bridged, a major challenge to survey folk attributions of epistemic justification remains unaddressed.

5. Folk attributions of epistemic justification: the way forward

The folk conceptual gap raises a considerable challenge. Can this challenge be addressed? If so, how? I will argue that while there are compelling reasons for a strongly pessimistic answer to these questions, different strategies to survey folk attributions of epistemic justification may warrant some cautious optimism. My aim is not to offer the perfect strategy to survey folk attributions of epistemic justification: more modestly, my main goal is to emphasise that the challenge raised by the folk conceptual gap is hard to address and thus worth taking seriously. I shall do so by pointing out the merits of pessimism, as well as the limitations of optimism.

5.1 Between strong pessimism and cautious optimism

The challenge raised by the folk conceptual gap turns out to be quite effective in casting serious doubt on the very possibility of surveying folk attributions of epistemic justification. Building on these initial doubts, a pessimistic line of response to the challenge will hold that folk attributions of epistemic justification simply cannot be surveyed. This pessimistic line of response enjoys a good deal of plausibility, and it can be motivated on the basis of two types of considerations.

Firstly, on fairly general grounds, we should not expect every phenomenon of philosophical interest to be amenable to empirical surveys. In a similar fashion, experimental epistemologists should not expect lay people to be able to make judgements on every relevant epistemological concept. Epistemic justification could plausibly be a key instance of these phenomena and concepts: given its technical nature, it may simply be unsuited to be surveyed experimentally.

Secondly, this pessimistic line of response can be further motivated by appeal to a local version of the so-called expertise defence (Kauppinen 2007; Ludwig 2007; Williamson 2011; Egler & Ross 2020). According to the advocates of a global version of the expertise defence, philosophers generally display special competence and privileged expertise in evaluating thought experiments and vignettes. As a result, trained philosophers are less prone to be influenced by other irrelevant and distorting factors (e.g., pragmatic implicatures or conceptual confusions). By contrast, given their lack of philosophical training, lay people do not possess such special competence and privileged expertise. Because their resulting judgements are more likely to be negatively affected and distorted, experimental surveys should not be given too much weight. Building on the global version of the expertise defence, supporters of a local version of the expertise defence will hold that the folk conceptual gap cannot be bridged exactly because lay people lack the specific training in epistemology which is necessary to competently evaluate whether a belief is epistemically justified. Since they have no training on the subtleties of the epistemic evaluation of belief, their judgements on this matter are intrinsically unreliable and too prone to errors and equivocations (e.g., lay people untrained in epistemology may too easily interpret justification in a pragmatic or moral sense). The fact that lay people are not trained to correctly apply the concept of epistemic justification provides reasons for strong pessimism on the possibility of survey folk attributions of epistemic justification: absent such essential training, it would appear that the folk conceptual gap simply cannot be bridged. Summing up, pessimism about the prospects of bridging the folk conceptual gap gains traction from the observation that some philosophical concepts may be simply unsuited to be surveyed experimentally, and from an independently supported local (and thus weaker) version of the expertise defence. Overall, strong pessimism seems warranted: taken together, the motivations for such a strong pessimism further highlight that the challenge raised by the folk concept is hard to meet.

However, even if well-supported and seemingly compelling, such a strong pessimism can (and should) be resisted: it would be premature to think that folk attributions of epistemic justification simply cannot be surveyed. The following observation begins to motivate a more optimistic stance towards the prospect of surveying folk attributions of epistemic justification. While epistemic justification may be a technical notion, lay people nevertheless seem to engage in genuinely epistemic assessments of belief which are distinct from knowledge attributions. In ordinary talk, lay people often ask about the reasons for belief, and criticise beliefs for being irrational or unreasonable. In certain contexts, lay people assess beliefs depending on whether they are supported by the available evidence: in a criminal trial, a lay jury may wonder whether the evidence adduced suffices to prove the defendant’s guilt beyond reasonable doubt, and in order to combat climate-change denial, a lay person might defer to scientists and appeal to the body of scientific evidence in support of anthropogenic climate change. These linguistic considerations are hardly decisive, but they are at the very least suggestive of the following: while lay people may lack the epistemologist’s technical concept of epistemic justification, they nevertheless can make judgements on the genuinely epistemic normative status of beliefs. These judgements inform epistemological theorising, and by adopting different strategies and a more cautious approach, they may also be investigated empirically. I now outline such strategies and approaches in detail.

5.2. Concept explication and alternative wording

In what follows, I will motivate a more cautiously optimistic approach to address the challenge raised by the folk conceptual gap. I will recommend two tentative strategies to better survey folk attributions of epistemic justification and also make sure to appreciate their most significant drawbacks. The upshot will be that while the challenge raised by the folk conceptual gap remains hard to address, adopting these strategies provides some reasons for cautious optimism.

A possible strategy attempts to remedy Immediate Testability , one of the untenable assumptions outlined in Section 3. Recall that a major factor contributing to the folk conceptual gap has to do with the design of the experimental studies surveying folk attributions of epistemic justification: these attributions are surveyed immediately without providing any preliminary explication of the relevant concept of epistemic justification. Accordingly, one way to bridge the folk conceptual gap is the concept explication strategy, which consists in providing the participants of the experimental studies with reliabilist or evidentialist explications of epistemic justification. For example, the surveys might include an explication that emphasises the importance of reliability and truth-conduciveness for epistemic justification (Goldman 1979) or the role of conscious mental states (Conee & Feldman 1985). As a result, participants will be better positioned to assess whether the belief under consideration is epistemically justified.

While it might improve on Immediate Testability , this strategy runs into a number of further difficulties. Firstly, it does not seem practically feasible, as it remains unclear how to include a reliabilist or evidentialist explication of epistemic justification which is informative but also succinct. Perhaps ostensive definitions might do better: basic examples of epistemically justified perceptual or testimonial beliefs could give an intuitive sense of epistemic justification. However, this by itself still would not explicate the notion sufficiently well. In general, it will be very hard to explicate it within the space of a standard experimental survey.

On the top of this first worry, there’s a second (and perhaps deeper) worry: crucially, it will also be very hard to find a principled way to explicate the notion of epistemic justification that can avoid the experimenter bias. This bias refers to the experimenters’ unconscious dispositions that may affect the experiments in order to secure previously hoped for and more favourable results. The experimenter bias is pervasive in many areas of science, and it calls into question the viability of the concept explication strategy. ¹⁴ To be clear, the experimenter bias does not pose a problem for the current surveys of folk attributions of epistemic justification. However, this bias becomes more worrisome and pressing once an informative explication of the concept of epistemic justification is offered to the participants. To better articulate the threat posed by the experimenter bias, suppose again that the concept of epistemic justification is explicated in reliabilist or evidentialist terms. Either way, the choice very much risks to cue and influence the participants, who will likely adjust their assessments to the way the concept is explicated. More vividly put, reliabilist explications centred around truth-conduciveness risk to secure empirical results which would favour externalist theories of epistemic justification, and the same applies, mutatis mutandis, to any evidentialist explication emphasising the connection between epistemic justification and internal experiences. These difficulties should not be surprising: as the discussion of Theoretical Neutrality previously showed, we currently lack any principled and theory-neutral way to explicate the notion of epistemic justification. Thus, the concept explication strategy is not only practically unfeasible but also likely to be affected by the experimenter bias. These are serious limitations for the strategy.

A different approach clarifies further the probe questions on epistemic justification. This is the alternative wording strategy, which consists in formulating the probe questions of the surveys without overidealising the conceptual repertoire of the participants. Its key insight is to employ terminology and concepts that lay people can be reasonably expected to understand. To flesh out this strategy, I shall propose alternative wordings based on purported necessary conditions for epistemic justification which are more conceptually friendly insofar as they sufficiently match ordinary talk and thought: notions like rationality, reasonableness and evidence can fit the bill.

It is often noted that epistemic justification bears a connection to (epistemic) rationality. Indeed, epistemologists of an internalist stripe take the two to be synonymous (e.g., Huemer 2001, Smithies 2012, Dogramaci 2015). However, ‘rational’ is not obviously a technical expression: Cohen develops semantic and meta-semantic arguments for this point, concluding that ‘if you ask sophisticated English speakers whether a certain belief is rational, they will understand what they are being asked’ (2016: 844). Cohen’s conclusion is disputable, and admittedly lacks clear empirical support. Still, if we grant it some prima facie plausibility, then ‘rational’ displays an advantage over ‘justified’. This tentatively suggests that ‘rational’ or ‘reasonable’ may provide slightly better (though still imperfect) wordings for the justification question. A different and potentially more helpful way to cash out epistemic justification is in terms of evidence: epistemically justified beliefs are supported by the evidence. This thesis can be developed in many different ways, depending on the nature of evidence and evidential support. ¹⁵ But regardless of these exact details, ‘evidence’ is also a more familiar term and conceptually friendly notion: as mentioned above, it figures prominently also in scientific and legal domains. ¹⁶

While by no means uncontroversial, these remarks have noteworthy consequences for the project of surveying folk attributions of epistemic justification. On the one hand, it is unreasonable to expect lay people to immediately and correctly comprehend questions on unqualified justification. On the other hand though, lay people may still be reasonably expected to understand the assessment of the rationality (or reasonableness) of a belief in light of the available evidence. For sake of illustration, here’s how one justification question may be reworded:

Is Harry justified in believing that the ticket has lost? Yes/No (Ebert et al. 2018).

Alternative wording #1. Given the available evidence, is Harry’s belief that the ticket has lost rational? Yes/No.

Alternative wording #2. Given the available evidence, is Harry’s belief that the ticket has lost reasonable? Yes/No.

Alternative wording #3. Given the available evidence, is Harry’s belief that the ticket has lost justified? Yes/No.

Different combinations of these alternative wordings are possible, and the other justification questions can be similarly rephrased. This alternative wording strategy remains underdeveloped, but it still provides a better alternative to the current probe questions.

To be maximally clear, these necessary conditions on epistemic justification remain controversial. Many will reject the identification of epistemic justification and rationality (e.g., Lyons 2013: 9), and just as many will reject accounts of epistemic justification in terms of evidence (e.g., Goldman 1999: Section 3). However, just for the purpose of surveying folk attributions of epistemic justification, these adjustments will sharpen the justification question, as they employ terminology and concepts we can reasonably expect lay people to possess and understand. As already noted, aiming for theoretical neutrality is unrealistic. Because it’s a technical notion, surveying folk intuitions on epistemic justification requires to take a previous stand on substantive features of epistemic justification. This is a necessary but worthwhile trade-off: dropping ambitions of theoretical neutrality promises to yield more informative and precise data. ¹⁷

5.3. Methodological reform

I care to emphasise that I am not offering these strategies as a naive solution to the problems affecting the current surveys of folk attributions of epistemic justification: I am aware that my recommended probe questions may incur the very same difficulties faced by the probe questions they are meant to replace. Drawing the participants’ attention to the available evidence might achieve progress, but talk of ‘rationality’ and ‘reasonableness’ is also potentially ambiguous between a practical and epistemic sense. Perhaps these worries can be addressed by developing and combining more fine-grained versions of the alternative wording and concept explication strategies. For example, the probe questions may ask explicitly to bracket moral and pragmatic considerations, or instead of a strict definitional explication, participants might be provided with additional cues to better focus on the epistemic sense of justification. But even so, lingering ambiguities will inevitably remain, and further challenges are to be expected.

However, not all is lost. Once these difficulties are fully acknowledged, the project of surveying folk attributions of epistemic justification can be properly reassessed. Rather than constituting a definitive mark against the success of the project, these difficulties motivate a more cautious approach, as well as a general methodological reform. In light of a deeper appreciation of the complex subject matter of their surveys, experimental epistemologists should take a leaf from philosophy of science and adopt the more fruitful methodology of evidential triangulation, which consists in the use of multiple methods and independent sources of evidence to measure and detect the same phenomenon (Campbell & Fiske 1959; Schickore & Coko 2013; Kuorikoski & Marchionni 2016). Evidential triangulation is especially apt for experimental surveys of folk attributions of epistemic justification: experimental epistemologists may pursue different versions of both the concept explication and the alternative wording strategies, and then assess a larger body of data gathered by using independent sources of evidence and distinct methods. These methods should encompass a variety of strategies, including but not limited to different explications of epistemic justification and alternative probe questions rephrased in terms of evidence, reasonableness, and rationality.

This methodological reform opens fruitful avenues for future research, and it also further highlights the value of the three assumptions I have discussed. In general, keeping the focus on these assumptions helps to assess experimental surveys of folk attributions of epistemic justification. Here is a concrete (but quick) illustration of this point. One could maintain the same vignettes of the current surveys but then use a wider range of probe questions to test for the robustness of the responses. Crucially, diverging responses and confidence rates resulting from terminological changes in the probe questions would yield a more informed perspective on Folk Understanding . Alternatively, one could try to survey attributions of epistemic justification made by philosophy students while controlling for the views of their lecturer. Suppose that laypeople and philosophy students answered the questions differently. This would suggest that some expertise does underlie (folk) attributions of epistemic justification, and the case against Immediate Testability would then be stronger. Also, similarities between the answers of the students and the views endorsed by their lecturers would tentatively show that attributions of epistemic justification are influenced by substantive theories of epistemic justification. Such a result would count against Theoretical Neutrality . ¹⁸ These hypotheses are very speculative, but hopefully they illustrate the value of the three assumptions discussed in this paper.

Let’s take stock. Here’s the main upshot: the challenge raised by the folk conceptual gap remains difficult to meet. While my remarks are tentative, they nonetheless provide some reasons for cautious optimism. A more thorough development of my proposed strategies promises to address this difficult challenge. However, such a task is best left for another day.

6. Conclusion

Recent experimental surveys of folk attributions of epistemic justification are missing the mark: they underestimate a folk conceptual gap and overestimate the conceptual repertoire available to lay people. In this paper, I have drawn the attention to this gap and explained the problematic assumptions of these surveys. However, mine are not knockdown objections to the project of surveying folk attributions of epistemic justification. In a more constructive spirit, I also suggested some tentative strategies to address the challenge raised by the folk conceptual gap. This is a serious challenge: my recommended strategies are thus worthy of serious consideration, as they promise to inform future experimental surveys of folk attributions of epistemic justification. ¹⁹