Illusory intuitions: Challenging the claim of non-exclusivity

Abstract

A person who arrives at correct solutions via false premises is right and wrong simultaneously. Similarly, a person who generates “logical intuitions” through superficial heuristics can likewise be right and wrong at the same time. However, heuristics aren't guaranteed to deliver the logical solution, so the claim that system 1 can routinely produce the alleged system 2 response is unfounded.

De Neys sets out two of the key challenges for dual-process theories as they have been traditionally conceived, the detection of response conflict and specification of the mechanism that controls the switch between system 1 (S1) and system 2 (S2) processing. In this commentary, we focus on first of these concerns and in particular the role of S1 in the generation of the “alleged S2 response”; the claim that reasoners routinely generate “logical intuitions.” Evidence for logical intuitions has been claimed across a range of paradigms, where there is evidence that conflict between knowledge-based and rule-based responses can be detected automatically, without the engagement of deliberative reasoning. According to De Neys, the proposal that S1 can generate a logical response solves one of the fundamental challenges for the dual-process framework; how can conflict be detected without the prior engagement of S2 in calculating the normative response? The solution to this quandary is to posit non-exclusivity, the idea that the generation of a logical response is not the unique purview of S2.

For the past decade, our own work has similarly suggested that reasoners show intuitive sensitivity to the logical validity of simple deductive reasoning arguments. One of the most convincing pieces of evidence comes from belief–logic instructional paradigm, where, across multiple studies, we have shown that a conflict between the logical and belief status of a conclusion influences judgments of conclusion believability as much, if not more than conclusion believability influences logical judgments (Handley, Newstead, & Trippas, 2011; Howarth, Handley, & Walsh, 2016, 2019; Trippas, Thompson, & Handley, 2017). The impact of conflict on belief judgments indicates that the logical inference is drawn automatically and intuitively and hence interferes with fast belief judgments. These findings appear to support De Neys claim for non-exclusivity, that S1 can generate logical intuitions.

However, our most recent research, together with parallel findings from other labs, suggests that the picture is not quite so straightforward. These findings suggest that “logical intuitions” may have little to do with formal logic, but instead reflect sensitivity to superficial structural features (Ghasemi, Handley, Howarth, Newman, & Thompson, 2022a; Meyer-Grant et al., 2022). Converging evidence from research on the liking-logic task and the instructional paradigm show that logical intuition effects emerge because on valid arguments there is a match between the polarity of the premises and the conclusion, which is not present on invalid versions of the same arguments. Ghasemi et al. (2022a) tested this explanation by using invalid arguments in which such a match was present, and showed that “logical intuition” effects are equally as strong on these invalid argument forms. They concluded that reasoners are not intuitively sensitive to logical validity in a formal sense, instead they are picking up on structural cues that reflect the repetition of elements in the premises. Our most recent research further shows that training in logical principles improves discrimination between valid and invalid logical forms under logical instructions, but does not reduce the propositional matching effect under belief instructions, providing convincing evidence that logical intuitions arise because of sensitivity to non-logical features rather than logical validity per se (Ghasemi, Handley, & Stephens, 2022b).

De Neys argues that although the S2 response may be generated by S1 processes, the equivalence is situated at the response level. Hence, an equivalent response generated by the intuitive and deliberative systems does not imply that the response was generated by the same mechanism or has the same features. Perhaps S1 does in fact rely on heuristics that draw on surface features but these co-vary with the logical status of the conclusion, hence on average deliver an intuition that aligns with the S2 response? So, does it matter if the intuitive response is generated by heuristics? We argue that it does, because a heuristic is no guarantee to a logical conclusion. In fact, more than half a century of work shows that heuristics regularly lead to systematic errors in reasoning and judgment tasks (Tversky & Kahneman, 1983) and the alignment of the output of a heuristic mechanism with the logically correct response can often be a matter of chance or clever experimental design (Evans & Lynch, 1973; Handley & Evans, 2000). We have recently run a series of studies in which the output of a matching heuristic and the logical response were misaligned. In these circumstances, matching dominates S1 outputs, whereas logic dominates S2 (Ghasemi, Handley, & Howarth, 2023). What these studies illustrate is that there is no guarantee that a response based upon superficial problem features will align with the formal logical response. You might get it right for the wrong reason, but you are as likely to get it wrong for the wrong reason also.

Is there a way of reconciling our findings with De Neys model? We think that there is at least one resolution which draws upon alternative normative accounts of human thinking that do not rely on formal logic as a normative standard. Such accounts are framed within the new paradigm in the psychology of reasoning which edifies the usage of heuristics as an adaptive mechanism, sensitive to probabilistic logic or information gain (Oaksford & Chater, 2020). An intriguing possibility is that logical intuitions, while arising through the application of simple heuristics, nevertheless respect the probabilistic structure of the environment and hence deliver outputs that have a rational basis. Such outputs will often align with an S2 response that draws upon deductive logic and hence the non-exclusivity principle will often, but not always hold. Perhaps intuitive reasoners do indeed sometimes get it right, but for a different reason, not the wrong one.