Hostname: page-component-cd9895bd7-hc48f Total loading time: 0 Render date: 2024-12-25T19:54:51.047Z Has data issue: false hasContentIssue false
  • English
  • Français

Cognitive abilities and superior decision making under risk: A protocol analysis and process model evaluation

Published online by Cambridge University Press:  01 January 2023

Edward T. Cokely  and
Colleen M. Kelley
Edward T. Cokely*
Affiliation:
Max Planck Institute for Human Development, Center for Adaptive Behavior and Cognition
Colleen M. Kelley
Affiliation:
Department of Psychology, Florida State University
*
*Address: Edward Cokely, Center for Adaptive Behavior and Cognition, Max Planck Institute for Human Development, Lentzeallee 94, 14195 Berlin, Germany. Email: [email protected].
Rights & Permissions [Opens in a new window]

Abstract

Individual differences in cognitive abilities and skills can predict normatively superior and logically consistent judgments and decisions. The current experiment investigates the processes that mediate individual differences in risky choices. We assessed working memory span, numeracy, and cognitive impulsivity and conducted a protocol analysis to trace variations in conscious deliberative processes. People higher in cognitive abilities made more choices consistent with expected values; however, expected-value choices rarely resulted from expected-value calculations. Instead, the cognitive ability and choice relationship was mediated by the number of simple considerations made during decision making — e.g., transforming probabilities and considering the relative size of gains. Results imply that, even in simple lotteries, superior risky decisions associated with cognitive abilities and controlled cognition can reflect metacognitive dynamics and elaborative heuristic search processes, rather than normative calculations. Modes of cognitive control (e.g., dual process dynamics) and implications for process models of risky decision-making (e.g., priority heuristic) are discussed.

Keywords

Risky choiceintelligenceworking memorynumeracycognitive controldual process theoryinformation searchrationalityexpected valueprotocol analysispriority heuristic
Type
Research Article
Information
Judgment and Decision Making , Volume 4 , Issue 1 , February 2009 , pp. 20 - 33
Creative Commons
Creative Common License - CCCreative Common License - BY
The authors license this article under the terms of the Creative Commons Attribution 3.0 License.
Copyright
Copyright © The Authors [2009] This is an Open Access article, distributed under the terms of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.

1 Introduction

Human decision-making is constrained by its bounded rationality and does not always follow normative prescriptions (Gigerenzer, Todd, & the ABC Research Group, Reference Gigerenzer and Todd1999; Kahneman, Reference Kahneman2003; Payne, Bettman, & Johnson, Reference Payne, Bettman and Johnson1993; Simon Reference Simon1990). Nevertheless, individual differences in cognitive abilities and skills predict normatively superior judgment and decision-making (Frederick, Reference Frederick2005; Peters & Levin, Reference Peters and Levin2008; Peters, Vastfjall, Slovic, Mertz, Mazzocco, & Dickert, Reference Peters, Västfjäll, Slovic, Mertz, Mazzocco and Dickert2006; Stanovich & West, Reference Stanovich and West1998; Reference Stanovich and West2000; Reference Stanovich and West2008). A variety of theories, such as dual-process theories, attribute the individual differences to deliberative processes (Baron, Reference Baron1985; De Neys, Reference De Neys2006; Evans, Reference Evans2008; Frederick, Reference Frederick2005; Kahneman, Reference Kahneman2003; Kahneman & Frederick, Reference Kahneman and Frederick2007; Sloman, Reference Sloman1996; Stanovich & West, Reference Stanovich and West1998; Reference Stanovich and West2000); however, the link between decision processes and abilities is largely uninvestigated. What are the cognitive processes that give rise to the relationship between cognitive abilities and superior decision making under risk?

Previous research has examined individual differences in decision making under risk in lotteries with known probabilities. For low stakes lotteries normative expected utility processes are assumed to be approximated with calculations that multiply probabilities by potential gains/losses, i.e., expected-value calculations (Frederick, Reference Frederick2005; Payne, Samper, Bettman, & Luce, Reference Payne, Samper, Bettman and Luce2008). Frederick has demonstrated that expected-value choices are associated with scores on the cognitive reflection test, which is designed to measure one's reliance on more consciously controlled processes rather than automatic first impressions (e.g., Stanovich and West's, Reference Stanovich and West2000, deliberative System 2 rather than intuitive System 1). The three-problem cognitive reflection test, which is known to correlate with other general cognitive ability measures, consists of mathematical problems for which an immediate intuitive response is incorrect. Frederick demonstrated that higher scoring individuals did not exhibit the clear non-normative risk asymmetry for gains and losses predicted by prospect theory (Kahneman & Tversky, Reference Kahneman and Tversky1979). When given a choice between a gain of $100 versus a 75% chance of a $200 gain, prospect theory predicted risk aversion and the selection of the certain $100.Footnote 1 However, people with higher cognitive reflection scores more often selected options with the higher expected values (i.e., the probability multiplied by the potential risky gain — $150) as compared to lower scoring individuals.

There are several candidate mechanisms that may account for the link between cognitive abilities and superior decision making under risk. For example, one can make expected-value choices by performing expected-value calculations. Frederick (Reference Frederick2005) suggests that computation of expected values may play a role, although he notes that it is not likely the only factor. More generally, Stanovich and West (Reference Stanovich and West2000) suggest that individual differences in normative judgments and decisions often arise from working memory capacity limitations on computation, implying that high ability individuals may make expected-value choices via expected-value calculations.Footnote 2 Other research indicates that individual differences in risky decision making may also arise from variations in one's general knowledge and understanding of probabilities — i.e., one's numeracy (Peters & Levin, Reference Peters and Levin2008; Peters et al., Reference Peters, Västfjäll, Slovic, Mertz, Mazzocco and Dickert2006). People high in numeracy, particularly the ability to comprehend and transform probabilities, are less affected by attribute framing because they can readily transform items such as 74% correct into 26% incorrect and translate percentages to frequencies and vice versa. Thus, numeracy may allow better risky choices as a result of a more accurate subjective sense of the size of gains and losses or other probability related trade-offs.

1.1 Process models of risky choice

Theories describing the actual cognitive processes commonly used for decision making under risk tend to be imprecise (Brandstätter, Gigerenzer, & Hertwig, Reference Brandstätter, Gigerenzer and Hertwig2006; Reference Schooler and Hertwig2008; Johnson, et al. Reference Johnson, Schulte-Mecklenbeck and Willernsen2008; Payne et al., Reference Payne, Bettman and Johnson1993; Payne & Braunstein, Reference Payne and Braunstein1978; Selart et al., Reference Selart, Kuvaas, Boe and Takemura2006). Risky choice models are typically as-if models, as in the case of prospect theory, which does not describe the exact cognitive operations of choice but holds only that people act as-if they evaluate losses with a steeper utility curve (Johnson et al., Reference Johnson, Schulte-Mecklenbeck and Willernsen2008). One exception to as-if modeling is the priority heuristic which is a parameter free choice-outcome and cognitive process model (Brandstätter et al., Reference Brandstätter, Gigerenzer and Hertwig2006). According to the priority heuristic, decisions between sure versus risky options are the result of considering simple reasons for a decision in a fixed order, until a stopping rule is met. First, people consider minimum gains. If the minimum gains differ by 1/10 or more of the maximum gain (1/10 of the maximum gain rounded to the closest prominent number) consideration stops and people choose the option with the higher minimum gain. If necessary, they consider a second reason, the probability of the minimum gain. If the probabilities of the two options differ by 1/10 or more of the probability scale, consideration stops and people choose the option with the higher probability minimum gain. If necessary, they will consider a third reason and choose the option with the higher maximum gain. A similar set of reasons and stopping rules occur for choices between losses.

The priority heuristic has accurately described majority choice-outcomes in several theoretically important datasets (but for critical reviews see Birnbaum, Reference Birnbaum2008; Hilbig, Reference Hilbig2008; Johnson et al., Reference Johnson, Schulte-Mecklenbeck and Willernsen2008). Some evidence also supports the priority heuristic process model as latencies to choose between two options have been greater for choices that require three considerations compared to one consideration (Brandstätter et al., Reference Brandstätter, Gigerenzer and Hertwig2006). However, the priority heuristic is silent on the potential cognitive processes that may mediate the relationship between cognitive abilities and superior decision-making. Given that the priority heuristic is designed to predict potentially non-normative majority choices we hypothesized that it may predict many participants’ choices and choice processes, although it would be unlikely to predict behavior of high ability individuals.

1.2 Heuristic search

Heuristics and simple considerations are common and often effective bases for judgment and choice (Gigerenzer et al., Reference Gigerenzer and Todd1999; Payne, Bettman, Coupey, & Johnson, Reference Payne, Bettman, Coupey and Johnson1992; Payne et al., Reference Payne, Bettman and Johnson1993; Tversky & Kahneman, Reference Tversky and Kahneman1974). We hypothesized that the relationship between cognitive abilities and decision-making under risk would not necessarily arise from expected-value calculations, but could result from simple considerations of reasons as in the priority heuristic, and simple transformations of probability information as in the research by Peters et al. (Reference Peters, Västfjäll, Slovic, Mertz, Mazzocco and Dickert2006; Peters & Levin, Reference Peters and Levin2008). Theory suggests that variation in superior decision making does not necessarily need to rely on the exact use of calculations based on normative models but can result from greater reflectiveness or thoroughness in decision making (Baron, Reference Baron1985; Reference Baron1990). Variation in risky choice performance has been linked to differences in duration and type of information search (Mann & Ball, Reference Mann and Ball1994; Payne & Braunstein, Reference Payne and Braunstein1978; Selart et al., Reference Selart, Kuvaas, Boe and Takemura2006). Working memory measures are also known to predict strategic differences in elaboration during encoding (Bailey, Dunlosky, & Kane, Reference Bailey, Dunlosky and Kane2008; Cokely, Kelley, & Gilchrist, Reference Cokely, Kelley and Gilchrist2006; Guida, Tardieu, & Nicolas, Reference Guida, Tardieu and Nicolas2008; McNamara & Scott, Reference McNamara and Scott2001) and differences in the number of hypotheses generated during probability judgment (Dougherty & Hunter, Reference Dougherty and Hunter2003; Thomas, Dougherty, Sprenger, & Harbison, Reference Thomas, Dougherty, Sprenger and Harbison2008). Therefore, we hypothesized that elaborative heuristic search — i.e., more thorough exploration and representation of the problem space — would often be positively related to superior risky decision making. To test the elaborative heuristic search hypothesis and to more precisely trace cognitive processes, we conducted a protocol analysis.

2 Experiment

Our experiment was designed to examine individual differences in decision processes. Process tracing was performed with retrospective verbal reports (Ericsson & Simon, Reference Ericsson and Simon1980) in which participants verbally reported the exact thoughts they remembered in the order in which they occurred, immediately following their choices. When people consciously and deliberately consider information, such as comparing minimum gains or transforming information into different probabilities, these processes should be observable in participants’ protocols (Evans, Reference Evans2008; Sloman, Reference Sloman1996). Verbal reports have previously been effectively used in related studies of choice (Rettinger & Hastie, Reference Rettinger, Hastie, Schneider and Shanteau2003; Payne Reference Payne1976; but for potential limitations see De Neys & Glumicic, Reference De Neys and Glumicic2008; Reisen, Hoffrage, & Mast, Reference Reisen, Hoffrage and Mast2008). To illustrate this methodology, both an expected-value calculation and the priority heuristic process predict that participants should consider distinct types of information when making their choices. Verbalization of an expected value or an attempt to estimate one (e.g., “75% of $200 is definitely more than $100”) would provide evidence of expected-value type processes. Similarly, the priority heuristic makes predictions about what information will and won’t be considered for different lotteries, and in what order (Brandstätter et al., Reference Brandstätter, Gigerenzer and Hertwig2006) (Table 1). These predictions allowed us to develop a coding system to quantify the types and amounts of considerations that were consistent and inconsistent with processing products predicted by the priority heuristic and expected-value calculations. Protocol analysis codes were also derived from previous research (Rettinger & Hastie, Reference Rettinger and Hastie2001; Reference Rettinger, Hastie, Schneider and Shanteau2003) and a pilot study (Table 2).

Table 1: Example of discrete model predictions and predicted verbal reports for a sample choice.

Table 2: Coding system for protocol analysis including examples of each consideration, mean considerations per trial (and standard deviations), and total observed considerations.

We hypothesized that protocol analysis would reveal a positive relationship between expected-value type choices and elaborative heuristic search (Baron, Reference Baron1985; Payne, Reference Payne1976; Selart et al., Reference Selart, Kuvaas, Boe and Takemura2006; Simon, Reference Simon1990), operationalized as the total number of different types of simple considerations verbalized (excluding expected-value calculations and ambiguous codes), regardless of output order (Table 2).Footnote 3 We also hypothesized that elaborative heuristic search would at least partially mediate the relationship between cognitive abilities and superior decisions.Footnote 4 More elaborative and thorough search processes were expected to include variations in the number of considerations (e.g., consider maximum gains and probabilities versus considering only maximum gains) as well as explorations of different aspects of problems (e.g., interpret the large difference between potential gains as a potential loss). Such variations could help some participants avoid overlooking valuable information or oppose the influence of framing effects (Peters et al, Reference Peters, Västfjäll, Slovic, Mertz, Mazzocco and Dickert2006).

2.1 Participants

Eighty undergraduate students from introductory psychology courses at Florida State University participated in partial fulfillment of course requirements and were tested individually. Four cognitive reflection scores were excluded as participants had seen the test in another experiment. Four working memory scores and two verbal reports were lost because of equipment failure. Seven participants did not receive numeracy scores due to a procedural error.

2.2 Materials

Ability measures included: (1) the operation span — a working memory capacity task that partially mediates relationships predicted by traditional intelligence instruments (Turner & Engle, Reference Turner and Engle1989); (2) the cognitive reflection test (CRT) which assesses differences in cognitive impulsivity (System 1) versus more deliberative thinking (System 2) (Frederick, Reference Frederick2005); (3) a numeracy scale measuring understanding of numerical probabilities (Lipkus, Samsa, & Rimer, Reference Lipkus, Samsa and Rimer2001; see Peters and Levin, Reference Peters and Levin2008 for the 11 item scale).

2.3 Decision making under risk

The stimuli included 40 choice problems with hypothetical gains/losses presented in US dollars. Each choice consisted of one certain option and one risky option, balanced such that expected value and priority heuristic models made unique predictions on exactly half of the trials. Expected-value ratios of lotteries were on average near the indifference point (M = 2.07, range = .15 to 5.3, relative to the certain option) a range in which the priority heuristic is expected to predict choices (for discussion see Brandstätter et al., Reference Brandstätter, Gigerenzer and Hertwig2006; Reference Brandstätter, Gigerenzer and Hertwig2008). Expected-value calculations predicted equal numbers of risky choices for gains and losses; priority-heuristic predictions were asymmetric favoring risky choices for losses, but not gains. Priority heuristic also predicted that 60% of choices would involve less search (i.e., a single consideration of the minimum possible gains/losses relative to 10% of the maximum gain/losses) while the other 40% of choices required the maximum number of considerations (i.e., all possible steps of the priority heuristic). Risky option probabilities ranged from 1%-80% (Appendix).

2.4 Procedure

Participants were tested individually. Responses were recorded by a head-mounted microphone. Verbal report instructions and warm up think-aloud problems were provided by an experimenter seated behind the participant. The experiment began with the cognitive reflection task followed by an example lottery. Participants were told that the experiment involved 40 such choices, all of which were presented in the same randomized order. Choices were presented from the top to the bottom of the screen with the first option (e.g., “A. gain $50”) displayed for two seconds before the second option appeared (e.g., “B. 50% to gain $400”). Choices remained on the screen until the participant made a selection and was prompted for a retrospective report. Lastly, participants completed the working memory span and numeracy measures, and were debriefed.

3 Results

Following Brandstätter et al. (Reference Brandstätter, Gigerenzer and Hertwig2006), a model competition was conducted. This analysis assessed the frequency with which each model predicted majority choices, across all choices. Binomial analysis indicated that expected-value calculations predicted majority choices significantly better than chance (M = .83, p = .001). A non-parametric test of equal proportions indicated that expected value also significantly outperformed the priority heuristic, χ 2 = 12.17, p = .001, d = 1.3, which predicted at chance levels (M = .45, p > .5). A variety of subsequent analyses of the priority heuristic converged to suggest that in the current task environment the priority heuristic was an inaccurate process and choice-outcome model (see also Birnbaum, Reference Birnbaum2008; Hilbig, Reference Hilbig2008; Johnson et al., Reference Johnson, Schulte-Mecklenbeck and Willernsen2008).Footnote 5

Because abilities are known to influence choice, and given evidence on the limits of majority choice aggregation analyses (Regenwetter, Grofman, Popova, Messner, Davis-Stober, & Cavagnaro, Reference Regenwetter, Grofman, Popova, Messner, Davis-Stober and Cavagnaro2008), we examined individual model-prediction-accuracy scores. Subsequent analyses compared the proportion of expected value consistent choices averaged across all choices for each individual. A one sample t test indicated that expected-value calculations strongly predicted participant choices (M = .72, SD = .12) above chance levels, t (79) = 16.02, p = .001, d = 1.9. The proportion of choices consistent with expected value was significantly related to CRT, r (74) = .27, p = .02, and numeracy, r (71) = .28, p = .02 (Table 6). A mixed model analysis of variance (ANOVA) with risk type (certain, risky) by choice type (gain, loss) by working memory span quartile (low, high) also indicated that working memory was associated with differences in choices, F (1, 36) = 7.70, p = .01, d = .8. High working memory span participants made significantly more expected-value type choices (M = .79, SD = .13) as compared to low span participants (M = .70, SD = .10).

3.1 Protocol analysis

Verbal reports were analyzed by two raters blind to model and judgment performance (Table 2). A randomly-selected subset of verbal reports (13%) were scored by both raters and indicated high inter-rater agreement on the number of considerations, r (8) = .97, p = .01, and substantial agreement on specific consideration codes (kappa = .63). The total number of considerations verbalized was also related to the mean choice reaction time, r (67) = .46, p = .001, Footnote 6 indicating that individuals who retrospectively verbalized more considerations also took longer to make their judgments. Unless otherwise noted, seven participants were excluded from subsequent analysis because more than 50% of their verbal protocols were unrevealing (e.g., “A is better; I like B”).Footnote 7

Three individuals verbalized expected-value calculations (or estimations) nearly exclusively (95–100% of all trials; see Table 3 for examples). The frequency of verbalized expected-value calculations was significantly related to expected-value type choices, r (69) = .25, p = .03; however, expected-value calculations were unrelated to cognitive ability variables (Table 6). The remaining participants exhibited a clear relationship between the number of considerations verbalized and expected-value choices excluding any ambiguous or expected-value verbalizations, β = .60, t = 15.90, p = .001, R 2 = .36 (Figure 1).Footnote 8 Individuals who made the most expected-value choices (top quartile) verbalized about twice as many considerations per trial (M = 1.78, SD = .52) as did those who made fewer (bottom quartile) expected-value choices (M = .94, SD = .35). Across all participants, the number of considerations verbalized was also significantly related to CRT, r (72) = .23, p = .05; numeracy, r (69) = .36, p = .01; and working memory span, r (72) = .25, p = .04 (see Tables 4 and 5 for examples of verbal protocols; see Tables 6 and 8 for intercorrelations among variables).Footnote 9 For example, across 40 trials, excluding expected-value or ambiguous verbalizations, individuals with higher working memory span scores (top quartile) verbalized significantly more considerations (M = 60.8, SD = 20.8) as compared to those with lower (bottom quartile) working memory scores (M = 47.1, SD = 14.5), F (1, 30) = 4.51, p = .04, d = .08.

Table 3: A sample of protocol analysis revealing expected-value calculation or estimation.

Figure 1: A linear regression with elaborative heuristic search (i.e., the number of verbalized considerations) predicting each participant's overall proportion of expected-value choices (ambiguous and expected-value verbalizations are not included). The line is based on the regression.

Table 4: A sample of coded protocol analysis from individuals with lower working memory, numeracy, and/or cognitive reflection scores (i.e., bottom quartile).

Table 5: A sample of coded protocol analysis from individuals with higher working memory, numeracy, and/or cognitive reflection scores (i.e., top quartile).

Table 6: Intercorellations for main variables.

Notes:

* p < .05;

** p < .01

Table 7: Hierarchical linear regression analysis explaining expected-value choices.

Note:

* p < .05;

** p < .01

We next constructed a series of hierarchical linear regression models (Table 7) with the most complex (full) model using three predictors including (1) expected-value verbalizations; (2) all three ability measures; and (3) number of verbalized considerations. The full model was a strong predictor of expected-value choices, F (5, 53) = 22.23, p = .001, R 2 = .44. The number of verbalized considerations accounted for a moderate amount of unique variance, F (1, 53) = 8.15, p = .001, R 2change = .24. The number of verbalized considerations also fully mediated the relationships between all three cognitive ability measures and expected-value choices (ts < 1, see Table 7).

To what extent might these results reflect the influence of particular choices, such as choices on gains rather than losses or choices involving high versus lower monetary values? To assess independent relationships controlling for these potentially influential factors we conducted a multilevel analysis. First, we constructed independent regression equations for each participant, predicting each participant's responses across all 40 choice trails. Individual level regression equation coefficients (i.e., unstandardized β coefficients) were computed for each of the following variables (1) expected-value model choice predictions; (2) priority heuristic model choice predictions; (3) gain versus loss trails (to assess and control for potential asymmetries in responding); and (4) the highest absolute monetary value for each choice (to assess and control for potentially non-uniform influences of declining marginal utility).Footnote 10 Next, we examined the intercorrelations between the individual level regression coefficients, all cognitive abilities, and the number of verbalized considerations (Table 8).Footnote 11

Table 8: Intercorrelations of ability, elaborative heuristic search, and individual level regression coefficients (indicated by β).

Note:

p < .05;

p < .01

As expected, results revealed reliable relationships between the expected-value choice coefficients and all cognitive ability measures including the cognitive reflection test, r (66) = .29, p = .02; numeracy, r (62) = .29, p = .02; and working memory span, r (66) = .27, p = .03. The number of verbalized considerations was also significantly related to the expected-value choice factor, r (69) = .45, p = .001. Lastly, a hierarchical linear regression was constructed, following the previous analyses but predicting the expected-value individual level coefficients with (1) expected-value verbalizations, (2) all three ability measures, and (3) number of verbalized considerations. The full model was again a strong and significant predictor, F (5, 53) = 4.22, p = .003, R 2 = .29. The number of verbalized considerations also accounted for unique variance beyond other variables, F (1, 53) = 10.06, p = .003, R 2change = .14, and again fully mediated the influence of all three cognitive abilities (ts < 1).

4 Discussion

A very small minority of our sample (about 5%) consistently verbalized expected-value processes during decision making (Payne & Braunstein, Reference Payne and Braunstein1978). The vast majority of expected-value choices were instead associated with simple heuristic-type decision processes. These decision processes were similar to the component considerations in the priority heuristic (see Table 2), although the priority heuristic was otherwise an inaccurate process and choice-outcome model. Consistent with the elaborative heuristic search hypothesis we found a relationship between the number of considerations verbalized and expected-value choices. Elaborative heuristic search also mediated the relationships between cognitive abilities and expected-value choices.Footnote 12 These results demonstrate that neither deliberative thinking nor cognitive abilities are necessarily associated with normative calculations, even when associated with normatively superior decision performance.

4.1 Dual-process models and modes of cognitive control

The elaborative heuristic search captured by protocol analysis in the current experiment may, in part, result from differences in top-down, early selection cognitive control mechanisms used during the task (Jacoby, Kelley, & McElree, Reference Jacoby, Kelley, McElree, Chaiken and Trope1999). The prevailing theoretical framework emphasizes a late correction cognitive control interpretation of dual process dynamics. That is, when controlled processes (System 2) do not compute an answer they are assumed to primarily operate by monitoring and correcting the output of automatic processes (Kahneman, Reference Kahneman2003). In contrast, early selection cognitive control uses controlled processing (System 2) to generate goals, strategies, and mental contexts that qualitatively alter the output of automatic processes (System 1) before critical impressions are yielded (Jacoby, Shimizu, Daniels, & Rhodes, Reference Jacoby, Shimizu, Daniels and Rhodes2005). For example, if some participants approached the task with the mindset of playing a game (e.g., “I feel lucky”) they would likely generate different search processes as compared to those construing choices in terms of their actual spending power (e.g., “the probability is low but I don’t even have $7000 dollars”). Spending-power type considerations (i.e., considering values small or large) were found to be significant predictors of expected-value choices [r (76) = .41, r (76) = .36, respectively] and were also strongly related to the overall number of considerations [r (76) = .62, and r (76) = .68, respectively]. Moreover, related research indicates that other judgment and decision biases — e.g., the endowment effect and non-rational discounting in intertemporal choice — can be accounted for by one's initial memory query and the resulting constraints on memory search and accessibility (see query theory and the preferences-as-memory framework; Johnson, Haubl, & Keinan, Reference Johnson, Haubl and Keinan2007; Weber, Johnson, Milch, Chang, Brodscholl, & Goldstein, Reference Weber, Johnson, Milch, Chang, Brodscholl and Goldstein2007).

A common assumption of dual process theories is that controlled cognition (System 2) reflects more rule-based, abstract and decontextualized reasoning whereas more automatic and impulsive cognition (System 1) is driven by associations, personal relevance, and situational-contextual information (cf. fundamental computational bias, Stanovich & West, Reference Stanovich and West2000; but see also Evans, Reference Evans2008).Footnote 13 Interestingly, in the current experiment more deliberation was associated with more personalization and contextualization during reasoning — as opposed to abstract rule based expected-value calculations — which was evidenced by more elaborative heuristic search and consideration of more concrete real world implications of choices (for other links between context, abilities, and performance see Delaney & Sahakyan, 2007; Morsanyi & Handley, Reference Morsanyi and Handley2008). Given that elaborative heuristic search accounted for unique variance beyond cognitive abilities, beneficial elaborative search processes may not require an exceptional cognitive capacity or skill. Instead, superior risky decision performance may partially reflect a cognitive style that is typical of (but not necessarily limited to) individuals with higher working memory span. Such a metacognitive style could generally bring more world knowledge to bear on many problems and thus may be less prone to compartmentalization and impulsive choice (Baron, Reference Baron1985; Stanovich and West, Reference Stanovich and West2000). Additionally or alternatively, these cognitive style factors may be driven by more crystallized knowledge or skill mechanisms. For example, more numerate individuals could derive more affective meaning from the consideration of probabilities and the comparison of options (Peters et al., Reference Peters, Västfjäll, Slovic, Mertz, Mazzocco and Dickert2006, Experiment 4), which could motivate more elaborative search.

Broadly, our results are consistent with general notions of reflectiveness suggesting that cognitive abilities are associated with more careful, thorough, and elaborative — but not necessarily normative — cognition (Baron, Reference Baron1985). Our results further suggest that early selection cognitive control mechanisms may play a role in reflectiveness and superior task performance. Indeed, individuals who score higher on cognitive ability measures are known to spend more time preparing for tasks (Sternberg, Reference Sternberg1977) and also more elaborately and strategically encode information, deliberatively building cognitive representations that better support subsequent task performance (Baron, Reference Baron and Underwood1978; Cokely et al., Reference Cokely, Kelley and Gilchrist2006; Ericsson & Kintsch, Reference Ericsson and Kintsch1995; Hertzog & Robinson, Reference Hertzog, Robinson, Wilhelm and Engle2005; McNamara & Scott, Reference McNamara and Scott2001; Vigneau, Caissie, & Bors, Reference Vigneau, Caissie and Bors2005). However, we caution against an interpretation that higher performing individuals (or better decision processes) always search or reflect more (for a discussion of “less is more” in decision making see Gigerenzer et al., Reference Gigerenzer and Todd1999). Research unambiguously demonstrates that abilities and expertise are associated with adaptive cognition, such that superior performers will tend to rely on less elaborative search when it is advantageous (Bröder, Reference Bröder2003; Ericsson, Prietula, & Cokely, Reference Ericsson, Prietula and Cokely2007; Fasolo, Misuraca, & McCelland, Reference Fasolo, Misuraca and McClelland2003, Mata, Schooler, & Rieskamp, Reference Mata, Schooler and Rieskamp2007; Shanteau, Reference Shanteau1992).

4.2 Models of risky choice

Expected value was a reliable as-if choice outcome model. Yet process data indicated that even in highly simplified lotteries expected value was only an as-if model, which showed little relation to actual cognitive processes (Payne & Braunstein, Reference Payne and Braunstein1978). The priority heuristic also proved to be an inaccurate process (and choice-outcome) model. This limitation may reflect the large individual differences in elaborative search elicited by the current task environment. These data provide further evidence on the limitations and boundary conditions of the priority heuristic (Birnbaum, Reference Birnbaum2008; Hilbig, Reference Hilbig2008; Johnson et al., Reference Johnson, Schulte-Mecklenbeck and Willernsen2008). It should be noted that this limitation is apparent only because the priority heuristic makes very exact predictions at both the cognitive process and choice-outcome levels, which is a useful and unique feature among risky choice models. Results indicate that more precise process modeling of risky choices with the priority heuristic or another computational model would require at least one parameter that creates variation in search and stopping rules. However, accurate modeling of psychologically plausible mechanisms for the regulation of heuristic search will require greater specification and research at the intersection of task environments and cognitive capacities (Bröder, Reference Bröder2003; Gaissmaier & Schooler, Reference Gaissmaier and Schooler2008; Gaissmaier, Schooler, & Rieskamp, Reference Gaissmaier, Schooler and Rieskamp2006; Gaissmaier, Schooler, & Mata, Reference Gaissmaier, Schooler and Mata2008; Schooler & Hertwig, Reference Schooler and Hertwig2005).

5 Conclusions

People higher in working memory span, cognitive reflectiveness, and those with greater skill in comprehending and transforming probabilities often made choices consistent with expected value; however, protocol analyses revealed that they did not commonly use expected-value calculations to arrive at those choices (Payne & Braunstein, Reference Payne and Braunstein1978; Payne et al., Reference Payne, Bettman and Johnson1993). Instead, cognitive abilities were related to relatively simple yet elaborative heuristic search processes. The results accord with examples showing that good decisions can be made with simple processes (Gigerenzer & Goldstein, Reference Gigerenzer and Goldstein1996; Gigerenzer et al., Reference Gigerenzer and Todd1999), although results also provide additional evidence that even heuristic search processes can require conscious, deliberative efforts (Simon, Reference Simon1990). The current results serve as a reminder that individual differences cannot be ignored by judgment and decision researchers as majority choice does not necessarily reflect a single decision process that can be accurately assessed or modeled at the level of group means (Cokely & Feltz, Reference Cokely and Feltz2009; Feltz & Cokely, Reference Feltz and Cokely2008; Regenwetter et al, Reference Regenwetter, Grofman, Popova, Messner, Davis-Stober and Cavagnaro2008).

Theoretically, our results indicate that the relationship between cognitive abilities and superior risky choices can reflect differences in relatively simple yet elaborative heuristic-type processes. Nevertheless, the cognitive and metacognitive dynamics that regulate search and stopping are not well understood. The current data provide some indication that these dynamics are likely to be complex and multiply determined, potentially reflecting the influence of early selection cognitive control processes. Further research is needed to identify the variety of mechanisms that give rise to individual differences in decision performance. Critically, a higher fidelity understanding of these mechanisms will require theoretical models to address the interplay of (1) individual differences (e.g., abilities, traits, motivation, expertise), (2) cognitive processes, and (3) the environmental factors that shape strategy selection and efficacy (Alter, Oppenheimer, Epley, & Eyre, Reference Alter, Oppenheimer, Epley and Eyre2007; Botella, Pena, Contreras, Shih, & Santacreu, Reference Botella, Pena, Contreras, Shih and Santacreu2009; Galesic, Garcia-Retamero, & Gigerenzer, Reference Galesic, Garcia-Retamero and Gigerenzer2008; Karlsson, Juslin, & Olsson, Reference Karlsson, Juslin and Olsson2008; Payne et al, Reference Payne, Bettman and Johnson1993; Reiskamp & Otto, Reference Rieskamp and Otto2006; Rieskamp & Hoffrage, Reference Rieskamp and Hoffrage2008; Simon, Reference Simon1990).

Appendix

Experimental choice options which were presented randomly, once as a gain and once as a loss.

Footnotes

*

We thank the Society for Judgment and Decision Making for a student research prize, awarded for a poster presentation based on this research at the 21st annual meeting of the Society (2007). This research was completed as part of Edward Cokely's dissertation submitted to The Florida State University. We would like to acknowledge the other members of the committee including K. Anders Ericsson, Neil Charness, Joyce Ehrlinger, and Michelle Bourgeios. We offer special thanks to Henrik Olsson and to Gerd Gigerenzer for extensive comments on earlier versions of the manuscript. We also thank Tres Roring, Ainsley Mitchum, Lael Schooler, Jeffery Stevens, Jörg Rieskamp, Linnea Karlsson, Mirta Galesic, Bettina Von Helversen, Cari Zimmerman, Mark Fox, Katy Nandagopal, Mike Tuffiash, and Steven Sloman for their comments. We are indebted to Amanda Walsh, Alicia Eddy, Ashly Baker, Carolina Avila, Tristan McCain, and Richard Molina for assistance in testing subjects and transcribing protocols.

1 The exact predictions of prospect theory depend on the model parameters used.

2 Stanovich and West (Reference Stanovich and West1998) have also shown that metacognitive factors account for unique variance in judgment and decision performance.

3 Retrospective reports were selected as the concurrent reports used during pilot studies were often unrevealing. Because retrospect reports rely on memory they are not as reliable as concurrent reports concerning the ordering of cognitive events (Ericsson & Simon, Reference Ericsson and Simon1980). Therefore, we used a conservative data analysis approach focusing only on the type and number of considerations verbalized, but not the order of output. The verbalized content of interest is likely to be at least moderately reliable as verbal protocols immediately followed decision making (which lasted only a few seconds) and the number of verbalized considerations was found to correlate with participants’ overall decision latencies (see results).

4 In a pilot study, as part of the first author's doctoral dissertation, we found that the majority of participants did not have sufficient math skills to calculate expected values when explicitly instructed to do so. Nonetheless, many of these participants still made many choices consistent with expected-value predictions.

5 Additional individual model-prediction-accuracy scores (i.e., analyses comparing the proportion of priority heuristic consistent choices averaged across all choices for each individual) indicated that priority heuristic was less accurate (M = .42, SD = .09) compared to chance, t (79) = -7.40, p = .001, d = .9, or as compared to expected value, F (1, 79) = 428.95, p < .001, ηp 2 = .84. A univariate ANOVA indicated a significant search difference (one reason, three reasons) in reaction times, F (1, 3151) = 19.63, p = .001. However, this difference was in the opposite direction of that predicted by the priority heuristic. One-reason choices tended to take longer (M = 13.3sec, SD = 9.82sec) than three-reason choices (M = 11.79sec, SD = 9.16sec). Protocol analysis also revealed that key comparisons (i.e., considering the difference in minimum gains relative to the maximum gain) predicted on 100% of all trials were reported on fewer than 1% of trials, whereas processes that were never predicted were among the most frequent verbalizations (e.g., recoding probabilities, considering probabilities low or high; see Table 2). Priority heuristic choices were unrelated to all cognitive ability measures (p > .20).

6 Reaction time was related to number of verbalized considerations; however, latencies showed only an unreliable trend in the expected direction with decision performance, F (1, 3151) = 3.11, p = .08. Non-expected-value choice latencies (M = 12.53 sec, SD = 9.16 sec) were similar to expected-value choice latencies (M = 13.20 sec., SD = 9.85 sec.).

7 Inclusion of these data in additional hierarchical regressions did not significantly change results.

8 Verbalizations of expected-value calculations are never included in estimates of elaborative heuristic search.

9 The correlations presented in Table 6 include the seven participants whose verbal reports were ambiguous and so the strength of some correlations may be underestimated.

10 Unreported analyses also investigated the influence of a fifth factor, an interaction between (3) & (4), which was found to be trivial and unrelated to all other variables including abilities and elaborative heuristic search.

11 Each individual level regression coefficient factor represents the estimated unique influence of that variable controlling for variance attributable to all other individual level coefficient variables.

12 One reviewer suggested a potential concern that completing the CRT before making choices might influence choice processes and outcomes (cf. Hsee & Rottenstreich, Reference Hsee and Rottenstreich2004). Although we suspect this is unlikely the presence of this type of effect cannot be ruled out. Nonetheless, such an effect would not undercut the theoretical implications linking abilities, performance, and elaborative heuristic search process.

13 Evans notes that “the notion that System 2 is in some sense rule-based is compatible with the proposals of most dual process theorists” (p. 261, Evans Reference Evans2008). However, Evans’ modification for a general dual system theory (i.e., a dual type theoretical framework) notes that even if abstract reasoning requires the use of System 2 it would be a mistake to assume that concrete contexts preclude its application, as is apparent in the current protocol data.

References

Alter, A. L., Oppenheimer, D. M., Epley, N., & Eyre, R. N. (2007). Overcoming intuition: Metacognitive difficulty activates analytic reasoning. Journal of Experimental Psychology-General, 136, 569–576.CrossRefGoogle ScholarPubMed
Bailey, H., Dunlosky, J., & Kane, M. J. (2008) Why does working memory span predict complex cognition? Testing the strategy affordance hypothesis. Memory & Cognition, 36, 1383–1390.CrossRefGoogle ScholarPubMed
Baron, J. (1978). Intelligence and general strategies. In Underwood, G. (Eds.). Strategies of information processing, (pp. 403–450). Academic Press.Google Scholar
Baron, J. (1985) Rationality and intelligence. New York, NY: Cambridge University Press.CrossRefGoogle Scholar
Baron, J. (1990). Reflectiveness and rational thinking: Response to Duemler and Mayer (1988). Journal of Educational Psychology, 82, 391–392.CrossRefGoogle Scholar
Botella, J., Pena, D., Contreras, M. J., Shih, P. C., & Santacreu, J. (2009). Performance as a function of ability, resources invested, and strategy used. Journal of General Psychology, 136, 41–69.CrossRefGoogle ScholarPubMed
Brandstätter, E., Gigerenzer, G., & Hertwig, R. (2006). The priority heuristic: Making choices without trade-offs. Psychological Review, 133, 409–432.CrossRefGoogle Scholar
Brandstätter, E., Gigerenzer, G., & Hertwig, R. (2008). Risky choice with heuristics: Reply to Birnbaum (2008), Johnson, Schulte-Mecklenbeck, and Willemsen (2008), and Rieger and Wang (2008). Psychological Review, 115, 281–289.CrossRefGoogle Scholar
Bröder, A. (2003). Decision making with the “adaptive toolbox”: Influence of environmental structure, intelligence, and working memory load. Journal of Experimental Psychology-Learning Memory and Cognition, 29, 611–625.CrossRefGoogle ScholarPubMed
Birnbaum, M. H. (2008). New tests of cumulative prospect theory and the priority heuristic: Probability-outcome tradeoff with branch splitting. Judgment and Decision Making, 3, 304–316.10.1017/S1930297500000875CrossRefGoogle Scholar
Cokely, E. T., & Feltz, A. (2009). Individual differences, judgment biases, and theory-of-mind: Deconstructing the intentional action side effect asymmetry. Journal of Research in Personality, 43, 18–24.CrossRefGoogle Scholar
Cokely, E. T., Kelley, C. M., & Gilchrist, A. L. (2006). Sources of individual differences in working memory: Contributions of strategy to capacity. Psychonomic Bulletin & Review, 13, 991–997.CrossRefGoogle ScholarPubMed
De Neys, W. (2006). Dual processing in reasoning - Two systems but one reasoner. Psychological Science, 17, 428–433.CrossRefGoogle ScholarPubMed
De Neys, W., & Glumicic, T. (2008). Conflict monitoring in dual process theories of thinking. Cognition, 106, 1248–1299CrossRefGoogle ScholarPubMed
Dougherty, M. R. P., & Hunter, J. E. (2003). Hypothesis generation, probability judgment, and individual differences in working memory capacity. Acta Psychologica, 113, 263–282.CrossRefGoogle ScholarPubMed
Ericsson, K. A., & Kintsch, W. (1995). Long-term working-memory. Psychological Review, 102, 211–245.CrossRefGoogle ScholarPubMed
Ericsson, K. A., Prietula, M. J., & Cokely, E. T. (2007). The making of an expert. Harvard Business Review, 85, 114–121.Google ScholarPubMed
Ericsson, K. A., & Simon, H. A. (1980). Verbal reports as data. Psychological Review, 87, 215–251.CrossRefGoogle Scholar
Evans, J. (2008). Dual-processing accounts of reasoning, judgment, and social cognition. Annual Reviewof Psychology, 59, 255–278.CrossRefGoogle ScholarPubMed
Fasolo, B., Misuraca, R., & McClelland, G. H. (2003). Individual differences in adaptive choice strategies. Research in Economics, 57, 219–233.CrossRefGoogle Scholar
Feltz, A., & Cokely, E. T. (2008). Do judgments about freedom and responsibility depend on who you are? Personality differences in intuitions about compatibilism and incompatibilism. Consciousness and Cognition. doi: http://dx.doi.org/10.1016/j.concog.2008.08.001.CrossRefGoogle Scholar
Frederick, S. (2005). Cognitive reflection and decision making. Journal of Economic Perspectives, 19, 24–42.CrossRefGoogle Scholar
Galesic, M., Garcia-Retamero, R., & Gigerenzer, G. (2008). Using Icon Arrays to communicate medical risks: Overcoming low numeracy. Health Psychology. doi: http://dx.doi.org/10.1037/a0014474CrossRefGoogle Scholar
Gaissmaier, W., & Schooler, L. J. (2008). The smart potential behind probability matching. Cognition, 109, 416–422.CrossRefGoogle ScholarPubMed
Gaissmaier, W., Schooler, L. J., & Rieskamp, J. (2006). Simple predictions fueled by capacity limitations: When are they successful? Journal of Experimental Psychology: Learning, Memory, and Cognition, 32, 966–982.Google ScholarPubMed
Gaissmaier, W., Schooler, L. J., & Mata, R. (2008). An ecological perspective to cognitive limits: Modeling environment-mind interactions with ACT-R. Judgment and Decision Making, 3, 278–291.CrossRefGoogle Scholar
Gigerenzer, G., & Todd, P. M. and the ABC Research Group (1999). Simple heuristics that make us smart. Oxford: Oxford University Press.Google Scholar
Gigerenzer, G., & Goldstein, D. G. (1996). Reasoning the fast and frugal way: Models of bounded rationality. Psychological Review, 103, 650–669.CrossRefGoogle ScholarPubMed
Guida, A., Tardieu, H., & Nicolas, S. (2008). The personalisation method applied to a working memory task: Evidence of long-term working memory effects. European Journal of Cognitive Psychology, doi: http://dx.doi.org/10.1080/09541440802236369.CrossRefGoogle Scholar
Hertzog, C., & Robinson, A. E. (2005). Metacognition and intelligence. In Wilhelm, O. & Engle, R. W. (Eds.),Understanding and measuring intelligence, pp. 101–123. London: Sage.CrossRefGoogle Scholar
Hilbig, B. E. (2008). One-reason decision making in risky choice? A closer look at the priority heuristic. Judgment and Decision Making, 3, 457–462.CrossRefGoogle Scholar
Hsee, C. K., & Rottenstreich, Y. (2004). Music, pandas, and muggers: On the affective psychology of value. Journal of Experimental Psychology-General, 133, 23–30.CrossRefGoogle ScholarPubMed
Jacoby, L. L., Kelley, C. M., & McElree, B. D. (1999). The role of cognitive control: Early selection versus late correction. In Chaiken, S. & Trope, Y., (Eds.), Dual-Process Theories in Social Psychology, pp. 383–402. New York, US: The Gilford Press.Google Scholar
Jacoby, L. L., Shimizu, Y., Daniels, K. A., & Rhodes, M. G. (2005). Modes of cognitive control in recognition and source memory: Depth of retrieval. Psychonomic Bulletin & Review, 12, 852–857.CrossRefGoogle ScholarPubMed
Johnson, E. J., Haubl, G., & Keinan, A. (2007). Aspects of endowment: A query theory of value construction. Journal of Experimental Psychology: Learning Memory and Cognition, 33, 461–474.Google ScholarPubMed
Johnson, E. J., Schulte-Mecklenbeck, M., & Willernsen, M. C. (2008). Process models deserve process data: Comment on Brandstätter, Gigerenzer, and Hertwig (2006). Psychological Review, 115, 263–272.CrossRefGoogle ScholarPubMed
Kahneman, D. (2003). A perspective on judgment and choice - Mapping bounded rationality. American Psychologist, 9, 697–720.CrossRefGoogle Scholar
Kahneman, D., & Tversky, A. (1979). Prospect theory: An analysis of decision under risk. Econometrica, 47, 263–291.CrossRefGoogle Scholar
Kahneman, D., & Frederick, S. (2007). Frames and brains: Elicitation and control of response tendencies. Trends in Cognitive Sciences, 11, 45–46.CrossRefGoogle ScholarPubMed
Karlsson, L., Juslin, P., & Olsson, H. (2008). Exemplar-based inference in multi-attribute decision making: Contingent, not automatic, strategy shifts? Judgment and Decision Making, 3, 244–260.CrossRefGoogle Scholar
Lipkus, I. M., Samsa, G., & Rimer, B. K. (2001). General performance on a numeracy scale among highly educated samples. Medical Decision Making, 21, 37–44.CrossRefGoogle ScholarPubMed
Mata, R., Schooler, L. J., & Rieskamp, J. (2007). The aging decision maker: Cognitive aging and the adaptive selection of decision strategies. Psychology and Aging, 22, 796–810.CrossRefGoogle ScholarPubMed
Mann, L., & Ball, C. (1994). The relationship between search strategy and risky choice. Australian Journal of Psychology, 46, 131–136.CrossRefGoogle Scholar
McNamara, D. S., & Scott, J. L. (2001). Working memory capacity and strategy use. Memory & Cognition, 29, 10–17.CrossRefGoogle ScholarPubMed
Morsanyi, K, & Handley, S. J. (2008). How smart do you need to be to get it wrong? The role of cognitive capacity in the development of heuristic-based judgment. Journal of Experimental Child Psychology, 99, 18–36.CrossRefGoogle Scholar
Payne, J. W. (1976). Task complexity and contingent processing in decision-making — information search and protocol analysis. Organizational Behavior and Human Performance, 16, 366–387.CrossRefGoogle Scholar
Payne, J. W., Bettman, J. R., Coupey, E., & Johnson, E. J. (1992). A constructive process view of decision-making — multiple strategies in judgment and choice. Acta Psychologica, 80, 107–141.CrossRefGoogle Scholar
Payne, J. W., Bettman, J. R., Johnson, E. J., 1993. The adaptive decision maker. New York, NY: Cambridge University Press.CrossRefGoogle Scholar
Payne, J. W., & Braunstein, M. L. (1978). Risky choice — examination of information acquisition behavior, Memory & Cognition, 6, 554–561.CrossRefGoogle ScholarPubMed
Payne, J. W., Samper, A., Bettman, J. R., & Luce, M. F. (2008). Boundary Conditions on Unconscious Thought in Complex Decision Making. Psychological Science, 19, 1118–1123.CrossRefGoogle ScholarPubMed
Peters, E. & Levin, I. P. (2008). Dissecting the risky-choice framing effect: Numeracy as an individual-difference factor in weighting risky and riskless options. Judgment and Decision Making, 3, 435–448.CrossRefGoogle Scholar
Peters, E., Västfjäll, D., Slovic, P., Mertz, C. K., Mazzocco, K., & Dickert, S. (2006). Numeracy and decision making. Psychological Science, 17, 407–413.10.1111/j.1467-9280.2006.01720.xCrossRefGoogle ScholarPubMed
Regenwetter, M., Grofman, B., Popova, A., Messner, W., Davis-Stober, C. & Cavagnaro, D. (2008). “Behavioral social choice: A status report.” Philosophical Transactions of the Royal Society B: Biological Sciences. doi: http://dx.doi.org/10.1098/rstb.2008.0259CrossRefGoogle Scholar
Reisen, N., Hoffrage, U., & Mast, F. W. (2008). Identifying decision strategies in a consumer choice situation. Judgment and decision making, 3, 641–658.CrossRefGoogle Scholar
Rettinger, D. A., & Hastie, R. (2001). Content effects on decision making. Organizational behavior and human decision processes, 85, 336–359.CrossRefGoogle ScholarPubMed
Rettinger, D. A., & Hastie, R. (2003). Comprehension and decision making. In Schneider, S. L. & Shanteau, J. (Eds.), Emerging perspectives on judgment and decision research, pp. 165–201. New York, NY: Cambridge University Press.CrossRefGoogle Scholar
Rieskamp, J., & Hoffrage, U. (2008). Inferences under time pressure: How opportunity costs affect strategy selection. Acta Psychologica, 127, 258–276CrossRefGoogle ScholarPubMed
Rieskamp, J., & Otto, P. E. (2006). SSL: A theory of how people learn to select strategies. Journal of Experimental Psychology: General, 135, 207–236.CrossRefGoogle ScholarPubMed
Schooler, L. J., & Hertwig, R. (2005). How forgetting aids heuristic inference. Psychological Review, 112, 610–628.CrossRefGoogle ScholarPubMed
Selart, M., Kuvaas, B., Boe, O., & Takemura, K. (2006). The influence of decision heuristics and overconfidence on multiattribute choice: A process-tracing study. European Journal of Cognitive Psychology, 18, 437–453.CrossRefGoogle Scholar
Simon, H. A. (1990). Invariants of human behavior. Annual Review of Psychology, 41, 1–19.CrossRefGoogle ScholarPubMed
Sloman, S. A. (1996). The empirical case for two systems of reasoning. Psychological Bulletin, 199, 3–22.CrossRefGoogle Scholar
Shanteau, J. (1992). How much information does an expert use? Is it relevant? Acta Psychologica, 81, 75–86.CrossRefGoogle Scholar
Sternberg, R. J. (1977). Component processes in analogical reasoning. Psychological Review, 84, 353–378.CrossRefGoogle Scholar
Stanovich, K. E., & West, R. F. (1998). Individual differences in rational thought. Journal of Experimental Psychology: General, 127, 161–188.CrossRefGoogle Scholar
Stanovich, K. E., & West, R. F. (2000). Individual differences in reasoning: Implications for the rationality debate. Behavioral and Brain Sciences, 23, 645–665.CrossRefGoogle ScholarPubMed
Stanovich, K. E., & West, R. F. (2008). On the relative independence of thinking biases and cognitive ability. Journal of Personality and Social Psychology, 94, 672–695.CrossRefGoogle ScholarPubMed
Thomas, R. P., Dougherty, M. R. P., Sprenger, A. M., & Harbison, J. I. (2008). Diagnostic hypothesis generation and human judgment. Psychological Review, 115, 155–185.CrossRefGoogle ScholarPubMed
Turner, M. L., & Engle, R. W. (1989). Is working memory capacity task dependent? Journal of Memory & Language, 28, 127–154.CrossRefGoogle Scholar
Tversky, A. & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185, 1124–1130.CrossRefGoogle ScholarPubMed
Vigneau, F., Caissie, A. F., & Bors, D. A. (2005). Eye-movement analysis demonstrates strategic influence on intelligence. Intelligence, 34, 261–272.CrossRefGoogle Scholar
Weber, E. U., Johnson, E. J., Milch, K. F., Chang, H., Brodscholl, J. C., & Goldstein, D. G. (2007). Asymmetric discounting in intertemporal choice: A query-theory account. Psychological Science, 18, 516–523.CrossRefGoogle Scholar
Figure 0

Table 1: Example of discrete model predictions and predicted verbal reports for a sample choice.

Figure 1

Table 2: Coding system for protocol analysis including examples of each consideration, mean considerations per trial (and standard deviations), and total observed considerations.

Figure 2

Table 3: A sample of protocol analysis revealing expected-value calculation or estimation.

Figure 3

Figure 1: A linear regression with elaborative heuristic search (i.e., the number of verbalized considerations) predicting each participant's overall proportion of expected-value choices (ambiguous and expected-value verbalizations are not included). The line is based on the regression.

Figure 4

Table 4: A sample of coded protocol analysis from individuals with lower working memory, numeracy, and/or cognitive reflection scores (i.e., bottom quartile).

Figure 5

Table 5: A sample of coded protocol analysis from individuals with higher working memory, numeracy, and/or cognitive reflection scores (i.e., top quartile).

Figure 6

Table 6: Intercorellations for main variables.

Figure 7

Table 7: Hierarchical linear regression analysis explaining expected-value choices.

Figure 8

Table 8: Intercorrelations of ability, elaborative heuristic search, and individual level regression coefficients (indicated by β).