Hostname: page-component-586b7cd67f-vdxz6 Total loading time: 0 Render date: 2024-11-23T20:59:37.809Z Has data issue: false hasContentIssue false

Numeracy predicts preference consistency: Deliberative search heuristics increase choice consistency for choices from description and experience

Published online by Cambridge University Press:  01 January 2023

Nathaniel J. S. Ashby*
Affiliation:
Faculty of Industrial Engineering and Management, Technion – Israel Institute of Technology, Technion City, Haifa 32000, Israel
*
Rights & Permissions [Opens in a new window]

Abstract

Many people exhibit inconsistent preferences when they make choices based on descriptive summaries as compared to choices based on prior experiences. Theoretically, factors that promote more deliberative and skilled decision making (e.g., statistical numeracy) should also tend to promote more consistent choices and preferences regardless of information presentation formats. To test this hypothesis, in two studies I investigated individual differences in information search strategies (e.g., the amount and direction of information search) while estimating the degree to which numeracy predicted risky choice consistency across (a) decisions-from-description and (b) the decisions-from-samples. Results from Study 1 revealed that numeracy was generally associated with more extensive sampling and greater choice consistency across paradigms. Instead of reflecting differences in logical maximizing strategies (e.g., calculating expected values), analyses indicated that highly-numerate participants largely generated consistent choices by relying on heuristic-type processes (e.g., alternated between options less and exploring options with higher outcome variability more). In Study 2, the relationship between numeracy and choice consistency was eliminated when all participants were forced to make choices after experiencing the same fixed amount of sampling (i.e., provided the same amount of information). Taken together, the results of both studies converge with and extend previous findings (e.g., Skilled Decision Theory), indicating that numerate participants may generally exhibit more consistent preferences and choices via deliberatative heuristic search during decision making.

Type
Research Article
Creative Commons
Creative Common License - CCCreative Common License - BY
The authors license this article under the terms of the Creative Commons Attribution 3.0 License.
Copyright
Copyright © The Authors [2017] This is an Open Access article, distributed under the terms of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.

1 Introduction

Decisions are frequently influenced by numerical information. Sometimes this information is provided as a descriptive summary (e.g., the stated return from an investment or the risk of side effects from a medication), while in other situations it is gleaned from prior experience (e.g., memory of previous returns from stock or the amount of time a particular route home took). Interestingly, risky decisions based on experience or descriptive summaries do not always align (Reference Barron and ErevBarron & Erev, 2003; see for review, Reference Rakow, Newell and ZougkouRakow & Newell, 2010). As a means of understanding this divergence, there has been particular interest in the way information is sought out (see for review, Erev et al., 2010; Reference Mehlhorn, Newell, Todd, Lee, Morgan, Braithwaite and GonzalezMehlhorn et al., 2015). Research suggests that the amount of information collected, and to some extent the way it is used, is influenced by cognitive and numeric abilities (Reference Ashby and RakowAshby & Rakow, 2014; Reference LejarragaLejarraga, 2010; Reference Rakow, Newell and ZougkouRakow, Newell & Zougkou, 2010), emotional states (Frey, Hertwig & Reiskamp, 2014), age (Reference Frey, Mata and HertwigFrey, Mata & Hertwig, 2015), search strategy (Reference Hills and HertwigHills & Hertwig, 2010), and perspective (Reference Pachur and ScheibehennePachur & Scheibehenne, 2012). The current work extends these findings by providing an analysis of the role numeric ability (numeracy) plays in search (exploration), and its influence on preference stability (i.e., the extent to which the same choices are made) across descriptive and experiential formats. Specifically, I test whether individuals with greater numeric ability seek out more experience before making a decision and show greater choice consistency across decisions made based on descriptive information or experience.

Numeracy tends to be the single strongest predictor of general decision making skill, including the ability to understand and evaluate risk (i.e., risk literacy; Cokely et al, in press, 2012). Statistical numeracy in particular (i.e., the ability to solve problems involving probabilistic information), has been shown to be predictive of performance in decisions based on descriptive information (decisions-from-description; Peters et al., 2006, 2012). For example, individuals with higher numeric ability are less influenced by frame (e.g., when outcomes are framed as gains or losses; Reference Peters and LevinPeters & Levin, 2008), make more normatively consistent decisions (Reference Cokely and KelleyCokely & Kelley, 2009), perform better on the job (Reference Burks, Carpenter, Goette and RustichiniBurks, Carpenter, Goette & Rustichini, 2009), and are generally more adept at using numerical information when making decisions (Reference Cokely, Galesic, Schulz, Ghazal and Garcia-RetameroCokely et al., 2012; Reference Garcia-Retamero and CokelyGarcia-Retamero & Cokely, 2017; Reference Pachur and GalesicPachur & Galesic, 2013; Reference PetersPeters et al., 2012; Petrova et al., in press; Reference Reyna, Nelson, Han and DieckmannReyna, Nelson, Han & Dieckmann, 2009). In addition, there is evidence that numeracy influences how probabilities (Reference Patalano, Saltiel, Machlin and BarthPatalano et al., 2015; Reference Traczyk and FulawkaTraczyk & Fulawka, 2016) and outcomes (Reference Schley and PetersSchley & Peters, 2014) are treated, with numerate individuals appearing to treat them more objectively (i.e., treating 90% more like 90% rather than 90% ± some bias). Numeracy has also been found to be related to increases in the amount of information sought out in decisions-from-samples paradigms (where decision makers simulate playing available options before making a consequential choice; Reference LejarragaLejarraga, 2010) and choosing options with higher expected value (Reference Frey, Mata and HertwigEV; Reference Jasper, Bhattacharya, Levin, Jones and BossardJasper, Bhattacharya, Levin, Jones & Bossard, 2013). Given that individuals with greater statistical numeracy make more normatively consistent decisions when options are described and experienced, one might suspect that they would also show greater preference consistency across descriptive and experiential formats.

This paper tests the hypothesis that individuals with higher statistical numeracy (referred to as numeracy from here on) generally support skilled decision making by searching for more information (Reference Cokely and KelleyCokely & Kelley, 2009; Reference Jasper, Bhattacharya and CorserJasper, Bhattacharya & Corser, 2017; Reference LejarragaLejarraga, 2010), and as a result will tend to express greater consistency in choices (e.g., revealed preferences) across decisions made from description and experience. In addition, I test whether numeracy influences the rate of alternation (i.e., switching from one option to another on consecutive samples) during information search, influencing choice. Specifically, Hills and Hertwig (2010) reported that individuals who alternated less made more EV maximizing choices (i.e., choosing the option with the highest EV). Given that numerate individuals have been found to make choices that align with an EV maximization strategy, one might predict that they also alternate between options less during search.

Second, I examine whether numeracy increases sampling from riskier options (i.e., options containing outcomes that occur with smaller probabilities). Such a bias in sampling is of interest because greater sampling from riskier options is required if one wishes to see all possible outcomes they may contain.Footnote 1 Furthermore, previous research indicates that one of the largest contributors to the divergence between risky choices made from experience and description (when rare events are involved) is the result of limited sampling from riskier options (Reference Glöckner, Hilbig, Henninger and FiedlerGlöckner, Hilbig, Henninger & Fiedler., 2016; Reference Hadar and FoxHadar & Fox, 2009; Reference Hau, Pleskac, Kiefer and HertwigHau, Pleskac, Kiefer & Hertwig, 2008; Reference Rakow, Demes and NewellRakow, Demes & Newell, 2008). Therefore, in order for more numerate individuals’ choices to align across decision formats as I predict, they should draw more representative samples from the riskier option.

Lastly, given that previous studies have reported that numerate individuals are more likely to choose higher EV options in both descriptive and experiential formats in higher stakes gambles with larger EV ratios (e.g., where one option returns twice the amount of another on average; Cokely & Kelly, 2009; Reference Jasper, Bhattacharya, Levin, Jones and BossardJasper et al., 2013; Reference Peters, Västfjäll, Slovic, Mertz, Mazzocco and DickertPeters et al., 2006), I examine whether such effects are found when stakes and EV ratios are smaller. To test these predictions, I present participants with the same choices in decisions-from-description and decisions-from-samples (Hertwig, Barron, Weber & Erev, 2005) and assess their numeracy (Reference Cokely, Galesic, Schulz, Ghazal and Garcia-RetameroCokely et al., 2012).

2 Study 1

2.1 Participants

One-hundred and eighty participants (M age = 34.37; 41% male) were recruited from Amazon Mechanical Turk and completed the study in full: The study was run in a session along with several unrelated studies. Participants reported relatively diverse educational backgrounds: Approximately 12% of participants reported having a high-school education or GED, 28% some college, 12% a 2-year degree, 35% a 4-year degree, 9% a masters, 2% a professional degree, and 1% a doctoral degree. As an attention check participants had to click an invisible box instead of “continue” on an instruction page. Participants who failed this attention check were not allowed to take part in the study. Participants received $0.80 as well as a bonuses contingent on their choices during the task (see below). The experimental session lasted approximately 30 minutes and participants earned $3.00 on average.

2.2 Materials and Procedure

After providing informed consent and answering demographic questions (e.g., age, gender, education, etc.) participants were presented with the adaptive version of the Berlin Numeracy Test (Reference Cokely, Galesic, Schulz, Ghazal and Garcia-RetameroCokely et al., 2012)Footnote 2: This test has undergone extensive validation in large diverse and representative samples, and correlates robustly with other (longer) numerical ability, reasoning, and decision making skill inventories (e.g., Cokely et al., in press). The Berlin Numeracy Test provides an efficient means of sorting people into quartiles (normed against college educated adults from industrialized countries). In the task, participants are presented with narrative math problems that become harder (require greater numerical ability/reasoning) to solve as they are answered correctly (two to three questions are asked). Scores range from one to four with higher scores indicating greater statistical numeracy.

After taking part in unrelated tasks, participants were told that they would be making choices across several different pairs of options, and that the points (pts) they earned from these choices would be converted to a bonus payment (40 pts = $0.01). Participants were not shown the outcomes of their choices but they were added to their bonus earnings. They then made consequential choices between the fully described (see Figure 1 for a screenshot of a trial) pairs of options in Table 1 (presented in random order for each participant). Pairs consisted of a safer (less outcome variability) option paying out two modest amounts with nearly equivalent probabilities and a riskier (more outcome variability) option paying out a larger amount about a quarter of the time and a smaller amount more frequently: In half of the pairs the riskier option provided a higher EV than the safer option.

Figure 1. Screenshot of a decisions-from-description trial. Participants clicked on the option they wanted to play and the resulting outcome (not shown to participants) was added to their bonus earnings.

Table 1. Table 1. Outcomes (O1-O2) in points and their respective probabilities (P1–P2) for the riskier (R) and safer options (S) and their expected values (EV) in Study 1. Also the difference between EVs (D EV = EVR – EVS) and the proportion of participants selecting the riskier option in the described (RD) and experiential (RE) formats in Study 1. Pluses (+) in the S2 column indicate option pairs that were included in Study 2.

Intermixed with the target pairs were 11 pairs of options drawn from Reference Holt and LauryHolt and Laury (2002). These pairs were constructed to assess individual differences in risk aversion, but were employed in the current study to serve as distractors and to screen out inattentive participants (see Table A1 in the Appendix). Specifically, in one pair the safer option paid out 200 pts with certainty while the riskier option paid out 385 pts with certainty, while in another pair the safer option paid out 160 pts with certainty while the riskier option paid out 10 pts with certainty. Participants who chose the option of lower value (e.g., choosing the 10 pts over the 160 pts) in these pairs are excluded from all analyses since they were either not paying attention to the task or had not understood the task.

Following a brief personality questionnaire used as a filler taskFootnote 3 (Reference Gosling, Rentfrow and SwannGosling, Rentfrow & Swan, 2003), participants selected between the same pairs of options in a decisions-from-samples paradigm (Hertwig et al., 2005); pairs were presented in random order for each participant. In this task participants were told that they would be presented with two options and that they would need to sample from the options (i.e., simulate playing them) in order to see what outcomes they provided before making one consequential choice (see Figure 2). Participants were free to sample from each option zero to 100 times: The upper bound of a 100 was known to participants and rarely reached. As with the choices in the decisions-from-description task the outcomes of the consequential choices in the decisions-from-samples task were not told to participants but were added to their earnings. After making their last consequential choice participants were told of their total earnings across the two tasks and thanked for their time.

Figure 2. Example of the course of a decisions-from-samples trial. Participants first explored (sampled from) the options by clicking on them to simulate plays (Panel 1), seeing the outcome of each sample for half a second (Panel 2). When ready participants indicated they wished to make their one consequential choice (Panel 3) and then selected the option they wanted to play once with consequence (Panel 4). This sequence was repeated until all option pairs were encountered.

2.3 Exclusions

Observations were excluded for two reasons that were established before the study was conducted. First, trials where a participant did not sample from each option at least once (658 observations across 76 participants – four participants never sampled) were removed, as were their corresponding choices in the decisions-from-description format. These exclusions are made since samples of size zero indicate a complete lack of engagement in the task. Sample sizes of zero are included in the analyses examining the number of samples drawn.Footnote 4 Next, 26 participants who chose either (or both) of the dominated options (e.g., choosing 200 pts over 385 pts) in the Holt-Laury pairs were excluded (one participant chose an inferior option and never sampled). Thus, the final analysis includes 2,775 observations from 151 participants.

2.4 Results

2.4.1 Numeracy

Participants were fairly distributed across possible numeracy scores: Numeracy scores of one (low numeracy: 36 participants, 39 including three who never sampled in any trial), two (49 participants), three (24 participants), and four (high numeracy; 42 participants) were observed.

2.4.2 Sample size

Overall, individuals sampled a relatively small proportion of the available information (M = 12.49; CI95%[11.87, 13.12]) as is common in the decisions-from-samples paradigm (see for example studies conducted in the lab with higher pay and larger outcomes, Reference Ashby and RakowAshby & Rakow, 2014; Reference Hertwig and PleskacHertwig & Pleskac, 2010). The number of samples drawn across the two options ranged from zero to 200 (95% range [0, 31]; see the left panel of Figure 3). To test whether numeracy was related to the amount of information sought out the average number of samples drawnFootnote 5 was regressed on numeracy. Replicating previous findings (Reference LejarragaLejarraga, 2010), numeracy was positively related to the number of samples drawn, b = 2.09, t(152) = 1.99, p = .048, R 2 = .03 (see the right panel of Figure 3 and Table A2 in the Appendix). Removing observations where each option was not sampled from at least once reduced this relationship to non-significance, p = .17. Thus, it appears that the increase in sampling that is related to numeracy in the current study is primarily driven by less numerate individuals who did not sample from each option at least once before making a decision.

Figure 3. The histogram in the left panel displays the distribution of samples drawn across both options in each trial. The scatterplot in the right panel displays the relationship between participants’ average number of samples and their numeracy score; dashed line represents the identity line.

2.4.3 Exploration Strategy

Alternation Rate

To test whether numeracy influenced the rate of alternation (selecting different options on consecutive samples) I regressed the average rate of alternation on numeracy. As predicted, the rate of alternation was negatively related to numeracy, b = –.07, t(149) = –3.05, p = .003, R 2 = .06 (Figure 4 and Table A2 in the Appendix).

Figure 4. Scatterplot showing the relationship between participants’ average alternation rate and their numeracy score. Dashed line represents the identity line.

Sampling biases

To test whether numeracy increased the proportion of samples drawn from riskier options I created a variable indexing the proportion of total samples drawn from the riskier option for each individual and pair of options:

I then regressed the average proportion on numeracy and found a significant positive relationship, b = .02, t(149) = 3.49, p = .001, R 2 = .08 (Figure 5 and Table A2 in the Appendix). As predicted, more numerate individuals explored riskier options to a greater extent than less variable options.Footnote 6

Figure 5. Scatter plot displaying the relationship between the average proportion of samples a participant drew from the riskier option and their numeracy score. Dashed line represents the identity line.

2.4.4 Choice consistency

To test the prediction that the choices of those with greater numeric ability would align across decision formats to a greater extent I generated a variable indicating the average rate of choice alignment across the described and experiential formats (i.e., picking the riskier option in a pair in both the decisions-from-description and decisions-from-samples tasks). I then regressed this variable on numeracy.Footnote 7 As predicted, the relationship between numeric ability and the rate of choice consistency was significant, with more numerate individuals showing greater consistency across formats, b = .03, t(149) = 1.99, p = .048, R 2 = .03 (see the left panel of Figure 6 and Table A2 in the Appendix).

Figure 6. Scatter plots showing participants’ average rate of choice alignment across the decisions-from-description and decisions-from-samples formats and their numeracy score in Study 1 (free sampling, left panel) and Study 2 (fixed sampling, right panel). Dashed lines represent identity lines.

2.4.5 EV Maximization

In order to test whether individuals with greater numeracy selected higher EV options to a greater extent, two variables were constructed: The average rate at which a participant selected the option with a higher objective EV in the decisions-from-description format, and the average rate at which they selected the option with a higher experienced EV (i.e., the mean of the outcomes sampled from) in the decisions-from-samples format. Neither the relationship between numeracy and the rate of selecting the higher EV option in decisions-from-description (upper left panel Figure 7 and Table A2 in the Appendix), p = .85, nor in decisions-from-samples (upper right panel Figure 7 and Table A2 in the Appendix), p = .76, were significant.Footnote 8

Figure 7. Scatter plots showing the relationship between participants’ average rate of selecting options with higher expected value (EV) and their numeric ability in Study 1 (Free Sampling — upper panels) and Study 2 (Fixed Sampling — lower panels). Plotted separately for choices in the decisions-from-description (left panels) and decisions-from-samples (right panels) paradigms. Dashed lines represent the identity line.

3 Study 2

One reason numerate individuals’ preferences were more consistent across formats may be that their exploration strategies differed (i.e., increased sampling, alternating less, and drawing a larger proportion of samples from more variable options).Footnote 9 For example, sampling more and drawing a greater proportion of samples from variable options should lead to objective and experienced option EVs aligning to a great extent, increasing the likelihood that the same option would be preferred across the two formats. In order to test whether the mechanism behind this increased choice consistency for numerate individuals stems from a more skillful exploration strategy I conducted a follow up study where sampling was the same for all participants. To the extent increased preference consistency in the previous study directly resulted from numeracy’s impact on exploration, the relationship between consistency and numeracy may be reduced or eliminated when sampling is equated.

3.1 Participants

One-hundred and forty-four new participants (M age = 37.78; 50% male) were recruited from Amazon Mechanical Turk and completed the study in full. Approximately 8% of participants reported having a high-school education or GED, 33% some college, 8% a 2-year degree, 33% a 4-year degree, 14% a masters, 2% a professional degree, and 1% a doctoral degree. As in the previous study an attention check was included and those who failed were not allowed to take part in the study. Participants received $0.75 as well as a bonuses contingent on their decisions as in the previous study. The experimental session, which contained other unrelated studies, lasted approximately 30 minutes and participants earned roughly $3.00.

3.2 Changes to method and materials

Three changes were made: First, participants encountered only five pairs of options, to reduce repetition (see column S2 in Table 1). Second, instead of sampling freely from the available options, participants were shown 100 samples from each option. Specifically, either the safer or riskier option (counter balanced across participants) was drawn with the outcome of each draw being shown for 300 milliseconds. Importantly, the outcomes drawn from each option aligned perfectly with the outcomes’ objective probabilities, eliminating any potential sampling error. For instance, for the first riskier option in Table 1 25 draws returned an outcome of 126 pts while 75 draws returned an outcome of 8 pts. Outcomes were dispersed randomly across draws. Lastly, the Holt-Laury pairs from Study 1 were not included.

3.3 Results

3.3.1 Choice Alignment

As in the previous study the average rate of choice alignment was regressed on numeracy. Counter to the previous study there was no robust relationship between the rate of choice alignment and numeracy (right panel Figure 6 and Table A2 in the Appendix), b = .03, t(142) = 1.29, p = .19, R 2 = .01.Footnote 10

3.3.2 EV Maximization

As in the previous study the relationship between the rate of selecting the higher EV option and numeracy was examined. As before the relationship was not significant for decisions-from-description (bottom left panel Figure 7 and Table A2 in the Appendix), b = –.01, t(142) = –.41, p = .68, R 2 < .01, nor for decisions-from-samples (bottom right panel Figure 7 and Table A2 in the Appendix), b = .01, t(142) = .47, p = .64, R 2 < .01.

4 Discussion

The current studies examined whether statistical numeric ability (numeracy) influenced pre-decision search and preference consistency across described (decisions-from-description) and experiential formats (decisions-from-samples). As predicted, numerate individuals drew larger samples before making their final consequential choices. In addition, they sampled from options with higher variability more (i.e., riskier options) and alternated between options less during search. More numerate individuals also made choices that were more consistent across formats. Study 2 employed fixed sampling and failed to find greater choice consistency among participants with higher numeracy scores. Thus, the current studies provide novel evidence indicating that numeric ability influences exploration strategies, which in turn leads to greater preference consistency across decision formats. Nevertheless, decision strategies do not appear to be based on EV maximization strategies (see below). These findings further demonstrate that numeracy is an important individual difference underpinning decisions under conditions of risk and uncertainty, and provide additional insight into decision strategies that skilled decision makers employ (see discussion of Skilled Decision Theory; Cokely et al., in press; Garcia-Retamero & Cokely, in press).

One area where individual differences in numeracy could prove particularly informative is in the description-experience gap (i.e., the propensity to overweight rare events in descriptive formats while underweighting them in experiential formats; Reference Barron and ErevBarron & Erev, 2003; Hertwig et al., 2014). Given that numerate individuals’ choices aligned across formats in the current studies it would be interesting to see if this alignment is also found when choices are between safer (certain) and very risky options (i.e., options with extremely rare outcomes). Given that numerate individuals sampled more in general, and drew a greater proportion of their samples from riskier options, one might expect them to be more consistent between the two formats. This should be the case if limited sampling is indeed the primary driver behind the description-experience gap (Reference Hadar and FoxHadar & Fox, 2009; Reference Glöckner, Hilbig, Henninger and FiedlerGlöckner et al., 2016; Reference Rakow, Demes and NewellRakow et al., 2008).

More numerate participants did not show a greater propensity to select options with higher expected value (EV). In addition, there was no evidence that more numerate individuals “played the odds” more through increased risk seeking — which may be a defensible strategy when stakes are low and differences between accepting risk or playing it safe are minimal. This indicates that the benefits of numeracy in terms of making “better” choices is not apparent in all circumstances (Reference Peters, Västfjäll, Slovic, Mertz, Mazzocco and DickertPeters et al., 2006). Moreover, the current results suggest that more numerate individuals might actually fare worse than their less numerate cohorts if one assumes there are “costs” of investing more time and effort while obtaining little monetary benefit in return (Reference LejarragaLejarraga, 2010). However, to the extent that preference consistency is valuable, then these additional search costs might be viewed as reasonable (e.g., necessary in order to achieve well-considered preferences). As such, the current studies suggest there may be many opportunities for future research to map the boundary conditions that determine when and why numeric ability benefits or harms decision making under risk, broadly defined.

From a pragmatic standpoint, the current studies suggest potential ways in which choices made by those of lower numeracy might be made more consistent with their higher numeracy peers. Specifically, a growing body of work, including the results of present study, suggests that less numerate individuals explore their options less in general (Reference Garcia-Retamero and CokelyGarcia-Retamero & Cokely, 2017; Reference Hess, Visschers, Siegrist and KellerHess, Visschers, Siegrist & Keller, 2011; Reference Okan, Galesic and Garcia-RetameroOkan, Galesic & Garcia-Retamero, 2015). In addition, the current study highlights a novel effect whereby numerate individuals allot a greater proportion of their samples to riskier (more variable) options. As noted previously, increased sampling from options, particularly those with greater outcome variability, leads to estimates of worth that are more likely to align with their EVs. Importantly, Study 2 found that insuring all participants encountered the same amount of information diminished the effect of numeracy on choice alignment. Thus, one way to reduce differences between the decisions made by those with lower and higher numeracy is to ensure equivalency in the information encountered and encoded into memory (e.g., through transparent visual aids; Garica-Retamero & Cokely, 2013, 2017). As such, future research should continue to investigate ways that less numerate individuals can be nudged toward more efficient and deliberative information search, so that they too might become more independent and skilled decision makers (Reference Cokely, Galesic, Schulz, Ghazal and Garcia-RetameroCokely et al., 2012; in press; Baron, 1985, 2000).

Appendix

Table A1: Outcomes (O1 – O2) and probabilities (P1 – P2) for the riskier and safer options drawn from Reference Holt and LauryHolt and Laury (2002) and used as fillers in the decisions-from-description task in Study 1. X’s in the last column indicate whether the riskier option had a higher expected value (EV). The first pair and the last pair (italicized) were used as attention checks: Data from participants who choose the dominated option were excluded from all analyses.

Table A2. Means and 95% confidence intervals in brackets for each dependent variable analyzed by numeracy score and study.

Note. Values calculated after all exclusion criteria applied and averaged within subject.

Table A3. Correlation matrix showing the relationships between each personality factor assessed and each dependent variable analyzed in Study 1.

Note. |r| ≥ .18 for p<.05; |r| ≥ .14 for p<.10.

Table A4 The objective expected values (EV) for the riskier and safer options and the corresponding average experienced EVs (i.e., the average of all outcomes sampled from each option) separately for each numeracy score in Study 1.

Footnotes

*

I would like to thank Daniel Ben-Eliezer, Tim Rakow, Chaim Salzer, Lukasz Walasek, and Eldad Yechiam for their insightful comments on earlier versions of this manuscript. Support was provided by a Technion Fellowship.

Note. Values calculated after all exclusion criteria applied and averaged within subject.

Note. |r| ≥ .18 for p<.05; |r| ≥ .14 for p<.10.

1 For example, if we have a fair six-sided die we would have to roll it six times, on average, to see a one. As we add sides to that die, decreasing the probability of a given side being shown, the number of rolls required to see it would increase.

2 Best practices for MTurk would be to employ the BNT-Schwartz described in Cokely et al. (2012).

3 Correlations between personality traits and the dependent variables analyzed in this study are reported in Table A3 in the Appendix.

4 In a similar vein one might argue that decisions made very quickly in the decisions-from-description task should also be removed as very fast decisions might also indicate a lack of interest in the task (e.g., choosing at random). Nevertheless, removing observations where decisions were made in less than three seconds (i.e., the bottom 10% of observations) had little effect on the results obtained. As such no exclusions were made based on decision times.

5 For most of the dependent variables analyzed in the current study a significant amount of skew was observed. Correcting for skew through log transformation did not greatly effect the results. As such, all reported analyses use uncorrected dependent variables to ease interpretation.

6 Another way to examine this relationship is to correlate the number of samples drawn from each option separately with numeracy. Doing so returns a similar conclusion: There was a positive relationship between the average number of samples drawn from riskier alternatives and numeric ability, r(150) = .17, p = .04, which was not present for the average number of samples drawn from safer options, r(150) = .06, p = .50. There was also a strong positive correlation between the average number of samples drawn from the riskier and safer options, r(150) = .91, p < .0001.

7 There was a positive relationship between the risky choice rate observed in each format in line with previous findings (Reference Koritzky and YechiamKoritzky & Yechiam, 2010), r(150) = .44, p < .0001.

8 To see if numeracy was related to the likelihood of picking the risky option (i.e., a preference or aversion to risk) we regressed the average risk rate on numeracy in each decision format. In neither the decisions-from-description, b = .01, t(149) = .42, p = .68, R 2 = .001, nor decisions-from-samples, b = -.01, t(149) = -.45, p = .65, R 2 = .001, was there a significant effect of numeracy on risk rate. A similar pattern was found in Study 2.

9 Some support for this explanation is found by regressing a participant’s average rate of choice consistency on numeric ability, the average number of samples drawn, the average alternation rate, and the average proportion of samples drawn from the riskier option. In this analysis the effect of numeracy is reduced to non-significance, p = .06. This suggests that differences in exploration partially mediates the relationship between choice consistency and numeracy.

10 Power analysis conducted using G*Power (Reference Faul, Erdfelder, Lang and BuchnerFaul, Erdfelder, Lang & Buchner, 2007) suggests the current study had low power to detect this effect (1-b = .25). To have 90% power to significantly detect such an effect 896 participants would be required.

References

Ashby, N. J., & Rakow, T. (2014). Forgetting the past: Individual differences in recency in subjective valuations from experience. Journal of Experimental Psychology: Learning, Memory, and Cognition, 40(4), 11531162.Google ScholarPubMed
Baron, J. (1985). Rationality and intelligence. Cambridge University Press.CrossRefGoogle Scholar
Baron, J. (2000). Thinking and deciding. Cambridge University Press.Google Scholar
Barron, G., & Erev, I. (2003). Small feedback-based decisions and their limited correspondence to description-based decisions. Journal of Behavioral Decision Making, 16(3), 215233.CrossRefGoogle Scholar
Burks, S. V., Carpenter, J. P., Goette, L., & Rustichini, A. (2009). Cognitive skills affect economic preferences, strategic behavior, and job attachment. Proceedings of the National Academy of Sciences, 106(19), 77457750.CrossRefGoogle ScholarPubMed
Cokely, E. T., Galesic, M., Schulz, E., Ghazal, S., & Garcia-Retamero, R. (2012). Measuring risk literacy: The Berlin numeracy test. Judgment and Decision Making, 7(1), 2547.CrossRefGoogle Scholar
Cokely, E. T., & Kelley, C. M. (2009). Cognitive abilities and superior decision making under risk: A protocol analysis and process model evaluation. Judgment and Decision Making, 4(1), 20-33.CrossRefGoogle Scholar
Cokely, E.T., Feltz, A., Ghazal, S., Allan, J., Petrova, P., & Garcia-Retamero (in press). Decision making skill: From intelligence to numeracy and expertise. The Cambridge Handbook of Expertise and Expert Performance, 2nd Edition.Google Scholar
Erev, I., Ert, E., Roth, A. E., Haruvy, E., Herzog, S. M., Hau, R., ... & Lebiere, C. (2010). A choice prediction competition: Choices from experience and from description. Journal of Behavioral Decision Making, 23(1), 1547.CrossRefGoogle Scholar
Faul, F., Erdfelder, E., Lang, A. G., & Buchner, A. (2007). G* Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior research methods, 39(2), 175191.CrossRefGoogle ScholarPubMed
Frey, R., Hertwig, R., & Rieskamp, J. (2014). Fear shapes information acquisition in decisions from experience. Cognition, 132(1), 9099.CrossRefGoogle ScholarPubMed
Frey, R., Mata, R., & Hertwig, R. (2015). The role of cognitive abilities in decisions from experience: Age differences emerge as a function of choice set size. Cognition, 142, 6080.CrossRefGoogle ScholarPubMed
Garcia-Retamero, R., & Cokely, E. T. (2013). Communicating health risks with visual aids. Current Directions in Psychological Science, 22(5), 392399.CrossRefGoogle Scholar
Garcia-Retamero, R., & Cokely, E. T. (2017). Designing visual aids that promote risk literacy: A systematic review of health research and evidence-based design heuristics. Human Factors, 0018720817690634.CrossRefGoogle ScholarPubMed
Gosling, S. D., Rentfrow, P. J., & Swann, W. B. (2003). A very brief measure of the Big-Five personality domains. Journal of Research in personality, 37(6), 504528.CrossRefGoogle Scholar
Glöckner, A., Hilbig, B. E., Henninger, F., & Fiedler, S. (2016). The reversed description-experience gap: Disentangling sources of presentation format effects in risky choice. Journal of Experimental Psychology: General, 145(4), 486.CrossRefGoogle ScholarPubMed
Hadar, L., & Fox, C. R. (2009). Information asymmetry in decision from description versus decision from experience. Judgment and Decision Making, 4(4), 317325.CrossRefGoogle Scholar
Hau, R., Pleskac, T. J., Kiefer, J., & Hertwig, R. (2008). The description–experience gap in risky choice: The role of sample size and experienced probabilities. Journal of Behavioral Decision Making, 21(5), 493518.CrossRefGoogle Scholar
Hertwig, R., Barron, G., Weber, E. U., & Erev, I. (2004). Decisions from experience and the effect of rare events in risky choice. Psychological Science, 15(8), 534539.CrossRefGoogle ScholarPubMed
Hertwig, R., & Pleskac, T. J. (2010). Decisions from experience: Why small samples?. Cognition, 115(2), 225237.CrossRefGoogle Scholar
Hess, R., Visschers, V. H., Siegrist, M., & Keller, C. (2011). How do people perceive graphical risk communication? The role of subjective numeracy. Journal of Risk Research, 14(1), 4761.CrossRefGoogle Scholar
Hills, T. T., & Hertwig, R. (2010). Information search in decisions from experience do our patterns of sampling foreshadow our decisions?. Psychological Science, 21(12), 17871792.CrossRefGoogle Scholar
Holt, C. A., & Laury, S. K. (2002). Risk aversion and incentive effects. American Economic Review, 92(5), 16441655.CrossRefGoogle Scholar
Jasper, J. D., Bhattacharya, C., & Corser, R. (2017). Numeracy predicts more effortful and elaborative search strategies in a complex risky choice context: A process-tracing approach. Journal of Behavioral Decision Making, 30(2), 224235.CrossRefGoogle Scholar
Jasper, J. D., Bhattacharya, C., Levin, I. P., Jones, L., & Bossard, E. (2013). Numeracy as a predictor of adaptive risky decision making. Journal of Behavioral Decision Making, 26(2), 164173.CrossRefGoogle Scholar
Koritzky, G., & Yechiam, E. (2010). On the robustness of description and experience based decision tasks to social desirability. Journal of Behavioral Decision Making, 23(1), 8399.CrossRefGoogle Scholar
Lejarraga, T. (2010). When experience is better than description: Time delays and complexity. Journal of Behavioral Decision Making, 23(1), 100116.CrossRefGoogle Scholar
Mehlhorn, K., Newell, B. R., Todd, P. M., Lee, M. D., Morgan, K., Braithwaite, V. A., ... & Gonzalez, C. (2015). Unpacking the exploration–exploitation tradeoff: A synthesis of human and animal literatures. Decision, 2(3), 191215.CrossRefGoogle Scholar
Okan, Y., Galesic, M., & Garcia-Retamero, R. (2015). How people with low and high graph literacy process health graphs: Evidence from eye-tracking. Journal of Behavioral Decision Making, 29(2-3), 271294.CrossRefGoogle Scholar
Pachur, T., & Galesic, M. (2013). Strategy selection in risky choice: The impact of numeracy, affect, and cross-cultural differences. Journal of Behavioral Decision Making, 26(3), 260271.CrossRefGoogle Scholar
Pachur, T., & Scheibehenne, B. (2012). Constructing preference from experience: the endowment effect reflected in external information search. Journal of Experimental Psychology: Learning, Memory, and Cognition, 38(4), 11081116.Google ScholarPubMed
Patalano, A. L., Saltiel, J. R., Machlin, L., & Barth, H. (2015). The role of numeracy and approximate number system acuity in predicting value and probability distortion. Psychonomic Bulletin & Review, 22(6), 18201829.CrossRefGoogle ScholarPubMed
Peters, E. (2012). Beyond comprehension the role of numeracy in judgments and decisions. Current Directions in Psychological Science, 21(1), 3135.CrossRefGoogle Scholar
Peters, E., & Levin, I. P. (2008). Dissecting the risky-choice framing effect: Numeracy as an individual-difference factor in weighting risky and riskless options. Judgment and Decision Making, 3(6), 435448.CrossRefGoogle Scholar
Peters, E., Västfjäll, D., Slovic, P., Mertz, C. K., Mazzocco, K., & Dickert, S. (2006). Numeracy and decision making. Psychological Science, 17(5), 407413.CrossRefGoogle ScholarPubMed
Petrova, D., Garcia-Retamero, R., Catena, A., Cokely, E., Carrasco, A. H., Moreno, A. A., & Hernández, J. A. R. (2016). Numeracy Predicts Risk of Pre-Hospital Decision Delay: a Retrospective Study of Acute Coronary Syndrome Survival. Annals of Behavioral Medicine, 1-15.Google Scholar
Rakow, T., Demes, K. A., & Newell, B. R. (2008). Biased samples not mode of presentation: Re-examining the apparent underweighting of rare events in experience-based choice. Organizational Behavior and Human Decision Processes, 106(2), 168179.CrossRefGoogle Scholar
Rakow, T., & Newell, B. R. (2010). Degrees of uncertainty: An overview and framework for future research on experience-based choice. Journal of Behavioral Decision Making, 23(1), 114.CrossRefGoogle Scholar
Rakow, T., Newell, B. R., & Zougkou, K. (2010). The role of working memory in information acquisition and decision making: Lessons from the binary prediction task. The Quarterly Journal of Experimental Psychology, 63(7), 13351360.CrossRefGoogle ScholarPubMed
Reyna, V. F., Nelson, W. L., Han, P. K., & Dieckmann, N. F. (2009). How numeracy influences risk comprehension and medical decision making. Psychological Bulletin, 135(6), 943973.CrossRefGoogle ScholarPubMed
Schley, D. R., & Peters, E. (2014). Assessing “economic value” symbolic-number mappings predict risky and riskless valuations. Psychological Science, 25(3), 753761.CrossRefGoogle ScholarPubMed
Traczyk, J., & Fulawka, K. (2016). Numeracy moderates the influence of task-irrelevant affect on probability weighting. Cognition, 151, 3741.CrossRefGoogle ScholarPubMed
Figure 0

Figure 1. Screenshot of a decisions-from-description trial. Participants clicked on the option they wanted to play and the resulting outcome (not shown to participants) was added to their bonus earnings.

Figure 1

Table 1. Table 1. Outcomes (O1-O2) in points and their respective probabilities (P1–P2) for the riskier (R) and safer options (S) and their expected values (EV) in Study 1. Also the difference between EVs (DEV = EVR – EVS) and the proportion of participants selecting the riskier option in the described (RD) and experiential (RE) formats in Study 1. Pluses (+) in the S2 column indicate option pairs that were included in Study 2.

Figure 2

Figure 2. Example of the course of a decisions-from-samples trial. Participants first explored (sampled from) the options by clicking on them to simulate plays (Panel 1), seeing the outcome of each sample for half a second (Panel 2). When ready participants indicated they wished to make their one consequential choice (Panel 3) and then selected the option they wanted to play once with consequence (Panel 4). This sequence was repeated until all option pairs were encountered.

Figure 3

Figure 3. The histogram in the left panel displays the distribution of samples drawn across both options in each trial. The scatterplot in the right panel displays the relationship between participants’ average number of samples and their numeracy score; dashed line represents the identity line.

Figure 4

Figure 4. Scatterplot showing the relationship between participants’ average alternation rate and their numeracy score. Dashed line represents the identity line.

Figure 5

Figure 5. Scatter plot displaying the relationship between the average proportion of samples a participant drew from the riskier option and their numeracy score. Dashed line represents the identity line.

Figure 6

Figure 6. Scatter plots showing participants’ average rate of choice alignment across the decisions-from-description and decisions-from-samples formats and their numeracy score in Study 1 (free sampling, left panel) and Study 2 (fixed sampling, right panel). Dashed lines represent identity lines.

Figure 7

Figure 7. Scatter plots showing the relationship between participants’ average rate of selecting options with higher expected value (EV) and their numeric ability in Study 1 (Free Sampling — upper panels) and Study 2 (Fixed Sampling — lower panels). Plotted separately for choices in the decisions-from-description (left panels) and decisions-from-samples (right panels) paradigms. Dashed lines represent the identity line.

Figure 8

Table A1: Outcomes (O1 – O2) and probabilities (P1 – P2) for the riskier and safer options drawn from Holt and Laury (2002) and used as fillers in the decisions-from-description task in Study 1. X’s in the last column indicate whether the riskier option had a higher expected value (EV). The first pair and the last pair (italicized) were used as attention checks: Data from participants who choose the dominated option were excluded from all analyses.

Figure 9

Table A2. Means and 95% confidence intervals in brackets for each dependent variable analyzed by numeracy score and study.

Figure 10

Table A3. Correlation matrix showing the relationships between each personality factor assessed and each dependent variable analyzed in Study 1.

Figure 11

Table A4 The objective expected values (EV) for the riskier and safer options and the corresponding average experienced EVs (i.e., the average of all outcomes sampled from each option) separately for each numeracy score in Study 1.

Supplementary material: File

Ashby supplementary material

Ashby supplementary material 1
Download Ashby supplementary material(File)
File 651.3 KB
Supplementary material: File

Ashby supplementary material

Ashby supplementary material 2
Download Ashby supplementary material(File)
File 104.9 KB