New tests of cumulative prospect theory and the priority heuristic: Probability-outcome tradeoff with branch splitting

Abstract

Previous tests of cumulative prospect theory (CPT) and of the priority heuristic (PH) found evidence contradicting these two models of risky decision making. However, those tests were criticized because they had characteristics that might “trigger” use of other heuristics. This paper presents new tests that avoid those characteristics. Expected values of the gambles are nearly equal in each choice. In addition, if a person followed expected value (EV), expected utility (EU), CPT, or PH in these tests, she would shift her preferences in the same direction as shifts in EV or EU. In contrast, the transfer of attention exchange model (TAX) and a similarity model predict that people will reverse preferences in the opposite direction. Results contradict the PH, even when PH is modified to include a preliminary similarity evaluation using the PH parameters. New tests of probability-consequence interaction were also conducted. Strong interactions were observed, contrary to PH. These results add to the growing bodies of evidence showing that neither CPT nor PH is an accurate description of risky decision making.

1 Introduction

This paper compares three models that attempt to describe risky decision making. These models are cumulative prospect theory (CPT) (Tversky & Kahneman, 1992), Birnbaum’s (1999) transfer of attention exchange model (TAX), and the priority heuristic (PH) of Brandstätter, Gigerenzer, and Hertwig (2006). The PH model is based on the idea that people compare one attribute at a time, such as the minimum prizes. In addition, the similarity model of Rubinstein (1988) as modified by Leland (1994) is also relevant to these studies, although these studies were not designed to test that model.

Birnbaum (1999; 2004b; 2008b) reviewed a number of critical tests that refute any rank dependent utility (RDU) model (Quiggin, 1993) including rank and sign-dependent utility (Luce & Fishburn, 1991; 1995; Luce, 2000), CPT, and Expected utility (EU). Birnbaum (2008a; 2008b) noted that many of the same tests that refute CPT also contradict the priority heuristic. For example, the priority heuristic predicted fewer than half of the modal choices analyzed by Birnbaum (1999), by Birnbaum (2004a), and by Birnbaum and Navarrete (1998).

Some of these choices included cases where 90% or more of the participants satisfied stochastic dominance but the priority heuristic predicts indifference. In other choices, significantly more than half of the participants (about 70% of undergraduates) violated stochastic dominance, but the priority heuristic predicts that people should satisfy it.

Brandstätter, Gigerenzer, and Hertwig (2008a) responded that properties of these choices may have induced people to use other heuristics drawn from a person’s “adaptive toolbox.” Presumably, decision makers first decide what rule to use, then they either apply that rule or choose to use another rule. The mechanism that decides what rule to use has not yet been specified; it is described instead with lists of “triggering conditions,” which are estimated from data like parameters. Brandstätter et al. (2008a) concluded that the priority heuristic does not apply when there is a stochastic dominance relation in the choice.

In addition, Brandstätter et al. (2008a) argued that certain choices reviewed by Birnbaum (2008a) used gambles that differed in expected value (EV). Brandstätter et al. (2008a) presented a figure to show that the priority heuristic is not accurate when expected values (EVs) differ, which led them to suppose that two strategies are at work, one for “easy” choices (that differ in EV) and one for “harder” choices where EVs are nearly equal. From the data, Brandstätter et al. (2008a) estimated that, when the ratio of EV exceeds 2, people act as if they choose the gamble with the higher EV. Brandstätter et al. consider EV ratio as a proxy for the “difficulty” of a choice, but do not necessarily hold that people actually compute ratios of EV. They argued that the priority heuristic is accurate for “difficult” choices in which EVs are nearly equal.

However, Birnbaum (2008c) noted that EV ratios in Birnbaum and Navarrete (1998) had been inside the region where PH is supposed to apply; in that study, the priority heuristic failed to reproduce even half of the modal choices correctly. Brandstätter et al. (2008a) replicated part of that study and their results confirmed that the priority heuristic reproduced fewer than half of the modal choices that they chose for replication (Birnbaum, 2008c). To account for the results, Brandstätter et al. noted that Birnbaum and Navarrete (1998) used many choices in which both gambles of a choice had the same probability distribution and in some choices two branches had the same probability. PH was not accurate for such choices, so Brandstätter, et al. (2008a) theorized that people use a “toting up” heuristic for choices in which two branches had the same probability. In some of the choices in Birnbaum and Navarrete (1998), there was a common probability-consequence branch in both choices, which was theorized to trigger editing rules and other heuristics that were called up to account for the failures of the priority heuristic.

The arguments of Brandstätter et al. (2008a) might also provide excuses for previous failures of CPT as well.

This paper devises a new type of test that avoids the exceptions stated above. In these tests, one alternative does not stochastically dominate the other, there are no common probability-consequence branches, probabilities of the consequences are not equal, and expected values are nearly equal. In addition, unlike previous tests, the new tests use shifts in expected value and expected utility to “help” predictions of PH and CPT. That is, expected value and expected utility are both manipulated such that, if a person shifts his or her judgments in the same direction as the changes in EU or EV, his or her choices will appear consistent with PH and CPT. However, the choices are designed so that the TAX model with parameters typical of previous research predicts that people will shift their choices in the opposite direction of EV, EU, CPT, and PH. To understand how the new test was devised, see Appendix A, which shows that the property tested can be deduced as a theorem from any rank dependent utility model, including CPT, and that the property also follows from the PH; and see Appendix B, which shows that the TAX model can systematically violate the property.

1.1 New test: Probability-outcome tradeoff and coalescing

The following choices illustrate the new test. In each case, the participant chooses the urn from which a ticket will be drawn at random from 100 otherwise identical tickets. The value printed on the ticket drawn determines the prize.

According to any RSDU, CPT, RDU, EU, or EV model, if a person prefers R over S, then that person should prefer R ^′ over S ^′, apart from random error. (In Appendix A, it is proved that, under any model in this class,

with . and all probabilities between 0 and 1.)

Three data patterns are compatible with the property, R ≻ S ⇒ R ^′ ≻ S ^′: SS ^′, SR ^′, and RR ^′; however, no one should show the pattern RS ^′ (except by error), according to any of these models. With parameters estimated by Tversky and Kahneman (1992), the CPT model predicts that people should prefer R over S and R ^′ over S ^′: the RR ^′ pattern. The certainty equivalents according to that model are CE(R) = 50.9, CE(S) = 42.7, CE(R ^′) = 51.2, and CE(S ^′) = 42.3. However, it is important to keep in mind that CPT with any functions and parameters for utility and weighting implies the general property that rules out RS ^′.

According to the priority heuristic (PH), a person first compares lowest consequences of a gamble and chooses the gamble with the higher lowest consequence if they differ by more than 10% of the largest consequence in either gamble, rounded to the nearest prominent number ($10 in this case). But the lowest consequences are equal in both choices ($7). When the lowest consequences are not sufficiently different, the person supposedly chooses the gamble with the smaller probability to get the lowest consequence, if these differ by 0.1 or more. A person should therefore choose S and R ^′ in these two choices because they have the lowest probabilities of getting the worst prize; that is, the PH implies the pattern SR ^′.

If the probabilities of the lowest consequences differed by less than 0.1, the person is theorized to next compare the highest prizes and choose by that criterion. When there are more than two branches and the first three comparisons yield no decision, the person next compares the probabilities to win the highest prize and decides on that basis alone, if there is any difference. And if all four criteria yield no choice, the person chooses randomly. When gambles have three or more branches, the PH assumes that people never examine intermediate branches.

To account for previous failures of PH, Brandstätter et al. (2008a) postulated that people might edit choices using the editing rules of prospect theory (Kahneman & Tversky, 1979). Suppose people first rounded off consequences in the example above that differ by less than $5 and combined these branches with approximately equal consequences, and then applied the priority heuristic. If so, then people would choose S over R and S ^′ over R ^′; i.e., the SS ^′ pattern.

One can easily construct other lexicographic semiorder models in which the difference threshold for probability is greater than 0.1 or where highest prizes are compared before probabilities. In either of these models, a person would choose R over S and R ^′ over S ^′ (RR ^′) because of the difference in the highest consequences of the two gambles. Because they argued against these alternatives, Brandstätter et al. (2006, 2008a) would not find these variations of their model to be attractive. Nevertheless, this study will investigate these possibilities as well.

Brandstätter et al. (2008b) cited the similarity heuristic of Rubinstein (1988) and Leland (1994) as another heuristic people might use. However, the similarity heuristic of Leland (1998) assumes that people first choose by EU, an assumption criticized by Brandstätter et al. (2006).

Brandstätter et al. (2008a; 2008b) modified the similarity model of Rubinstein (1988) and Leland (1994), and proposed that this modified similarity evaluation might precede the application of the priority heuristic. Besides excluding Leland’s use of EU as the first step, they apparently reject the idea in Rubinstein (1988) and Leland (1994) that similarity may involve a nonlinear transformation between objective payoffs and a subjective scale of similarity. Instead, they theorized that people skip Leland’s (1994) first step and apply the second and third steps using the same parameters as in the priority heuristic to the objective cash values and probabilities.

According to this modified similarity plus priority heuristic (described more precisely in Appendix C), people first check for transparent dominance and take the dominant alternative if there is one (Steps 1–10, Appendix C). Next, they compare consequences and probabilities to see if one gamble is favored by a cash difference of at least 10% of the maximal prize, rounded to nearest prominent number ($10 in this case) and the other gamble is not favored by a cash difference as large as $10 nor favored by a probability difference greater than or equal to 0.1. They also check to see if one gamble is favored by a probability difference greater than or equal to 0.1 and the other is not favored by such a difference or a difference in consequences as large as $10 (Steps 11–20, Appendix C). If this modified similarity heuristic determines no preference, it is theorized that people next use the priority heuristic (Steps 21–29, Appendix C).

In Choice 1, we see that the risky gamble has a higher best prize but the safe gamble has a higher probability to win the best prize. Because these conflicting differences exceed $10 and 0.1, this choice cannot be resolved by the similarity evaluation, so people should use the priority heuristic (last nine steps in Appendix C). In Choice 2, note that the risky gamble has a higher best prize but the safe gamble has the higher middle consequence (and both differences exceed $10), so the modified similarity heuristic also implies that people should apply the priority heuristic in Choice 2 as well. Thus, the modified similarity heuristic plus priority heuristic makes the same predictions as the priority heuristic in these choices because of the absence of a “no-conflict” solution.

Table 1 presents a summary of these predictions, PH implies SR ^′; editing (rounding and combining) + PH implies SS ^′; certain other lexicographic semiorders imply RR ^′ (such as the priority heuristic with the assumption that probability differences must exceed 0.2 to be decisive). In addition, if people used the same parameters to examine similarity before applying the PH, they should also follow the pattern, SR ^′. But no version of the PH has yet been proposed that predicts RS ^′; indeed, the PH assumes that people never examine middle branches of gambles with more than two branches. It is the consequences on the middle branches that would justify choosing R and S ^′ in this case.

Table 1: Predicted choice combinations of different models in the new test of probability-consequence tradeoff with branch splitting. PH = priority heuristic; EV = expected value; CPT = cumulative prospect theory; LS = lexicographic semiorder.

According to the TAX model with parameters taken from prior research, people should prefer R over S and S ^′ over R ^′; i.e., the pattern RS ^′. There are also reasonable parameters such that TAX could imply other patterns such as SS ^′ or RR ^′. TAX thus differs from the CPT model and the heuristic models above in that it can predict RS ^′ instead of SR ^′. Intuitively, the reason that TAX makes these predictions is that when a probability-consequence branch is split, the splinter branches have greater total weight than when they are combined. By splitting branches leading to higher (or lower) consequences, one can improve (or diminish) a gamble, respectively.

Although TAX and CPT have the same number of parameters, TAX can also handle the data pattern, SR ^′, because EU is a special case of TAX. A “control” test is therefore included such that TAX with the same parameters implies SR ^′. The model and parameters should be able to predict cases in which we should observe SR ^′ and predict cases in which we should find RS ^′ (Appendix B).

The similarity model of Leland (1998) begins with an evaluation of EU, followed by a check that tests for transparent dominance. Next, a person chooses one alternative if it is superior and dissimilar on one or more attributes and if the other alternative is not noticeably better on any attribute. If we theorize that $90 is superior and dissimilar to $8 but that $92 is similar to $66 (and assuming .8 is similar to .6 and .1 is similar to .3 in probability), people might choose R in the first choice; and, if $63 is dissimilar and better than $8 and if $92 is similar to $66, people might choose S ^′ in the second choice. This model makes similar predictions to TAX, except for the “control” choices.

1.2 Interactive independence

This study also includes new tests of a second property, called interactive independence proposed by Birnbaum (submitted) as a test of the family of lexicographic semiorders. In a lexicographic semiorder, any attribute that is the same in both gambles can be changed in both gambles without reversing preference. For example, consider Choices 3 and 4:

Assuming no interaction between probability and consequences,

According to PH, people should choose the “safe” gamble in both cases because the lowest consequence of the “safe” gambles always exceeds the lowest consequence of the “risky” gamble by $15, and this exceeds 10% of the highest consequence ($10). Thus, PH implies the data pattern, S ^′′ S ^′′′. Other heuristic models in which the difference threshold exceeds $15 or in which people examine the highest consequences first could imply the pattern, R ^′′ R ^′′′, but no lexicographic semiorder implies the interactive pattern, R ^′′ S ^′′′.

Previous tests of interactive independence employed cases in which the ratio of EVs exceeded 2 (Birnbaum, submitted; Birnbaum & Gutierrez, 2007). In the new tests presented here, the EV ratio is always less than 2, so the priority heuristic is supposed to apply, according to Brandstätter et al. (2006; 2008a).

More generally, interactive independence of probability and consequences holds that

where x ^′> x > y > y ^′≥ 0. According to TAX or CPT, people should violate interactive independence by showing interactions between probability and consequences. With parameters from previous research, both of these models imply the pattern, R ^′′ S ^′′′, as do many other models, including EU. Interactive independence should be satisfied by any lexicographic semiorder (Birnbaum, submitted) as well as the priority heuristic and the similarity models of Rubinstein (1988) and Leland (1994), apart from the step in which EU is compared.

According to the similarity model (Rubinstein, 1988; Leland, 1994), if there is no transparent dominance relation, as is the case in Choices 3 and 4, people next look for a contrast favoring one gamble that is great enough to be dissimilar. Let x ^′ ≻_$ x indicate that the cash value x ^′ is preferred to x and dissimilar, and let x ^′ ∼_$ x indicate that the difference in cash is not great enough to be dissimilar. Because the differences are the same in both choices, people either choose the “risky” gamble in both cases (assuming $100 ≻_$ $50 but $20 ∼_$ $5 in the example), or people should choose the “safe” gamble in both cases (if $100 ∼_$ $50 but $20 ≻_$ $5), or people should be indifferent in both choices (if $100 ≻_$ $50 and $20 ≻_$ $5 or if $100 ∼_$ $50 and $20 ∼_$ $5).

However, the Leland (1994) model includes EU theory, so it can violate interactive independence, given suitable parameters for the utility function and EU threshold. In addition, with free parameters for the similarity relations, the Leland (1994) model can imply the pattern RS ^′ for Choice Problems 1 and 2. For example, suppose that $66 ∼_$ $92 but $8 ≺_$ $90, 0.8 ∼_P 0.6, and 0.1 ∼_P 0.3, where ∼_P indicates that the probabilities are not different enough to be dissimilar. If so, people should prefer R over S in Choice 1. Similarly, with the assumptions above and $63 ≻_$ $8, people should choose S ^′ over R ^′ in Choice 2. Leland (personal communication, February 26, 2008) predicted this pattern of results correctly before seeing the results.

2 Method

The method was similar to Birnbaum (1999). Embedded among a series of decision tasks were 30 choices of this study. These choices were presented twice, separated by other tasks that required about 10 minutes intervening between repetitions. Each choice was displayed via computer as follows:

Participants clicked a button beside the gamble they would rather play in each choice. They were informed that 3 participants (about 1 per fifty) would play one of their chosen gambles for real cash, so they should choose carefully.

Main design. The main design included eight choices (four pairs of choices) constructed from the following recipe:

and

Levels of consequences () were ($95, $90, $55, $52, $12, $12, $11), ($100, $98, $52, $50, $12, $12, $10), ($100, $98, $50, $48, $4, $3, $2), and ($92, $90, $66, $63, $8, $8, $7) in the four tests, respectively. Levels of probabilities (p, q, r, s, r ^′, s ^′) were (0.65, 0.90, 0.05, 0.05, 0.30, 0.10), (0.20, 0.30, 0.10, 0.10, 0.10, 0.30), (0.10, 0.20, 0.05, 0.40, 0.30, 0.10), and (0.70, 0.80, 0.10, 0.10, 0.10, 0.20). These levels (x > x ⁻ > y > y ⁻ > z ^′ ≥ z ⁺ > z ≥ 0) ensure that R ^′ dominates R and S dominates S ^′ by first order stochastic dominance.

Control Choices: Two additional “control” choices were constructed from the same splitting manipulation as follows: S = ($50, 0.2; $12, 0.1; $10, 0.7) versus R = ($100, 0.05; $90, 0.05; $0, 0.9) and S ^′ = ($50, 0.15; $40, 0.05; $0, 0.8) versus R ^′ = ($100, 0.1; $12, 0.2; $10, 0.7). For these choices, however, TAX with parameters taken from previous research implies that people should choose S and R ^′, as do CPT, EU, EV, and PH.

Interactive independence design. The second design, testing interactive independence, included choices of the following type:

where , and p is manipulated between choices. All ten choices used (x ^′, y ^′) = ($100, $5). Five trials used (x, y) = ($50, $20), where p = 0.1, 0.3, 0.5, 0.7, and 0.9. Five others used (x, y) = ($70, $20), with the same 5 levels of probability.

In addition to 20 trials comprising the main designs, there were four warmup choices and six filler choices. These were presented in random order, restricted so that no two trials from the same design appear on successive trials. Complete materials are available from the URL: http://psych.fullerton.edu/mbirnbaum/SPR_07/choice_gambles_F_07_01v2.htm

Participants were 167 undergraduates who participated as one option toward an assignment in Introductory Psychology; 69% were female and 96% were 21 years of age or younger.

3 Results

Table 2 reports the number of people who showed each choice pattern for two replications of eight choices of the main design. From these data, the rates of “error” and the “true” probabilities of choosing the “risky” gamble can be estimated for each choice. The error rate for a choice is estimated from preference reversals when the same choice is repeated, using the model described in Appendix D, which allows each person to have a different true pattern of preferences and each item to have a different error rate.

Table 2: Replication data used to estimate true probability and error rates for each choice. Entries under SS, SR, RS, and RR are the observed numbers of people who showed each combination of choices on the two replications. For example, 120 people chose the risky gamble in both replicates of Choice 12 (first row of the table). The chi-squares in the right-most column evaluate the fit of the true and error model to these frequencies. All are acceptable fits.

^* Choices marked with asterisk had the risky gamble presented in the first position, with the safe gamble in the second position. Unmarked choices had the opposite arrangement. Choices 28 and 30 are the “control” choices in which all models agree.

The first row of Table 2 shows that 120 of the 167 participants chose the “risky” gamble on both presentations of the choice between S = ($66, 0.8; $8, 0.1; $7, 0.1) and R = ($92, 0.6; $90, 0.1; $7, 0.3), and only 15 chose S both times. According to PH, most people should have chosen S because it has a lower probability of yielding the smallest prize, and the difference in probability is 0.2, which exceeds 0.1. Instead, 90% of the participants are estimated to prefer R, and estimated “error” rate is 11% on this choice.

The second row of Table 2 shows that 80 people chose S ^′ = ($66, 0.7; $63, 0.1; $7, 0.2) over R ^′ = ($92, 0.7; $8, 0.2; $7, 0.1) on both presentations compared to 43 who chose R ^′ both times. The PH predicts that most people should prefer R ^′ because of the 0.1 lower probability to get the lowest prize. However, 66% of the participants are estimated to truly prefer S ^′, with an “error” rate of 16%.

Of the eight modal choices in the main design, PH is correct in only one case, Choice 14, where 52% are estimated to prefer the “risky” choice. PH correctly predicts the modal choices of the two “control” choices, as do TAX and CPT with their prior parameters.

The last column of Table 2 contains statistical tests of the true and error model; none is significant (α = 0.05), indicating that the model of error can be retained as descriptive.

Table 3 shows the results of the true and error model extended to choice combinations (Appendix D). This model assumes that each participant has one and only one of the four possible preference patterns for each test of the main design. Each participant exhibited one of sixteen possible observed data patterns (one of four possible patterns in each replication). Parameters were estimated by fitting the frequencies of these sixteen observed data frequencies so as to minimize, G = 2 ∑_i=1¹⁶ O_i · ln(O_i /E_i), where O_i and E_i are respectively the obtained and predicted frequencies of the sixteen possible data patterns.

Table 3: Estimated True probabilities of each response pattern in the new tests of probability-outcome tradeoff with coalescing, monotonicity, and transitivity. The true and error model is tested by χ²(10); all four show acceptable fits. The CPT and PH models imply that no one should show the RS ^′ pattern of reversal, except by error. The χ²(1) statistics in the right-most column test this hypothesis (that p _{RS ^′} = 0); all are large and significant, indicating systematic evidence against CPT and PH.

^* Choices marked with asterisk have the risky gamble presented first, with the safe gamble in the second position. Entries in bold show estimates of p _{RS ^′} these should be zero according to PH and CPT.

According to CPT or PH, no one should show the choice pattern RS ^′, except by “error”; i.e., p _{RS ^′} = 0. According to TAX, however, people should indeed show this pattern. For the first test in Table 3, the observed frequencies of the 16 data patterns, SS ^′SS ^′, SS ^′SR ^′, … , RR ^′RR ^′ are 4, 3, 11, 2, 2, 6, 2, 1, 9, 4, 56, 11, 1, 2, 19, and 34. The modal pattern was RS ^′RS ^′, shown by 56 of the 167 participants. The first row in Table 3 displays the estimate of p _{RS ^′} = 0.62; that is, most people (62%) are estimated to violate the predictions of CPT and PH on this test.

In all four tests in the main design, the estimated probability of RS ^′ (bold font), which is inconsistent with any RDU or CPT or PH model is higher than that of the SR ^′ pattern, which is predicted by PH and compatible with RDU and CPT/RSDU. For example, in the first test, 62% are estimated to have RS ^′ as their true data pattern compared to 5% with the opposite reversal. A fair proportion of cases (29% in the first row) are consistent with the pattern RR ^′, which is predicted by CPT with its prior parameters and which is also compatible with TAX (with other parameters) and with certain lexicographic semiorders, such as the one in which people compare highest consequences first.

Suppose people edited choices by rounding off consequences differing by less than $5 and combined branches leading to rounded consequences by adding their probabilities. If a person were to edit first and then apply PH, she or he would show the SS ^′ pattern. However, Table 3 shows that only 4% were estimated to show this pattern in the first row, and in no case did the sum of SR ^′ and SS ^′ reach a majority in any row.

These analyses were conducted for each individual separately by adding the four tests of the main design (with two repetitions each) for each person. It was found that 117 of the 167 people (70%) showed more reversals of the type RS ^′ than of SR ^′; 32 (19%) had more of SR ^′, and the rest (11%) split evenly or showed no reversals. Thus, significantly more than half of individuals show the pattern predicted by TAX and which contradicts both the PH and CPT.

The last column in Table 3 presents statistical tests of the hypothesis that the true probability of the RS ^′ pattern is 0, as implied by both PH and CPT. In all four cases, there is significant, systematic evidence violating those models. The magnitude of the violations appears to be larger in the first two tests in Table 3, where probability to win the highest consequence in S is high (0.8 and 0.9) and relatively smaller in the second two tests in Table 3.

With “control” choices, in contrast, the estimate is that 83% truly prefer S = ($50, 0.2; $12, 0.1; $10, 0.7) over R = ($100, 0.05; $90, 0.05; $0, 0.9), with an error rate of 14%, whereas only 5% truly prefer S ^′ = ($50, 0.15; $40, 0.05; $0, 0.8) over R ^′ = ($100, 0.1; $12, 0.2; $10, 0.7), with an error rate of 7%. These are consistent with PH, CPT, and TAX. This choice apparently creates difficulty for the Leland (1994) model because people should choose R over S because $90 is presumed superior and dissimilar to $12. (See Appendix E).

Table 4 presents the results of choices testing interactive independence, with the information arranged as in Table 2. For example, in the first row, it is shown that 99 people chose the “risky” gamble in both replications of Choice 7, and that 33 chose the “safe” gamble in both replications of this choice. The estimated proportion of people who truly prefer S = ($50, 0.9; $20, 0.1) over R = ($100, 0.9; $5, 0.1) was estimated by the true and error model to be 0.24. According to the priority heuristic, most people should have chosen the safe gamble, S, in every row in Table 4 because S always has a lowest prize of $20, which exceeds the lowest prize of R ($5) by more than $10. Instead, most people chose R in Choice 7 (first row). Similar results are observed in Choice 25 in the same table.

Table 4: Replication data used to estimate true probability and error rate for each choice in the tests of interactive independence. Entries under RR, RS, SR, and SS show observed frequencies of each combination of choices on the two replications. According to EV + PH, the estimated choice percentages should be the same in all rows. According to either TAX or CPT, the probabilities of choosing the “safe” gamble should increase within each series, showing evidence of interaction between probability and prizes. The chi-squares in the right-most column evaluate the fit of the true and error model; only the first is significant.

The probability to win the higher prize (which is the same in both gambles of each choice in Table 4) changes from 0.9 to 0.1 in each set of five successive choices in the table. According to any lexicographic semiorder, probability should have no effect in Table 4, because probability is the same in both gambles of each choice. Instead, estimated choice probabilities range from 0.24 to 0.87 in the first series (first five rows of Table 4) and from 0.29 to 0.84 in the second series (last five rows), contrary to this prediction. Tests of the true and error model are presented in the last column. Only the first one is significant, making one significant test out of the twenty tests in Tables 2 and 4. When the same type of error model is fit to these data as in Table 3, it is estimated that 63% of participants truly switched from R in Choice 7 to S in Choice 9, contrary to the family of lexicographic semiorders, as well as the modified similarity heuristic plus priority heuristic.

Because EV ratios never exceed 2, these data refute the “as if EV” + PH model analyzed by Brandstätter et al. (2006, 2008a). Instead, the data show an interaction between probability and consequences: when probability to win the better consequence is high, people tend to choose the risky gamble with the higher best consequence. When probability to win is lower, people tend to choose the safe gamble with the better lowest consequence.

4 Discussion

Neither PH nor CPT provides an accurate description of the data in the main design (Tables 2 and 3). The new test does not use common branches, equal probabilities, large ratios of expected value, or changes in EU that oppose PH and CPT. Therefore, no model using EV, EU or CPT can account for the shifts in preference because the reversals of preference observed in Table 3 are opposite the changes in EV, EU, and CPT. The results are compatible with the TAX model in which splitting of the branch leading to the highest consequence improves the gamble and splitting of the lower branch lowers the evaluation of the gamble.

Tests of interactive independence (Table 4) show evidence of interaction that refutes PH and all lexicographic semiorder models. Evidence of interaction persists in these new tests despite the fact that EV ratios are less than 2 in all cases. Both CPT and TAX can account for this interaction, as do many other models, including EU.

Among the models considered here, PH is least accurate because it fails to describe results in Tables 2, 3, or 4. CPT correctly predicts the pattern in Table 4, but it fails to predict the results in Tables 2 and 3. The similarity model of Leland (1994, 1998) is more accurate than CPT because it can account for the data in the main design, and by means of its EU feature it could potentially handle the results in Table 4. TAX is most accurate because it correctly predicts the patterns in all three tables. In addition, the TAX model with previous parameters correctly predicted the “control” choices, where quantitatively stronger manipulations reversed the pattern observed in Tables 2 and 3.

Marley and Luce (2005) have shown that a family of gains decomposition utility (GDU) models has properties similar to those of TAX. Models in this family that violate coalescing remain consistent with results of Tables 2 and 3, and GDU can also account for Table 4. See Birnbaum (2007) for tests of certain of those models.

After seeing these results, a reviewer proposed a “striking difference” heuristic for Choices 12 and 20 of Table 3. This new heuristic introduces a new parameter defining a “striking difference,” but as shown in Appendix E, no single value of the new parameter reconciles all of the data in Tables 2, 3, and 4, nor does it account for the “control” choices.

This study was not designed to test the similarity model of Leland (1998); however, that model remains compatible with these data, if we allow different parameters for different participants and allow EU to handle the results of Table 4. Unlike the striking difference model, the Leland model allows a nonlinear transformation to utility, so that similarity need not be a function of objective differences in consequences.

The simplest interpretation of these results combined with previous evidence is that neither CPT nor PH, with or without their editing rules of rounding and combination, is an accurate model of risky decision making. Nor is the PH plus the modified similarity heuristic compatible with these data. It seems possible that someone might construct yet another excuse consisting of a new “heuristic” that is “triggered” by the conditions of this experiment in order to explain why those theories failed to predict these results. But at this point, given the cumulative evidence against these models, the number of post hoc excuses seems to have grown too large, in my opinion, to retain those models as plausible descriptions.

These results do not disprove the “adaptive toolbox” approach because that approach cannot in principle be refuted. The “adaptive toolbox” approach assumes that people have different mental tools for handling different kinds of problems. The idea that people can evaluate a combination of stimuli by more than one operation is plausible and consistent with evidence (e.g., Birnbaum, 1982; Mellers, Ordóñez, & Birnbaum, 1992); the theory that people have only one way to compare or combine stimuli is not at issue. Instead, the issue appears to be that proponents of the “adaptive toolbox” view have restricted what is permitted to go in the toolbox. They have argued that people lack tools that have simple mathematical descriptions and allowed that the only tools available to humans and animals are those that have simple verbal descriptions. So far, the heuristic models have been successful in describing data that were already available previous to the construction of the heuristics, but the constructed models have not yet had success in predicting the results of a new experiment designed to test their implications.

For example, lexicographic semiorder models imply that once a decision is reached for any reason, no amount of difference in other attributes should have any effect. This property, termed priority dominance (Birnbaum, submitted), has not had success in handling real data. The lexicographic models also imply that small changes in two attributes, each of which is too small to reverse a decision, cannot combine to reverse a decision reached for another reason. This property, termed integrative independence, has also been disproved (Birnbaum & LaCroix, 2008). As shown in Birnbaum (submitted), Birnbaum and Gutierrez (2008), and in this study (Table 4), data violate the implication of interactive independence, which is implied by the priority heuristic, lexicographic semiorders, and the similarity evaluation model (apart from the use of EU as in Leland, 1994; 1998). Furthermore, the priority heuristic, lexicographic semiorders, and the similarity model can predict violations of transitivity, but such violations have been rare when the data are analyzed by the true and error model (Birnbaum & Gutierrez, 2007).