Additional deliberation reduces pessimism: evidence from the double-response method

Abstract

We conduct a laboratory experiment to investigate the impact of deliberation time on behavior under risk and uncertainty. Towards this end we ask our participant to make quick, intuitive evaluations of a number of lotteries and report resulting certainty equivalents. Yet, we invite them to modify these initial decisions, whenever they find, after (additional) deliberation, that they do not precisely represent their preference. Both certainty equivalents are incentivized (a double-response method). The choice of evaluated lotteries allows us to semi-parametrically estimate the value function and the probability weighting function within the paradigm of the cumulative prospect theory. The main finding is that deliberation raises the probability weighting function (reduces pessimism), especially in the case of lotteries involving unknown probabilities.

1 Introduction and brief review of related experiments

A typical choice involves uncertain consequences and must be made very quickly. This may be particularly true in the dynamic landscape of the digital era, with new bits of information arriving continuously and requiring swift yet efficient processing. It appears clear, therefore, that studying impact of time pressure (TP) on decision making under risk and uncertainty is of utmost importance.

The dual-system approach (Stanovich and West 2000, Kahneman 2011) has been proposed as a theoretical framework that may help us understand the role of time pressure in decision making. Roughly, it proposes that two separate modes of decision making exist. System 1 is crude, intuitive, emotional, unconscious, and old from evolutionary viewpoint (‘the reptilian brain’). System 2 is more precise, deliberative, conscious and only evolved later (‘the mammalian brain’). Critically, System 2 tends to be much slower, meaning that its role is severely limited under strict time pressure. In particular, that means that scarcity of deliberation time results in filtration, i.e., only the most salient aspects of the situation are taken into account (Maule et al. 2000). In the context of decision making under risk, this means that greater deviations from the normatively correct model of expected utility maximization are hypothesized, and indeed often observed, under time pressure (Hogarth 1980; Kruglanski and Freund 1983). In particular, losses (as opposed to gains) have been proposed to be salient, and thus overweighed under time pressure (Ben-Zur and Breznitz 1981; Huber and Kunz 2007).

It is also plausible that outcomes are taken into account, but sensitivity to probability drops under time pressure (Dror et al. 1999; Young, et al. 2012, Experiment 3; Nursimulu and Bossaerts 2014 even observed objective probabilities being used under the strictest time limit and oversensitivity to probability when more time was available). Young et al. (2012) also found (in their Experiments 1 and 2) that the probability weighting function for gains was more elevated under time limit, which corresponds to greater risk attractiveness. Greater risk acceptance under time pressure was also reported by Madan et al. (2015) in a study involving decisions from experience rather than description. In contrast, Kocher et al. (2013) found that their participants continued to avoid risks in positive prospects but switched from risk seeking to risk aversion for negative prospects when time limits were introduced. Yet another pattern of findings, arising from an unusually large (n > 1700) international project was recently reported by Kirchler et al. (2017). These authors looked at several binary choices between sure amounts and 50/50 gambles, estimating measurement noise separately from risk preference. They observed that, compared to time delay, time pressure strengthened risk aversion in the domain of gains and risk seeking in the domain of losses. Interestingly, Saqib and Chan (2015) reported precisely opposite (and very strong) results, also for 50/50 gambles, with time pressure reversing standard risk aversion in gains and risk seeking in losses. These authors run hypothetical experiments using the online mTurk platform.

Because our study involves both risky and uncertain outcomes, it is also related to ambiguity aversion literature. The relevant papers typically looked at intuitive/affective vs. deliberative mode of decision making as a trait rather than result of an exogenous manipulation. Rubinstein (2013) found no correlation between (unconstrained) response time and choices in Ellsberg Paradox. Butler et al. (2014) found using a representative survey and large-scale behavioral experiments that individuals reporting being prone to use more intuitive (rather than also deliberative) reasoning style are less often averse to ambiguity (and also to well-defined risk), but see Bergheim and Roos (2013). Recently, Baillon et al. (2018) elicited winning probabilities that were considered as good as winning contingent on natural events (stock exchange index change being within a specified range), with and without time pressure. They found that time pressure does not affect ambiguity aversion but appears to increase ambiguity generated-insensitivity, a tendency analogous to inverse-S probability weighting.

To sum up, the related studies do not show a very clear behavioral pattern and further research is certainly needed. A more comprehensive review can be found i.a. in Ordóñez and Benson (2015).

In the current study we are trying to establish how time pressure, resulting in restricted deliberation, affects risk posture, particularly probability weighting. We use a well-established method of Abdellaoui et al. (2011) to semi-parametrically elicit entire probability weighting function and value function. Therefore, we obtain a comprehensive picture of participants’ preference under risk. We apply the double-response method of Krawczyk and Sylwestrzak (2018), which involves observing incentivized responses both after short and after longer deliberation in a given situation from the same participant. In other words, the decision is made both under strict time pressure and under very weak time pressure. This allows a detailed insight into how time pressure causally affects contents of decisions under risk in specific individuals. In contrast, between-subject studies only allow comparing aggregate distributions and have to deal with the severe selection due to subjects failing to respond at all (Tinghög et al. 2013), an effect recently showed to be important in the context of decision making under risk by Kocher et al. (2019) Still, to understand the impact of participants’ willingness to behave consistently under long vs. short deliberation time and similar effects we also conduct control sessions with no time pressure. Our main finding is that, particularly in the case of ambiguity, having time for deliberation reduces the initial pessimism, bringing participants closer to using objective probabilities. As a result, risk aversion is reduced, consistent with the findings reported (for the gain domain) by Kirchler et al. (2017).

In contrast, our results are opposite to those of experiments 1 and 2 in (Young et al. 2012), who also asked for certainty equivalents of random lotteries and found them to be higher under time pressure. Among the differences between our designs one stands out as a plausible candidate for this discrepancy. Young et al. asked their subjects “What is the smallest amount of money you would be willing to accept rather than the bet?” (while our question was “How much is this lottery worth to you?”). It is well known that exogenously imposing time pressure and verbally emphasizing speed rather than accuracy reduce subjects’ confidence that they are making the correct choices or judgments (Vickers et al. 1985). It seems natural that when the confidence is low, subjects choose to report a fairly high number responding to the Young et al.’s question: the subject may be inclined to type in the smallest amount she is fairly confident she likes better than the bet (which is to say, a relatively high number, because her confidence is limited).

2 Methods

2.1 Materials

The experiment consisted of the main decision task and a short questionnaire. It was computerized using PhP, with printed instructions (see online appendix at http://coin.wne.uw.edu.pl/mkrawczyk/gawryluk_krawczyk_appendix.pdf).

2.1.1 Decision task

The design was based on that of Abdellaoui et al. (2011). In each round, the participants were asked to evaluate lotteries involving drawing from virtual, Ellsberg-like urns. Two types of urns were used: the known and the unknown. The known urn always contained one ball of each of eight colors. In the unknown urn there were also eight colored balls, but participants did not know the particular composition. For example, there could be three blue balls, zero green balls, etc. In each case each ball had the same probability to be drawn. The same was true of each color in the known urns only.

At the beginning of each round a clock would start counting down from 60 s. Participants saw the graphical representation of the urn (known or unknown) and the information how much money they could win when particular color was drawn, see Fig. 1. They were asked to type in the amount they considered just as good as the lottery. They did so twice in each round. Upon confirming their initial (and typically rapid) choice, participants saw their selected amount displayed below the picture and were invited to rethink it and amend it or type in the same one again if they were sure it correctly represented their preference. Thus, each participant was allowed to change his or her mind at most once per round. Once the second amount (the same as the first one or a different one) was typed in and confirmed or when the time was gone, the participant could move on to the second round.

Fig. 1 Decision screen

At the end of the experiment the computer randomly selected: one round, one second of this round (1–60) and an amount of money X that could be obtained for sure, drawn from the range of possible payoffs in this round (e.g., from 40 to 100 in Fig. 1) to determine participants’ payoffs. If the participant in the selected second of the round had been indicating that her certainty equivalent of the lottery was lower than X, she received X. If she indicated it was higher than X, the lottery was played out and she received one of the possible outcomes (a Becker-DeGroot-Marschak procedure). In the case a second was selected in which no decision had been made yet, the lottery or X would be assigned randomly. Thereby participants were motivated to make their first decision as quick as possible and then to indicate if they changed their mind after consideration.

For example, assume that the computer drew round 1, in which the participant could win 100 PLN with probability 4/8, or 40 PLN otherwise, as shown in Fig. 1. Furthermore, the 25th second of this round was selected and sure amount X equal to 70. Suppose that in the fifth second of Round 1 the participant had evaluated the lottery at 64 and then she amended it to 68 in the 21st second. Because 68 is smaller than X = 70, in the 25th second of the round she was revealing that she liked X better than the lottery. She would thus receive 70 for sure. Now suppose she had typed in 72 in the third second and only changed it to 65 in the 30th second. Then, in the 25th second she was indicating that she liked the lottery better, so the lottery would be played out. In such a case, she would win 100 PLN with probability 4/8 and 40 PLN otherwise.

This double-response method has previously been applied by Krawczyk and Sylwestrzak (2018) to binary choices aimed at eliciting social preference. A closely related design was also pursued by Agranov et al. (2015) investigating a guessing game and Kessler et al. (2017), who looked at modified binary dictator games and prisoner’s dilemmas. Beyond the domain studied, there are two main differences. First, in these studies, participants could change their opinion many times, not just once. In the study by Agranov et al. (2015) this in some cases led to seemingly nonsensically large number of changes; further, Kessler et al. (2017) ended up reporting the initial and the final choices only. This suggests that allowing just dual response, naturally corresponding to the dual-system theories, may be a reasonable design choice. Second, compared to these studies, we only mildly punished indecision: as mentioned before, one of the options was implemented randomly, whereas in the case of Agranov et al. (2015) and Kessler et al. (2017), participants earned zero if a second was randomly selected in which there was no decision yet. This design choice is related to the first difference: the possibility that some participants make an instant; essentially random initial choice out of fear of receiving nothing has more bearing on the data if only one decision change is allowed.

On top of the double-response treatment described above, a no time pressure treatment was additionally run, in which participants took as much time as they wanted to make a decision. Only the final decision mattered (and was incentivized using an analogous Becker-DeGroot-Marschak procedure). This treatment obviously provided less useful evidence on the effect of deliberation and was, therefore, run with a smaller group of participants only. Its main purpose was to verify if final decisions of the double-response treatment are similar in terms of timing and contents to decisions taken under no time pressure at all.

At the end of the experiment participants received information on the round, second of the round and amount of money X that would determine their payoff. Then they completed Frederick’s (2005) cognitive reflection test (CRT) and a short post-experimental questionnaire (sex, age and field and year of study). These are very briefly discussed in the online appendix, which can also be consulted for the detailed description of the procedure.

2.1.2 Decision task parameters; estimation of probability weights; hypotheses

In total, participants made their decisions in 32 paid rounds, preceded by two practice rounds, with possible payoffs ranging from 0 PLN to 100 PLN (ca. 24 euro), see table C1 in the online appendix for the parameters of all lotteries. Some of the rounds were identical up to the coloring of the winning balls, allowing assessment of consistency in participants’ choices. Half of the participants played the 13 rounds with known urns first and then 19 rounds with unknown urns (we will refer to this condition as “Known First”), whereas the order was reversed for the other half (“Unknown First”).

It can be noted that some of the tasks involved 50/50 chances to get different non-zero rewards. Following the semi-parametric method of Abdellaoui et al. (2011), the certainty equivalents from these rounds were used to estimate the parameter of power value function, with the weight of 0.5 as an additional parameter. Non-linear least squares method was applied. Resulting values of each possible reward, together with certainty equivalents provided in remaining rounds (involving probabilities other than 50/50 for 100 PLN and zero otherwise) were subsequently used to calculate weights for these probabilities. Indeed, if for some $j\in \left(1,2,3,5,6,7,8\right)$ we denote the amount reported to be as good as 100 PLN with probability j/8 by CE (standing for the certainty equivalent), we conclude that w(j/8) = (CE/100)^ρ, where ρ is the individual parameter of the power value function. The seven probability weights can then be used to estimate parameters of Prelec’s (1998) two-parameter probability weighting function,

$w\left(p\right)=exp\left(-\beta \left(-ln\left(p\right){)}^{\alpha }\right)\right).$

Objective probabilities, w(p) = p results from $\alpha =\beta =1$ . Lower values of α (often reported in previous studies) correspond to low sensitivity to changes in probability away from the reference points of absolute impossibility and absolute certainty (inverse-S). Higher values of α would signify lower sensitivity near these thresholds. Low values of β characterize elevated probability weighting curves (optimism) while high β means pessimism. This procedure was applied separately to each participant’s choices under specific conditions (initial choices for known urns, final choices for known urns, initial choices for unknown urns, final choices for unknown urns). As a result we have four separate sets of estimates of r, w(p)’s (for each $j\in \left(1,\dots ,8\right)$ ), α, and β for each participant making decisions under double-response and two for each participant making decisions under no time pressure.

Analogously, we consider an alternative specification, under which the probability weights are subjected to simple linear regression on the unit interval, $w\left(p\right)=c+sp$ . Following Abdellaoui et al., we focus on $a=1-s$ as an index of likelihood insensitivity and $b=1-s-2c$ (which is the difference between the “dual” intercept arising when we flip the picture 180° and the standard intercept) as an index of pessimism.

In view of existing literature, we hypothesize that subjects will initially focus predominantly on the (most salient) lowest possible outcome (the lower outcome of the lottery), especially when probabilities are not explicitly given, only considering other variables later. In accordance with the attentional drift–diffusion model (which has been shown to correctly explain choices and decision times in other domains, see Krajbich et al. 2012) subjects will tend to accumulate evidence in favor of the option currently being the focus of their attention. In the context of our experiment this means that the initial choices will be more risk-avoiding (high b and β) and show less sensitivity to the probability (low a/α) than final choices, especially in the case of unknown urns.

2.2 Participants

In total, 184 volunteers took part in our experiment, see Table 1 for the distribution by treatment. As is typical for such experiments, some made decisions in the main task that were very difficult to justify. Applying a relatively mild criterion,¹ we excluded 33 participants, leaving 151 for further analysis. Of these, about 60% were male, 32% studied economics, 41% studied other fields and 27% were non-students. Mean age was 28.62 (SD = 11.953). The distribution of these variables in the entire pre-exclusion sample of 184 was similar, with a slightly higher fraction of females and non-economists. All the participants had been recruited using ORSEE (Greiner, 2015). None of them had participated before in a similar study. The experiment was conducted at the University of Warsaw Laboratory of Experimental Economics and lasted up to about 45 min. Earnings ranged between 5 PLN and 105 PLN with a mean of 56 PLN including a guaranteed 5 PLN show-up fee. These are relatively generous payoffs, much beyond what a simple student job would pay.

Table 1 Number of participants in each treatment

Treatment	Number of participants
Double response–known first	56
Double response–unknown first	57
No time pressure–known first	20
No time pressure–unknown first	18

3 Results

3.1 Manipulation check: response times

The logic of the experimental design was that participants in the double-response condition make their first (initial) decision quickly and then they have enough time to change their mind after consideration. Indeed, median times were about 6 s for the initial decisions and 10–13 s, depending on the treatment for the final decisions. The latter were not significantly different (using Mann–Whitney U tests) from those under NTP, neither for the known urns ( $Z=-0.266,p=0.790$ ), nor the unknown urns ( $Z=-1.497,p=0.135$ ).

3.2 Decision changes

Overall, participants changed 23% of their initial decisions: 25% decisions made for known urns and 21% for unknown urns. Most of the participants changed their mind once or a few times over the 32 rounds. Participants who more often changed their first decision needed more time between the initial and the final decision than participants who tended to repeat their first choice ( $\rho =0.374,p<0.001$ ). Very large changes were rare. On average, the absolute value of the difference between the final and the initial valuation was 1.33 PLN or nearly 5.78 PLN for non-zero changes. This corresponds to about 10% of the expected value of a typical lottery.

3.3 Probability weights

Table 2 shows summary statistics as well as p values of tests for differences in probability weights by treatment (see table C1 in the online appendix for certainty equivalents of all lotteries). The following observations can be made. First, there is strong heterogeneity in the data. Second, central tendency diverges substantially from objective probabilities in all the treatments. Specifically, low probability of 0.125 tends to be overweighted and $p\text{s}\ge 0.5$ are underweighted. Third, the double-response procedure does not seem to radically distort final responses: those made under DR are not different from those under no time pressure.² Fourth, weights for unknown urns are generally smaller than for known urns, the difference being most pronounced for larger probabilities. All of these largely replicate the findings of Abdellaoui et al. (2008). Crucially, deliberation time under DR also matters, as final choices are systematically different from initial choices. Specifically, for unknown urns, final weights are generally higher (greater optimism, closer to objective probabilities). In contrast, for known urns, the effect is less pronounced and only shows up for higher (underweighted) probabilities.

Table 2 Probability weights by treatment

p	Urns	Treatment	Median	Mean	interquartile range	t test w(p) = p	Wilcoxon test: Initial = Final	Mann–Whitney: DR-Final = NTP
0.125	K	DR: Initial	0.130	0.254	0.333	0.000	0.541	0.714
		DR: Final	0.126	0.255	0.355	0.000
		NTP	0.133	0.233	0.314	0.011
	U	DR: Initial	0.105	0.186	0.229	0.004	0.147	0.611
		DR: Final	0.120	0.197	0.256	0.001
		NTP	0.155	0.202	0.272	0.029
0.250	K	DR: Initial	0.228	0.281	0.300	0.203	0.377	0.927
		DR: Final	0.250	0.292	0.366	0.090
		NTP	0.217	0.279	0.307	0.439
	U	DR: Initial	0.222	0.248	0.292	0.923	0.041	0.690
		DR: Final	0.237	0.274	0.302	0.277
		NTP	0.197	0.238	0.269	0.671
0.375	K	DR: Initial	0.344	0.341	0.326	0.128	0.414	0.928
		DR: Final	0.352	0.352	0.264	0.303
		NTP	0.328	0.360	0.313	0.662
	U	DR: Initial	0.300	0.347	0.390	0.260	0.008	0.660
		DR: Final	0.335	0.371	0.375	0.882
		NTP	0.280	0.330	0.320	0.181
0.500	K	DR: Initial	0.419	0.410	0.282	0.000	0.116	0.864
		DR: Final	0.473	0.426	0.315	0.001
		NTP	0.435	0.425	0.329	0.036
	U	DR: Initial	0.339	0.361	0.286	0.000	0.014	0.986
		DR: Final	0.381	0.385	0.271	0.000
		NTP	0.418	0.374	0.271	0.000
0.625	K	DR: Initial	0.582	0.533	0.408	0.001	0.037	0.670
		DR: Final	0.600	0.564	0.324	0.016
		NTP	0.599	0.541	0.318	0.031
	U	DR: Initial	0.500	0.487	0.332	0.000	0.104	0.580
		DR: Final	0.511	0.505	0.381	0.000
		NTP	0.483	0.473	0.299	0.000
0.750	K	DR: Initial	0.696	0.634	0.357	0.000	0.020	0.775
		DR: Final	0.739	0.670	0.322	0.001
		NTP	0.723	0.671	0.263	0.034
	U	DR: Initial	0.622	0.583	0.380	0.000	0.083	0.414
		DR: Final	0.661	0.606	0.355	0.000
		NTP	0.604	0.571	0.360	0.000
0.875	K	DR: Initial	0.855	0.796	0.251	0.000	0.051	0.240
		DR: Final	0.880	0.815	0.231	0.002
		NTP	0.817	0.768	0.178	0.010
	U	DR: Initial	0.800	0.707	0.307	0.000	0.045	0.528
		DR: Final	0.824	0.728	0.327	0.000
		NTP	0.756	0.698	0.449	0.000

3.4 Estimated probability weighting functions and value functions

The fact that decisions on unknown urns were more optimistic than initial decisions is also reflected in estimated parameters of probability weighting functions; see Table 3 for the Prelec’s α–β parameterization (the simple intercept-slope parameterization leads to the same conclusion).

Table 3 Estimated parameters for the Prelec (1998) probability weighting functions and the value function

Urn	Treatment	Median			Wilcoxon test: Initial = Final			Mann–Whitney: DR-Final = NTP
		Ρ	α	β	ρ	α	β	ρ	α	β
K	DR: Initial	1.142	0.916	1.064	0.515	0.336	0.118	0.607	0.079	0.847
	DR: Final	1.082	0.948	1.025
	NTP	1.094	0.803	1.054
U	DR: Initial	1.158	0.826	1.258	0.075	0.581	0.002	0.592	0.222	0.751
	DR: Final	1.142	0.806	1.210
	NTP	1.199	0.653	1.148

The median estimated probability weighting functions by treatment are represented graphically in Figs. 2 and 3.

Fig. 2 Median individual probability weighting functions: known urns

Fig. 3 Median individual probability weighting functions: unknown urns

As can be seen, in the case of known urns, median participant is close to using objective probabilities, while she is markedly pessimistic in the case of unknown urns. In both cases, differences between final, and initial decisions are subtle, generally involving less pessimism in the latter case.

4 Conclusion

In this paper we used a novel method of identifying within-subject changes in decisions under risk and uncertainty after (additional) deliberation. While data shows substantial heterogeneity (as is typical in similar tasks), the following general patterns can be clearly observed. First, most participants are quite consistent in their choices in that they do not make many (large) changes. Second, however, a non-trivial minority of choices do get updated. Third, most of these changes involve reporting a higher certainty equivalent of the random lottery (more risk acceptance) after than before deliberation. Fourth, the pattern is stronger in the case of “unknown” urns (involving ambiguous chances of success). Fifth, the pattern does not seem to apply to a specific probability range or a specific subset of participants only. Instead, the “pessimism” coefficient β (or b) revealed in the final choices tends to be generally lower than the one shown immediately. Importantly, because initial choices under ambiguity tend to be very pessimistic, deliberation usually pushes them towards rationality.

One possible interpretation of the general tendency to come up with a more positive evaluation of a gamble over time observed in this experiment could be based on the fact that it reduces the relative role of emotions (which tend to be very quick). Volumes of research generally indicate that negative emotions tend to be stronger and more prominent than positive emotions (see Baumeister et al. 2001 for a review). In general, emotions’ valence does not suffice to determine its impact on the propensity to take risk (Lerner and Keltner 2000). For example, fear and anger (both negative emotions) tend to generate opposing action tendencies in a risky situation. Note, however, that our design required subjects to report the certainty equivalent for a risky lottery, making the latter the natural focus of thoughts and emotions. The valence of these emotions was plausibly sufficient to determine their impact on the WTP. The presumably dominant negative emotional reaction towards the lottery, no matter if mostly fearful or angry, could by one reason why initial evaluations were relatively low.

Our findings may have important economic consequences. For example, consumers should be advised not to take insurance decisions hastily and allowed by law to revise their initial decision after deliberation, i.e., nullify the contract at zero or low cost. That is because excessive focus on the emotionally salient loss event (of unknown probability) is likely to push them towards signing the contract even at clearly unattractive terms.