When less is more in the recognition heuristic

Abstract

The “less is more effect” (LIME) occurs when a recognition-dependent agent has a greater probability of choosing the better item than a more knowledgeable agent who recognizes more items. Goldstein and Gigerenzer (2002) define α as the probability that a correct choice is made on the basis of recognition alone and β the probability that a correct choice is made when both items are recognized (via additional cues). They claim that a LIME occurs if α > β (α > 1/2) and α and β remain constant as the number of recognized items, n, varies. In fact, it can be shown that neither of these parameters generally remains constant as n varies, and neither of them are simple functions of n. Therefore, a new theoretical basis for the LIME is needed. This paper provides mathematical results for understanding when the LIME can occur and elucidates implications of these results. The major findings presented here are as follows:

• Demonstrations that the LIME can occur when α ≤ β and fail to occur when α > β, and derivation of the conditions for these co-occurrences;

• A new characterization of the conditions under which the LIME occurs;

• Generalizations of this characterization to handle imperfect recognition; and

• Characterization of when the LIME occurs as more items become recognized.

The primary implication of these results is that the advantage of the recognition cue depends not only on cue validities, but also on the order in which items are learned. This realization, in turn, suggests that research in this area should incorporate a more dynamic focus on learning and memory processes, and the effects of reputational information.

1 Introduction

In choosing between two items, an agent who recognizes one item but not the other may use this recognition cue to make the choice, whereas one who recognizes both items must use other cues and one who recognizes neither must guess. The “less is more effect” (LIME) occurs when a recognition-dependent agent has a greater probability of choosing the better item than a more knowledgeable agent who recognizes more items. This paper provides some new mathematical results for understanding when the LIME can occur and elucidates implications of these results.

Many researchers investigating the descriptive validity of the recognition heuristic report high usage rates. Goldstein and Gigerenzer (2002) reported a 90% usage rate. Serwe and Frings (2006) found that 88% of their lay and 93% of their amateur samples used the recognition heuristic in choosing tennis match winners. Newell and Shanks (2004) reported 88% usage in a stock-market setting. Pachur and Biele (2007) found that the recognition heuristic accounted for 90% of the forecasts in their study, more than four other candidate mechanisms. Finally, Pohl (2006) observed that additional cue knowledge increased the usage of the recognition heuristic over cases where recognition of an object did not carry any other knowledge with it.

However, empirical evidence for the LIME is equivocal, at least on face value. Goldstein and Gigerenzer (2002), Serwe and Frings (2006), and Scheibehenne and Bröder (2007) are definitely in the “yes” camp, Pohl (2006) finds that the LIME is possible but claims only small effect-sizes, Andersson, Edman and Ekman (2005) and Ayton and Onkal (1997) present “less is as good as more” evidence, and Pachur and Biele (2007) are decidedly in the “no” camp. Simulation studies based on real ecologies lend some support to the prospect of LIMEs (e.g., Goldstein & Gigerenzer, 2002, and Dougherty, Franco-Watkins & Thomas, 2008). Moreover, Schooler and Hertwig (2005) and Pleskac (2007) present simulation results suggesting that imperfect recognition may actually increase the likelihood of a LIME. Matters are further complicated by shortcomings in some of the studies and an apparent lack of consensus on the requirements for a test of the LIME. These exigencies, combined with the results presented in this paper, render the corpus of empirical studies problematic and inconclusive. I shall return to this matter toward the end of this paper.

Goldstein and Gigerenzer (2002) define α as the probability that a correct choice is made on the basis of recognition alone and β the probability that a correct choice is made when both items are recognized (via additional cues). They claim that a LIME occurs if α > β (α > 1/2) and α and β remain constant as the number of recognized items, n, varies. This view has been widely accepted and used as a guide for when to expect the LIME (e.g., Pachur & Biele 2007). Pleskac (2007) concurs with Goldstein and Gigerenzer and makes an analogous claim under conditions of imperfect recognition.

However, Goldstein and Gigerenzer assume that α and β remain constant as the number of recognized items, n, varies. In fact, neither of these parameters necessarily remains constant as n varies, and neither of them is a simple function of n. We shall see demonstrations of these assertions shortly, and indeed Goldstein and Gigerenzer allowed that the assumption is not realistic. We shall see how various modifications of this assumption lead to the absence or presence of a LIME.

A sufficiently rigorous approach to this problem begins by distinguishing between the probability, β, of correctly choosing between pairs of recognized items using the knowledge cue, and the probability, v_c, of correctly choosing between any pair of items using the knowledge cue (i.e., v_c is the knowledge cue validity). To begin, I will demonstrate that the LIME can occur when α < β. In Table 1 we have 10 items of which 6 are recognized. The left-most column shows the rank of each item on the outcome and the fifth (Cue Rank) column shows their ranks on a knowledge cue to be used for choosing between two recognized items. For purposes of simplification and clarity, throughout this paper I will restrict discussion to a rank-order knowledge cue with no ties.

Table 1: LIME when α < β

First, let us determine α. From Table 1, the number of correct choices is the sum of the 0-entries in the “Recog.” column whose ranks is greater (i.e., worse) each of the 1 entries: C_r = 4 + 4 + 3 + 3 + 2 + 1 = 17. The number of incorrect choices is the sum of the 1-entries whose rank is greater than each of the 0 entries: D_r = 4 + 2 + 1 = 7. The result is α = 17/(7+17) = .708.

We use a similar procedure to compute the probability of making a correct choice using the knowledge cue, i.e., the knowledge cue validity v_c. The C_c column in Table 1 shows the number of items ranked worse than the item in each row that would be correctly identified by comparing that item’s cue-rank with that of the other items. For example, the first item has cue-rank 1 so by using the cue to compare it with the other 9 items we would correctly choose the first item as the better-ranked. In contrast, the third item has cue-rank 9, so we would make only 1 correct choice in comparing its cue-rank with those of the items that actually are ranked worse. The D_c column shows the corresponding number of incorrect choices. There are C_c = 30 correct and D_c = 15 incorrect choices, resulting in a cue validity v_c = 30/(30+15) = .667. Likewise, from the last two columns in Table 1, the probability of choosing correctly between pairs of recognized items by using the knowledge cue is β = C_cr/(C_cr + D_cr) = 14/(14+1) = .933.

Note that v_c ≠ β. That is, we have an example of the fact that the probability of making a correct choice between pairs from the 6 recognized items is not the same as the probability of making a correct choice when all 10 items are recognized. Moreover, both α and β can vary depending on the order in which the remaining items are learned (i.e., become recognizable). For example, if the next item learned is item 6 or 10 then the result will be β = .857, whereas if the next is item 3 or 8 then the result will be β = .809. Likewise, if item 6 is learned next α = .714 whereas if item 10 is learned next α = .524. These examples show variation in α and β as n varies, and they demonstrate that both parameters can take different values for alternative collections of recognized items having the same n.

Moreover, there is no generalized relation between the range of possible values of β and v_c. Assuming v_c ≥ 1/2 (i.e., use any negative cue in reverse), there is always at least one pair of items whose rank-order matches the order of the cue, so that if only those two items have been learned then β = 1. Conversely, if v_c < 1 then there is always at least one pair whose rank-order and cue-order are reversed so that if only those two items have been learned then β = 0. By the same argument, α can range from 0 to 1 depending on the order in which items are learned.

Now, we shall build up the probability of making a correct choice between pairs of items in Table 1, initially following Goldstein and Gigerenzer. For those pairs where one item is recognized and the other isn’t, we use the recognition cue and have

where N is the total number of items and n is the number of recognized items. The probability of a correct choice when both items are unrecognized (i.e., where a guess must be made) is

Finally, the probability of a correct choice when both items are recognized is

Summing these terms gives Goldstein and Gigerenzer’s (2002) formula. They denote P(correct) by f(n), so using their notation and plugging in the appropriate numbers yields f(n) = .756. Thus, we have the LIME because v_c = .667 < f(n) = .756, but we also have α = .708 < β = .933, so we observe that if β is allowed to vary (and thus differ from v_c) a LIME can occur when α < β.

When α and β are not constant, not only can the LIME occur when α < β, but the condition α > β does not guarantee a LIME. A counter-example can be constructed by modifying the one in Table 1. Suppose the knowledge cue ranks for the 10 objects are {5,4,3,2,1,6,7,8,9,10}. Then the knowledge cue validity is v_c = 35/(35 + 10) = .778. Now suppose the 6 recognized objects have outcome ranks {1,2,3,4,9,10}. Then α = 16/(16 + 8) = .667 and β = 9/(9 + 6) = .6, and α > β is satisfied. However, both α and β are less than v_c so no weighted sum of them and 1/2 is going to exceed v_c. Indeed, f(n) = .622, so the LIME does not occur. I shall address the issue of how common are occurrences of the LIME when α < β and no LIME when α > β in sections to follow.

Finally, we need to distinguish among various definitions of the LIME. Goldstein and Gigerenzer point out that there are at least three versions: One comparing more and less knowledgeable agents, another comparing performance in different domains, and a third comparing performance as an agent learns new items. The version we have been discussing is the first kind, v_c < f(n), which Katsikopoulos (2010) calls the “full experience” LIME. But another is f(n) > f(n+1), which can occur regardless of whether v_c < f(n). Let us call this a “local LIME.” The difference between the two is simply that v_c = f(N).

The next section of this paper investigates the co-occurrence of the LIME and α < β. The third lays out the conditions under which the LIME can occur under conditions of perfect and imperfect recognition. The fourth deals with the effect of learning items, and there is a brief concluding section. All technical arguments (theorems and proofs) are relegated to the Appendix.

2 When do the LIME and α < β co-occur?

In this section I will demonstrate that the co-occurrence of the LIME and α < β is likely to be quite commonplace. My purpose is twofold: First, to enhance our understanding of their co-occurrence and, second, to develop a perspective that extends our understanding of the LIME and performance of the recognition heuristic generally. To begin, I will alter the Goldstein-Gigerenzer notation by using v_r = α and v_cr = β. Thus, all cue validities will be denoted by v with an appropriate subscript. Throughout this paper, without much loss of generality, we will limit the treatment of the knowledge cues to a single cue with ranks and no ties.

The Goldstein-Gigerenzer formula for f(n) is

(1)

where Q_r = (N − n )(N − n − 1). This can be rewritten as

(2)

where γ_r = 2v_r−1 and γ_cr = 2v_cr−1. These γ parameters are Goodman and Kruskal’s gamma coefficient of association. For instance, γ_cr = (C_cr−D_cr)/(C_cr + D_cr). Equation (2) shows that deviations of f(n) from 1/2 may be written as a weighted sum of gamma coefficients. It will prove useful at times to interpret the LIME in these terms.

First, substituting (1 + γ_c)/2 for v_c, from equation (2) we may express the LIME as

(3)

Second, α < β iff γ_r < γ_cr. Combining this latter inequality with the LIME inequality above and rearranging terms, we satisfy both the LIME and α < β iff

(4)

It is also possible for the LIME to occur even when v_r < v_c under this condition:

(5)

It certainly is possible for these inequalities to be satisfied under conditions that are quite ordinary. In particular, it can be shown (see Theorem 1 in the Appendix) that when equations (4) or (5) are satisfied if n < N then it is always the case that v_c < v_cr. This result reveals that the LIME and α < β always can co-occur for some appropriate n if the recognition heuristic moderates the knowledge cue validity so as to increase it within the subset of recognized items. Thus, the knowledge cue “piggy-backs” on the recognition heuristic. Returning to the simple example in the Introduction, we can see that the LIME and α < β co-occur and, indeed, v_c = .667 < v_cr = .933.

It is all very well to show that the LIME and α < β can co-occur once, but can they repeatedly co-occur as more items become recognized? Suppose we have 20 items ranked 1,2,..., 20 and let the knowledge cue have ranks 2, 5, 10, 12, 19, 20, 4, 6, 3, 9, 8,7 , 1, 14, 13, 11, 15, 18, 17, 16. The knowledge cue validity is v_c = .663. Now let the order in which these items become recognized be 1, 3, 7, 4, 6, 12, 14, 2, 19, 18, 9, 15, 20, 5, 8, 11, 10, 13, 16, 17. Figure 1 plots the resulting values for f(n) as items become recognized with v_c represented by a horizontal line at .663, with v_c < f(n) and therefore the LIME on 12 occasions. Figure 2 plots the cue validity within the recognized items by the recognition validity at each turn. On 10 occasions α < β and 6 of those co-occur with the LIME.

Figure 1: Number of recognized items by probability of correct choice

Figure 2: Cue validity of recognized items by recognition validity

Of course, existence proofs and demonstrations do not indicate whether this co-occurrence is common or not, so let us turn to simulations to pursue this point. The simulations randomly sampled 20 replicates 10,000 times from a trivariate standard normal distribution and converted them to a vector of ranks (x ₁,x ₂,x ₃), where x ₁ is the outcome rank, x ₂ is the knowledge cue rank, and x ₃ is the order of learning rank. The pairwise correlations were set to all possible combinations of {.3, .5, .7}, plus an additional 9 combinations with r ₁₃ (the correlation between outcome and order of learning ranks) set to 0, resulting in 36 runs.

The results are summarized in Figure 3, which displays the proportion of runs where α < β out of those in which v_c < f(n), i.e., P(α < β|v_c < f(n)). This proportion ranges from about .05 to .43, so this co-occurrence is not uncommon for mid-range cue validities. Moreover, it is apparent that r ₁₃, the correlation between outcome and order of learning ranks, drives P(α < β|v_c < f(n)). Lower r ₁₃ predicts higher P(α<β|v_c<f(n)), with the maximum achieved when r ₁₃ = 0. In contrast, r ₁₂ (the correlation between outcome and the knowledge cue rank) and r ₂₃ (the correlation between the order of learning and the knowledge cue rank) have negligible effects. Recall that r ₁₃ is a proxy for the cue validity of order of learning which in turn determines the recognition cue validity for each value of n. Therefore, this finding tells us that the co-occurrence of the LIME and α < β is inversely related to the order of learning validity.

Figure 3: P(α<β|v_c<f(n)) by r ₁₂ and r ₁₃

3 A New General Condition for the “full experience” LIME

3.1 Perfect Recognition

We now return to examining the LIME itself. The simulations described earlier may be used to gain intuition about how the LIME is influenced by the cue validities of the knowledge cue and the order of learning, with a “chance” benchmark in which the order of learning is uncorrelated with outcome rank. Figure 4 shows the resulting P(v_c<f(n)), the proportion of trials in which the LIME occurred, as a function of r ₁₂ and r ₁₃. As we might expect, higher r ₁₃ predicts a higher probability of the LIME, and for constant r ₁₃ a lower r ₁₂ predicts more frequent LIMEs. This latter trend reflects the fact that although it is possible for the LIME to occur when α < β, it is easier for it to occur when the opposite is true. As r ₁₂ declines it is more likely that α > β and therefore also more likely that the LIME will occur.

Figure 4: P(v_c<f(n)) by r ₁₂ and r ₁₃

It should be clear that the LIME can occur “by chance,” in the sense that an arbitrary order of learning can sometimes produce the LIME. In the Table 1 example, if item 8 or item 10 is the last item to be learned then just before it is learned f(n) will be .733, both instances of the LIME (recall that v_c = .667). However, if item 3 or item 6 is the last to be learned then f(n) will be .644 or .667, neither of which exceeds v_c. So, conditional on all items but 3, 6, 8, and 10 having been learned, if each of the remaining four is equally likely to be the last learned then in the last learning stage the probability of the LIME is .5.

Now in Figure 4 note that when r ₁₃ = 0, P(v_c<f(n)) does not fall to a negligible level. In fact, for r ₁₂ = .3 the probability of the LIME is around .15 to .20. It can be driven higher still by allowing a negative correlation between the order of learning and the knowledge cue rank. For r ₁₂ = .3 and r ₂₃ = −.5, for instance, the simulation resulted in P(v_c<f(n)) = .256. At least some occurrences of the LIME are an artifact of random variability in recognition cue validity despite the absence of order-of-learning validity. Therefore, in evaluating the accuracy of the recognition heuristic, it seems advisable to benchmark any empirical findings against appropriate “null” models that track the occurrence of the LIME when the order-of-learning validity is zero.

Is there another general condition restricting when the LIME can occur? This condition can be stated simply but it requires a small addition to the machinery that has been built up so far. Denote by v_cnr the probability of choosing correctly between recognized and unrecognized items by using the knowledge cue (rather than the recognition cue), and let v_cnn be the probability of correctly choosing between two unrecognized items by using the knowledge cue. Obviously these are “counterfactual” constructions in the sense that the partially ignorant agent cannot use the knowledge cue to choose between items unless both are recognized. Nevertheless, v_cnr and v_cnn permit us to decompose v_c into its three components:

where again Q_r = (N − n )(N − n − 1). From this expression and equation (1) the LIME condition may be written as

(6)

This version of the LIME reveals that if v_cnn ≥ 1/2 then the LIME occurs only if

(7)

Recall that in the Goldstein-Gigerenzer notation v_r = α. So this really is where the recognition cue’s validity must exceed that of the knowledge cue, namely in choices between a recognized item and an unrecognized item. Schooler and Hertwig’s (2005) implementation of the recognition heuristic in the ACT-R framework uses no knowledge cue and instead assumes guessing when both objects are recognized. That is, they explicitly restrict v_cr and v_cnn to 0.5 and thus implicitly v_cnr also is 0.5. Consequently their simulation obtains a LIME simply by recognition performing above chance level. Conversely, if v_cnr ≥ v_r the LIME occurs only if v_cnn < 1/2. Finally, it should be evident that because v_cr (i.e., β) is common to both f(n) and v_c, the occurrence of α > β without the LIME may be quite frequent. Indeed, it is no surprise that Pachur and Biele (2007) failed to find a LIME even when the α > β condition was satisfied.

Clearly the order in which items become recognized is crucial in determining whether the LIME will occur. If this order perfectly matches the order of the outcome ranks then of course at each step v_r = 1 and the LIME is maximally likely. On the other hand, if pairs of items become recognized whose ranks are equally above and below the median rank then v_r = .5 and the LIME is unlikely to occur. The order in which items become recognized acts like another cue with the order of learning determining the ranks of this cue.

Accordingly, let v_o denote the validity of the order in which items become recognized. At the point where n items have become recognized we may decompose v_o in the same way as v_c using an obvious notation, so that we write v_o as

Restrictions on v_o impose further restrictions on the conditions under which the LIME occurs. If v_o ≤ v_c, then v_cnr < v_r only if γ_c.r > γ_o.r, where γ_c.r is the partial gamma coefficient for the knowledge cue with recognition partialed out and γ_o.r is the corresponding partial gamma for the order of learning (see Corollary 1 in the Appendix). It may seem counter-intuitive that the LIME could occur even when the order of learning validity is lower than that of the knowledge cue and v_cnn ≥ 1/2, but that is unmistakably what this result says. Nor is it difficult to construct such examples.

Table 2 displays one such example with 10 items of which 5 are recognized, v_c = .778 < f(n) = .8 and therefore the LIME, and yet v_cnn = .6 and v_o = .667 < v_c. The LIME is achievable here because v_cnr = .8 < v_r = .88 , and this inequality in turn is achievable because γ_c.r = .486 > γ_o.r = .371.

Table 2: LIME when v_o ≤ v_c

3.2 Imperfect Recognition

Pleskac (2007) extends the study of the recognition heuristic by introducing a condition that recognition is imperfect, i.e., people do not always recognize the items they have experienced. Thus, instead of just recognized and unrecognized items, there are hits (items correctly identified as having been experienced), misses (items incorrectly identified as not experienced), true rejections (items correctly identified as not experienced), and false alarms (items incorrectly identified as experienced). The result is 10 distinct pairs of items, each with their own decision rule (Pleskac, 2007, Table 1).

Pleskac assumes that the cue validity of experience, denoted by A, is independent of recognition ability, so he replaces α with A. Likewise, he replaces β with B, the validity of the knowledge cue among the experienced (instead of the recognized) items. I have summarized the components of f(n) in his scheme in Table 3, using the abbreviations H for hits, M for misses, T for true rejections, and F for false alarms.

Table 3: Probabilities of correct choices for f(n)

* G = zA+(1−z)/2

For (M,M), (T,M), and (T,T) pairs a decision maker must guess, so the probability of a correct choice from these three pairs is 1/2. Pleskac (2007: 384) argues that the probabilities of correct choices from the (H,M) and (F,T) pairs also must be 1/2. By definition, the probability of a correct choice from the (T,H) pairs is A, the probability of a correct choice from the (F,M) pairs is 1 − A, and the probability of a correct choice from the (H,H) pairs is B.

As Pleskac (2007: 384-5) points out, choices involving the (F,H) pairs benefit partly from experience. Rather than repeating his argument here, suffice it to say that the proportion of correct choices for these pairs is zA + (1 −z)/2, where z is the proportion of experienced items that would be chosen over the false-alarm items on the basis of some choice heuristic. In Pleskac’s setup the knowledge cues are binary (either positive or negative) and his version of this heuristic is that the experienced item must have at least one positive cue value. For the time being, we will leave this heuristic unspecified.

Pleskac claims that the LIME can occur only if A > B. In a recent paper Katsikopoulos (2010) disproves this claim, showing that the LIME can co-occur with A < B even allowing Gigerenzer and Goldstein’s assumption. We can extend the argument from section 2 to specify when the LIME can co-occur with A < B. Denoting the hit-rate by h and the false-alarm rate by f, Theorem 2 in the Appendix provides the following characterization of the LIME under imperfect recognition:

(8)

where n_e is the number of items experienced, and γ_A and γ_B have the obvious meanings. When h=1 and f=0 (i.e., under perfect recognition) this equation reduces to equation (3) with n_e = n, γ_A = γ_r, and γ_B = γ_cr. Theorem 2 proves that the LIME can co-occur with A < B iff

(9)

When h=1 and f=0 this equation reduces to equation (4) with the same substitutions as above.

Now, following Katsikopoulos (2010), let α _e = (A − 1/2)(h − f + zhf) + 1/2, and β _e = (B − 1/2)h ² + 1/2.

Thus, α _e and β _e are analogous to α and β under imperfect recognition. Theorem 2 also shows that the LIME can co-occur with α _e < β _e iff the γ_B(h − f + zhf) term in equation (9) is replaced with γ_Bh ². This is a more severe requirement than equation (9), so if the LIME co-occurs with α _e < β _e it also co-occurs with A < B but the converse does not hold.

A higher value of h and a lower value of f make the inequalities in equations (8) and (9) easier to satisfy, and therefore the LIME more likely to occur. The h − f + zhf term is not positive when f ≥ h/(1−hz), in which case the inequalities cannot hold if γ_c, γ_B and γ_A all are positive. Katsikopoulos (2010) presents a new version of the LIME when f ≥ h/(1−hz), whereby f(n) declines as n increases until n becomes sufficiently large. We will not consider this condition here; a full investigation of the LIME under imperfect recognition is beyond our scope.

Clearly a higher value of z also increases the likelihood of the LIME. Thus, the heuristic driving z when the knowledge cue is ranked instead of binary should be of interest to researchers in this area. A simple heuristic would be to choose the experienced item over the false-alarm item if the knowledge cue rank of the experienced item is better than some benchmark known to the decision maker. On the other hand, a rational decision maker who believes that A > 1/2 should set z = 1.

What form does the general condition for the LIME in equation (7) take under imperfect recognition? To determine this, we begin by assuming that the validity of the knowledge cue differs only across the same three subsets of item pairs as in perfect recognition. This assumption is simply the counterpart of the foregoing assumption regarding the experience cue validity, A, namely that these cue validities are conditionally independent of the agent’s recognition ability. Thus, in Table 4 the knowledge cue validity is B for choices between pairs of experienced items, B ₁ for choices where one item is experienced and the other not, and B ₂ when both items are not experienced.

Table 4: Probabilities of correct choices for the knowledge cue

Corollary 2 in the Appendix shows that if the appropriately weighted sum of B ₂ and B is 1/2 or greater, then the general condition in equation (7) generalizes to the inequality

(10)

When h=1 and f=0 this inequality reduces to equation (7) with γ_A = γ_r and γ_B1 = γ_cnr. As before, higher values of h and z and a lower value of f make this inequality easier to satisfy, and therefore the LIME more likely to occur.

We now will relax the assumption that the experience and recognition are conditionally independent, by allowing the probability of a correct choice between (H,M) pairs to differ from 1/2. This probability is denoted by Q in Table 5. The motivation for relaxing this assumption is to consider the influence that memory effects such as primary or recency might have on the LIME. If the higher-ranked experienced items are more likely to be recognized then Q > 1/2. If the earlier-experienced items are more highly ranked then a primacy effect will result in Q > 1/2, whereas a recency effect would yield Q < 1/2.

Table 5: Probabilities of correct choices without the independence assumption

* G = zA+(1−z)/2

Relaxing the conditional independence assumption also affects the probability of a correct choice between (F,M) pairs because the knowledge cue validity for the misses is no longer A. Instead, it is qA, where if Q > 1/2 then 0 < q < 1 whereas if Q < 1/2, q > 1. Corollary 3 in the Appendix shows that the LIME condition in equation (10) generalizes to

(11)

where γ_Q = 2Q − 1. For small n_e this inequality is dominated by the comparison between γ_A and γ_B1, whereas large n_e it is dominated by γ_Q versus γ_B. Here, the generalized condition for the LIME no longer holds. It is possible for γ_Q to exceed γ_B sufficiently to enable the LIME to occur when equation (10) is violated.

4 Learning and forgetting

In order for a local LIME to occur, f(n) must either rise and then fall or vice-versa. That is, there must be a local “more-is-more” effect (or MIME) followed by a local LIME or vice-versa as items are learned or the reverse sequence if items are being forgotten. There is a corresponding local LIME and MIME pair if we consider removing or adding an item to the collection of items, but we will not deal with that case here. We shall also consider only the case of perfect recognition.

Learning and forgetting items will generally change f(n) but not v_c. The conditions under which the direction of change in f(n) can switch sign are of interest, because that is the event that signals a local MIME followed by a local LIME or vice-versa. We will focus on the case where one more item is learned. The results for the case where one item is forgotten differ only in minor respects that are not of interest here.

Let v _r1 denote the new probability of correct choices between a recognized and unrecognized pair using the recognition heuristic when one more item has been learned. Likewise, let v _cnr1, v _cnn1, and v _cr1 denote the new probabilities of correct choices using the knowledge cue between a recognized and unrecognized pair, two unrecognized items, and two recognized items respectively. Each of these probabilities will have their corresponding γ parameters as before. Now, consider the change in the proportion of correct choices as one more item is learned: f(n)−f(n+1). Theorem 3 in the Appendix shows that f(n)−f(n+1) = 0 for n < N−1 under the following conditions:

(12)

where δ_r = γ_r−γ_r1, δ_cr = γ_cr−γ_cr1, and

When n = N−1, f(n)−f(n+1) = 0 iff

For n < (N−1)/2, δ_r0 > 0 so equation 12 implies that if δ_r < 0 then δ_cr > 0. Moreover, even for intermediate values of n ≥ (N−1)/2 it turns out that δ_r0 is close to 0. Thus, generally equation (12) suggests that in order for a local MIME-LIME sequence to occur, δ_cr and δ_r will tend to have opposite signs so that an increase in the recognition validity will be offset by a decrease in the knowledge-cue validity among recognized items and vice-versa.

Now because v_c does not change, we also must ascertain the conditions for it to remain constant as one more item is learned. Theorem 4 shows that v_c remains constant for n such that n < N−1 under the following conditions:

(13)

where δ_cnr = γ_cnr−γ_cnr1, δ_cnn = γ_cnn − γ_cnn1, and

When n = N−1, v_c remains constant when one more item is learned iff

Equation (13) suggests a quasi-hydraulic relation between δ_cr and both δ_cnr and δ_cnn that accords with the commonsense supposition that as an additional item is learned any change in v_cr will be compensated by a net opposite change in the weighted sum of v_cnr and v_cnn due to the fact that v_c does not change. For intermediate values of n, it turns out that δ_cnr0 is close to 0. Consequently, δ_cnn tends to have a larger effect on δ_cr than δ_cnr0 does.

5 Discussion

The main results presented in this paper may be summarized as follows.

1. 1. When we relax the assumption that α and β do not change as n varies, the LIME does not depend on the condition that α > β. This condition can occur without the LIME. Likewise, the LIME can co-occur with α < β and, indeed, with α < v_c or even when the order of learning validity is less than the knowledge cue validity (v_o < v_c). Moreover, these co-occurrences can arise under conditions that arguably are neither unusual nor bizarre. The main requirement is that the recognition heuristic moderates the knowledge cue validity so as to increase it within the subset of recognized items. Simulations revealed that the higher the order of learning validity, the less likely the LIME is to co-occur with α < β. An analogous result was obtained under imperfect recognition, thereby extending Katsikopoulos’ (2010) finding that the LIME can co-occur with A < B to include co-occurrence with α _e < β _e.

2. 2. In the new general condition for the LIME, equation (6) implies that if v_cnn > 1/2 then the LIME can occur only if v_cnr < v_r, i.e., if the recognition cue validity exceeds the knowledge cue validity within the set of recognized items. This new condition for the LIME was generalized to deal with imperfect recognition, with the additional finding that a higher hit-rate and lower false-alarm rate increase the likelihood of the LIME.

3. 3. Under imperfect recognition when the assumption of conditional independence between recognition and experience is relaxed, v_cnr < v_r is no longer required because the LIME can occur if the knowledge cue validity for (H,M) pairs sufficiently exceeds the knowledge cue validity for (H,H) pairs. The latter comparison carries greater weight as more items are learned and/or as hit-rate increases.

4. 4. If the LIME occurs then at some point as more items are learned or forgotten there must be a local MIME followed by a local LIME, or vice-versa. Equation (12) suggests (but does not strictly imply) that when the local MIME-local LIME sequence occurs, the change in v_cr will be negatively associated with change in v_r.

The results generalize to a binary knowledge cue or an ordinal cue with tied ranks (here I have assumed an ordinal knowledge cue with no tied ranks), and also to a weighted sum of cues. Equivalent examples to those from Table 1 onward using a binary knowledge cue are available from the author on request. Tied ranks sever the analogy with the γ coefficient of association but do not invalidate the results. When the knowledge cue is used to make a choice, I assume guessing is used if the two items are tied on the knowledge cue. Letting T_c denote the number of tied pairs, v_c = (C_c + T_c/2)/(C_c + D_c + T_c) and an analogous formula holds for v_cr, i.e., β. Now, 2v_c − 1 no longer is γ_c but instead equals Somers’ (1962) d_xy, an asymmetric measure of ordinal association (Somers’ measure is related to Kendall’s τ_b by d_xyd_yx = τ_b ²). Thus, all results in this paper expressed in terms of validities remain as they are, and dealing with ties simply means that all results expressed in terms of γ coefficients have Somers’ d_xy substituted for γ.

The findings presented here apply to any binary characteristic whose possession by an item is not fixed but can vary either through assignment by a perceiver or environmental changes. Not only does this include the recognition cue, but any other binary status cue (e.g., membership in a group, organization or club that carries with it relevant knowledge cues and without which those cues are absent). These findings describe how effective status cues earn their keep.

The results also point toward four programmatic recommendations regarding future work on the recognition heuristic. First, despite the demonstrations via analytical results and simulations that α > β is not required for the LIME, it is not known how often α < β and the LIME co-occur in real environments, how often α > β occurs without the LIME, or whether these co-occurrences depend on n and/or N. All three merit further investigation.

Second, the results highlight the importance of the order in which items are learned. No account of the recognition heuristic can be complete without an understanding of the effects of the order of learning, and therefore those aspects of reputational systems and learners determining that order. Population-level models of the recognition heuristic and predictions of its accuracy should incorporate at least an expected order of learning, and preferably an appropriate distributional model of that order.

The implications of these results are compatible with certain other criticisms of empirical research on the recognition heuristic. Dougherty et al. (2008) raise the problem of determining the reference class and ecology within which cue validities are evaluated, and Gigerenzer, Hoffrage and Goldstein’s (2008) response refers to a general confusion between cue validity and ecological validity. The main point is that inferring a LIME via between-agent comparisons requires agents in the same ecology (e.g., German citizens reading German newspapers should not be compared with American citizens reading American newspapers) who are making choices within the same reference class of objects (e.g., f(n) for American cities cannot sensibly be compared with f(n) for German cities).

The results in this paper imply that f(n), α, and β for one set of n recognized objects in a particular ecology and reference class will not necessarily be identical for a different set of n recognized objects, even for the same agent. Moreover, to establish that a LIME has occurred by comparing between agents requires the ecological validity of the knowledge cues to be identical for those agents. Thus, unconfounded between-agent comparisons (agent 1 knows n ₁ objects and agent 2 knows n ₂ objects, where n ₁ > n ₂) require not only that both agents be located in the same ecology and reference class, but also v _c1 = v _c2, preferably because both agents use the same knowledge cues in the same way, and the n ₂ objects are a subset of the n ₁ objects. Of course, this is unlikely to hold for agents in real environments. For example, Dougherty et al. (2008: 208) suggest that as agents learn more objects they may also learn more effective cues.

The current empirical literature on the recognition heuristic generally is flawed or subject to influences that researchers have not taken into account. For example, Serwe and Frings (2006) compare the predictive accuracy of aggregated rankings of Wimbledon tennis players based on mere recognition with the ATP rankings of these players, so they are not actually evaluating the performance of the recognition heuristic in conjunction with knowledge and guessing. On the other hand, Pohl (2006) and Pachur and Biele (2007) use methods that do this, but the remaining potential confounds in their studies are, first, that the knowledge cue validity v_c will be unique for each individual and therefore will have an unknown effect on each person’s β, and second, the sets of objects recognized by subjects whose n is small may not be subsets of the sets of objects recognized by subjects whose n is large.

Within-agent (agent 1 = agent 2) comparisons satisfy nearly all of the aforementioned requirements if the collection of objects remains stable for the duration of the comparisons. Between-agent comparisons or comparisons of mean f(n) for different values of n are vulnerable to confounds except in very restricted or controlled ecologies. A clear recommendation for studying the LIME in its “pure” form with effects due solely to n is tracking agents over time as they learn or forget objects in environments with stable collections of objects.

The order of learning is determined not only by reputational systems but also by learners. The effectiveness of the recognition heuristic therefore hinges not only on aspects of the social environment but also how individuals interact with and learn from that environment, and retain what they have learned. Pleskac (2007) and Katsikopoulos (2010) have made inroads on this topic. Both Katsikopoulos’ paper and the results at the end of section 4 suggest possible joint effects of memory processes (the example used here is primacy versus recency effects) and the reputational system on the performance of the recognition heuristic. Empirical studies would benefit from taking on a more dynamic approach than most recognition heuristic studies, studying how people learn and remember (or forget) about a collection of items.

To date, agent learning or forgetting in regard to the recognition heuristic has been investigated in simulations (e.g., Goldstein & Gigerenzer 1999 and Dougherty et al. 2008) but not empirically. Dougherty et al.’s methods come close to satisfying the requirements for investigating the LIME that have been derived from the results presented here. However, like others in this domain, they have erroneously assumed that β remains constant as n varies and is the same for different collections of n recognized items.

A third direction for future research is the extension of the issues raised in this paper to group inferences. Reimer and Katsikopoulos (2004) present several analytical results characterizing the LIME under various combination rules such as majority-rule. They assume that α and β do not change as n varies, so their findings merit further investigation whereby this assumption is relaxed.

Fourth, collections of items and their ranks on outcomes often are unstable. Ranks can change, of course, because items can improve or decline, even if only through stochastic artifacts such as regression toward the mean. Perhaps more importantly, items may drop out or new ones appear. The disappearance of old items and appearance of novel ones will affect both the order of recognition validity and knowledge cue validity, and therefore the performance of the recognition heuristic. These effects were hinted at but not dealt with here. Understanding them will require the same reorientations described above, namely greater attention to the order in which items are learned (or forgotten), to the joint effects of learner and environment characteristics, and to dynamics in general.

Finally, a few remarks are in order on the limitations and utility of formal analysis as utilized in this paper. As in any mathematization, some idealizations and simplifications have been made. Chief among these is the assumption that the properties of the knowledge cue do not change as more items are learned or forgotten. As Dougherty et al. (2008) observe, it is plausible that this assumption may not hold. On the other hand, the setup in this paper avoids simplifications in earlier analyses that have misguided researchers, most importantly the assumption that α and β are invariant under changes in n or for different collections of n recognized items. I would argue that this new analysis does not commit what Lewandowsky (1993) termed “irrelevant specification.”

Regarding utility, the approach in this paper does what formal analyses and models should (Fum, Missier, & Stocco, 2007). First, it highlights determinants of how the recognition heuristic performs that have been overlooked. It does this by deriving the influence of the order of learning and by introducing “counterfactual” constructs such as v_cr, neither of which are obvious in verbal descriptions of the recognition heuristic. Second, it provides guidelines for researchers concerning methods, novel phenomena to investigate, and when the LIME is possible and when it is not.