Can Identity Theory Improve Survey Design?

Abstract

Question effects are important when designing and interpreting surveys. Question responses are influenced by preceding questions through ordering effects. Identity Theory is employed to explain why some ordering effects exist. A conceptual model predicts respondents will display identity inertia, where the identity cued in one question will be expressed in subsequent questions regardless of whether those questions cue that identity. Lower amounts of identity inertia are found compared to habitual inertia, where respondents tend to give similar answers to previous questions. The magnitude of both inertias is small, suggesting they are only minor obstacles to survey design.

Among the observed phenomenon in survey responses is the “question-ordering effect”, where the answer to a specific question is influenced by which questions were asked previously (Kalton, Collins, and Brook, 1978; Schuman, Presser, and Ludwig, 1981; Kalton and Schuman, 1982; Schuman, 1992; Moore, 2002; Tourangeau, Couper, and Conrad, 2004; Wang et al., 2014). For example, surveys containing discrete-choice experiments may find that the random prices assigned to goods in the first set of choices impact responses to subsequent choice sets (Su et al., 2017; Yamazaki et al., 2013; Boyle, Johnson, and McCollum, 1997). That is a case where the questions themselves influence future responses. Question-order effects can also refer to cases where the response to a question influences future responses, even though the questions are unrelated (Hafner-Fink and Uhan, 2011; Carlsson, Mørkbakb and Olsenc, 2012; Gehlbach and Barge, 2012).

Question-order effects are a variety of behaviors referred to in the social and behavior sciences as anchoring. Anchoring occurs when individuals use unrelated information to help them make decisions. In addition to question-order effects on surveys, anchoring effects have been detected in experiments and market outcomes and are used in marketing strategies to increase revenues (Alevy, Landry, and List, 2015; Araña and León, 2007; Bokhari and Geltner, 2011; Cameron and Quiggin, 1994; Li, Maniadis, and Sedikides, 2021; Tversky and Kahneman, 1974).

The basic idea is that subjects do not have fixed preferences for goods or answers to survey questions, and must construct these preferences immediately prior to when the decision is made. To aid in this construction, people often rely on information that is not directly related to the question being posed. This study concerns a form of anchoring where information from previous survey questions that is mostly irrelevant to the current question influences their answer, referred to as the question-order effect.

Survey designers sometimes randomize the order of questions to mitigate question-order effects, but Strack (1992) aptly observes that it is not the order of questions per se that influences results but the questions themselves. Randomization does not eliminate bias; it just ensures whatever bias exists is not the product of one specific ordering of questions. Instead, undesirable ordering effects are often best avoided by thinking carefully about how one question may alter answers to subsequent questions and choosing an order that is most consistent with the research objectives (Schwartz and Sudman, 1992).

Research has identified a number of causes for question-ordering effects, and they are usually psychological processes where a question “primes” subjects to answer a specific way to subsequent questions (Chapman and Johnson, 1999; Frederick, Kahneman, and Mochon 2010). Consider information-availability effects whereby a question makes certain types of information more mentally accessible, such that this information is used in forming answers to subsequent questions (Strack, 1992).

We hypothesize an alternative source of ordering effects or anchoring not yet considered by researchers. This source emanates from Identity Theory, which states individuals have numerous potential identities they can activate at any one time. The identities they choose to express at any one time depends on the context, and when the context cues them to express multiple identities in a short period, they will blend the identities together in an attempt to portray a consistent persona (Burke and Stets, 2009). An example would be a survey where respondents are first asked if they believe citizens have a moral obligation to vote, and then asks them if they voted in the previous election. Mostly everyone will agree with the first statement, and even if they did not vote in the previous election, they are more likely to answer “yes” to the second question to avoid appearing hypocritical. This is a specific case of a consistency effect described by Moore (2002) where respondents attempt to make answers to two questions consistent with one another, resulting in different answers than if the questions were asked in isolation at different points in time.

Previous research has shown that Identity Theory can manifest itself in responses to an individual survey question (Brenner and DeLamater, 2016) but has not considered how Identity Theory might explain patterns of responses to a sequence of questions. We hypothesize that, if presented with a series of survey questions that each cue a different identity, respondents may be more likely to combine those identities as opposed to lurching from one discrete identity to another. That is, the manner in which an identity is expressed in the second question of a survey may be influenced by the identity expressed in the first question. We refer to this as identity inertia, meaning an identity expressed in one survey question will be expressed to some degree in subsequent questions, regardless of whether those subsequent questions specifically cue that identity.

This study develops a mathematical model of identity inertia and then tests predictions from this model in an online survey. The empirical results detect identity inertia in some but not all possible cases. In these occurrences, the magnitude of the effect is small, and a different type of inertia stemming from habit is more pronounced.

Identity Theory in Survey Responses

The concept of ‘identity’ in Identity Theory refers to, “the set of meanings that define who one is when one is an occupant of a particular role in society, a member of a particular group, or claims particular characteristics that identify him or her as a unique person,” and its main objective is to describe the relationship between a person and society (Burke and Stets, 2009, p. 3). The theory states that any one individual will possess multiple identities because they serve several different roles in society, interact with different social groups, and describe themselves using different characteristics across various contexts. These identities are a mechanism by which the individual is linked to others in society, and the “meanings” associated with an identity is determined not just by the person but society as well.

The idea of a person possessing multiple identities has been used in economic studies. Behavioral economists frequently describe individuals as possessing two different ways of thinking: automatic and deliberative—more commonly described as “thinking fast and slow” (Kahneman, 2011). Alós-Ferrer and Strack (2014) show that this has led to multiple selves models, as in Jamison and Wegener (2010). Studies measuring food safety, animal welfare, and wildlife preservation have been shown to differ depending on whether survey respondents are primed to activate their “consumer” or “citizen” self (Alphonce, Alfnes, and Sharma, 2014; Blamey, Common, and Quiggin, 1995; Norwood, Tonsor, and Lusk, 2019). Descriptions explaining hypothetical bias often describe how values are shaped by context (see Penn and Hu, 2018) in ways that can easily be described as the same people donning different identities. For example, one might argue that survey respondents place the highest prominence on the “citizen” self where they are willing to make sacrifices for a good cause, but when actual sacrifices are requested, they express their “consumer” self instead, where their willingness to sacrifice is diminished.

Social desirability bias is a major concern in survey design and occurs when respondents provide misleading information about their true behaviors and instead provide responses they believe others will deem favorable (Crowne and Marlowe, 1960, 1974; Fisher, 1993; King and Bruner, 2000; Leggett et al., 2003; Lusk and Norwood, 2010; Norwood and Lusk, 2011; Plant, Devine, and Brazy, 2003; Tourangeau and Bradburn, 2010; Cerri, Thøgersen and Testa, 2019). Previous research (Brenner and DeLamater, 2016) has demonstrated how Identity Theory can help explain social desirability bias in surveys. When asked about behaviors associated with identities holding high prominence or importance for individuals, they tend to overstate the extent to which they activate that identity in various contexts. Note that there are times when questions about actual and ideal behavior are asked sequentially in a survey, and this may cause an individual to carry over the weight of the first decision to the next even if the question prompts another behavior.

We refer to the phenomenon of blending identities expressed in recently answered questions from the identity cued by the current question as identity inertia. This is a hypothesized phenomenon, and the objective of this study is to empirically test for identity inertia in survey responses. This requires a conceptual model of how identity inertia manifests in survey responses and a survey design that can empirically test for the inertia. The next section describes this conceptual model.

A Conceptual Model of Identity Inertia

Brenner and DeLamater (2016) applied Identity Theory to individual survey questions. We now extend their work by creating a conceptual model of how Identity Theory might explain patterns of responses to a series of survey questions. This is a mathematical model of Identity Theory in the spirit of familiar economic models. It should be noted that Identity Theory is rarely stated in mathematical terms as it is here, but we do so here for conformity with traditional economic models.

Let s _{i, c} be the ith potential self the individual can activate, where there are N potential identities in any context c. s _{i, c} equals one if activated and zero otherwise. Let P _i be the prominence factor for identity i, and γ _{i, c} be its salience factor in context c, where an increase in either factor increases the likelihood of the identity being activated. Prominence ranks the identities in terms of desirability absence of context, thus the absence of a c subscript, whereas salience factors describe how an identity is more or less likely to be activated in a specific context. It is the combination of prominence and salience factors which determine the identity or combination of identities a person chooses to express in any given context (Burke and Stets, 2009).

The desirability of any one identity, s _{i, c} , depends on the values of P _i and γ _{i, c} in much the same way that the desirability of a good in a consumption problem depends on the exogenous utility function parameters and prices. While the Identity Theory literature typically uses prominence and salience as descriptors and not exogenous factors, we do so here as a proxy for the exogenous factors that determine behavior, like genetics, the environment, and the specific context of a decision.

A person can choose to display a single identity or a medley of multiple identities, so we depict the decision variables of the model to be the weight (w _{i, c} ) the person places on identity i in context c. The activated portfolio of identities is then represented by the vector S _c = [w _{1, c}   …  w _{N, c} ]. They cannot activate all identities fully at one time, so the weights are constrained such that $\sum _{i\,=\,1}^{N}w_{i,c}=1$ and 0 ≤ w _{i, c} ≤ 1∀i. For example, if w _{1, c} = 0.25 and w _{3, c} = 0.75, the individual has activated identities 1 and 3, with identity 3 displayed with three times the intensity of identity 1, but with less intensity than if it was the only identity expressed (in which case w _{3, c} = 1).

The optimal selection of w _{i, c} is a function of the prominence and salience characteristics specific to the person and the context. The objective function is initially conceptualized as f(w _{1, c} ,…,w _{N, c} |P _i ,γ _{i, c} ) where a higher value of f(.) denotes a more desirable identity expression (this will be simplified shortly). One attribute of Identity Theory is how an individual manages identities when multiple identities are expressed. This will be particularly important for survey design because different questions will cue the respondent to activate different identities. As the respondent projects these multiple identities, they must reconcile some of those differences in the identity expression. A core tenet of Identity Theory is that, when expressed as a group, the activation of an identity has implications for how other identities are expressed (Burke and Stets, 2009).

This suggests that the prominence and salience factors interact in such a way that the function mapping their values to the chosen identity portfolio is complex. For our purposes, we simplify max _{w i, c} f(w _{1, c} ,…,w _{N, c} |P _i ,γ _{i, c} ) to be a combination of a linear objective function plus a penalty term, where the person is penalized if they express multiple identities in ways that are inconsistent with one another. The objective function is specified to take the linear form $\max _{w_{i,c}} \sum _{i\,=\,1}^{N}\left(P_{i}w_{i,c}+\gamma _{i,c}w_{i,c}\right)$ .

A penalty term is then added to account for an individual’s attempt to reconcile their multiple selves as they go through the process of answering a sequence of survey questions. It is this penalty term that gives rise to identity inertia. To account for respondents’ hypothesized desire for expressing a consistent identity portfolio as they move from one statement to the next, we include the penalty term $-\beta \sum _{i\,=\,1}^{N}\left(w_{i,c}-\overline{w}_{i}\right)^{2}$ , where β is a positive parameter, w _{i, c} is the expression of identity i in context c, and $\overline{w}_{i}$ is an indicator of how strongly identity i was expressed in the recent past. The larger the term, $\sum _{i\,=\,1}^{N}\left(w_{i,c}-\overline{w}_{i}\right)^{2}$ , the less consistent the identity portfolios expressed in the survey. The larger the value of β, the more consistent identity expression matters to the person, and the greater the amount of identity inertia across survey questions.

The complete model showing how respondents choose which identities to express on a given survey question is then as follows

(1) \begin{align}{{{\max }_U}}_{{w_{i,c}}} &= {\!\sum\nolimits _{i \,= \,1}^N {\left( {{P_i}{w_{i,c}} + {\gamma _{i,c}}{w_{i,c}}} \right)} - \lambda \left( {1 - \sum\nolimits _{i \,= \,1}^N {{w_{i,c}}} } \right) - \beta \sum\nolimits _{i \,= \,1}^N {{{\left( {{w_{i,c}} - {{\overline w}_i}} \right)}^2}}}.\\ &\quad {\rm{subject}}\;{\rm{to}}:{\rm{1}} \ge {{{w}}_{{{i}},{{c}}}} \ge {\rm{0}}\forall {{i}}\end{align}

The major implication of this model is that due to the penalty term and the assumption β > 0, the individual will display identity inertia as they complete a questionnaire. Each specific question in isolation may cue a unique identity portfolio but the individual will be hesitant to switch from one distinct identity to the next and will instead display an identity portfolio that is both suitable for the question being answered and consistent with the portfolio activated in previous questions. Identity inertia results in answers that depend on the questions previously asked, providing one theoretical explanation for the ordering effect observed in surveys.

Let us simplify the model to assume only two possible identities: the ideal self and the common self. The weight of the ideal self, w ₁, describes the person’s aspirations while the weight of common self, w ₂, describes a set of frequent behaviors in daily life. These are separate identities because the person only rarely lives up to their ideal self. Suppose they are answering a sequence of two questions and the first question cues their ideal self. The optimal identity portfolio for the first question is then (w ₁,w ₂) = (1,0), and as they move to the second question the value of $\left(\overline{w}_{1},\overline{w}_{2}\right)$ = (1,0). Because this is the first question there is no identity inertia.

Suppose the second statement cues their common self, and the respondent must optimize (1) given the values of $\overline{w}_{1}\gt \overline{w}_{2}$ . The c subscript is dropped for simplification, though c = 1 could be used for statement 1 and c = 2 for statement 2.

The slope of the isoquant for (1) at the second statement is given in below in (2) and shown in Figure 1 for a specific set of parameters

Figure 1. Isoquant of objective function for identity expression.

(2) $${{\partial {w_1}} \over {\partial {w_2}}} = {{ - \left[ {{P_2} + {\gamma _2} - 2\beta \left( {{w_2} - {{\overline w}_2}} \right)} \right]} \over {\left[ {{P_1} + {\gamma _1} - 2\beta \left( {{w_1} - {{\overline w}_1}} \right)} \right]}}.$$

While isoquants are typically convex in economic theory, the isoquant here, in fact, is circular. This is due to w _i being restricted to 0 or 1. To see this, recall the formula for a circle is (x−h)² + (y−h)² = r ² (where x and y are coordinates, h is a constant, and r is the radius) and note the isoquant can be written as

(3) $$-G=\left(w_{1}-{a_{1} \over 2}\right)^{2}+\left(w_{2}-{a_{2} \over 2}\right)^{2},$$

where

(4) $$a_{i}={\left[P_{i}+\gamma _{i}+2\beta \overline{w}_{i}\right] \over \beta }$$

and

(5) $$G=\left(\overline{w}_{1}\right)^{2}+\left(\overline{w}_{2}\right)^{2}+{U \over \beta} -\left(a_{1} \over 2\right)^{2}-\left(a_{2} \over 2\right)^{2}.$$

Further details are provided in Appendix A. This circular isoquant ensures that the optimal identity expression will include positive values for both w ₁ and w ₂ in the second question. There will be an interior solution, like point B in Figure 1. That is, while the second question in isolation only cues the second identity, the fact that the first identity was expressed in the previous question ensures that at least a portion of the first identity will be expressed in the second question as well. There is identity inertia, where identities expressed in the recent past will tend to be expressed in the present in order to display consistency, regardless of whether the present context explicitly cues that identity.

To see the significance of this result, consider how the isoquant changes if β equals zero, meaning the individual has no desire to display a consistent persona across survey questions. When there is no penalty for inconsistency the isoquant of (1) is linear with a slope of $-\left[P_{2}+\gamma _{2}\right] \over \left[P_{1}+\gamma _{1}\right]$ . This results in a corner solution, shown in Figure 1, where the isoquant of the objective function is on the most upper right coordinate as possible while still touching the identity constraint w ₁ + w ₂ = 1, shown by point A. Here, the respondent expresses only the second identity, where (w ₁,w ₂) = (0,1).

This main purpose of this model is to illustrate how identity inertia works, not to predict identity inertia, as it is built into the model by assuming β > 0. The nature of identity inertia is determined mostly by the value of β and how $\overline{w}_{i}$ is calculated. Theory does not dictate how $\overline{w}_{i}$ is determined, so some process must be assumed. It seems reasonable to assume that if $\overline{w}_{i}$ is an individual’s memory of how often identity i was expressed in the past, and that more recent expressions of the identity matter more than expressions further in the past. Also, there is no reason to believe the process determining $\overline{w}_{i}$ differs according to the value of i. As such, we propose the following equation

(6) $${\rm{Memory}}\;{\rm{of}}\;{\rm{previous}}\;{\rm{expressions}}\;{\rm{of}}\;{\rm{identity}}\,i = {{{\overline w}}_{{{i}},{{t}}}}{{ = }}\sum\nolimits_{{{k \,= \,1}}}^{{{t - 1}}} {{\rho ^{{k}}}} {{{w}}_{{{i}},{{t - k}}}}.$$

where w _{i, t} is the degree to which identity i is expressed in survey question t. The value of ρ is assumed to be 0 ≤ ρ ≤ 1. This equation achieves three things for modeling identity inertia at survey question t. First, because the right-hand side sums over all previous t−1 survey questions it accounts for the entire sequence of questions asked in the survey. Second, $\overline{w}_{i,t}$ becomes larger the more times identity i was activated in previous questions. Third, it gives more weight to recent expressions of identity i.

The main objective of this study is to test for identity inertia in survey responses. This is achieved by designing a survey with questions that sometimes elicits respondents’ ideal self, sometimes their common self, and tests whether the memory variable in (6) has a statistically significant effect on survey responses. This survey design is discussed in the next section.

Methods

The purpose of this section is to describe a statistical test used for detecting identity inertia in surveys. A survey is administered where respondents are asked the extent to which they agree or disagree with a series of statements. Some statements are written to cue the respondent’s ideal self and some for the common self. These are referred to as ideal-activating (IA) and common-activating (CA) questions, respectively. The survey design randomizes whether IA or CA questions come first, as well as how many IA or CA are asked in a sequence. A “memory” variable is constructed that increases in value the greater the number of IA questions answered and decreases in value the number of CA questions. To test for identity inertia, a statistical model is constructed to determine whether this memory variable has a statistically significant effect on agreement with statements.

The effectiveness of this test depends on the ability of the questions to activate the desired identity. That is, the IA (CA) questions must truly cue a respondent’s ideal (common) self for the test to be valid. Two different types of IA/CA questions are used. One concerns general behaviors we believe activates the targeted identity with certainty for most of the respondents and are referred to as non-food issues. The other set concerns food issues and will only invoke the targeted identity for respondents whose ideal self includes humane treatment of farm animals.

Respondents

Data are collected via an online survey through Qualtrics from August to October 2019. Qualtrics provided financial incentives to individuals completing the survey. A representative sample of nearly 2600 was drawn from the U.S. population. After filtering out incomplete responses to the main statements, the sample contains 2354 respondents. Table 1 presents summary statistics of six demographic variables collected from the survey. Additional variables collected but not reported on the table include: average number of residents in household (2.74), average number of children in household (2.52), average number of unemployed in household (1.77), and average number of fully employed in household (1.30).

Table 1. Summary statistics of demographic variables (N = 2600)

Notes: Summation of percentages may not equal 100 due to rounding. Summation of frequencies may not equal 2600 due to missing responses.

While many statistics are similar to the demographic profile of the nation, the sample is not representative of the general public as it is an opt-in survey where individuals must volunteer to participate in the survey in return for various forms of compensation. Yet, because the purpose of the study is to analyze respondent behavior and is not intended to illustrate the attitudes of the nation as a whole, there is no need for a perfectly representative sample or sample balancing.

Survey Format Summary

Respondents were randomly assigned to one of four treatments, each containing different survey questions. All questions were posed as statements where the individual indicated their level of agreement from strongly disagree to strongly agree on a 7-point Likert scale. Respondents were always asked the non-food questions set of statement before the food related questions. Within a group of IA or CA questions the order of the statements is randomized.

Figures 2 and 3 present the nature of the experimental design as well as the questions asked. With food questions, respondents always answered three IA and three CA questions, but depending on the treatment they either answered zero or three buffer statements. Of the non-food questions, some are shown two or six IA statements first, followed by two or six CA statements, and some are given the reverse order. In the treatment with non-food questions, respondents had either zero, two, or six buffer questions (explained subsequently). If buffer questions were provided in either non-food or food statements, they were provided between the IA and CA questions. The IA questions are inspired from the statements used in the Marlowe–Crowne self-reported social desirability scale (Crowne and Marlowe, 1960) in that they describe behaviors most people wish to but fail to always emulate. With the CA food statements, half of respondents in the entire sample saw extended versions these statements to include consequences for these specific issues.

Figure 2. Summary of survey treatments.

Figure 3. Survey questions and question format.

Non-Food Issues

The non-food IA questions deal with general behaviors that contemporary American society approves of. One IA non-food question asks respondents whether they agree that ‘citizens have a moral obligation to vote’. This statement is used under the assumption that most respondents will aspire to be a regular voting citizen. A statement like this was thought to generate the highest degree of support even among those who rarely vote. This question is also used because it has a clear common-self counterpart that asks whether the respondent agrees that ‘I always vote’ (see non-food CA questions in Figure 3). This concerns an actual behavior in a real-life setting where there is a cost to identity expression. It is presumed that many Americans will agree that citizens have a moral obligation to vote but that, if they were being honest, would have to admit they do not always vote (about 40% of eligible voters did not vote in the 2016 presidential election). It is hypothesized that the more IA questions a respondent answers before seeing the statement ‘I always vote’, the more likely they are to agree that they always vote because not doing so would signal two conflicting identities. Confirmation of this hypothesis is interpreted to be confirmation of identity inertia. The remainder of the IA and CA questions in the non-food category have similar features. The IA question concerns something the respondent aspires to be (e.g., honest) but rarely achieves in full (e.g., complete honesty).

For some respondents, a set of “buffer” questions are asked between the IA and CA questions. These contain statements like “I prefer to watch reality shows on television over crime shows” that are relatively neutral in the identity they activate. While even the most mundane question requires some selection of identities, what makes the buffer questions unique is that they do not obviously appeal to the person’s ideal self. The greater the number of buffer questions between a set of IA and CA (or CA and IA) questions, the less impact identity inertia is hypothesized to have on the results.

Before describing the food-related questions, let us begin building an empirical model of the survey responses. Denote Y _it as the expressed agreement by the ith individual to the tth statement where 1 = strongly disagree, 4 = neither agree nor disagree, and 7 = strongly agree (see Figure 3 for an example). The value of Y _it will depend on whether it is an IA or CA question, so let I _it be an indicator variable equaling one if an IA question is asked and zero otherwise, and C _it and B _it be indicator variables for CA and buffer questions, respectively.

A test for identity inertia requires a variable measuring the extent to which previous questions are IA, CA, or buffer questions. This is the “memory” variable akin to (6) in the previous section. We elect to use one variable that increases in value the greater number of previous IA questions, decreases in value the greater the number of previous CA questions, and is reduced in absolute value by previously answered buffer questions (if many buffer questions are asked the memory variable should approach zero as if wiping clean memory of IA questions). This variable is calculated by:

(7) $$M_{it}\!=\!\sum\nolimits _{k\,=\,1}^{t-1}\rho ^{k}(I_{i,t-k}\!-\!C_{i,t-k}).$$

The variable M _it stands for ‘memory’ of past questions. Its value increases the more IA questions are asked in the past, decreases the more CA questions are asked, and is brought closer to zero by the asking of a buffer question, for which I _it = C _it = 0. The value of ρ must be between zero and one and is chosen ex-ante to be 0.8 to represent the ability to recall previous responses to past questions dampens as more questions are asked. Alternative values of ρ are not considered until the first set of hypothesis tests to prevent the deterioration of statistical power due to multiple testing. We set M _i1 = 0 for all i = 1 due to fact that it is the first survey question.

If identity inertia is manifested in the responses, Y _it will be influenced by the variable M _it , and since the effect of M _it might differ for IA and CA questions it is interacted with I _it and C _it . A statistical model can be expressed in terms of a latent linear response, where observed original responses Y _it are generated from the latent continuous response as

(8) $$Y_{it}^{*}=\beta _{1}I_{it}+\beta _{2}C_{it}+\beta _{3}B_{it}+\beta _{4}I_{it}M_{it}+\beta _{5}C_{it}M_{it}+\beta _{6}B_{it}M_{it}+\tau _{i}+\epsilon _{it},$$

where β _i are parameters to be estimated, τ _i is an individual-specific error term that is fixed for each respondent but distributed normally $(\tau _{i}\sim N(0,\sigma _{v}^{2}))$ across respondents, and ϵ _it is a stochastic error term distributed according to the logistic distribution. If there is identity inertia whereby previous IA and CA questions impact responses to any one question, then β ₄ and/or β ₅ will be both statistically significant and positive. This is because both IA and CA questions refer to behaviors most respondents would engage in if they activated their ideal self, and the memory variable M _it increases the more a respondent’s ideal self is activated in previous questions.

An additional variable is needed. If identity inertia is detected it is desirable to have a benchmark as to whether the effect is large, so we deliberately include another documented effect: acquiescence bias. This occurs when people generally prefer to agree to statements than disagree (Tourangeau and Bradburn, 2010). With a reversed valence in the question, a higher value of Y _it indicates less desirable behavior that is not consistent with the ideal self, as opposed to the other questions where a higher value of Y _it is more consistent with the ideal self. As such, for observations with a reversed valence, the value of Y _it is reformatted to 7 − Y _it . This is done so that a higher value of Y _it always indicates greater agreement with a desirable behavior. Nevertheless, because of the acquiescence bias, Y _it will likely be a lower value for reversed valence observations, on average. If V _it is an indicator variable for questions with reversed valence the empirical model now becomes

(9) $$Y_{it}^{*}=\beta _{1}I_{it}+\beta _{2}C_{it}+\beta _{3}B_{it}+\beta _{4}I_{it}M_{it}+\beta _{5}C_{it}M_{it}+\beta _{6}B_{it}M_{it}+\beta _{7}V_{it}+\tau _{i}+\epsilon _{it}.$$

If the absolute value of β ₄ or β ₅ is greater than the absolute value of β ₇, then it can be said (in this specific case) that the ordering effect is larger than the acquiescence effect. This variable also becomes particularly useful in separating ordering effects due to anchoring from those due to Identity Theory, although this was not foreseen when it was originally included in the survey design.

Food Issues

After the individual answers a set of non-food questions, they are presented with a set of IA and CA questions regarding a food-related issue: animal welfare. The topic of animal welfare is chosen because it was a recent animal welfare survey that helped inspire this research. In 2017, the Animal Sentience Institute administered a survey reporting that almost half of Americans said they support a ban on slaughterhouses and one-third support a ban on animal farming (Anthis, 2017). Given that less than 7% of Americans are vegetarian or vegan (Lusk and Norwood, 2016), those results seem unrealistic—so unrealistic one of the authors replicated the survey but found the same results. That number of people truly wanting to eliminate meat is highly doubtful, so the source of this strange result might be the function of the survey design.

Why would so many meat-eaters say they wish to ban slaughterhouses and animal farming? One explanation might be the ordering of the questions, which was not randomized. Instead, respondents were first given a number of questions that, for some, would seem to cue their ideal self. These include the following statements: (1) People should consume fewer animal-based foods (meat, dairy, and/or eggs) and more plant-based foods (fruits, grains, beans, and/or vegetables), (2) I have some discomfort with the way animals are used in the food industry, and (3) Farmed animals have roughly the same ability to feel pain and discomfort as humans. These questions do not ask about the respondents’ actual eating behaviors but the behaviors they may aspire to. While not everyone aspires to be a vegetarian, health professionals have been urging Americans to eat more plants for some time, climate change groups are increasingly blaming animal agriculture for global warming, and the geist of the last few decades is toward a greater concern for animal treatment. Perhaps some respondents in Anthis (2017) agreed with these first set of questions because it asked about their aspirations and cued their ideal self, but when confronted with a question about their actual eating habits—cueing the common self—the desire to not appear hypocritical induced them to activate a mixture of their ideal and common self. As such, many of them claim they are against the raising and slaughter of animals for food consumption when their eating habits say otherwise. That is, perhaps identity inertia is responsible for the large support on livestock and meat production bans? This thought and exposure to Identity Theory prompted the generated hypothesis that Identity Theory causes some forms of ordering effects.

To test this, respondents were administered three questions about livestock treatment that cue the ideal self and three that cue the common-self, with the order of the selves activated randomized (unlike the Animal Sentience survey). Some respondents also answer a few buffer questions between the two identity types. The exact questions used are shown in Figure 3, and all six questions used for the ideal and common statements regarding animal welfare are pulled verbatim from the Animal Sentience Survey (Anthis, 2017).

The statistical model explaining answers to the food-related questions are the same as the model for the non-food questions, except there are no observations with reversed valence. Because the food and non-food questions concern such different topics, separate parameters are estimated for each. If we let the g and f subscript denote the non-food and food-related questions, respectively, the empirical model for the survey responses can be written as

(10) $${Y_{it}^{*}\!=\!\sum\nolimits_{k\,=\,g,f}\left(\beta _{1,k}I_{it}+\beta _{2,k}C_{it}+\beta _{3,k}B_{it}+\beta _{4,k}I_{it}M_{it}+\beta _{5,k}C_{it}M_{it}+\beta _{6,k}B_{it}M_{it}\right)+\beta _{7,g}V_{it}+\tau _{i}+\epsilon _{it}}.$$

One final variable is needed for the food questions. It might be that many people indicated support for banning slaughterhouses and eliminating animal farming in the Animal Sentience survey because they are not sure what such actions imply for the food supply. Without animal farming and slaughter, there can be no meat consumption. To test this, roughly half of the subjects are given the CA statements exactly as they appeared on the Animal Sentience survey, while the other half contain an addition shown in square brackets in Figure 3. For example, some see the statement “I support a ban on slaughterhouses” while others see the statement “I support a ban on slaughterhouses and will stop eating meat”. The addition to each of the three CA questions is intended to clarify the consequences of the measure proposed. Let A _it be an indicator variable identifying questions with this addition. The statistical model now becomes

(11) \begin{align}Y_{it}^{*}\! &= \!{\sum\nolimits _{k{\kern 1pt} = {\kern 1pt} g,f}}\left(\beta _{1,k}I_{it}+\beta _{2,k}C_{it}+\beta _{3,k}B_{it}+\beta _{4,k}I_{it}M_{it}+\beta _{5,k}C_{it}M_{it}+\beta _{6,k}B_{it}M_{it}\right)+\beta _{7,g}V_{it}\\ &\quad +\beta _{8,f}A_{it}+\tau _{i}+\epsilon _{it}.\end{align}

Note that the memory variable M _it is not reset to zero when the survey transitions from non-food to food issues, so whatever identity inertia exists in the non-food issues will extend to the food issues as well. As mentioned previously, the IA food questions may only activate the ideal self for a subset of the sample, as not everyone aspires to valuing animal welfare and reducing meat from their diet. To account for this, (11) is also estimated for a subset of the data including only respondents who responded, “somewhat agree”, “agree”, or “strongly agree” to any one of the three IA food questions. This reduces the number of respondents from 2354 to 1941.

Habitual Inertia

A type of inertia different than identity inertia could be present in the survey responses. If a subject indicates high levels of agreement to one set of questions, they may tend to also provide high levels of agreement to the next set of questions simply due to habit. We refer to this as habitual inertia. To help differentiate between effects from identity and habitual inertia, for some of the survey respondents, the CA questions for non-food statements are reversed in valence, meaning instead of the statement “I always vote” it says, “Sometimes I fail to vote”. If the IA questions are asked first, habitual inertia will tend to result in higher levels of agreement with these statements, while identity inertia will lead to lower levels of agreement. Then, as the dependent variable for these questions are transformed to equal new value = 8 − old value, for these questions, habitual inertia will lead to lower dependent variable values while identity inertia will lead to higher values.

Hypotheses

The variable M _it measures the type of questions individual i faced previously to question t. The average value of M _it in the data is approximately zero because of the balanced nature of the randomization of IA and CA questions. Its standard deviation is 1.55, and its minimum and maximum values are −3.41 and 3.41, respectively (the minimum and maximum have similar absolute values due to the balanced randomization). Our model predicts that if identity inertia exists, as the value of M _it rises, indicating more IA questions in the recent past, the higher the value of Y _it , indicating greater agreement with the statement. This would mean a person is likely activating more of their ideal self. Conversely, the lower the value of M _it , the lower the value of Y _it and activation of the common self, if identity inertia is present. As such, we hypothesize that a statistically significant and positive value of β _{4, g} , β _{5, g} , β _{4, f} , or β _{5, f} indicates identity inertia.

For observations with reversed valence, since the coefficients β _{5, g} and β _{5, f} correspond to the interaction terms for CA questions and the memory variables, habitual inertia will act to reduce and identity inertia will act to increase the value of these coefficients. As such, negative values for these coefficients indicate a dominance of habitual inertia, positive values indicate dominance of identity inertia, and insignificant coefficients indicate either the absence or the canceling out of both types of inertia. However, for observations without reversed valence habitual inertia will act to increase the values of β _{4, g} , β _{5, g} , β _{4, f} , or β _{5, f} . Consequently, if habitual inertia is detected for the reversed valence observations it suggests the coefficients for the non-reversed observations cannot be used as a pure test of identity inertia.

There are a few other hypotheses not directly relevant to Identity Theory. The acquiescence effect should result in a statistically significant and negative value for β _{7, g} , if the respondents generally prefer to agree rather than disagree with statements, even if agreement and disagreement imply the same attitude. The additional information that banning factory animal farming would lead to higher food prices and banning slaughterhouses and animal farming would, for the most part, eliminate meat as a food should reduce agreement, thus we hypothesize that β _{7, f} should also be statistically significant and negative. Finally, we hypothesize that β _{1, k} > β _{2, k} for all k, as we believe individuals will be more eager to express their ideal self than their common self.

Results

All models are estimated with STATA 16.1 (StataCorp, 2019). Estimates of the ordered logit coefficients along with their p-values are shown in Table 2. Any coefficient with a p-value less than 0.05 is deemed statistically significant. Recall that identity inertia is detected if β _{4, g} , β _{5, g} , β _{4, f} , or β _{5, f} are statistically significant and positive. The table shows positive values and p-values less than 0.05 for β _{5, g} and β _{5, f} , implying that the greater the number of questions respondents have answered cueing their ideal self (relative to the number cueing their common self), the greater the agreement on questions cueing the common self. That is, identity inertia is indeed detected when the respondent had already seen questions concerning their ideal self (like whether voting is a moral virtue) and is then asked about their common self (like whether they actually voted). This conclusion is reached regardless of whether the full sample or the subset of the sample is used (description of the subset provided shortly).

Table 2. Results of random-effects ordered logit and random-effects generalized least squares estimations

^a Coefficient is normalized to zero for model identification.

Notes: The number of respondents for the full sample and the subset of the sample is 2354 and 1941, respectively. The subset includes only those responses who indicated agreement (as opposed to disagreement or neither) to at least one IA question concerning food issues.

Identity inertia is not detected in questions invoking the ideal self, but as the coefficients are positive but insignificant, this may not be a rejection of identity inertia but an inability to detect it. Note that the significant and positive value of β _{6, g} suggests identity inertia for buffer questions, but this is not something identity inertia predicts (but habitual inertia does).

Likelihood ratio tests conclude that β _{1, g} > β _{2, g} and β _{1, f} > β _{2, f} at the 5% level, as expected, indicating people indicate higher agreement to IA than CA questions. Table 2 shows the coefficient β _{7, g} is insignificant at the 5% level, indicating the acquiescence bias detected in other studies is not present here. This was going to be used as a benchmark for gauging the size of any identity inertia effect found, so this suggests the identity inertia effect might be large compared to other known biases. As expected, the coefficient β _{7, f} is statistically significant and negative, meaning that once individuals are informed about the implications of actions like banning slaughterhouses, they are less eager to do so.

The last column in Table 2 also provides random-effects generalized least square (GLS) estimates. This is similar to ordinary least squares (OLS) except that the stochastic error is assumed to have a component that is fixed for each respondent but random across respondents. While GLS is generally not preferred for discrete dependent variables, the coefficients are easier to interpret and facilitates an understanding of the magnitude of the effects. First note that coefficients which are statistically significant in the ordered logit estimation are usually significant in the GLS estimation as well. The GLS coefficients provide a direct interpretation of how the coefficients increase or decrease the ratings of agreement to the questions. Recall that β _{5, g} and β _{5, f} are positive and significant in the ordered logit estimation, which is evidence for the identity inertia effect. The dependent variable takes values of 1 for strong disagreement to 7 for strong agreement. The GLS coefficients for β _{5, g} and β _{5, f} are 0.036 and 0.040, respectively, and the standard deviation of the memory variable is about 1.55. As such, an increase in the memory variable of one standard deviation only increases the dependent variable for by about (0.04) (1.55) = 0.062. This is a very small effect, suggesting that even in the case where identity inertia is detected, its impact on survey responses is negligible, and researchers can probably design surveys without worrying about identity inertia effects.

Overall, the results in Table 2 suggest that identity inertia of the kind predicted in our conceptual model can arise—but will not always arise—in survey responses, but is too small for concern. To evaluate the extent to which this finding is robust, we conduct a number of sensitivity analyses. The estimates in Table 2 do not contain any explanatory variables for the demographics of the respondents. To test how doing so would impact the results, the ordered logit was estimated multiple times with different sets of demographic explanatory variables at each estimation. For example, one estimate adds six dummy variables for age to the model in Table 2, another adds eight categories for marital status, and another adds 61 demographic variables reflecting the subjects’ age, marital status, income, ethnicity, household size, employment status, education attainment, and number of children.

A total of nine different models with different sets of demographic variables were estimated, and in all but one the coefficients β _{5, g} and β _{5, f} were significant and positive, suggesting the detection of identity inertia for CA questions is robust to alternative model specifications. However, the magnitude of the effects remained small for all models, reinforcing the idea that it is not a practical problem in survey design.

Recall the possibility of habitual inertia, where respondents continue indicating high levels of agreement simply because they did so in the past. If the model is estimated using only the observations with reversed valence, the coefficient β _{5, f} will be affected by both identity and habitual inertia but in opposite directions. If the coefficient is significant and positive, then identity inertia is the stronger force; if significant and negative then habitual inertia is a stronger force; and if insignificant the two effects are either absent or cancel each other.

When the ordered logit in Table 2 is estimated using only the observations with reversed valence (including only non-food questions) the coefficient β _{5, f} is significant and negative, suggesting that most of the effects in the memory variable are actually influenced by habitual inertia and not identity inertia. The magnitude of the effect is rather small though. The average of the memory variable is close to zero and the standard deviation is 1.55, so the GLS results in Table 3 suggest that a one standard deviation increase in the memory variable decreases the dependent variable by about (−0.114) (1.55) = −0.18. Given that the dependent variable’s range is [0,7], this is a relatively small change, one that would rarely alter the response of any one survey respondent.

Table 3. Results of random-effects ordered logit and random-effects generalized least squares estimations using only observations with reversed valence for common activating non-food questions (N = 1941)

^a Dropped for model identification.

We thus conclude that when the IA questions cue the individual to indicate agreement, CA questions that follow tend to have greater agreement, even when identity inertia would predict the opposite. As such, we conclude that while a strong theoretical case for identity inertia is made, we cannot detect identity inertia effects in the data, and whatever influence identity inertia may have it is small compared to ordering effects caused by habitual inertia—and even the impacts of habitual inertia are small in magnitude.

Recapitulation

Even if readers are encountering the vocabulary of Identity Theory for the first time, they are (perhaps unknowingly) already aware of what the theory argues and what it implies for survey design. The notion that we express different personas in different contexts is self-evident, like when a professor has their child as a student and plays the professor role in class and the parent role at home. Ordering effects caused by Identity Theory are intuitive as well. With some reflection, the reader will likely recall conversations where the order of the questions is deliberately manipulated to create identity inertia for achieving a desired result.

That is the essence of this study: that the ordering of survey questions can manipulate the individual to answer a certain way. This study performs two actions to advance this well-known phenomenon and motivate its explicit consideration in survey design. First, we provide a theoretical explanation for some (but not all) ordering effects in surveys. Second, we provide a set of vocabulary researchers can use in describing why they order questions a particular way in surveys, as articulating why survey questions are listed in a specific order will not only help researchers defend their methods, but will aid researchers in ensuring the survey design is consistent with their research objectives.

What this study cannot do is empirically detect identity inertia in the survey data, though. Instead, we find ordering effects are caused more by habitual inertia, where individuals tend to agree (disagree) with statements more if they agreed (disagreed) with previous statements. Either identity inertia is either not empirically prevalent or their detection is made difficult by an overwhelming influence of other ordering effects. Future research testing for identity inertia should place greater emphasis on separating ordering effects caused by identity inertia and those caused by habitual inertia.