An item response theory and factor analytic examination of two prominent maximizing tendency scales

Abstract

The current study examines the construct validity of the Maximization Scale (MS; Schwartz et al., 2002) and the Maximization Tendency Scale (MTS; Diab et al., 2008) as well as the nomological net of the maximizing construct. We find that both scales of maximizing suffer psychometrically, especially in their proposed dimensionality. Using confirmatory factor analysis and item response theory (IRT) we identify and remove three problematic items from the MTS and six problematic items from the MS. Additionally, we find that the MS appears to be measuring difficulty and restlessness with the search for the best alternative, whereas the MTS is more focused on the search for the best option, regardless of choice difficulty. We then examined these revised scales in relation to other psychological constructs in the nomological net for maximizing and found that maximizers may not be unhappy but are generally distressed in the decision-making context. Finally, we suggest that future maximizng research use revised form of the MTS that seems to us to be most consistent with the original concept of maximizing/satisficing.

1 Introduction

The conceptualization and measurement of the maximizing construct has received considerable attention in the last ten years (Schwartz et al., 2002; Nenkov et al., 2008; Diab, et al., 2008; Lai, 2010; Rim, et al., 2011). Traditional economic models of choice theorized that individuals pursue a maximization goal in decision-making contexts. However, in an evaluation of the existing data and economic models Simon (1956) stated, “Evidently, organisms adapt well enough to ‘satisfice’; they do not, in general, ‘optimize’ .” Building on this research, Schwartz et al. attempted to define the psychological effects of maximizing for those who pursue maximizing goals. Specifically, they theorized that in environments with a lot of choice, individuals with a maximizing goal would likely be unhappy and regret their decisions. Schwartz et al. developed a 13-item measure of maximization (Maximization Scale; MS) and found that scores from the MS correlated positively with depression, perfectionism, and regret and correlated negatively with happiness, life satisfaction, optimism, and self-esteem.

Since the development of this scale, there has been considerable debate about the validity of the measure and the development of alternative measures. Diab et al. (2008) developed a new maximization scale (Maximizing Tendency Scale; MTS) and found that the MTS did not correlate with constructs such as depression, life satisfaction, and neuroticism. However, their scale did correlate positively with regret. Nenkov et al. (2008) modified the MS and created a 6-item scale that was found to have better psychometric characteristics than the original 13-item scale. Lai (2010) also developed a new scale of maximizing and found that it correlated positively with optimism and need for cognition, but the correlation between her measure of maximizing and regret was inconsistent across samples. Finally, Rim et al. (2011) examined both the MS and MTS using item response theory (IRT) and found that both scales had weakness in measuring the maximizing construct. They also found that the MTS was not unidimensional as proposed by Diab et al. (2008). However, Rim et al. did not discuss removing problematic items. The purpose of the current study is to examine both the MS and MTS using exploratory and confirmatory factor analysis, and then use polytomous IRT to resolve the problems found in the scales. In addition, we test whether these solutions can answer the question of whether maximizers are happy or unhappy.

1.1 Construct validity of maximizing

Schwartz and colleagues (2002) have changed the perspective of the maximizing and satisficing constructs by departing from both economic models description of maximizing choice strategy (von Neumann & Morgenstein, 1944), and Simon’s (1955; 1956) view that all decision makers would satisfice in order to adapt to their environment. Schwartz et al.’s revised perspective is that both maximizing and satisficing represent choice-behavior tendencies performed by decision makers depending on their standing on the maximizing construct. In addition, Schwartz et al. (2002) focused on the degree to which maximizing is associated with regretting decisions. They proposed that satisficers and maximizers differ in their sensitivity to regret because of differences in investment and goals in the decision making process. For maximizers, the potential for regret can increase as a consequence of two factors. The first is the potential for failing to find the best option after spending much time and effort in searching for the very best alternative. The second is the potential for failing to choose the very best option in spite of the amount of available choice in the market place. Therefore internalizing the failure as reflecting the decision makers’ inability to optimally make a decision would yield great dissatisfaction. On the other hand, satisficers have the goal of finding a good enough alternative that has crossed the decision maker threshold, consequently, the time and effort spent by satisficers during the choice process is much more modest. Thus, satisficers are likely to experience less dissatisfaction, not only because their investment is modest, but also because their goal does not elicit unrealistic expectations.

Schwartz and colleagues performed a series of correlational studies to provide evidence for the differentiation of these two groups (maximizers and satisficers), not only with reference to the choice tendency, but also in relation to a variety of other psychological constructs. The other dimensions in the nomological net of maximizing were subjective happiness, which assesses dispositional happiness (Lyubomirsky & Lepper, 1999); depression (Beck & Beck, 1972); life orientation, which assesses dispositional optimism (Scheier & Carver, 1985); satisfaction with life (Diener, et al., 1985); dispositional neuroticism (John, et al., 1991); self-oriented perfectionism (Hewitt & Flett, 1990; 1991); and self-esteem (Rosenberg, 1965).

Findings for the validity of the MS, based on the relationships between maximizing and the aforementioned constructs in the nomological net showed that maximizers experience less satisfaction, happiness, optimism, and self-esteem than satisficers. In addition to that, maximizers were found to experience more regret, depression, and tendency towards perfectionism than satisficers. No gender difference was found in four out of the seven samples, and in the other three samples, males were more likely than females to be maximizers. Schwartz et al. argued to have demonstrated support for the satisficing/maximizing construct, but some of the psychometric results of the MS were not strong.

These less than optimal psychometric results have motivated researchers to re-evaluate the scale measuring the maximizing/satisficing construct. A particular re-evaluation of the MS and its psychometric properties came from Diab and colleagues (Diab et al., 2008). More specifically, Diab et al. have indicated that there are psychometric and conceptual irregularities with the MS. First, they indicated that the MS falls short of commonly accepted psychometric standards. Second, they suggested that there was not a clear connection between the theory of maximizing and satisficing and the MS. As reported by Diab et al. (2008), even though the theoretical basis for the original maximizing scale is Simon’s (1955) definition of maximizing representing the optimization goal, many of the items that compose the MS seem to diverge from this definition. For instance, items such as having “difficulty writing letters to friends” and “preference for ranking things like movies” do not seem to fit conceptually with an optimizing goal definition. Third, Diab and colleagues argued that the Schwartz et al. (2002) conclusion that the tendency to maximize was correlated with being less happy was a reflection of how the construct was measured, and not a reflection of the construct itself.

Diab et al. (2008) addressed the above criticisms by developing the MTS. This scale was expected to better represent and measure the constructs of maximizing and satisficing. The MTS has three items from the original MS and six new items that tap into the definition of maximization as an “optimization goal”. Furthermore, Diab et al. examined the correlation between MTS and measures of indecisiveness, avoidance, regret, neuroticism, and life (dis)satisfaction. Results showed clear differences between the original MS and the new MTS. First, they found that the MTS demonstrated substantially greater internal consistency reliability than the MS (MS α = .58; and MTS α = .80). As predicted, the MTS was largely unrelated to maladaptive personality and decision-making constructs. More specifically, MTS did not correlate with indecisiveness, avoidance, neuroticism, and life (dis) satisfaction, except regret. Although, the correlation between tendency to maximize and regret was lower for the MTS (r = .27) than observed for the original MS (r = .45). In sum, Diab and colleagues presented a different version of maximizing that revealed better psychometric properties, and brought to the literature new findings. However, the two most important distinctions between both scales are theoretical. First, Schwartz et al. (2002) regard the construct as multidimensional, involving multiple goals and aspects of the decision maker, and Diab et al. (2008), on the other hand, view the maximizing construct as a unidimensional measure that reflects the goal of finding the very best. Second, the findings for the validity of the MS suggested that the construct of maximizing is associated with unhappiness, however, the MTS was not related to unhappiness or other maladaptive measures, except regret.

It is interesting that the MS and the MTS, which ostensibly measure the same construct, produce different results regarding the relationships between maximizing and the other constructs in the proposed nomological net. Although the work thus far has been focused on improving the measurement of the maximizing construct, questions still remain as to our conceptual understanding of the construct itself. Given the diverging results from the previous work, it may be plausible that maximizing and satisficing are separate constructs rather than polar ends of a single construct continuum. We will focus on better understanding of the maximizing and satisficing perspective through both MS and MTS.

For Schwartz et al. an important difference between maximizing and satisficing is in the choice goal. They even suggested that, “a satisficer often moves in the direction of maximizing without ever having it as a deliberate goal” (Schwartz et al., 2002, p. 1178). Therefore, satisficers could be maximizers, as measured in terms of effort, time, or even aspiration level, but they differ from maximizers in the sense that they do not have the goal of optimizing (i.e., finding the very best option). If that is the case, the psychometric measure developed to assess the distinction between these two groups of people should focus on the goal of optimization. Examining the MS, the questions do not seem to be associated with the goal of optimization. In fact, most of the questions seem to tap into the difficulty of finding the best choice (e.g., “I treat relationships like clothing, I expect to try a lot on before I get the perfect fit” see also questions 7, 8, and 10). Some items also appear to be addressing restlessness towards one’s current choice or state (e.g., “I often fantasize about living in ways that are quite different from my actual life”; “No matter how satisfied I am with my job, it’s only right for me to be on the lookout for better opportunities” see also questions 1 and 2).

It has been shown in the literature that unhappiness is associated with restlessness (Levenson & Neuringer, 1971; Arens, 1982). Considering that, it should not be a surprise that the MS is positively associated with life dissatisfaction. The question now would be why unhappiness or restlessness would necessarily be associated with people who have the choice goal of finding the best option? One answer provided by Schwartz (2004) indicates that there are too many alternatives available for people to choose from, which leads to feelings of anxiety and frustration. This process has been labeled as choice overload, which is operationalized in studies of choice by increasing the the number of options to choose from. On the other hand, there is controversy in the literature on the existence of choice overload. Some studies have found evidence for it,whereas other studies have not found such evidence (Scheibehenne, et al., 2010). In fact, Scheibehenne et al. presented a meta-analytic review of choice overload and found little evidence for such an effect. The absence of choice overload raises problems for the previous question on the association among unhappiness, restlessness, and maximizing tendencies to remain unanswered, but also inquiries on the association between the MS and the concept of optimization as a goal.

Doubts about the association between maximizing and dissatisfaction became even more evident after Diab et al.’s (2008) findings that the construct of maximizing was not associated with life dissatisfaction. Therefore, we believe the association between MS and life dissatisfaction is probably due to the difficulty and restlessness incorporated in all the questions of the scale. What would then explain the lack of association between the MTS and life dissatisfaction? Examining the MTS questions, we can make some inferences. First, none of the items addresses difficulty with the choice or restlessness. By restlessness, we mean the constant search for something better, even if one is supposedly happy with one’s choice. Second, most of the questions focus on the idea of optimizing and not settling. As a result, we believe that the MTS measures an individual’s tendency to search for the best option and is silent on the issue of choice difficulty. It could be that individuals who score high on the MTS enjoy complex thinking and therefore do not find optimization to be stressful. In conclusion, we believe it is reasonable to think that MS measures frustration with decision difficulty and restlessness when searching for the best option whereas MTS just measures the individuals’ tendency to search for the best option.

1.2 Development and refinement of the maximizing scales

Basing their definition of a maximizer from Simon’s conceptualization of the term, Schwartz et al. (2002) began the creation of a measuring instrument for the constructs of maximization and regret. Beginning with a 42-item questionnaire, the number of items was soon reduced to 22 based on item reliability and face validity. Schwartz et al. conducted a principal components analysis (PCA) with the goal of establishing a scale where all the items would load on to a single component. However, the analysis produced a three component model with 4-items that had low item-total correlations that were then eliminated. This realization lead to the construction of the 13-item scale that Schwartz and colleagues divided into three components that they felt could accurately measure the characteristics of maximizers. These components are choice difficulty, difficulty with large numbers of options, and high standards.

Nenkov et al. (2008) conducted an analysis of the reliability, factor structure, and validity of the Schwartz et al. MS and created a short form of the scale. Following the original analysis, they found that several of the items qualified for removal, as they scored low based on commonly accepted psychometric standards. Nenkov et al. also questioned the factor structure of the original scale. They performed a confirmatory factor analysis (CFA) on three newly developed factors, alternative search, decision difficulty, and high standards and found that some items loaded on to more than a single factor and thus did not support the proposed model. They also found that six of the items had factor loadings below .50. Nenkov et al. concluded that, despite the scale having sufficient internal consistency and construct validity, it is not unidimensional. However, they did not report the correlation between the factors. They performed a second analysis to identify problematic items in the maximization scale and to develop a short form of it. Items were evaluated based on “external, internal and judgmental criteria” (p. 376). For the creation of a short scale, items were selected based on their assortment into the three dimensions. Results showed that the 6-item scale fit the proposed model significantly better while retaining good levels of reliability and nomological validity. Similar to the original 13-item scale, they found negative relationships between the new short scale of maximizing and life satisfaction, happiness,and found a positive relationship with regret. Thus, it appears, even with the 6-item scale, that those who are high on maximizing may be happy and regret their choices more than those who are low.

As stated previously, Diab et al. (2008) felt that the maximization scale was lacking in its reliability and validity. Diab et al. constrained the development of the MTS to searching for an optimal alternative. Specifically items were generated based on this definition, “I am uncomfortable making decisions before I know all of my options.” In addition, they stated that their scale was unidimensional, which they theorize is an important characteristic of measuring maximizing. Results of their analysis found that the MTS had considerably higher levels of internal consistency and did not correlate with life satisfaction, avoidance, neuroticism, and indecisiveness. However, their scale did correlate positively with regret. However, in their study the MS exhibited similar relationships found in previous studies. Therefore, it is possible as we stated that MTS and MS are not measuring the same construct.

Rim et al. (2011) reevaluated the MTS and MS, but differed from previous papers evaluating the scales by performing IRT analysis. Before performing IRT, they examined the factor structure of both scales using Exploratory Factor Analysis (EFA) and CFA. Results of the EFA indicated that a three-factor model fit the data well for the maximization scale. Results of the CFA indicate that a three-factor solution fits the data the best; however, their CFA revealed that two of the factors (decision difficulty and high standards) are likely not measuring the same underlying construct.

Diab et al. (2008) claimed that the MTS is unidimensional. Results of the EFA indicated that a three-factor model fit the data the best; however, there was evidence of overfactoring, which occurs when the major factor is estimated, but addtional factors are poorly estimated. Results from the CFA indicated that a three-factor model fit the data the best, but that the factor loadings were not interpretable and thus they concluded that the factor structure is unspecifiable.

Rim et al. (2011) applied IRT analysis to the MTS and MS, specifically using the graded response model (GRM; Samejima, 1969). The results of their analyses indicated that many of the items did not provide adequate discriminability. Specifically they set a standard of item discriminability and found that only four items between both of the scales met or exceeded this standard. In addition, they claimed that the current maximizing scales might in fact be measuring satisficing rather than maximizing.

1.3 Current study

In addition to addressing the construct validity of the both the MTS and MS we also wanted to address the psychometric properties of the scales. In the current study, we extend the work done by Rim et al. (2011) by using EFA, CFA, and IRT to revise the scales on the basis of their dimensionality and item information parameters. We do this for three reasons. First, by reducing the number of items in both scales we are seeking to reduce the number of factors in both scales. Specifically, by removing items that contain little or no information from the MTS we predict that the scale will better fit a one-factor solution. Second, by removing items that contain little information the new revised scales will be a more parsimonious scale of the maximizing construct. Although, removing any item from a scale, even bad items, reduces the overall information of the scale, the goal of this analysis is to produce the most parsimonious scale. Third, reducing the number of items in both these scales has practical value because both these scales can be given in experiments easily, where the experimental data can help resolve differences between these scales. Finally, we preserved the original response scale structure of the MTS and MS in our study as 5-category and 7-category (respectively) response scales to maintain consistency with their original forms.

We will first subject both the MTS and MS to an (EFA) and a (CFA) to determine their dimensionality in our sample. We will then use item response theory (IRT) to examine the information that each item contributes to the maximizing construct. Low information items will be removed from each scale in an effort to improve the overall reliability and construct validity of the measures. Finally, we will examine the MTS and MS and their revised versions in comparison to constructs commonly found in the maximizing literature.

2 Method

2.1 Samples

Participants were undergraduate students from a large Midwestern U.S. university (N = 751) and a large Northeastern university (N = 234) in the United States. No demographic information was collected. Participants with missing data on either the MS or MTS were removed from the data, leaving a total sample size of 948 individuals. The samples were randomly divided in half for the EFA and CFA models (N = 474 each). The IRT and correlation analyses used the combined sample of 948 individuals.

2.2 Measures

Maximizing Tendency Scale (MTS). The MTS (Diab et al., 2008) is composed of nine items that are designed to measure an individual’s tendency toward making optimal decisions. The items are rated on a 5-point response scale with options ranging from strongly disagree (1) to strongly agree (5). Higher scores indicate a greater tendency toward maximizing. The reliability of the MTS in our full (combined) sample was α = .78.

Maximization Scale (MS). Like the MTS, the MS (Schwartz et al., 2002) is composed of thirteen items that are designed to measure an individual’s tendency toward making optimal decisions. The items are rated on a 7-point response scale with response options ranging from completely disagree (1) to completely agree (7). Higher scores indicate a greater tendency toward maximizing. The reliability of the MS in our full (combined) sample was α = .71.

Indecisiveness. The indecisiveness scale (Frost & Shows, 1993) is composed of 15 items that are designed to measure compulsive indecisiveness. The items are rated on a 5-point response scale with options ranging from strongly disagree (1) to strongly agree (5). Higher scores indicate greater levels of indecisiveness. The reliability of the indecisiveness scale in our full (combined) sample was α = .84.

Avoidant Decision Making. The avoidant decision making measure (Scott & Bruce, 1995) is composed of five items that are designed to measure the extent to which an individual puts off making an important decision. The items are rated on a 5-point Likert–Type response scale with options ranging from strongly disagree (1) to strongly agree (5). Higher scores indicate greater levels of avoidance. The reliability of the avoidant decision making measure in our full (combined) sample was α = .90.

Regret. The Schwartz Regret Scale (Schwartz et al., 2002) is composed of five items that are designed to measure regret following a decision. The items are rated on a 7-point response scale with options ranging from completely disagree (1) to completely agree (7). Higher scores indicate greater levels of post–decision regret. The reliability of the regret scale in our full (combined) sample was α = .65.

Neuroticism. The Goldberg Neuroticism Scale (Goldberg et al., 2006) is composed of 20 items presented as short statements that would describe an individual as generally depressed, moody, doubt–filled, etc. Participants are asked to respond to each statement using a 5-point response scale ranging from strongly disagree (1) to strongly agree (5). Higher scores indicate greater levels of neuroticism. The reliability of the neuroticism scale in our full (combined) sample was α = .93.

Life Satisfaction. The Satisfaction With Life Scale (Diener et al., 1985) is composed of five items that are designed to measure the extent to which an individual is satisfied with with the current conditions in his or her life. The items are rated on a 5-point response scale with options ranging from strongly disagree (1) to strongly agree (5). Higher scores indicate higher levels of satisfaction. The reliability of the life satisfaction scale in our full (combined) sample was α = .85.

Depression. The Center for Epidemiological Studies Depression Scale (Cole et al., 2004) is composed of 20 items that are designed to measure depression-related feelings that an individual has been having in the past week. Responses based on frequency of the feelings and are categorized into five options that are scored from zero to four. Higher scores indicate more depression-related feelings. The reliability of the depression scale in our full (combined) sample was α = .93.

Subjective Happiness. The subjective happiness scale (Lyubomirsky & Lepper, 1997) is composed of four items designed to measure a general (dispositional) level of happiness. The items are rated on a 7-point response scale with response options tailored to each item. Higher scores indicate a greater levels of happiness. The reliability of the subjective happiness scale in our full (combined) sample was α = .82.

Optimism. The Optimism Scale (Scheier et al., 1994) is composed of six items designed to measure general feelings of optimism for future events. The items are rated on a 5-point Likert–Type response scale with options ranging from strongly disagree (1) to strongly agree (5). Higher scores indicate greater levels of optimism. The reliability of the optimism scale in our full (combined) sample was α = .80.

Need for Cognition. The Need for Cognition Scale (Cacioppo et al., 1984) is composed of 18 items that are designed to measure the extent to which individuals enjoy engaging in effortful cognitive tasks. The items are rated on a 5-point response scale with options ranging from extremely uncharacteristic of you (not at all like you) (1) to extremely characteristic of you (very much like you) (5). Higher scores indicate greater enjoyment with cognitive tasks. The reliability of the need for cognition scale in our full (combined) sample was α = .86.

2.3 Analyses

To test the dimensionality of the MS and MTS, we employed an ordinal exploratory factor analysis followed by an ordinal confirmatory factor analysis using LISREL v8.8 (Jöreskog & Sörbom, 2006). IRT analyses were performed using MULTILOG (Thissen, 1991). The IRT analysis was done using Samejima’s graded response model (GRM; Samejima, 1969; 1996). The GRM is an IRT model designed to assess constructs measured with scales using multiple ordered categories, or polytomous response scales (e.g., Likert-type scales), and is an extension of the two-parameter logistic model for binary response items.

Using the GRM, an individual’s likelihood of responding in a particular response category is derived using a two-step process which first identifies category boundary functions for j−1 response categories for each item. These functions are derived using Equation 1 (adapted from Embretson & Reise, 2000).

(1)

In Equation 1, P_ix (θ) is the probability that an individual with a trait (construct) level θ will respond positively at the boundary of category j for item i where x = j = 1 … m_i. Theta (θ) represents the individual’s trait (construct) level, a_i represents the item discrimination or slope, and b_ij represents the category location or difficulty parameter with respect to the trait continuum. Importantly, the values of b_ij should be successive integers reflecting increased difficulty in progressing through the response options in well-functioning items.

In the second step of the GRM, the probability of responding in a particular category is determined using category response functions, which are derived by subtracting P_ix(θ) from the following category. This process is illustrated in Equation 2 (adapted from Embretson & Reise, 2000).

(2)

The first category is determined by subtracting P _i1θ from 1.0, and the last category is equal to P_im(θ).

3 Results

3.1 Factor analysis of the MS

To test the dimensionality of the MS, we used our EFA sample (N = 474) and employed an ordinal EFA followed by an ordinal CFA using LISREL v8.8 (Jöreskog & Sörbom, 2006). We generated three separate exploratory factor analyses (one, two, and three factor solutions) to understand how the individual items loaded on the respective factors. A promax (oblique) rotation was employed in the three factor analyses. The single factor solution suggested that several items loaded inversely on the factor and at least one item did not load at all. The two-factor solution resulted in over-factoring and insufficient factor loadings for several items making the solution uninterpretable. Based on these results, the two-factor solution was not regarded as acceptable and was not included in the subsequent confirmatory factor analyses. The three-factor solution produced an interpretable factor structure, however several items had very low factor loadings. Factor loadings are presented in Table 1.

Table 1: Ordinal exploratory factor analysis (EFA) of the Maximization Scale (N = 474).

We followed the exploratory factor analysis with an ordinal confirmatory factor analysis with a separate, hold-out sample. Due to the nature of the response scales used in the MTS (categorical, Likert-type response options), we used weighted least squares (WLS) approximation and polychoric correlations with the asymptotic covariance matrix as the weight matrix in the models. To evaluate model fit, we used the root mean square error of approximation (RMSEA) and its 90% confidence interval as an absolute fit index, and the adjusted goodness-of-fit index (AGFI) and comparative fit index (CFI) as incremental indices. The following index combinations and cut-off values were used: An RMSEA value < .08 and both AGFI and CFI values > 0.95.

Neither the single-factor model nor three-factor model that included all of the original items fit the data well on all four measures of fit (see Table 2). The Nenkov et al. (2008) three-factor short scale fit the data well. However, it is not recommended to apply IRT analysis to factors with only two items, which is what comprises the Nenkov short scale. Therefore, given that the three-factor model replicates the original characterization of the data by Schwartz et al. (2002) and fit the data well using RMSEA and AGFI, we investigated the individual performance of the items within each factor as a unidimensional construct using a polytomous IRT model.

Table 2: Ordinal confirmatory factor analysis for the Maximization Scale (MS) (N = 474).

*** p<.001

3.2 Factor analysis of the MTS

To test the dimensionality of the MTS, we employed an ordinal exploratory factor analysis followed by an ordinal confirmatory factor analysis using LISREL v8.8 (Jöreskog & Sörbom, 2006). We generated three separate exploratory factor analyses (one, two, and three factor solutions) to understand how the individual items loaded on the respective factors. A promax rotation (non-orthogonal) was employed in the two and three factor analyses. The single factor solution suggested that all of the items loaded reasonably well on a single factor. However, items 1, 7, and 8 had the lowest factor loadings (relatively). In the two-factor solution, items 1, 7, and 8 clearly loaded on a second factor. Finally, the three-factor solution resulted in over-factoring and insufficient factor loadings for several items. Based on these results, the three-factor solution was not regarded as acceptable and was not included in the subsequent confirmatory factor analyses. Factor loadings are presented in Table 3.

Table 3: Ordinal exploratory factor analysis of the Maximizing Tendency Scale (N = 474).

We followed the exploratory factor analysis with an ordinal confirmatory factor analysis with a separate, hold-out sample. Using the same model fit criteria specified for the MS, we first tested the single-factor model and found that it had moderate fit for the data (see Table 4). All fit indices were in the proposed range except for the CFI. Next, we tested the two-factor model based on the exploratory factor analysis results and found nearly identical results. Although there was a drop in the overall model chi- square value, this drop was not sufficient to suggest that the two-factor model was a better fitting model for the MTS. Based on these results, we can conclude (with some reservation) that the MTS fits a unidimensional model. Next, we used a polytomous IRT model to investigate the performance of the individual items in the MTS.

Table 4: Ordinal confirmatory factor analysis for the Maximizing Tendency Scale (MTS) (N = 474)

** p<.001

3.3 IRT analysis of MS and MTS

We applied IRT to the MS and MTS to examine the amount of information that is contained in each item and then use this information along with the item discrimination (a parameter) to suggest removing certain items if they contain little information and have low item discrimination. Item discrimination is an inverse function of item information, which in IRT models is a measure of reliability (Embretson & Reise, 2000). Importantly, total information is an incremental additive function in which each item contributes to the overall reliability of the scale. Currently there is no set standard for what level of item discrimination is good enough to be considered a good item, nor is there a standard for what is an adequate level of information that an item must contain. We believe that both the item information and item discrimination need to be taken into account when determining which items should be removed from a scale. For the current study, the criterion we used to remove an item was that the item information curve for an item had to be relatively flat, be below 0.50 on item information and have an a parameter below 1.50. Because there is no standard in the literature when using IRT to remove items, researchers should clearly state the criteria they used to determine which items could be removed from a scale. Zickar et al. (2002) recommend that items with a parameters above 1.0 should be retained and Hafsteinsson, Donovan, and Breland (2007) recommend that for shorter scales the threshold should be increased to 2.0.

Using our IRT sample (N = 948) we fit each factor of the MS and then fit the MTS as a unidimensional construct using the GRM. Table 5 contains the item parameters for the MS and MTS. Figure 1 contains the item information curves for each item in the MS and Figure 2 contains the item information curves for each item in the MTS. Examination of the item parameters and information curves reveals that a number of items can be removed from both of these scales according to the criteria we set. Specifically, items 1, 7, and 8 can be removed from the MTS and items 3, 4, 5, 6, 10, 12 can be removed from the MS. As can be clearly shown from Figure 1 and 2 these items contain little information and are flat across all possible levels of the maximizing construct. Plateau-shaped information curves are not necessarily bad as this would indicate that the item is discriminating across a wide range of the latent trait. However, in the case of the items we removed these are not plateaued but rather completely flat lines relative to the other items indicating that no incremental information is being provided by these items.

Table 5: IRT Item parameters for the Maximizing Tendency Scale (MTS) and the Maximization Scale (MS)

Notes. a = item discrimination parameter; b_j = response category difficulty parameter

Figure 1: Item information curves for the Maximization Scale

Figure 2: Item information curves for the Maximizing Tendency Scale

Nenkov et al. (2008) revised the original 13-item Schwartz et al. (2002) scale and reduced the scale down to 6-items. There are some differences between the items they kept and the items that we kept from our IRT analysis. Their 6-item scale contains three factors and consists of Alternative Search (items 2, 4) Decision Difficulty (items 7, 9) and High Standards (items 11, 12). We are confident in the items we recommended for removal because IRT provides the researcher with item level analysis about the information that a certain item contains in regards to an underlying construct. Now that we have suggested removing items from these scales, we reexamine the factor structure of these scales and then we will examine the correlations between the original scales and our revised scales with a number of other constructs that have been shown to be related to maximizing in the past.

3.4 Factor analysis of the revised MS

Given the results of our exploratory factor analysis and the IRT analysis, we revised the MS to a three-factor, eight-item structure by removing items that loaded insufficiently on their respective factors and demonstrated low information functions (see Table 3). We conducted a CFA (using our CFA data set), on this revised structure and found that it met all of our criteria for model fit (see Table 1). Further, the reduction in the overall model chi-square statistic from the previous models suggests that our revised scale (Revised MS Short) is a more parsimonious version of the MS. We then tested a three-factor model based off Nenkov et al. factor structure with Factor 1 (Alternative Search) containing items 1 and 2; Factor 2 (Decision Difficulty) containing items 7, 8, 9; and Factor 3 (High Standards) containing items 11 and 13. The new revised scale is presented in Table 6.

Table 6: Revised Maximizing Tendency Scale (MTS) and Maximization Scale (MS) item-factor structures

3.5 Factor analysis of the revised MTS

Given the results of our CFA and the IRT analysis, we revised the MTS to a six-item structure by removing items 1, 7 and 8 (see Table 4). We conducted a CFA (again using our CFA sample) test on this revised structure and found that it met all of our criteria for model fit as a unidimensional model (see ’). Further, the reduction in the overall model chi-square statistic from the previous models suggests that our revised scale is a more parsimonious, unidimensional version of the MTS. The new revised scale is presented in Table 6.

3.6 Correlation analysis

The revised MS and MTS scales, and the original MS, MS Short (Nenkov et al. 2008) and MTS were subjected to correlation analysis with various other psychological constructs often linked to maximizing including: indecisiveness, avoidance, regret, neuroticism, life satisfaction, depression, happiness, optimism and need for cognition. Table 7 provides descriptive statistics, correlations and reliabilities for the various psychological constructs. It should be noted that with such a large sample size small correlations would be significant.

Table 7: Descriptive statistics, correlations, and reliabilities (N=948)

^* p < .05

^** p < .01

Notes. MS: Maximization Scale; MS-A: alternative search factor of the MS; MS-D: decision difficulty factor of the MS;

MS-H: high standards factor of the MS; MTS: Maximizing Tendency Scale.

Following Rim et al. (2011) and Nenkov et al. (2008) when using the MS Short we examined each factor separately. The reliabilities for the factors in the MS Short were all low. Although the reliabilities for the factors in the Revised MS Short were also low they were higher than the original MS Short. In regards to the MTS, our revised scale (.79) has slightly higher reliability than the original MTS (.78). Thus, it appears that removing low information items did not negatively affect the reliability of the scale.

Similar to previous research (Schwartz et al., 2002; Nenkov et al., 2008) the original 13-item MS significantly correlates positively with indecisiveness, avoidance, regret, neuroticism, and depression. The MS also significantly correlates negatively with happiness and optimism, but is unrelated to life satisfaction.

The Nenkov et al. (2008) MS was examined by each factor separately. The alternative search factor (Nenkov et al. (2008) MS-A) significantly correlated positively with indecisiveness, avoidance, and regret. However it was unrelated to neuroticism, life satisfaction, depression, happiness, optimism, and need for cognition. The decision difficulty factor (Nenkov et al. (2008) MS-D) correlated positively with indecisiveness, avoidance, regret, neuroticism, and depression. This factor also significantly correlated negatively with life satisfaction, happiness, optimism, and need for cognition. The high standards factor (Nenkov et al. (2008) MS-H) significantly correlated positively with regret, life satisfaction, happiness, optimism, and need for cognition. The high standards factor was significantly negatively correlated to indecisiveness, neuroticism, and depression. These results are in the opposite direction compared to the other two factors.

The Revised MS was also examined by each factor separately. The revised alternative search factor (Revised MS-A) significantly correlated positively with indecisiveness, avoidance, regret, and neuroticism. The only significant negative correlation was with need for cognition. It was unrelated to life satisfaction, depression, happiness, and optimism. The revised decision difficulty factor (Revised MS-D) correlated positively with indecisiveness, avoidance, regret, neuroticism, and depression. This factor also significantly correlated negatively with life satisfaction, happiness, optimism, and need for cognition. The revised high standards factor (Revised MS-H) significantly correlated positively with life satisfaction, happiness, optimism, and need for cognition. The high standards factor was significantly negatively correlated to indecisiveness, avoidance, neuroticism, and depression. The factor was also unrelated to regret. Similar to the Nenkov et al. (2008) MS-H these results are in the opposite direction compared to the other two factors. In addition, in almost every instance the correlations were stronger for the Revised MS-H than for the Nenkov et al. (2008) MS-H.

The original MTS and our revised MTS are negatively related to indecisiveness, avoidance, neuroticism, and depression. They are both positively related to life satisfaction, happiness, optimism and need for cognition. The original MTS is significantly related to regret, which Diab et al. (2008) also found. However, our revised scale is unrelated to regret.

4 Discussion

The purpose of the current study was two-fold: to address the construct validity of the maximizing construct, and to address the psychometric properties of the two prominent scales in hopes of clarifying conflicting findings. In regards to the construct validity of the maximizing construct, we conclude that the MS and MTS are measuring two distinct constructs. Specifically, the MS appears to be measuring difficulty and restlessness with the search for the best alternative, whereas the MTS is more focused on the search for the best option, regardless of choice difficulty. However, the high standards factor of the MS appears to be measuring a different construct than the other two factors. Indeed, the correlation between the high standards factor and the other two factors are quite low and in the case of the revised high standards factor it is unrelated to the decision difficulty factor. Whether using the original or the revised Short MS the alternative search and decision difficulty factors are negatively correlated with need for cognition, whereas the two versions of the MTS and the high standards factor from the MS are positively related to need for cognition. Thus, it is possible that if one enjoys complex thinking and does not find the search process stressful then maximizing will not have the negative consequences predicted by Schwartz et al. (2002). Finally, we believe that the MTS (and the Revised MTS) are more in line with Simon’s (1955; 1956) definition of maximization as an optimization goal.

In regards to the psychometric properties of the MTS and MS, both these scales suffer a number of shortcomings. However, contrary to Rim et al. (2011), the MTS was found to be unidimensional. In addition, removing three items resulted in a better overall fit. Therefore, future research should use the revised MTS. Although the MS provided good fit in regards to RMSEA, using other metrics the fit was poor. In regards to the IRT analysis, both scales did not fare well. Using our criterion for removal of items, we proposed that three items be removed from the MTS and that six items be removed from the MS. The proposed items contained little information in regards to the underlying construct and were low on item discrimination. Once these items were removed, the factor structure of both scales fared better. Using CFA, a three-factor model fit the data well for the MS and all the items had high factor loadings. Using CFA, the Revised MTS resulted in a single factor and a better overall fit than the original MTS. Nenkov et al. (2008) proposed an alternative 6-item MS to the original 13-item MS, which contains many of the same items that are in our revised scale. However, we believe that our revised 7-item MS better represents the construct of maximizing as defined by Schwartz et al. psychometrically because we were able to use IRT to perform item level analysis. Therefore, we were able to get rid of items that contained little information in regards to the construct of maximizing.

Schwartz et al. (2002) indicated that maximizers are generally unhappy. However, this appears to be true only when using the original 13-item scale. When using our revised MS, it appears that maximizers are not unhappy; however, maximization is still positively related to indecisiveness, avoidance, and regret. Thus, it appears that maximizing as measured by the MS is not related to life satisfaction or happiness, but rather the restlessness of maximizing. However, our Revised MTS is unidimensional, correlates positively with well-being, and is unrelated with regret. Thus, it appears that these two scales are not measuring the same construct.

Turner, Rim, Betz, and Nygren (2012) have recently proposed a new maximizing scale that consists of three factors (i.e., satisficing, decision difficulty, and alternative search) called the Maximization Inventory (MI). Turner et al. showed that the MI is superior psychometrically to the MS. The MI does not correlate highly with the MS and the MTS. In regards to the decision difficulty factor and the MTS the correlation is essentially zero. A great benefit of the MI is that one of the factors consists of items that measure satisficing. We believe this is an important advancement in the maximizing literature. The data do not support the assumption that maximizing and satisficing are on opposite ends of a continuum and therefore developing a satisficing measures is extremely important. Because their paper was published after our data collection we are unable to compare their scale with our revised scales quantitatively and therefore will focus on more qualitative issues.

First as a practical matter, the two factors in Turner et al. (2012) scale measuring maximizing uses 24-items, whereas the Revised MTS is only 6-items and the Revised Short MS is only 7-items. Although, there are tradeoffs between parsimony and construct deficiency, we believe the Revised MTS is not construct deficient for the construct it is stated to measure. Specifically, Diab et al. (2010) state that their scale is meant to measure the goal of optimization and we believe that it does measure that construct sufficiently and is a very parsimonious scale. However, as we have discussed the definition of what is maximizing is elusive. Therefore, if one wants to measure restlessness and the difficulty in maximizing then the MTS suffers from construct deficiency. The MI appears to be measuring restlessness and difficulty in maximizing, particularly with the items in the decision difficulty factor. Not surprisingly, the items in the decision difficulty and alternative search factors are very similar to the items in the items in the original MS-D and MS-A factors (Schwartz et al., 2002). However, their items are not about specific behaviors, but rather frame the items in more general behaviors. This is a benefit of the MI because the MS contain questions about specific behaviors that may now be outdated such as renting videos, and writing letters. Although they did not use the same regret scale that previous studies have used, including the current study, these factors were significantly related to regret.

Decision difficulty was negatively related to generalized self-efficacy, self-esteem, and optimism. However, it was unrelated to happiness. Turner et al. (2012) did not compare their scale to the shortened MS scale by Nenkov et al. (2008) and did not examine their scales against the MS broken down by the three factors. In addition, they did not use the same measures of well-being we used in our study, but overall decision difficulty showed similar results to the MS in our study. However, except for regret, alternative search was generally unrelated to the measures of well-being. Tentatively, it appears the MI is a more psychometrically sound measure of maximizing behavior as defined by Schwartz et al. (2002) and has the added benefit of measuring satisficing directly. However, our Revised MTS provides a psychometrically sound unidimensional and global measure of maximization as an optimization goal which is in line with Simon (1955; 1956) that is also more parsimonious than previous measures. Therefore, now that we have two psychometrically sound measures of maximizing, more experimental work needs to be conducted to examine the differences between these definitions of maximizing.

Finally, the current study highlights the use of IRT as an important tool that researchers should use when developing and validating scales. As an item-level analysis, IRT is uniquely positioned for understanding the quality of items with respect to their contribution to construct validity. Appropriate IRT models have also been shown to help attenuate some of the scaling issues that have long been raised when operationalizing psychological constructs (Kang & Waller, 2005; Morse, et al., 2012). These scaling issues, along with other known complications with Likert-type response data in the scale development and validation arena (Flora & Curran, 2004) led us to employ ordinal factor-analytic models. We recognize that this approach is viewed by some as unnecessarily complicated for data that generally abides by linear rules. However, we felt that this was the most conceptually appropriate approach for our data. The inclusion of IRT methodologies in the scale development process may help to alleviate some of the scaling tensions that still arise amongst psychometricians and thus lead to better scales that maximize information and validity.