Preliminary Investigation of the Reliability and Validity of the Clinical Perfectionism Questionnaire in a Clinical Sample

Abstract

Perfectionism is a risk and maintaining factor across psychopathology and has been proposed to be a transdiagnostic process. The aim of this study was to examine the reliability and validity of the Clinical Perfectionism Questionnaire (CPQ) in 32 adults (75% female, M age = 35.54 years, SD = 9.71) with a range of psychological disorders, presenting for treatment of clinical perfectionism. There was evidence that the CPQ was correlated with established measures of perfectionism and theoretically related constructs including self-criticism and dichotomous thinking. The CPQ was also able to predict treatment outcome. The internal consistency was not adequate in the current study; however, the sample size was small. Future studies should examine the psychometric properties of the CPQ in a larger sample of individuals with a range of psychological disorders.

Perfectionism has been proposed to be a transdiagnostic process that is important in a range of psychological disorders (Egan, Wade, & Shafran, 2011). This argument is based on evidence it is: (1) elevated across eating disorders, anxiety disorders and depression compared to healthy controls; (2) a risk and maintaining factor across disorders; and (3) associated with higher comorbidity of psychological disorders (Egan et al., 2011). A recent meta-analysis showed cognitive behaviour therapy (CBT) for perfectionism results not only in reductions in perfectionism but also a range of psychopathologies (Lloyd, Schmidt, Khondoker, & Tchnaturia, 2015).

Despite numerous studies showing treatment is effective, the majority of measures have not been designed to assess change in perfectionism after treatment. The most widely used measures of perfectionism are the Multidimensional Perfectionism Scales (FMPS; Frost, Marten, Lahart, & Rosenblate, 1990; HMPS; Hewitt & Flett, 1991). The FMPS consists of six subscales: concerns over mistakes (CM), personal standards (PS), parental expectations (PE), parental criticisms (PC), doubts about actions (DA), and organisation (O). The HMPS consists of three subscales: self-oriented perfectionism, other-oriented perfectionism, and socially prescribed perfectionism. These scales have consistently been found to divide into two higher order factors, namely positive striving and maladaptive evaluative concerns (e.g., Dunkley, Blankstein, Masheb, & Grilo, 2006). However, the MPS scales were designed to measure the multidimensional components of perfectionism rather than changes in perfectionism, and a valid measure that is able to assess change is required in order to assess the efficacy of treatment for perfectionism. The focus of this study was on investigating in a clinical sample the psychometric properties of a scale designed to measure change in perfectionism, the Clinical Perfectionism Questionnaire (CPQ; Fairburn, Cooper, & Shafran, 2003).

In order to inform cognitive behavioural models and treatment of perfectionism, Shafran, Cooper, and Fairburn (2002) put forward a clinically relevant definition of perfectionism, clinical perfectionism, which they defined as ‘the overdependence of self-evaluation on the determined pursuit of personally demanding, self-imposed standards in at least one highly salient domain, despite adverse consequences’ (p. 778). Central to the definition of clinical perfectionism is the notion that an individual bases their self-worth almost exclusively on their pursuit of personally demanding standards. The 12-item Clinical Perfectionism Questionnaire (Fairburn et al., 2003) was designed to measure the construct of clinical perfectionism.

A number of studies have examined the psychometric properties of the CPQ and have found acceptable internal consistency and validity among university students, the general community, and adults with eating disorders (Chang & Sanna, 2012; Dickie, Surgenor, Wilson, & McDowall, 2012; Egan et al., 2016; Steele, O'Shea, Murdock, & Wade, 2011; Stoeber & Damian, 2014). For example, the CPQ has shown good convergent validity with other measures of perfectionism in non-clinical (Chang & Sanna, 2012; Dickie et al., 2012; Stoeber & Damian, 2014) and clinical eating disorder samples (Egan et al., 2016; Steele et al., 2011). Egan et al. (2016) found in a non-clinical sample that the CPQ showed convergent validity through high correlations with measures of perfectionism and psychopathology and acceptable internal consistency. Egan et al. (2016) also found in a sample of adults with eating disorders that the CPQ had acceptable internal consistency and construct validity, where the CPQ had a significant correlation with clinician's ratings of eating disorders. They also reported the CPQ showed evidence of discriminative validity through the eating disorder group having higher CPQ scores than a community control group (Egan et al., 2016). Several studies have reported a two-factor structure of the CPQ in community (e.g., Dickie et al., 2012; Egan et al., 2016; Stoeber & Damian, 2014) and clinical eating disorder samples (Egan et al., 2016). For example, in a sample of university students Stoeber and Damian (2014) reported a two-factor structure they stated captured perfectionistic strivings and perfectionistic concerns.

Despite several studies that have evaluated the CPQ (Chang & Sanna, 2012; Dickie et al., 2012; Egan et al., 2016; Steele et al., 2011; Stoeber & Damian, 2014), to date, the psychometric properties have only been evaluated in clinical samples of individuals with eating disorders and not in a wider sample including people with anxiety disorders and depression. It is important to establish the psychometric properties of the CPQ in a clinical sample with a wider range of diagnoses, given CBT for perfectionism has been proposed to be a transdiagnostic intervention that can reduce psychopathology across disorders (Egan et al., 2011). Given CBT for perfectionism is applied across disorders, and a valid measure of change in perfectionism is required to assess the efficacy of treatment, the aim of this study was to assess the reliability and validity of the CPQ in a clinical sample with a range of disorders. It was hypothesised that the CPQ would have acceptable internal consistency, convergent validity with related measures, and predictive validity by predicting treatment outcome.

Method

Participants

The sample consisted of 32 adults (75% female; age 19–57 years, M = 34.54, SD = 9.71) who met criteria for a range of DSM-IV psychological disorders. The sample was participating in a randomised controlled trial (RCT) assessing the efficacy of an 8-week CBT for clinical perfectionism. Primary diagnoses were an anxiety disorder (72%), depressive disorder (19%), or eating disorder (9%), further specified as generalised anxiety disorder (n = 11), social anxiety disorder (n = 9), major depression (n = 5), eating disorder not otherwise specified (n = 2), panic disorder (n = 2), bulimia nervosa (n = 1), dysthymia (n = 1), and obsessive-compulsive disorder (n = 1). The majority had co-occurring psychological disorders: = two diagnoses (41%), = three diagnoses (28%), and ≥ four diagnoses (6%). The majority were employed in full-time work (53%) or were university students (34%).

The inclusion criteria for the RCT and hence present study were an elevated score (≥22) on the CM subscale of the FMPS, a stable (3-month) medication regimen if receiving psychotropic medication, and lack of concurrent psychotherapy. The cut-off of 22 on the CM subscale of the FMPS was based on previous research by Egan and Hine (2008), where 22 was the mean score of clinical samples included on a review of perfectionism (Shafran & Mansell, 2001). The cut-off of 22 on CM has also been used in other studies assessing the efficacy of CBT for perfectionism (Steele et al., 2013). A further inclusion criteria for the current study was a DSM-IV mood, anxiety, or eating disorder diagnosis. Exclusion criteria were high suicide risk, current psychosis, or alcohol and/or substance dependence.

Procedure

This study was approved by the Curtin University Human Research Ethics Committee (HR120/2010) and participants provided written informed consent. The participants were recruited from a mail-out to all mental health and general practitioners in the Perth metropolitan area. Advertisements were also distributed to metropolitan universities and to members of a triathlon club.

Participants were telephoned and screened for eligibility using the Mini International Neuropsychiatric Interview-Screen (MINI-Screen). The individual answers either yes or no to experiencing the symptom. Four MINI modules — Suicidality, Alcohol Dependence/Abuse, Substance Dependence/Abuse (Non-Alcohol), and Psychotic Disorders and Mood Disorder with Psychotic Features — were administered at the telephone screen to assess exclusion criteria. Participants attended a clinical interview and completed the self-report measures. If the participant endorsed a MINI-Screen item on the telephone, the corresponding module of the ADIS or Mini International Neuropsychiatric Interview (MINI) was administered at the clinical interview to determine diagnosis. The Anorexia Nervosa and Bulimia Nervosa modules of the MINI were administered at the clinical interview if the participants endorsed the corresponding items on the Mini-Screen on the telephone.

Measures

Mini International Neuropsychiatric Interview (MINI; Sheehan & Lecrubier, 2009)

The MINI is a brief structured interview that assesses 16 Axis I disorders. Sheehan and colleagues (1997) found the MINI to have excellent reliability and validity and stated that it is a useful tool to screen for DSM-IV Axis I disorders in clinical trials. The MINI-Screen assesses initial signs of DSM-IV psychopathology.

Anxiety Disorders Interview Schedule for DSM-IV (ADIS-IV; Brown, DiNardo, & Barlow, 1994)

The ADIS-IV was administered to determine DSM-IV diagnoses. The ADIS-IV assesses diagnostic criteria for each anxiety disorder, depressive disorder, and diagnoses that frequently co-occur with anxiety and depressive disorders and has good reliability and validity (Grisham, Brown, & Campbell, 2004).

Clinical Perfectionism Questionnaire (CPQ; Fairburn et al., 2003)

The 12-item self-report CPQ was developed to measure clinical perfectionism over the previous month. Items — for example, ‘Over the past month, have you felt a failure as a person because you have not succeeded in meeting your standards?’ — are scored using a 4-point Likert response format (1 = not at all to 4 = all of the time). Prior to summing items, Items 2 and 8 are reverse scored. Previous studies have reported strong correlations between the CPQ and EC and PS components of the FMPS (r = .76; r = 73), and small-to-moderate positive correlations between the CPQ and the three subscales of the HMPS (self-oriented perfectionism, r = .49; other-oriented perfectionism, r = .28; socially prescribed perfectionism, r = .51; Chang & Sanna, 2012; Steele et al., 2011). Furthermore, literature has suggested the CPQ consists of two factors, positive striving and EC (Dickie et al., 2012; Egan et al., 2016; Stoeber & Damian, 2014). The internal consistency of the CPQ in the current study can be seen in Table 1.

TABLE 1 Summary of Intercorrelations, Means, Standard Deviations, and Reliabilities for Study Measures (N = 32)

Note: CPQ = Clinical Perfectionism Questionnaire, EC = Evaluative Concerns, consisting of a composite of the Frost Multidimensional Perfectionism Scale (FMPS), PS = personal standards subscale of the FMPS, SC = self-criticism subscale of the Dysfunctional Attitudes Scale, DT = general subscale of the Dichotomous Thinking in Eating Disorders Scale.

* p < .05 ** p < .01.

Frost Multidimensional Perfectionism Scale (FMPS; Frost et al., 1990)

The 35-item self-report FMPS measures perfectionism on the subscales concern over mistakes, personal standards, parental expectations, parental criticism, doubts about actions, and organisation. To assess the validity of the CPQ, previous studies have used the PS subscale and Evaluative Concerns (EC), where EC is the sum of concern over mistakes and doubts about actions (Dickie et al., 2012; Egan et al., 2016; Steele et al., 2011). Dunkley and colleagues (2006) have argued that ‘clinical perfectionism consists of two distinct dimensions. One dimension, tapped by PS perfectionism variables, reflects the determined pursuit of self-imposed standards. The second dimension, tapped by EC perfectionism measures, reflects the extremely vulnerable self-evaluation and critical maintaining pathology of clinical perfectionism’ (p. 66). Therefore, to remain consistent with previous research, the 13 items reflecting EC and the seven-item PS subscale of the FMPS were used in the current study.

Frost and colleagues (1990) reported that the FMPS has good convergent validity and internal consistency. The internal consistency of PS subscale in this study (α = .58) was lower than other studies, where it has been found to have excellent internal consistency (e.g., Frost et al., 1990).

The Dichotomous Thinking in Eating Disorders Scale (DTEDS; Byrne, Allen, Dove, Watt, & Nathan, 2008)

The general subscale of the 11-item self-report DTEDS measured general dichotomous thinking. Confirmatory factor analysis has yielded a two-factor solution consisting of four items relating to food, eating, weight and dieting, and seven items reflecting global dichotomous thinking.

The Dysfunctional Attitudes Scale (DAS; Weissman & Beck, 1978)

The self-criticism (DAS-SC) subscale of the DAS measured self-criticism. The DAS consists of 40 items, with 15 of them loading on the DAS-SC subscale. Participants rate their self-criticism using a 7-point Likert scale. The DAS-SC correlates significantly with commonly used measures of perfectionism and symptoms of depression (Dunkley & Kyparissis, 2008).

Results

The intercorrelations, means, standard deviations and reliabilities of the measures are reported in Table 1.

The internal consistency of the CPQ in the current study was not acceptable (α <.7). Convergent validity was examined by observing the correlations of the CPQ with related measures using Pearson's correlations. Moderate positive correlations were observed with measures of perfectionism, between the CPQ and EC (r = .43, p = .01), and between the CPQ and PS (r = .44, p = .01). There was a significant correlation between the CPQ and another measure of perfectionism the DAS-SC (r = .45, p = .01, but not between the CPQ and DTEDS-general (r = .31, p = .08).

The predictive validity of pretreatment CPQ on post-treatment CM was evaluated with a moderated hierarchical linear regression model. Pretreatment CPQ and group (intervention, control) were entered on Step 1, and the CPQ × group interaction was entered on Step 2. The interaction was significant, t(24) = 21.78, p < .001, indicating that the relationship between pretreatment CPQ and post-treatment CM varied as a function of group. One-tailed tests showed that the relationship was significant for the intervention group, r(11) = .531, p = .031, but not for the control group, r(13) = .300, p = .139, suggesting that pretreatment CPQ scores predicted the impact of the intervention on post-treatment CM rather than simply predicting the post-treatment CM per se.

In order to make a comparison of the mean score and standard deviation on the CPQ in the current study to the literature, see Table 2, in which it can be seen that scores are within a similar range to other clinical samples in the literature.

TABLE 2 Descriptive Statistics of the Clinical Perfectionism Questionnaire in the Current Study and Literature

Note: Egan et al. (2016) = Egan, Shafran, Lee, Fairburn, Cooper, Doll, Palmer, & Watson, 2016; Dickie et al. (2012) = Dickie, Surgenor, Wilson, & McDowall, 2012; Shafran et al. (2006) = Shafran, Lee, Payne, & Fairburn, 2006; Steele et al. (2011) = Steele, O'Shea, Murdock, & Wade, 2011; Riley et al. (2007) = Riley, Lee, Cooper, Fairburn, & Shafran, 2007; Egan, van Noort et al. (2014) = Egan, van Noort, Chee, Kane, Hoiles, Shafran, & Wade, 2014; Handley et al. (2015) = Handley, Egan, Rees, & Kane, 2015.

Discussion

The aim of the study was to assess the reliability, construct validity and predictive validity of the CPQ in a clinical sample with a range of psychological disorders. Significant correlations between the CPQ and FMPS were observed. The CPQ was also found to predict treatment outcome following an intervention. Internal consistency was not adequate in the current study. This finding is inconsistent with previous research that has found adequate reliability of the CPQ in community (α = .72–.83) and eating disorder (α = .82–.83) samples (Chang & Sanna, 2012; Egan et al., 2016; Steele et al., 2011). Small sample sizes can affect internal consistency and lead to insufficiently precise reliabilities (Charter, 2003). This could also be an explanation for the poor internal consistency on the PS subscale of the FMPS, which has been well documented to have excellent reliability (e.g., Frost et al., 1990). Further research is required with a larger sample to establish internal consistency in a clinical sample with a range of disorders.

The moderate correlation observed between the CPQ and the perfectionism measures of EC and PS on the FMPS is consistent with previous literature (Dickie et al., 2012; Egan et al., 2016; Stoeber & Damian, 2014). Previous literature has reported that the combination of EC and PS is maladaptive in clinical samples (Bardone-Cone et al., 2007; Egan et al., 2011; Steele et al., 2011) and that the CPQ captures these two constructs (Stoeber & Damian, 2014).

Convergent validity was evaluated with measures of theoretically related constructs proposed to maintain the cycle of clinical perfectionism; namely, self-criticism and dichotomous thinking. The results partially supported the proposed hypotheses: the CPQ was significantly correlated with self-criticism but not dichotomous thinking. This significant relationship between clinical perfectionism and self-criticism is consistent with previous research and was expected, given that the self-criticism scale has previously been used as a measure of perfectionism (Egan et al., 2016; Steele et al., 2011). However, the finding of no significant relationship between the DTEDS-general and CPQ is in contrast to other research that has found that dichotomous thinking is related to perfectionism in mixed clinical groups and eating disorders (Egan, Piek, Dyck, & Rees, 2007; Egan et al., 2016; Lethbridge, Watson, Egan, Street, & Nathan, 2011). Given the relatively small sample size, but observed moderate correlation, it is likely that there is an association beyond chance, but simply not a large association in this study due to the sample size.

The major limitation of the study is that the sample size was small and was not adequate to accurately estimate internal consistency; therefore, findings should be interpreted accordingly. Future research should evaluate the CPQ in a larger sample of people with a range of psychological disorders.

Previous studies that have assessed the psychometric properties of the CPQ have been conducted either using community or female eating disorder samples (Chang & Sanna, 2012; Dickie et al., 2012; Egan et al., 2016; Steele et al., 2011). The strength of this study was that a range of psychological disorders were included; however, further research is required in larger clinical samples. Clinical perfectionism is a construct of relevance to a number of psychological disorders and future research should further investigate the reliability and validity of the CPQ in a larger, representative clinical sample.