DOSPERT+M: A survey of medical risk attitudes in the United States

Abstract

Background:

The Domain-Specific Risk Taking scale (DOSPERT) has been recommended as a tool for measuring risk attitudes in medical studies, but does not contain items specific to health care. Butler, et al. (2012) developed a medical risk domain subscale for DOSPERT.

Objective:

To characterize medical risk attitudes in a nationally-representative U.S. sample using the full DOSPERT scale with the medical risk domain add-on (DOSPERT+M), and examine associations with other risk domains.

Methods:

Members of a nationally-representative online panel (KnowledgePanel®) were randomized to complete pairs of DOSPERT+M tasks (risk attitude, risk perception, expected benefits). We explored relationships among domains through correlational and factor analysis; we tested the hypothesis that the medical risk domain and DOSPERT’s health/safety domains were not highly correlated.

Participants:

Three hundred forty-four panelists.

Results:

The medical risk domain subscale had low inter-item reliability in the risk-taking task and moderate inter-item reliability in the other tasks. Medical risk domain scores were poorly correlated with the DOSPERT health/safety domain. Exploratory factor analysis largely recovered the expected DOSPERT domain structure.

Conclusion:

Attitudes toward risky medical activities may constitute a distinct domain from those measured by the standard DOSPERT items. Additional work is required to develop a medical risk subscale with higher inter-item reliability.

1 Introduction

The Domain-Specific Risk Taking scale (DOSPERT) (Blais & Weber, 2006; Blais & Weber, 2009; Weber, Blais, & Betz, 2002) is a widely used instrument that measures risk attitudes (willingness to engage), risk perceptions, and expected benefits for a set of potentially risky activities organized into five domains: ethical, financial, health/safety, recreational, and social risk taking. DOSPERT is one of a small number of instruments recommended in Harrison, et al.’s (2005) review of instruments for assessing health-related or clinical risks. However, DOSPERT does not include a domain that focuses on health care activities; the health/safety scale is limited to preventive behaviors.

In recognition of this limitation and other concerns with the health/safety subscale (Blais & Weber, 2009; Young et al., 2008), Butler, et al. (2012) developed and provided initial psychometric evidence for an add-on medical risk domain consisting of six activities, modeled after the other DOSPERT domains. Adding the new domain’s items to the standard DOSPERT items produces a 36-item instrument that we refer to as DOSPERT+M. Table 1 lists the medical risk domain items.

Table 1: DOSPERT+M medical risk domain subscale items.

The goal of this study was to characterize risk attitudes, including medical risk attitudes, in a nationally-representative U.S. sample using DOSPERT+M, and to examine the associations among domains. We specifically hypothesized that the DOSPERT health/safety scale would not be strongly associated with the DOSPERT medical scale.

2 Method

2.1 Instrument

The 36 DOSPERT+M items, each describing a risky activity, can be presented in any of the three standard DOSPERT tasks (risk-taking, risk perception, or expected benefit). In the risk-taking task, participants indicate their likelihood of participating in the activity on a 1-7 scale from “extremely unlikely” to “extremely likely”. In the risk perception task, participants indicate how risky they believe the activity is on a 1-7 scale from “not at all risky” to “extremely risky”. In the expected benefit task, participants indicate the benefits they believe they would receive from each activity on a 1-7 scale from "no benefits at all" to "great benefits". To reduce respondent fatigue, each participant was randomized to receive two of these three tasks presented in a randomized order.

2.2 Participants

DOSPERT+M was administered using the web-enabled KnowledgePanel®, a probability-based panel designed to be representative of the United States population. KnowledgePanel®consists of approximately 50,000 members aged 18 and older and includes persons living in cell phone only households as well as non-internet households. All participants completed the DOSPERT+M survey online; participants without computers or internet access complete the survey using computers provided by Knowledge Networks (now GfK). Prior to taking the survey each participant was asked a series of demographic questions related to age, gender, education level, geographic location, marital status, and socioeconomic status. Data collection took place between December 2, 2011 and December 18, 2011.

2.3 Data analysis

We examined the relationship among risk-taking, risk perception, and expected benefits in each domain by computing correlations among domain subscale scores in each task. Because participants were randomized to complete two of three tasks, each intra-domain (inter-task) correlation is based on a different group of participants.

To explore the hypothesis that the medical domain subscale measures a different construct than the health/safety domain subscale, we tested the correlation between subscale scores on the two domains in each task. In addition, we fit three confirmatory factor analysis models to the 12 items of the two scales in each task. The first model (“one factor”) assumed all items loaded onto a single “health and medical” factor. The second model (“two distinct factors”) assumed the health/safety items loaded onto a health factor and the medical items loaded onto a medical factor, with the possibility of correlation between the factors. The third model (“two entangled factors”) also assumed two factors, but allowed all of the items but one in each domain to potentially load onto either factor (one item in each domain was fixed to each factor to make the model identifiable). The goal of the confirmatory factor analysis was to compare the goodness of fit of these three models; if the medical domain subscale and health/safety domain subscales are distinct constructs, we expect to see better fit of the “two distinct factors” model than the (nested) “one factor” model, but little additional fit improvement from the (non-nested) “two entangled factors” model. We examined RMSEA and AIC for each model and compared nested models using chi-squared tests of the difference in likelihood ratios (MacCallum, Browne, & Sugawara, 1996; Schreiber, et al., 2006).

Finally, we conducted exploratory factor analyses for each task using all 36 DOSPERT+M items, extracting six or seven factors by maximum likelihood methods and using an oblimin rotation, and examined the factor pattern matrix and inter-factor correlations (Fabrigar, et al., 1999). The goal of the exploratory factor analyses was to further investigate how the medical risk items were related to other DOSPERT items.

We also conducted three multiple linear regressions to determine whether variance in the medical domain task scores (risk-taking, risk perception, and expected benefits) was associated with demographic factors. Age, gender, ethnicity (dummy coded to compare White non-Hispanic, Black non-Hispanic, Hispanic, and Multiracial/Other non-Hispanic), cohabitation (married/living with partner or not), and education (some college or no college) were included as predictors.

Analyses were conducted using R 2.15 for factor analyses with the sem, semPlot, psych, and GPArotation packages (R Core Team, 2013; Fox, Nie, & Byrnes, 2013; Epskamp, 2013; Bernaards & Jennrich, 2005; Revelle, 2013)

3 Results

3.1 Participants

The overall within-survey response rate for KnowledgePanel®is 65% with some minor variation depending on survey length and topic. The DOSPERT+M study sampled 538 participants of which 350 completed the survey for a response of 65%. Of those 350 completed surveys, 344 were considered sufficiently complete for inclusion. The six removed were excluded due to the fact that they refused to answer a majority (19 or more) of the substantive survey questions. The median time for completing the survey was nine minutes. Table 2 presents the demographic characteristics of study participants.

Table 2: Demographic characteristics.

Twenty-five respondents were excluded from analysis based on response patterns: 11 respondents indicated that giving blood was riskier than donating a kidney, and 14 (different) respondents gave the same response to every item in at least one task. Repeating the analyses with these respondents included did not result in substantial differences except where the inclusion of respondents with no variance interfered with model fitting.

3.2 Medical risk subscale item-level statistics

Table 3 presents the range, median, mean, and standard deviation of responses for each of the medical risk subscale items in each task. There was considerable variance in the responses for each item in each task, with the exception of risk perceptions for giving blood (M2) which heavily clustered around the low end of the scale. Figures 1-3 show histograms of each item in each task on the main diagonals, with pairwise scatterplots (lower triangle) and correlations (upper triangle).

Table 3: Medical risk subscale item-level statistics.

Figure 1: Medical subscale item distributions - risk-taking task

Figure 2: Medical subscale item distributions - risk perception task

Figure 3: Medical subscale item distributions - expected benefits task

3.3 Relationships among tasks

Within the medical subscale, risk-taking scores were not significantly correlated with risk perception scores (r=-.12) in the subsample that completed those two tasks. Risk taking was significantly positively correlated with expected benefits scores (r=.69, p<.01) in the subsample that completed those two tasks. Benefit and risk perception scores were negatively correlated (r=-.41, p<.01) in the subsample that completed those two tasks, which may suggest evaluation based on a common underlying feeling (Alhakami & Slovic, 1994). A similar pattern of associations was found for the financial and social subscales; for recreation, ethical, and health/safety subscales, risk-taking and risk perception were also significantly negatively correlated (r=-.28, r=-.49, and r=-.46, respectively).

3.4 Relationships among domains

Table 4 presents the correlations among domain subscale scores in each task. Medical domain scores were least strongly correlated with the health/safety and ethics domain scores, and most strongly correlated with social domain scores.

Table 4: Correlations among domains and Cronbach’s α for each domain.

^* p < .05

^** p < .01

Note: Cronbach’s α on main diagonals, correlations on off-diagonals.

3.5 Confirmatory factor analysis

In each task, the “two distinct factors” model provided a significantly better fit than the “one factor” model (likelihood ratio difference test χ²(1)=56, 103, and 146, for risk-taking, risk perception, and expected benefits, respectively, p<.001 in all cases; AIC=157, 162, 145, respectively). The “two entangled factors” model did not improve significantly on the “two distinct factors” model (AIC=159 vs. 157 for risk-taking, AIC=174 vs. 162 for risk perception, and AIC=156 vs. 145 for expected benefits). Figure 4 shows the standardized path coefficients (factor loadings) for the “two distinct factors” model for each task. The RMSEA for the “two distinct factors” models varies from 0.06 to 0.08 across the tasks, representing acceptable but not excellent fit of the models to the data (risk-taking RMSEA=0.07, 95% CI [0.05-0.09], risk perception RMSEA=0.08 [0.06-0.10], expected benefit RMSEA=0.06 [0.04-0.08]). Item M6 (general rather than local anesthesia for wisdom tooth removal) contributed most to misfit of the model, particularly in the risk-taking task. Correlations among the latent factors for the medical and health/safety items were low.

Figure 4: Confirmatory factor analysis model for risk-taking

3.6 Exploratory factor analysis

Tables 5-7 present the pattern matrices from exploratory factor analyses of each task extracting 6 factors; similar tables for 7-factor extraction appear in the Appendix. RMSEA values for the 6-factor solutions (with 95% confidence intervals) were: risk-taking 0.04 [0.03-0.05], risk perception 0.04 [0.03-0.05], expected benefits 0.05 [0.04-0.06]. RMSEAs for the 7-factor solution were not significantly lower (risk-taking 0.04 [0.03-0.05], risk perception 0.04 [0.03-0.05], expected benefits 0.04 [0.03-0.05]).

In the risk-taking task (Table 5), each DOSPERT+M domain’s items largely loaded together on different factors, with the ethics domain items spread across two factors, one of which also included most of the financial domain items. Medical and health/safety domain items did not load onto the same factor. Item M6 (general rather than local anesthesia) did not load onto any factor.

Table 5: Exploratory factor analysis of the risk-taking task.

Note: Loadings ≥0.2 shown; loadings ≥0.35 shaded.

Table 6: Exploratory factor analysis of the risk perception task.

Note: Loadings ≥0.2 shown; loadings ≥0.35 shaded.

Table 7: Exploratory factor analysis of the expected benefit task.

Note: Loadings ≥0.2 shown; loadings ≥0.35 shaded.

Figure 5: Confirmatory factor analysis model for risk perception

Figure 6: Confirmatory factor analysis model for expected benefit

In the risk perception task (Table 6), the health domain items shared variance with the ethics and recreational items, and the financial items were spread across two factors representing gambling vs. investing items. Medical and health/safety domain items did not load onto the same factor; the correlation between one health/safety factor (factor 1, with ethics items) and the medical factor was 0.48; the correlation between the other health/safety factor (factor 2, with recreational items) and the medical factor was -0.29. Item M6 (general rather than local anesthesia) had weaker loadings.

In the expected benefit task (Table 7), the medical domain items shared variance with the financial and social items, and the financial and ethical items were spread across two factors (with financial representing investing vs. gambling). Medical and health/safety domain items did not load onto the same factor; the correlation between the medical and health/safety factors was 0.29.

3.7 Demographic predictors of medical domain scores

The regression models for medical risk-taking and medical risk perception scores did not account for significant variance in the scores (F(7,213)=0.89, p=0.5, and F(7,199)=1.5, p=0.2, respectively). For expected benefit of medical activities, the regression model reached significance (F(7,223)=2.6, p=0.014), and three demographic predictors were associated with expected benefit scores. Black non-Hispanic respondents had higher scores for expected benefit (B=0.60, SE=0.27, p=.028), as did female respondents (B=0.32, SE=0.15, p=.039). Older respondents also reported higher expected benefit (B=0.010 per year of age, SE=0.004, p=.026), contrary to the findings in the development study (Butler, et al., 2012). Table 8 compares the demographic predictors of medical domain scores with those for the other DOSPERT domains in each task, and suggests that associations between demographic predictors and the medical risk subscale are different than those for other subscales.

Table 8: Demographic predictors of domain scores.

(+) = significant (p < .05) positive association, (-) = significant negative association

4 Discussion

4.1 Summary of results

In this study using a U.S-representative panel sample, the DOSPERT+M items largely (although not perfectly) cluster into the expected domains, providing additional evidence of DOSPERT’s fundamental assumption of domain specificity in risk attitude. The pattern is similar to that reported in factor analyses of the original DOSPERT scale (Blais & Weber, 2009).

DOSPERT+M medical domain scores were not associated with health/safety domain scores. The items in these two domains did not load onto a common factor in either confirmatory or exploratory factor analysis of any task. DOSPERT+M risk-taking and risk perception scores were not associated with demographic factors; DOSPERT+M expected benefit scores were higher for women, Black respondents, and older respondents, on average.

The internal consistency of the medical domain subscale for the risk-taking task was unacceptably low for practical use as a measure of willingness to engage in medical risks. Although the consistency was acceptable in the risk perception and expected benefits tasks, a feature of DOSPERT is the use of the same items in all three tasks, and items in the subscale should be replaced or revised before it is applied as a risk measure in practice. The dental anesthesia item (M6) appears to be a clear candidate for replacement.

4.2 Limitations

We did not measure actual decision behavior, and we cannot conclude whether medical domain subscale scores are associated with behavior by patients facing actual medical decisions. This remains an area for future investigation.

Out of concern for survey fatigue, we did not present all three DOSPERT tasks to each participant, limiting our ability to conduct within-subject comparisons of responses to each task. Past studies using DOSPERT have often omitted the expected benefits scale for similar reasons, but we preferred to collect information on benefit perception.

4.3 Implications

DOSPERT, and the health/safety domain in particular, have been recommended as measures of risk attitude for use in studies of medical decisions. As responses to the health/safety domain do not appear to be related to responses to the DOSPERT+M medical domain, we recommend that studies seeking to measure risk attitudes in medical decisions include medical domain items as well as the health/safety items and assess their value in explaining or informing medical decisions.