Patient experience of negative effects of psychological treatment: results of a national survey

Abstract

Background

To make informed choices, patients need information about negative as well as positive effects of treatments. There is little information about negative effects of psychological interventions.

Aims

To determine the prevalence of and risk factors for perceived negative effects of psychological treatment for common mental disorders.

Method

Cross-sectional survey of people receiving psychological treatment from 184 services in England and Wales. Respondents were asked whether they had experienced lasting bad effects from the treatment they received.

Results

Of 14 587 respondents, 763 (5.2%) reported experiencing lasting bad effects. People aged over 65 were less likely to report such effects and sexual and ethnic minorities were more likely to report them. People who were unsure what type of therapy they received were more likely to report negative effects (odds ratio (OR) = 1.51, 95% CI 1.22–1.87), and those that stated that they were given enough information about therapy before it started were less likely to report them (OR = 0.65, 95% CI 0.54–0.79).

Conclusions

One in 20 people responding to this survey reported lasting bad effects from psychological treatment. Clinicians should discuss the potential for both the positive and negative effects of therapy before it starts.

A number of psychological and pharmacological interventions have been shown to be effective for the treatment of mental disorders. Although both the effects and side-effects of pharmacological treatments have been widely investigated, ¹ there is less information about negative effects of psychological treatments. ^2–4 Research trials of psychological therapies do not monitor or report negative effects adequately compared with pharmacological trials. ⁵ It is estimated that between 5 and 10% of people have higher levels of psychiatric symptoms following psychological treatment than before they start it, ⁶ but it is not possible to attribute this to the treatment itself. Very few studies have examined patient experiences of negative effects of psychological treatments, and those that have, have been small and restricted to selected groups of patients. ^7,8

The National Audit of Psychological Therapies is a large-scale examination of state-funded psychological therapy services for adults with depression and anxiety in England and Wales. ⁹ The audit comprises an examination of routine clinical records and a survey of people using a wide range of primary and secondary care services to evaluate their performance against agreed standards of care. We analysed data from the audit to determine the prevalence of patient-reported negative effects of psychological treatments and to identify factors that may influence the likelihood that patients experience these.

Method

Setting and participants

Data for the study were collected as part of the second round of the audit in 2012–2013. To identify eligible services we contacted medical directors and chief executives of all National Health Service (NHS) providers in England and Wales asking them to submit contact details for the psychological treatment services they provide. We supplemented this with data from the national Improving Access to Psychological Therapies (IAPT) programme in England, and contact details of services that participated in an earlier round of the audit. ¹⁰ A total of 220 services took part (approximately 60% of the 350–380 services that we estimate were eligible to take part in the audit at that time). All those aged 18 years or older who were on the case-load of participating services and receiving out-patient treatment for anxiety and/or depression on an agreed census date within the period 1 July to 31 October 2012 were invited to complete an anonymous service-user questionnaire that examined people's experience of the process and outcomes of treatment. All participants were given written information about the audit and invited to complete a paper or web-based questionnaire. Those who opted to complete a paper version were given a pre-paid envelope to return the questionnaire directly to the audit team. Demographic data were not collected from people who did not participate in the survey, but were available from the audit of clinical records that was conducted along with the survey.

Main outcome measure and covariates

As part of a parallel qualitative study examining patient experiences of negative effects of psychological therapies, we recruited patients who may have had these experiences through adverts in local newspapers and online fora. Patients who were interviewed made a distinction between short-lived unsettling or upsetting experiences that occurred during therapy, and longer-lasting negative effects. Some patients felt the term ‘negative’ did not properly capture how difficult their experience had been. Based on these accounts we asked all patients who took part in the survey to indicate whether they had experienced ‘lasting bad effects from the treatment’. People were asked whether they strongly or slightly agreed with this statement, whether they were not sure, or slightly or strongly disagreed with it.

The survey also included a series of questions on demographic factors (age, gender, sexual orientation and ethnicity), type of therapy received, and the person's experience of the process of care. The latter included questions on how long people had to wait before the start of treatment, the number of sessions they received, whether they thought they were referred at the right time, whether the time they waited was reasonable, whether they thought they had received enough information about treatment before it began and whether they were asked to give feedback on their progress in treatment. These items were based on guidelines for delivering psychological treatments and feedback from an expert group of service users and providers.

Statistical methods

The primary outcome was a self-report of having experienced ‘lasting bad effects from the treatment’. Having calculated the prevalence of those who agreed, disagreed or were neutral about whether they had experienced lasting bad effects from treatment, this item was converted into a dichotomous variable: whether people strongly or slightly agreed that they had experienced lasting bad effects or disagreed or were unsure whether this was the case. We then examined univariate associations between this variable and demographic factors (age, gender, ethnicity and sexual orientation) and factors associated with the process of care. Differences in levels of self-reported negative effects were examined between those receiving different forms of therapy. A multivariate analysis of factors associated with the likelihood of having experienced negative effects of therapy was subsequently conducted. To restrict the number of factors in this analysis, only variables showing some evidence of an association with the outcome in the univariate analysis (P<0.1) were included in the multivariate analysis. A backwards selection procedure was used to retain only the statistically significant variables.

A feature of the data was that patients were clustered within different services. Outcomes from patients from the same service may be more similar than outcomes from patients from different services. Therefore, to allow for this data structure, and the dichotomous nature of the outcome, all analysis was performed using multilevel logistic regression. Two-level models were used with patients nested within services. The analyses were implemented using the software package Stata (version 12.1).

Results

Of 220 psychological treatment services that took part in the audit, 184 (83.6%) collected data for the patient survey. Patient questionnaires were sent out to 76 950 people who were either receiving therapy or had recently completed it and 15 078 (19.6%) responded. Returns from ineligible patients who were aged under 18 or had not started therapy at the time of the survey were removed and data from 14 587 (19.0%) were included in this analysis. Characteristics of those who took part in the study are presented in Table 1 together with aggregate data from the audit of clinical records of people using the 220 treatment services during this period. Types of therapy received by patients are listed in Table 2. Among the 14 384 who provided information on the type of therapy they received, the most commonly reported treatment was cognitive–behavioural therapy (CBT, n = 7 340, 51.03%). Whereas most people reported receiving one therapy, 1208 (8.40%) reported receiving two or more, usually a low-intensity psychological treatment together with CBT, and 163 (1.13%) reported receiving three or more therapies. Most people received individual therapy, but 66 (4.59%) reported receiving group-based treatment either alone or combined with an individual therapy. A minority of patients indicated that they received another type of therapy not featured in the list they were presented with (n = 563, 3.77%) or reported that they were unsure what type of therapy they received (n = 161, 1.11%).

Table 1 Demographic characteristics of study participants and comparative data from the case-note audit

Demographic characteristics	Study sample, n (%)	Sample included in the case-note audit, n (%)	Difference in proportions (95% CI)
Age, years	14 148	122 740
18–24	1088 (7.69)	16405 (13.37)	−5.68 (−5.18 to −6.15)
25–34	2513 (17.76)	30117 (24.54)	−6.78 (−6.09 to −7.44)
35–44	3287 (23.23)	28796 (23.46)	−0.23 (−0.51 to 0.96)
45–54	3519 (24.87)	25359 (20.66)	2.57 (1.84 to 3.31)
55–64	2474 (17.49)	14269 (11.63)	5.86 (5.22 to 6.53)
65–74	980 (6.93)	5617 (4.58)	2.35 (1.92 to 2.90)
75+	287 (2.03)	2177 (1.77)	0.25 (0.02 to 0.51)
Gender	13 954	122 585
Female	9656 (69.20)	79157 (64.57)	4.63 (3.81 to 5.43)
Male	4298 (30.80)	43 428 (35.43)	–
Ethnicity	14 004	101 550
White	13 134 (93.79)	90 769 (89.38)	4.41 (3.95 to 4.84)
Asian	348 (2.49)	3736 (3.68)	−1.19 (−0.9 to −1.47)
Black	159 (1.14)	2788 (2.75)	−1.61 (−1.40 to −1.80)
Mixed	219 (1.56)	2181 (2.15)	−0.58 (−0.35 to −0.80)
Chinese/other	144 (1.03)	2078 (2.05)	−1.02 (−0.81 to −1.20)

Table 2 Likelihood of experiencing negative effects of treatment, type and form of therapy, taking into account clustering by service

	Reporting negative effect, n/N (%)	OR (95% CI)	P
Type of therapy
Cognitive–behavioural therapy
No	441/6840 (6)	1	<0.001
Yes	311/7340 (4)	0.64 (0.56–0.75)
Counselling
Yes	522/10136 (5)	1	0.18
No	230/4044 (6)	1.12 (0.95–1.32)
Psychodynamic therapy
No	720/13834 (5)	1	0.004
Yes	32/346 (9)	1.75 (1.19–2.58)
Cognitive analytical therapy
No	741/14011 (5)	1	0.56
Yes	11/169 (7)	1.20 (0.64–2.24)
Low-intensity therapy
No	724/13679 (5)	1	0.73
Yes	28/501 (6)	1.07 (0.73–1.59)
Humanistic therapy
No	743/13901 (5)	1	0.11
Yes	9/279 (3)	0.58 (0.30–1.13)
Solution-focused therapy
No	743/13931 (5)	1	0.21
Yes	9/249 (4)	0.65 (0.33–1.27)
Other
No	708/13661 (5)	1	0.16
Yes	44/519 (8)	1.64 (1.19–2.26)
Unsure
No	591/12198 (5)	1	<0.001
Yes	161/1982 (8)	1.71 (1.42–2.05)
Form of therapy
Individual	670/10136 (7)	1	0.88
Group	42/814 (5)	0.97 (0.70–1.34)
Both	24/500 (5)	0.90 (0.59–1.38)

Regarding treatment process, most people were referred to treatment by a family doctor or other healthcare professional, but 2041 (15.66%) referred themselves to the service (Table 3). Most people were seen by a therapist within 3 months of referral to the service (n = 10 114, 73.33%), and received fewer than 10 sessions of treatment (n = 10 229, 74.35%). Patient views about the process of treatment they received are presented in Table 4; most patients reported being referred at the right time, being offered the right number of sessions and being given sufficient information about treatment before it started.

Table 3 Likelihood of reporting negative effects of treatment, treatment processes, taking into account clustering by service

	Negative effect, n/N (%)	OR (95% CI)	P
Source of referral
Self-referral	95/2041 (5)	1	0.30
Someone else	572/10988 (5)	1.13 (0.90–1.41)
Time taken for treatment to start
Less than 1 month	203/4082 (5)	1	0.002
1–3 months	284/6032 (5)	0.94 (0.78–1.13)
4–6 months	119/2179 (5)	1.10 (0.87–1.39)
7–9 months	44/761 (6)	1.16 (0.82–1.63)
10–12 months	31/327 (9)	1.95 (1.30–2.91)
More than 12 months	34/430 (8)	1.58 (1.07–2.33)
Number of sessions
1–5	292/5442 (5)	1	<0.001
6–10	214/4787 (4)	0.81 (0.68–0.98)
11–15	89/1739 (5)	0.94 (0.73–1.20)
16–20	54/960 (6)	1.03 (0.76–1.40)
21–25	9/295 (3)	0.54 (0.38–1.97)
26 or more	49/535 (9)	1.65 (1.18–2.29)

Table 4 Likelihood of reporting negative effects of treatment, experiences of treatment, taking into account clustering by service

Experience of treatment	Negative effect, n/N (%)	OR (95% CI)	P
I was referred at right time
Unsure/disagree	225/3220 (7)	1	<0.001
Agree	525/10 750 (5)	0.71 (0.61–0.84)
The waiting time was reasonable
Unsure/disagree	270/4702 (6)	1	0.18
Agree	486/9417 (5)	0.90 (0.77–1.06)
Appointments scheduled at a convenient time
Unsure/disagree	100/1008 (10)	1	<0.001
Agree	655/13 136 (5)	0.48 (0.38–0.60)
Able to get there without difficulty
Unsure/disagree	83/863 (10)	1	<0.001
Agree	664/13 175 (5)	0.51 (0.40–0.65)
Received enough information about treatment before it began
Unsure/disagree	250/3118 (8)	1	<0.001
Agree	506/11 018 (5)	0.56 (0.47–0.65)
Receiving the right number of sessions
Unsure/disagree	293/4577 (6)	1	<0.001
Agree	451/9483 (5)	0.73 (0.63–0.85)
I am asked to give feedback on how helpful I am finding treatment
Unsure/disagree	240/4137 (6)	1	0.16
Agree	515/9983 (5)	0.89 (0.76–1.05)

Of 14 270 people who provided information about the impact of therapy, 763 (5.23%, 95% CI 4.88–5.60) strongly or slightly agreed that it had resulted in lasting bad effects, and an additional 1099 (7.70%, 95 CI 7.27–8.15) reported that they were unsure whether therapy had resulted in lasting bad effects. Relationships between demographic factors, type of therapy, process of care and the likelihood of reporting lasting bad effects taking into account clustering by service, are presented in Tables 2, 3, 4 and 5.

Table 5 Likelihood of experiencing negative effects of treatment, demographic factors, taking into account clustering by service

	Reporting negative effect, n/N (%)	OR (95% CI)	P
Age, years
18–24	68/1088 (6)	1	0.001
25–34	124/2513 (5)	0.77 (0.57–1.05)
35–44	167/3287 (5)	0.80 (0.60–1.08)
45–54	226/3519 (6)	1.03 (0.78–1.37)
55–64	128/2474 (5)	0.82 (0.60–1.11)
65–74	38/980 (4)	0.60 (0.40–0.91)
75+	5/287 (2)	0.26 (0.10–0.66)
Gender
Male	256/4298 (6)	1	0.01
Female	478/9656 (5)	0.82 (0.70–0.96)
Sexual orientation
Heterosexual	632/12874 (5)	1	0.002
Lesbian/gay	23/365 (6)	1.28 (0.83–1.97)
Bisexual/other	30/320 (9)	1.98 (1.35–2.92)
Ethnicity
White	649/13 134 (5)	1	<0.001
Asian	41/348 (12)	2.61 (1.85–3.67)
Black	16/159 (10)	2.16 (1.27–3.67)
Mixed	17/219 (8)	1.65 (1.00–2.74)
Chinese/other	19/144 (13)	2.86 (1.73–4.69)

In the multivariate analysis, factors associated with the likelihood of negative effects included some patient characteristics, types of therapy and aspects of the process of care (Table 6). Likelihood of reporting lasting bad effects of therapy was associated with receiving ‘other’ forms of treatment or being unsure what type of therapy the person had received.

Table 6 Multivariate model of factors associated with likelihood that patients reported lasting bad effects of treatment, taking into account clustering by service

Category	OR (95% CI)	P
Age, years
18–24	1	0.02
25–34	0.75 (0.54–1.04)
35–44	0.78 (0.57–1.07)
45–54	0.98 (0.72–1.33)
55–64	0.82 (0.59–1.15)
65–74	0.61 (0.39–0.96)
75+	0.28 (0.10–0.78)
Sexual orientation
Heterosexual	1	0.003
Lesbian/gay	1.31 (0.84–2.05)
Bisexual/other	1.97 (1.31–2.99)
Ethnicity
White	1	<0.001
Asian	2.07 (1.34–3.18)
Black	2.50 (1.41–4.42)
Mixed	1.54 (0.88–2.70)
Chinese/other	3.30 (1.93–5.63)
Appointments scheduled at a convenient time
Unsure/disagree	1	<0.001
Agree	0.61 (0.46–0.79)
Able to get there without difficulty
Unsure/disagree	1	0.05
Agree	0.74 (0.55–0.99)
Received enough information about treatment before it began
Unsure/disagree	1	<0.001
Agree	0.65 (0.54–0.79)
Time taken for treatment to start
<1 month	1	0.02
1–3 months	0.86 (0.71–1.06)
4–6 months	0.97 (0.75–1.25)
7–9 months	1.03 (0.71–1.49)
10–12 months	1.76 (1.15–2.69)
>12 months	1.18 (0.77–1.80)
Type of therapy: other
No	1	0.001
Yes	1.84 (1.29–2.63)
Type of therapy: not sure
No	1	<0.001
Yes	1.51 (1.22–1.87)

Discussion

Data from almost 15 000 people receiving psychological treatment for anxiety and depression in England in Wales suggest that about 1 in 20 think that it had a lasting bad effect. The likelihood of patients reporting bad effects from treatment varied according to both demographic and clinical factors. People over the age of 65 were less likely to report negative effects than younger patients. People from Black and minority ethnic groups and non-heterosexuals were more likely to report such problems. Patients were less likely to report lasting bad effects of treatment if they felt they had been given sufficient information about therapy before it started. Whereas the results of the multilevel univariate analysis suggested different levels of patient experience of bad effects among different types of therapy, multivariate analysis suggested that it is only those offered ‘other’ therapies or those who were unsure what type of therapy they received that were more likely to report this type of negative experience.

Strengths and limitations

Strengths of the study are that it is based on a large sample recruited from a broad geographical spread of services that included a wide variation of treatment modalities and settings. Our primary outcome was based on a question that was developed following in-depth interviews with patients who reported negative experiences of psychological treatments. However, the study has a number of limitations that need be taken into account when considering the results. These include a low response rate to the survey and a reliance on patient recall of information about the type and duration of treatment that they received. Although we do not have demographic data from those who did not respond to the survey, comparative data from the case-note audit suggests that the response rate may have been different in different groups of patients. Nonetheless, differences in the proportion of patients in different age groups and from different ethnic backgrounds who reported experiencing negative effects are greater than differences in the proportion of people from these groups who responded to the survey. Another limitation is we do not have information about diagnoses or other clinical details. Previous studies examining outcomes of in-patient mental healthcare indicate that people with some conditions, such as somatoform or personality disorders, may be more likely to deteriorate during treatment than others. ¹¹ Qualitative data about negative effects were not collected in the survey, but data from an ongoing analysis of in-depth interviews of people who report these experiences suggest that these include exacerbations of existing symptoms and emergence of new ones including anxiety, anger and loss of self-esteem. Finally, it is important to note that survey data were collected from people who were in treatment or had recently completed it and we do not know the extent to which these negative experiences subsequently resolved. However, even when negative experiences do not turn out to be lasting, they are unpleasant for the patient and have the potential to erode the patient's confidence in the therapist or therapy process and limit further engagement with the treatment.

Implications

A substantial minority of patients who responded to the survey reported that they experienced lasting bad effects from their treatment. To give informed consent to treatment it is important that patients are informed about possible costs and benefits. ¹² Although far more people reported beneficial effects than reported harm, it is important that people being referred to psychological therapy services are informed that a minority of people experience negative effects from treatment. With well over a million people receiving psychological treatment for common mental disorders in England alone over recent years, ¹³ these data imply that many thousands of patients could have experienced negative effects from treatment. Our finding that people who did not know what type of therapy they received or stated that they were not given enough information about treatment before it started were more likely to report lasting bad effects is noteworthy. It is possible that people who were properly informed about treatment before it started had a better sense of what was involved and more realistic expectations of what it might achieve.

The finding that ethnic and sexual minority groups are more likely to report negative effects is of concern. Although available evidence suggests psychological therapies are at least as effective among people from minority ethnic groups as they are among White patients in Western countries, ¹⁴ we are not aware of any previous research that has examined the likelihood of negative effects. Previous reports have highlighted the role that attitudes to homosexuality can have on patient experience of psychological therapies among gay people. ¹⁵ Our findings may indicate a need to place greater emphasis on the development of therapists' cultural competence during initial training and subsequent professional development activities. ¹⁶

There is some evidence that helping therapists become aware of poor response to treatment can help avoid negative treatment outcomes. ^17,18 More research is needed to establish whether this can reduce the incidence of negative experiences of treatment and it is clearly important for therapists and patients to discuss both the positive and negative effects of treatment during therapy. Future research should also include longitudinal studies that examine the course of negative effects of treatment and what can be done to help people who experience them.

Meanwhile, clinicians delivering psychological therapies should ensure that people feel that they have sufficient information about treatment before it starts and obtain informed consent to treatment by ensuring that people considering psychological treatment for their condition are aware that there is the potential for both positive and negative effects.