The effectiveness of remote therapy in two London IAPT services

Abstract

The COVID-19 pandemic increased population levels of depression and anxiety, and infection control measures obliged services to provide psychological therapies remotely. Evidence for the routine provision of psychological therapy via telephone and video-conferencing is limited. This study compared therapy outcomes for 5360 clients in two London Improving Access to Psychological Therapy (IAPT) services before and after homeworking produced a compete shift to remotely delivered therapy. Despite the psychological impacts of pandemic restrictions, and the use of a novel therapy modality in video-conferencing, recovery rates and net score change improved in both services, significantly in one. There was no significant worsening of outcomes for any demographic group or presenting disorder. The findings suggest that for those able to access it, therapy provided by telephone and video is a clinically effective option for IAPT services.

Key learning aims

1. (1) To assess the clinical effectiveness of delivering IAPT therapies remotely.

2. (2) To gain insight into the impacts of remote therapy on different client groups, including ethnicity, gender, age and presenting problem.

3. (3) To assess the impact of remote therapy on access to IAPT services.

Introduction

In March 2020, the UK government introduced national social distancing and lockdown rules as a response to the COVID-19 pandemic (Cabinet Office, 2020). There is growing evidence that the pandemic and subsequent lockdowns had a significant impact on the mental health of the general population (Chandola et al., 2020; Pierce et al., 2020). The mental health of young people (18–29 years), women, people from socially disadvantaged backgrounds, people with physical health conditions, ethnic minority groups, as well as those with pre-existing mental health conditions was particularly affected (Bu et al., 2020; Frank et al., 2020; O’Connor et al., 2021; Pierce et al., 2020). Adults reporting some level of depression doubled from pre-pandemic levels (Office for National Statistics, 2021). The implications for mental health care services were that providing accessible and effective treatment was more important than ever, at a time when pandemic public health measures presented new challenges to traditional delivery methods.

Improving Access to Psychological Therapy (IAPT) services are a key element of service provision for people with depression and anxiety problems, with 1.17 million people starting treatment in 2019–2020 (NHS Digital, 2020). The two services featured in this study are IAPT services provided by South London and Maudsley NHS Foundation Trust (SLAM). The services share a Clinical Director but operate independently with separate senior teams and staffing. The service models are broadly similar. Service A caters for an inner London borough with an ethnically diverse population and was ranked 42nd most deprived Local Authority on the 2019 Index of Multiple Deprivation (Ministry of Housing, Communities & Local Government, 2019). Service B operates in an outer London borough, also with an ethnically diverse population, and was ranked 102nd most deprived Local Authority (Ministry of Housing, Communities & Local Government, 2019).

Prior to the pandemic, both services offered cognitive behavioural therapy (CBT) and counselling for common mental health disorders in a range of media including in-person, telephone, and online CBT programmes. The main interventions followed the national IAPT stepped care model, and included online computerised CBT, individual and group guided self-help, 1:1 CBT and counselling. Following the national lockdown in March 2020, the services rapidly reorganised, all clinical staff moving to homeworking and providing all therapy remotely. Interventions that were previously offered in-person were now delivered by video-conferencing (VC), alongside ongoing provision of telephone and online therapy. Workshop activity was suspended. Clinical supervision was also provided remotely.

There is preliminary evidence that CBT programmes over telephone and VC are acceptable to clients and show no significant difference in clinical recovery compared with in-person therapy (Andrews et al., 2018; Barnett et al., 2021; Car et al., 2020; Cromarty, 2016; Mohr et al., 2012; Mullin et al., 2015). There is specific evidence that low intensity interventions are effective via telephone (Cromarty et al., 2016). There is, however, little precedent for the sort of overnight and complete shift from largely in-person therapy to entirely remote therapy which took place in these two services. High-intensity clinicians, in particular, had to adapt their approach and learn new ways of delivering interventions (Cromarty et al., 2020), which might have been expected to reduce treatment effectiveness. The overall shift could potentially have an impact on clinical outcomes, clients’ experience of care and access to care (Buckman et al., 2021). Similar changes were happening in many IAPT services in England (NHS England and NHS Improvement, 2020).

An umbrella review of systematic reviews suggested that remotely provided mental health services have the potential to be effective and acceptable, but highlighted limited evidence of the impact of large-scale implementation (Barnett et al., 2021). Any consideration of remote therapy also needs to take into account the risk of digital exclusion for service users lacking the skills, confidential space, equipment and financial resources to make use of web-based treatment (Barnett et al., 2021).

To the best of our knowledge, little is published about the impact of substantial remote working on the clinical efficacy of IAPT services. We were keen to establish whether providing therapy by video and telephone was clinically effective and, equally importantly, acceptable to both clients and therapists. As well as investigating overall effectiveness, we wanted to identify any exceptions: client groups who benefited more or less from the change. With this aim, we carried out a research project using electronic healthcare records in the two IAPT services to investigate the impact of remote working on (1) overall clinical recovery, (2) clinical recovery across different care steps, and (3) clinical recovery across different client groups, defined by ethnicity, age, gender and provisional diagnosis. A linked study assessed client and therapist satisfaction with therapy over video (Dowling et al., in press). Determining whether IAPT services can be offered remotely at scale without a negative impact on clinical outcomes, together with client and staff acceptability, are key considerations in future service planning. The diverse nature of the service populations may potentially make the findings more generalisable to the wider population of the UK.

Method

Settings and measures

IAPT services routinely collect a minimum dataset for all clients, including demographic information and scores on standardised clinical measures at each contact (Clark et al., 2018). A key measure of clinical outcome performance in IAPT services is the recovery rate, defined as the proportion of clients moving from clinical caseness at the start of treatment to non-caseness at the end of treatment. Caseness is based on client self-reported scores on standardised measures of symptoms of depression and anxiety. The caseness cut-off score is 9 on the Patient Health Questionnaire-9 (PHQ-9; Kroenke et al., 2001) and 7 on the Generalised Anxiety Disorder-7 questionnaire (GAD-7; Spitzer et al., 2006). A client scoring above either or both of these cut-off values at assessment is classified as a case, and a client scoring at or below both cut-off values at the end of treatment is classified as moving towards recovery (National Collaborating Centre for Mental Health, 2021). The proportion of clients who were at caseness at the start of treatment, and have completed treatment ‘moving towards recovery’, is known as the recovery rate. If clients present with symptoms of anxiety disorders such a social anxiety disorder, panic disorder, obsessive-compulsive disorder and post-traumatic stress disorder, then appropriate anxiety disorder specific measures (ADSMs) are used to determine clinical caseness and recovery instead of the GAD-7 (National Collaborating Centre for Mental Health, 2021). The PHQ-9 and GAD-7 are collected at every appointment regardless of provisional diagnosis. Completing treatment in an IAPT service also has a specific meaning: it includes all clients who have ended treatment and have also attended two or more appointments that are recorded as treatment contacts. This includes in outcome calculations clients who have left treatment without completing a full intervention, and therefore sets a challenging target for services (National Collaborating Centre for Mental Health, 2021).

Participants

The study included all clients discharged from the two services between May and July 2019 and May and July 2020, who also met the national IAPT definition of completing treatment. Clients who were not at ‘caseness’ at first contact were excluded from the analysis, on the basis that they could not be included in recovery calculations. The participant numbers are shown in Table 1.

Table 1. Analysis sample and exclusions

The 2020 clients were on average a little younger and more ethnically diverse than the 2019 clients, with a lower proportion from a White ethnic background (Table 2). In terms of presenting difficulty, there was an increase in the proportion of clients treated for obsessive compulsive disorder (OCD) and health anxiety, and fewer clients for recurrent depression. None of the demographic or clinical presentation changes were statistically significant (Table 2).

Table 2. Demographics of the sample groups by year

The initial severity of client symptoms was slightly lower in 2020 in both services. Service A clients had slightly lower starting scores on the PHQ-9 in 2020 (mean=14.69, SD=5.64) than in 2019 (mean=15.37, SD=5.55), t ₃₀₁₁=3.35, p=.001. They also had lower starting scores on the GAD-7 (mean=13.82, SD=4.36) than in 2019 (mean=14.33, SD=4.27), t ₃₀₁₁= 3.26, p=.001. In Service B, the 2020 starting PHQ-9 score (mean=15.56, SD=5.67) was lower than in 2019 (mean=16.15, SD=5.53), t ₂₃₄₅=2.51, p=.01. The 2020 GAD score was also slightly lower (mean=13.97, SD=4.56) than in 2019 (mean=14.28, SD=4.43), but not significantly so, t ₂₃₄₅=1.64, p=.10.

Procedure and design

We used the electronic healthcare records from the services to compare the clinical outcomes of clients who were discharged in May–July 2019 against those in May–July 2020, before and after the services moved to remote working. A matching period of time was used to avoid seasonal impacts on symptoms and outcomes (Lyall et al., 2018). All data were anonymised by Mayden, the software provider of the electronic patient record system IAPTUS that is used by both services. Datasets from the services were analysed and compared separately using IBM SPSS software.

The research team includes both clinicians and clinical leads working in the two services. During the analysis stage, a member of the research team (J.N.) extracted the data and provided descriptive comparisons to the research team on clients’ recovery rate between the two time periods.

Data analysis

The key data extracted were the starting and end scores for all clients on PHQ-9 and GAD-7 measures. GAD-7 scores were used rather than ADSMs because the ADSM data were incomplete. The measures were used to calculate the starting severity, size of score change and recovery rate. Comparisons between the starting severity and size of score change between the two different time periods were carried out using independent sample t-tests. Comparisons between recovery rates, a categorical variable, were carried out using a chi-square goodness of fit test. Overall comparisons were made, along with additional comparisons of selected demographic and clinical sub-groups. Missing data for any of the demographic or clinical variables were defined as unknown but were not excluded from the analyses.

Results

Therapy delivery

The dramatic impact of moving to homeworking is shown in Table 3, with in-person sessions reducing from around two-thirds of treatment activity in May to July 2019 to less than 2% in the same period in 2020. In Service B, in-person sessions were largely replaced by telephone sessions, with a higher proportion of the activity moving to video in Service A.

Table 3. Clinical session modality (includes all appointments in period, not just those for study group)

Attendance at sessions increased in both services between 2019 and 2020, from 83% to 87% of sessions in Service A and from 81% to 85% of sessions in Service B, with corresponding drops in the proportions of sessions cancelled or not attended.

Impact on therapy outcomes

Therapy type

Tables 4 and 5 summarise the impact of moving to remote therapy on therapy outcomes in the two services. Both services experienced an increase in recovery rate in the period of remote working, significantly so in Service A [χ (1, N=3013)=23.74, p<.001]. Both services also saw an increase in the average score change on the PHQ-9 and GAD-7 during treatment. In Service A the PHQ-9 reduction in 2020 (mean=–6.82, SD=5.97) was significantly greater than the reduction in 2019 (mean=–6.16, SD=6.24, t ₃₀₁₁=2.95, p=.003). Similarly, the GAD-7 reduction in 2020 (mean=–6.49, SD=5.52) was significantly greater than the reduction in 2019 (mean=–5.79, SD=5.69, t ₃₀₁₁=3.42, p=.001).

Table 4. Therapy outcomes for Service A, May to July 2019 vs 2020

Significance tested using Pearson chi-square test: **p<.005, *p<.05; significance tested using independent t-tests:**p<.005, *p<.05;does not equal Step 2 total because workshops not included; presenting problem based on ICD-10; depression severity as measured by the PHQ-9 score: mild <10, moderate 10–14, moderately severe 15–19, severe >19; anxiety severity as measured by the GAD-7 score: mild <10, moderate 10–14, severe >14.

Table 5. Therapy outcomes for Service B, May to July 2019 vs 2020

Significance tested using Pearson chi-square test, **p < .005, *p < .05; significance tested using independent t-tests,**p < .005, *p < .05;presenting problem based on ICD-10; depression severity as measured by the PHQ-9 score: mild <10, moderate 10-14, moderately severe 15-19, severe >19; anxiety severity as measured by the GAD-7 score: mild <10, moderate 10-14, severe >14.

Looking at service modalities, both services saw an increase in recovery rate for clients receiving Step 2 interventions, significantly so in Service A [χ (1, N=1697)=16.59, p<.001]. Both services also experienced an increase in average score reduction in the course of Step 2 treatment. Both changes are significant in Service A, where the PHQ9 reduction in 2020 (mean=–6.82, SD = 5.97) was greater than the reduction in 2019 (mean=–6.16, SD=6.24), t ₃₀₁₁=2.95, p=.003. Similarly, the GAD7 reduction in 2020 (mean=–6.49, SD=5.52) was significantly greater than the reduction in 2019 (mean=–5.79, SD=5.69), t ₃₀₁₁=3.42, p=.001. The recovery rate for counselling increased in both services in 2020, but without a significant increase in average score change.

In Service A, it was possible to break down Step 2 further into two main interventions – guided self-help and facilitated online CBT. This revealed that the outcome improvement at Step 2 in Service A was due to a significant improvement in recovery rate for online CBT alone [χ (1, N=427)=32.41, p<.001]. Online CBT also showed a significant improvement in mean score change. The PHQ-9 reduction in 2020 (mean=–5.0, SD=4.9) was greater than the reduction in 2019 (mean=–3.9, SD=5.4), t ₄₂₅=2.07, p=.039. Similarly, the GAD-7 reduction in 2020 (mean=–6.1, SD=5.16) was significantly greater than the reduction in 2019 (mean=–4.0, SD=5.34), t ₄₂₅=4.01, p<.001.

Demographics

Both female and male clients saw an increase in recovery rate in both services. For female clients in Service A there was also a significant increase in average score change. The PHQ-9 reduction in 2020 (mean=–6.89, SD=5.87) was greater than the reduction in 2019 (mean=–6.22, SD=6.18, t ₂₁₁₇=2.54, p=.01). Similarly, the GAD-7 reduction in 2020 (mean=–6.57, SD=5.51) was significantly greater than the reduction in 2019 (mean=–5.53, SD=5.55), t ₂₁₁₇=2.79, p=.005).

All ethnic groups (except ‘Other’ in Service B) had a higher recovery rate in the remote working condition, but this is only matched by a significant increase in average score change for clients from an Asian or Asian British background in Service A. For this client group, the PHQ-9 reduction in 2020 (mean=–8.59, SD=5.80) was greater than the reduction in 2019 (mean=–6.02, SD=7.67, t ₁₇₁=2.30, p=.01). Similarly, the GAD-7 reduction in 2020 (mean=–8.33, SD=5.04) was significantly greater than the reduction in 2019 (mean=–5.06, SD=6.20, t ₁₇₁=3.82, p<.001).

All age groups in Service B except the over-80s experienced an increase in recovery rate, as did clients under 60 in Service A. These increases were matched by significant increases in average score changes for 17- to 30- and 41- to 50-year-olds in Service A. For the 17–30 age group, the PHQ-9 reduction in 2020 (mean=–6.57, SD=5.70) was greater than the reduction in 2019 (mean=–5.86, SD=6.04, t ₁₄₄₆=2.30, p=.02). Similarly, the GAD-7 reduction in 2020 (mean=–6.61, SD=5.23) was significantly greater than the reduction in 2019 (mean=–5.75, SD=5.40, t ₁₄₄₆=3.08, p=.002). For the 41–50 age group, the PHQ-9 reduction in 2020 (mean=–7.46, SD = 5.70) was greater than the reduction in 2019 (mean=–6.04, SD=6.03, t ₄₀₁=2.21, p=.03). Similarly, the GAD-7 reduction in 2020 (mean=–6.67, SD=6.08) was significantly greater than the reduction in 2019 (mean=–5.46, SD=5.67, t ₄₀₁=2.05, p=.04).

Disorders

Recovery rates improved for several disorders in both services – depression, generalised anxiety disorder (GAD), panic disorder and recurrent depression. They worsened for health anxiety and social phobia in both services, and additionally for post-traumatic stress disorder and obsessive compulsive disorder in Service B. The only significant changes were increases in average score change on the PHQ-9 for depression clients, and the GAD-7 for GAD clients, in Service A. For clients with a diagnosis of depression, the PHQ-9 reduction in 2020 (mean=–7.90, SD=6.07) was greater than the reduction in 2019 (mean=–6.97, SD=6.61, t ₁₄₃₀=2.79, p=.005). For clients with GAD, the GAD-7 reduction in 2020 (mean=–7.71, SD=5.07) was significantly greater than the reduction in 2019 (mean=–6.12, SD=5.41, t ₇₂₂=4.09, p<.001).

Severity

Recovery rates were similar in 2020 regardless of the initial severity of presentation. Classifying clients by severity range for PHQ-9 and GAD-7 scores show no significant reductions in recovery rate between the years. In Service A there were significant improvements for clients with moderate [χ (1, N=899)=8.47, p=.004] and moderately severe [χ (1, N=895)=12.24, p<.001] starting PHQ-9 scores, and with moderate [χ (1, N=1049)=5.61, p=.018] and severe [χ (1, N=1460)=19.61, p<.001] GAD-7 scores.

Discussion

The context for the 2020 telephone and video treatments reported in this study was the COVID-19 pandemic, with its associated stressors, including restrictions on social interaction, direct and indirect health impacts, and economic consequences. These factors are generally assessed as increasing anxiety and depression levels. The researchers anticipated that a combination of this context, and the switch to what for most therapists and clients in this study was a novel way of delivering and receiving psychological therapy, would have a negative impact on treatment outcomes. Against this background, the standout finding from this study is that there was no significant worsening of recovery rates for any demographic group, disorder or treatment modality in either service.

The reduction in starting severity meant that more clients were likely to move towards recovery, so it was important to look at mean score change as well as recovery rates to evaluate outcomes. Both the recovery rates and mean score change on key disorder measures in these two IAPT services who switched to remote therapy have remained at least as good as in the prior year. Indeed, in one service there was an overall improvement in both. Outcomes have been as good across all demographic groups, all treatment types and most common mental health problems.

It is important to acknowledge the differences between the two cohorts of clients. The remote therapy cohort only included clients who were willing and able to access therapy from their homes by telephone or video. It may also include clients who accessed therapy because it was remote. These group differences may have had a significant impact on the outcomes and mean that our conclusions are necessarily tentative.

The cohort comparisons suggest that individual therapy in an IAPT service delivered by telephone or video was an effective alternative to face-to-face therapy for those clients able to access remote therapy during the pandemic. This effectiveness is sustained across different treatment steps. Although the 2020 cohort are less severe in initial presentation, the size of treatment effect is consistent across the two time periods. Delivering similar outcomes against the multiple stressors of the pandemic background further suggests that the treatment effects are robust. These findings are consistent with the results of surveys of therapists and clients from the same study (Dowling et al., in press), which found that both therapists and clients were pleasantly surprised by the effectiveness of therapy by video and found it satisfactory overall. The sustained outcomes mirror the national picture, with IAPT recovery rates for England in 2020–2021 slightly higher than in 2019–2020 (NHS Digital, 2020), covering a period in which use of remote therapy was expected to be extensive (NHS England and NHS Improvement, 2020).

Remote therapy has the potential to exclude some groups, but also to make therapy more accessible to others (Barnett et al., 2021). Clients with long-term physical health conditions and clients in full-time work might particularly benefit from the flexibility and convenience of video and telephone appointments. The comparable outcomes experienced by clients in this study suggests that services offering both remote and in-person therapy options could potentially extend access without reducing effectiveness.

There are a number of client groups in the study that have done less well with remote therapy: older clients, and clients with a diagnosis of social anxiety or health anxiety. The small sample sizes mean that these are indications rather than significant findings. The reduced impact for older clients may be consistent with less familiarity and ease with technology use in this group, although this may be a stereotypical view. The pandemic background may provide an explanation for sustained anxiety levels amongst clients presenting with health anxiety. In the absence of more specific data, there may be a case for prioritising in-person options for some or all of these client groups.

For clients with social anxiety, the restrictions on social interaction during the pandemic may have had a specific impact in reducing the opportunities for experimentation that are a key part of the CBT treatment protocol for social anxiety. In the qualitative research that was carried out as part of this study, some clients commented upon the loss of in-person experiments, whilst therapists noted greater ease in structuring and focusing sessions (Dowling et al., in press). It may be that remote working supports greater fidelity to treatment protocols and reduces opportunity for therapist drift (Waller, 2009). Increased session attendance, reported as a factor in better outcomes, may also be a key mechanism (Saunders et al., 2020). The finding that outcomes improved significantly in one service for online CBT, the delivery of which was unchanged by remote working, may suggest a specific impact on client use of therapy caused by the pandemic restrictions. This nuanced picture highlights the need for further research on the mechanisms that make remote CBT effective.

Limitations

The 2020 client cohort for these two services was different from the 2019 group. Observed differences were that the clients were slightly younger and more ethnically diverse, albeit non-significantly. Their self-reported depression and anxiety symptoms were slightly but significantly less severe at initial assessment. Most clients self-referred by the service websites and would have read messages informing them that all treatment would be by telephone or video. The awareness that therapy would be remote may have attracted additional clients who would not have opted for face-to-face therapy. Clients who did not have access to technology or confidential space, or who chose not to receive therapy remotely, are not part of the 2020 cohort. The combined effect may have been to select a group of clients more likely to benefit from remote therapy. The pandemic lockdown context could also have had an impact on how people made use of therapy sessions. Limitations on other activities might have resulted in greater focus on the therapy, with a corresponding increase in benefit.

A proportion of the clients discharged in May to June 2020 will have received some in-person sessions, which may have had an impact on their overall therapy experience.

The findings are from one NHS trust, with similar IAPT services covering an inner and outer London borough. They may not generalise to other services or locations.

Conclusion

The cohort comparisons suggest that delivering IAPT psychological therapy in two London services by telephone and video-conferencing was an effective alternative to face-to-face therapy for those clients able to access remote therapy during the pandemic.

Implications for future practice

The overall finding that clinical outcomes for two IAPT services were not worsened by providing therapy by telephone and video-conferencing rather than in-person has important implications for future service delivery. Together with the findings from the linked study of client and therapist satisfaction with remote therapy, it suggests that the remote provision of therapy could be a substantial ongoing part of IAPT service delivery. Further controlled studies of remote CBT would be of benefit to produce more evidence for future changes. It will also be important to establish that the acceptability of remote working and therapy is maintained post-pandemic. Service planning clearly needs to take into account the risks of digital exclusion.

Areas for further investigation include a focus on the client groups who seemed to do less well with remote therapy, with a specific focus on older clients and clients presenting with health anxiety and social anxiety. There is need for further research to isolate the treatment factors which make remote therapy effective. It would also be very helpful to know if a blend of in-person and remote therapy is even more effective and acceptable.