Early intervention in psychosis: health economic evaluation using the net benefit approach in a real-world setting

Abstract

Background

Early intervention in psychosis is a complex intervention, usually delivered in a specialist stand-alone setting, which aims to improve outcomes for people with psychosis. Previous studies have been criticised because the control used did not accurately reflect actual practice.

Aims

To evaluate the cost-effectiveness of early intervention by estimating the incremental net benefit (INB) of an early-intervention programme, delivered in a real-world setting. INB measures the difference in monetary terms between alternative interventions.

Method

Two contemporaneous incidence-based cohorts presenting with first-episode psychosis, aged 18–65 years, were compared. Costs and outcomes were measured over 1 year. The main outcome was avoidance of a relapse that required admission to hospital or home-based treatment.

Results

From the health sector perspective, the probability that early intervention was cost-effective was 0.77. The INB was €2465 per person (95% CI − €4418 to €9347) when society placed a value of €6000, the cost of an in-patient relapse, on preventing a relapse requiring admission or home care. Following adjustment, the probability that early intervention was cost-effective was 1, and the INB to the health sector was €3105 per person (95% CI −€8453 to €14 663). From a societal perspective, the adjusted probability that early intervention was cost-effective was 1, and the INB was €19 928 per person (95% CI − €2075 to €41 931).

Conclusions

Early intervention has a modest INB from the health sector perspective and a large INB from the societal perspective. The perspective chosen is critical when presenting results of an economic evaluation of a complex intervention.

There has been considerable debate in both academic and health policy circles about the merits or otherwise of specialist early intervention services for people suffering from psychosis. A number of studies have found that early intervention achieves savings over time through reductions in in-patient admissions.^1–4 A variety of methodological approaches have been used in previous studies. The majority used a historical treatment-as-usual (TAU) control,^1,2,5–9 three used decision analytic modelling,^10–12 one used data linkage¹³ and two used a randomised controlled trial (RCT) format.^3,4 Critics have questioned whether the TAU services in some of these studies are a reasonable reflection of current practice, arguing that community mental health teams (CMHTs) are able to deliver sophisticated treatments such as home-based treatment and assertive outreach treatment.^14,15 Another issue with previous studies is that those that used patient-level data focused on younger people, whereas recent policy changes have recommended extending early intervention beyond the youth population.¹⁶ There is an urgent need, therefore, to compare the costs and outcomes of a specialist early intervention service with a TAU that routinely offers home-based and assertive outreach treatments, and to include older individuals in the analysis.

Economic evaluation

Economic evaluation is a useful method for examining whether an intervention translates into the real world, thereby generating information relevant for service planning and policy makers. Cost-effectiveness evaluations typically report incremental cost-effectiveness ratios (ICERs). However, there are difficulties in interpreting the value of the ICER and in generating a measure of uncertainty, and they are not amenable to regression analysis.^17,18 Reformulating the cost-effectiveness question to generate net benefit estimates facilitates the interpretation of the results of an evaluation, and allows us to use regression techniques to adjust for sociodemographic and clinical differences between the TAU and intervention groups. Incremental net benefit (INB) measures the difference in monetary benefit between alternative interventions. A positive INB indicates that an intervention is cost-effective at a given willingness-to-pay threshold. To the best of our knowledge, only one other study has examined the cost-effectiveness of early intervention using the INB approach. However, that study compared a specialist early intervention service with a traditional TAU community model that did not deliver home-based and assertive outreach treatments.⁴

Aim

The aim of this study was to conduct a cost-effectiveness evaluation of early intervention in a real-world setting compared with best-practice TAU, using the INB framework. This study is presented according to the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) guidelines for economic evaluations.¹⁹

Method

The study sample consisted of two incidence-based cohorts presenting to five catchment area services in the Republic of Ireland between 2010 and 2012. Data were collected at first presentation and at 1 year of follow-up. Individuals presenting with first-episode psychosis (FEP) aged between 18 and 65 years were included. Exclusion criteria were intellectual disability, being on antipsychotic medication for >30 days before presentation and having a diagnosis of psychosis secondary to a general medical condition. The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008. Ethical approval for the study was obtained from the local ethics committees of the relevant services (identification 406 St John of God Provincial Ethics Committee). All participants gave written informed consent.

The early intervention cohort consisted of people from two urban and one predominantly rural catchment areas who were referred by their general practitioner or the local CMHT to an early intervention service, which operated as a specialist hub to three CMHT catchment areas. The early intervention service offered early detection and a phase-specific intervention strategy.^20,21 Referral to the early intervention service was via primary care or the CMHT. Everybody referred was registered with a CMHT and were offered a rapid assessment within 72 h to establish the presence of psychosis. Where possible, the assessor also interviewed a family member. Interpreters were used where required. The CMHT was responsible for care during any in-patient admission and subsequent medication management. Each participant was offered one or all of three phase-specific interventions. Cognitive–behavioural therapy for psychosis was delivered in a group format, over 12 sessions. Family education and intervention was delivered in a group format, over six sessions. A psychosocial intervention focusing on vocational or educational needs was delivered in individual sessions for as many as were required. A follow-up assessment was conducted at 1 year.

The TAU cohort consisted of people from two predominantly rural catchment areas who presented to their CMHT services, which offered home-based treatment teams (HBTs) and assertive outreach teams. Patients presenting to the TAU service were assessed by a research registrar following presentation (on average within 41 days), but otherwise received standard care.²²

In both cohorts, participants were assessed using the Structured Clinical Interview for DSM-5 to establish a diagnosis of psychosis.²³ Information on health service and resource use was collected using the Client Socio-Demographic Service and Receipt Inventory (CSSRI) for a 1-year period.^24,25 This information was supplemented by medical records and contact with the primary care service. Unit costs were derived from previously published studies and personal contact with the individual services or the finance department of the Health Service Executive. Where Irish cost data were not obtainable, we used data from the Personal Social Services Research Unit (PSSRU) and converted the costs to Euros using purchasing power parity (PPP) and the Consumer Health Index.²⁶ Lost productivity was valued at the average industrial wage.²⁷ Informal care was valued by the average wage of carers and child care providers based on the proxy good approach. Costs were reported in Euros for the year 2012.

The reference case takes the health sector perspective. Costs accruing to the health sector during the year following presentation with FEP were collected. These included mental health in-patient costs, general medical costs related to the FEP, home-based treatment costs, CMHT service costs, costs of primary care, and counselling, medication and investigation costs. Secondary analysis was from the societal perspective. The 1-year follow-up period allowed sufficient time for the outcome to occur. Therefore a discount rate was not applied to the costs or benefits.

The primary outcome was a relapse of psychosis sufficiently severe to require admission to hospital or the HBT. Relapse limits recovery and is distressing for the person and their family and carers. Between 30 and 70% of people with FEP will relapse.²⁸ Information on relapse was collected from hospital electronic records and CMHT records. The outcome was recorded as a dummy variable indicating relapse or no relapse. Information on lost productivity and employment status was collected from clinical interviews and/or medical records review.

Data were analysed in Microsoft Excel 2010 and Stata version 13.0 (Windows 7). Univariate analysis of outcome data were carried out using χ²-tests, parametric data were analysed using Student t-tests, and non-parametric data were analysed using Mann–Whitney U tests. Multivariate analyses of outcome data were carried out using logistic regression. As the cost data were highly skewed, cost data were analysed using a generalised linear model with a gamma family and a log link. Cost and outcome data were adjusted for sociodemographic and baseline clinical characteristics in the multivariate analyses. The net benefit statistic was generated using the equation NB = λ.E − C, where NB is the net benefit, E is the effectiveness (i.e. avoidance of a relapse requiring admission or HBT) and C are the service costs. λ is a theoretical unknown value placed on the outcome by society. Cost and effectiveness data were bootstrapped to 1000 replications using sampling with replacement to generate 95% confidence estimates. The proportion of replications greater than zero indicated the probability that early intervention was more cost-effective than TAU. The probabilities were used to generate cost-effectiveness acceptability curves. Sensitivity analyses of the costs and outcomes were conducted.

Results

Of 307 people presenting to services who fulfilled the inclusion criteria, 270 were eligible for follow-up at 1 year, 212 were assessed at 1 year and 201 people were included in the cost-effectiveness analysis. The reasons for non-inclusion in the cost-effectiveness analysis were incomplete Client Socio-Demographic Service and Receipt Inventory data (n = 12) or attendance at private in-patient services only (n = 2). Older people were more likely to be followed up (P = 0.01). Table 1 shows the sociodemographic and clinical characteristics of the sample. There were no statistically significant differences in baseline characteristics as regards gender, marital status, living independently at presentation, the proportion possessing a medical card, the proportion born in Ireland or the proportion with English as their first language. The TAU cohort were younger at presentation (28 years v. 33 years, z = −2.646, P = 0.008), and had a higher proportion in employment at baseline (47% v. 27%, χ² = 7.823, d.f. 1, P = 0.005). More of the early intervention cohort lived in urban areas (98% v. 39%, χ² = 87.34, d.f. 1, P < 0.001). The TAU cohort were more likely to live in areas with higher levels of deprivation (decile 9 v. decile 4, z = 5.554, P < 0.001).

Table 1 Baseline characteristics of the study sample

GAF, Global Assessment of Functioning Scale; IQR, Interquartile range; ROI, Republic of Ireland; SFI, Social Fragmentation Index; SSD, schizophrenia spectrum disorder; TAU, treatment as usual.

a. Possessed a medical card for part or all of year.

Cost-effectiveness results

From the health sector perspective, the intervention dominated (Table 2) since it cost €1681 (s.e. €3247) less and more relapses were avoided (0.10; s.e. 0.06). The unadjusted ICER was €17 078 saved per relapse avoided. The bootstrapped estimates of the ICER show that the intervention dominated in 63% of replications. Following adjustment for sociodemographic and clinical characteristics, early intervention dominated in 95% of replications (Fig. 1). From a societal perspective, the intervention dominated in 74% of replications. Following adjustment for sociodemographic and clinical characteristics, the intervention dominated in 95% of replications. The unadjusted ICER was €25 543 saved per relapse avoided. The confidence intervals were wide and not significant. From both perspectives, there were a small number of replications in the south-west quadrant, indicating that the intervention was both less effective and less costly than TAU. As the number of such replications is so small (3%), we do not consider this outcome as a realistic possibility.

Fig. 1 Cost-effectiveness plane.

Table 2 Incremental cost-effectiveness ratio

Costs are rounded up, and all other figures are rounded to two decimal places.

ICER, incremental cost-effectiveness ratio; N, normal based.

a. Bootstrapped to 1000 replications.

The intervention yielded an INB to the health sector of €1796 (s.e. €3376). This fell to €1200 (s.e. €5410) following adjustment for sociodemographic and clinical characteristics, even when society placed no value on avoiding a relapse requiring admission or HBT. When a value of €6000, the approximate cost of an in-patient relapse, was placed on avoiding such a relapse, the intervention resulted in an INB of approximately €2465 (s.e. €3389) to the health sector, which rose to €3105 (s.e. €5890) following adjustment for sociodemographic and clinical characteristics. The standard errors of the mean were large and the 95% confidence intervals were wide and not significant, reflecting a high degree of uncertainty around the cost data. When the value of λ was €0, the probability that the intervention was cost-effective was 0.71 in the unadjusted model. Following adjustment for sociodemographic and clinical characteristics, the probability that early intervention was cost-effective fell to 0.59.

From a societal perspective, the intervention resulted in an INB of €34 694 (s.e. €8994). This fell to €17 604 (s.e. €10 933) following adjustment for sociodemographic and clinical characteristics, even when society placed no value on avoiding a relapse requiring admission or HBT. When λ was valued at €6000, implementing the intervention resulted in an INB to society of €35 363 (s.e. €8081), or €19 928 (s.e. €11 212) following adjustment for sociodemographic and clinical characteristics. The probability that the intervention was cost-effective was 1, both before and after adjustment for sociodemographic and clinical characteristics. Supplementary Fig. 1, available at https://doi.org/doi:10.1192/bjp.2019.126, shows the probabilities that the intervention was cost-effective for a range of values of willingness to pay (λ).

A number of sensitivity analyses were carried out. Varying the proportion of relapses to 25% (the minimum relapse rate aimed for in the Initiative to Reduce the Impact of Schizophrenia (IRIS) guidelines²⁹) and adjusting for baseline costs had no effect on the results. Neither did restricting the analysis to those seen by clinical interview compared with those whose data was extracted from medical records. Including medical card status as one of the control variables had no effect on the results.

We identified a number of subgroups a priori for further analysis because of their relevance to policy makers and service managers. The results are presented in Table 3. Many early intervention services restrict by age (usually 15–35 years) and diagnosis (usually functional psychosis). We found that delivering early intervention to the functional psychosis (schizophrenia spectrum disorder) subgroup was highly cost-effective from both a health sector and a societal perspective. This was because the proportion of relapses was higher in this subgroup in the TAU cohort (0.18 v. 0.10). As the net benefit is affected more by costs when the effect size approaches zero, a bigger effect size in this subgroup demonstrated a different INB profile.

Table 3 Subgroup analyses

Δ, difference in means; GMC, general medical; INB, incremental net benefit; SSD, schizophrenia spectrum disorder.

a. Bootstrapped to 1000 replications.

b. Adjusted for age, gender, marital status, employment at baseline, diagnosis, GAF at baseline, the use of drugs, the presence of depression, catchment area and Social Fragmentation Index decile; set seed was the same as the unadjusted model to facilitate replication; cost data rounded up: all other figures rounded to three decimal places.

By contrast, restricting the analysis to the age 18–35 years subgroup revealed that early intervention was less likely to be cost-effective from the health sector perspective. The INB favoured traditional CMHT until the willingness to pay to avoid a relapse was over €15 000. However, the INB favoured the early intervention service from the societal perspective even when the willingness to pay to avoid a relapse was €0.

The final subgroup analysis tested the effect of including people with organic psychosis, or psychosis secondary to a general medical condition who have typically not been included in research on early intervention, despite the fact that people with psychosis secondary to a general medical condition will often present to early intervention services. Including the costs of treating the physical health cause of psychosis is reasonable as in a clinical setting it is not always immediately apparent that the cause of psychosis is medical rather than functional. These cases can generate high costs and the effect of early intervention is uncertain, as the psychosis is often resolved by treating the underlying medical condition. In our sample, although there were only four patients who presented with an organic psychosis (two from each cohort), the probability that early intervention was effective from a healthcare perspective fell substantially when they were included. See Table 3 for details of the subgroup analysis.

Discussion

Our analysis of early intervention in a real-world setting found that, compared with TAU, early intervention contributed a modest benefit to the health sector and a large benefit to society. The probability that early intervention was cost-effective from a societal perspective was 1 in the reference case and in most of the subgroup analysis. Our results highlight the need to look beyond the health sector when evaluating interventions such as early intervention. Our measure of lost productivity may well be an underestimate because it did not include days in education, or measure productivity losses by carers.

Although early intervention is part of the core mental health service in Canada, England and Australia, implementation is far from uniform across regions. Other countries trying to implement early intervention face significant challenges, not least how to finance and sustain services. Another challenge is the often strict delineation between child and adolescent mental health services and general adult mental health services, with no models of care for early intervention programmes that cross this divide. This study examined whether early intervention was cost-effective in a setting where there is no transitional youth model, and early intervention is delivered using a specialist hub within a community mental health setting to an adult population aged over 18 years. Early intervention services have traditionally been restricted to younger people, yet there are a substantial proportion of people who present for care for the first time over the age of 45 years. Our study included older people (up to 65 years) in the analysis. The oldest age at onset was 63 years in both cohorts. This is a particularly relevant question from a policy perspective, as the ‘Early Intervention Access and Waiting Time Standard’ published in 2016 is being widely implemented in England,¹⁶ and the National Clinical Programme in Ireland Model of Care is also delivered up to 65 years of age.

The importance of restricting our analysis by age was evident in our subgroup findings, where early intervention was much less likely to be cost-effective when we excluded people aged over 35 years. Like many other countries, there is a strict boundary in the Republic of Ireland between child and adolescent mental health services and adult mental health services, a boundary that straddles the age range in which many mental health disorders manifest themselves for the first time. Our finding that early intervention services are less likely to be cost effective in the under 35 years age group should be interpreted with this caveat in mind.

The study has a number of strengths. Previous studies of early intervention have compared early intervention with older models of TAU. By contrast, the TAU cohort in this study received best-practice community mental healthcare, including HBT and assertive community outreach, which are both designed to deliver acute care in the community. We also used a robust methodology for the case finding and evaluation of people presenting with FEP. Apart from the possibility that some patients in the areas covered by our analysis might have attended a private mental hospital for which we did not have data, both samples are epidemiological samples of FEP presenting to community mental health services in the five catchment areas included in the study. Each person attending the early intervention service received a comprehensive diagnostic interview and assessment by trained assessors with good interrater reliability. Of those presenting to the TAU catchment area, 80% were assessed at baseline with a comprehensive diagnostic interview and assessment by trained assessors, also with good interrater reliability.

The patient-level costs included in this study were collected in a structured, standardised manner. As a year is a long time for patient recall, the information provided by patients was supplemented with information from medical records and by contacting primary care practices. Evidence shows that using patient recall alone underestimates resource use and therefore costs.³⁰

As regards limitations, the study design is not as robust as that of an RCT and there are potential sources of observed and unobserved differences between the cohorts, which might bias the results. The TAU cohort was younger and from a predominantly rural and more deprived setting with higher levels of unemployment. The early intervention cohort was a mixture of individuals from a predominantly affluent area with a relatively older population and a smaller number from deprived or rural areas. Ethnicity was not examined in this study. Although the overall trend nationally has been an inward migration of other nationalities to the Republic of Ireland over the time period between the two studies, the population has remained relatively homogenous, particularly in the catchment areas examined. We could not use duration of untreated psychosis as a covariate as this was measured differently in each cohort.

We considered using propensity score matching to simulate the conditions of an RCT design.³¹ However, the optimal conditions for propensity score matching usually require more observations in the control group than the treatment group. In this case there were more observations in the treatment group and using propensity score matching would have excluded a large number of people from the analysis. The propensity match score was used as a covariate in the initial analysis; however, as this did not yield any extra information in comparison to including the covariates themselves, the score was ultimately not included in the final model.

The healthcare service in the Republic of Ireland is a complex, hybrid system funded by taxation, private health insurance and out-of-pocket expenditure. A total of 40% of the population possess a medical card, which means they have access to free public healthcare (both primary and hospital), but there are very long waiting lists in the public system, particularly for elective services. The rest of the population are entitled to use the public hospital system, albeit with some co-payments. That segment of the population must pay for primary healthcare. Almost 50% of the population have supplementary private health insurance, which enables them to access acute hospital services in the public sector with less delay and to use privately operated hospitals.³² Most mental health services in the Republic of Ireland are delivered publicly by the Health Service Executive, although there are two private hospitals that specialise in mental disorders. We excluded any person who was admitted to either of the private mental hospitals during the course of the study. The two services that form the basis for this study, the early intervention and TAU, are both publicly funded services and free to the individual at the point of use. It is very unlikely that having private health insurance has any effect on a person's access to either of these services.

The Republic of Ireland does not yet have a reliable comprehensive national source for unit cost data. The possibility remains that some of the costs used were biased but it is difficult to say a priori the direction of any such bias. The standard outcome measure of choice in economic evaluations is the quality-adjusted life-year (QALY). One advantage of the QALY is that the cost per QALY threshold in any country provides an obvious estimate of society's willingness to pay for a QALY.¹⁸ In choosing a primary outcome for this study, we were restricted to one that could be reliably extracted from clinical records at 1 year. This is a typical limitation of doing research in a real-world setting.

There was potential selection bias in the 1-year follow-up data. Because of differences in the ethical approval from each ethics committee, the tracing procedures in each cohort were different. Not all of the eligible early intervention sample was followed up at 1 year, whereas almost all the TAU sample had follow-up at 1 year either by clinical interview or by medical records. The effect of this potential selection bias was tested by examining the total sample in the reference case analysis, and then by reconducting the analysis only in those followed up by clinical interview in both samples. There were no statistically significant different sociodemographic or clinical characteristics between those followed up by clinical interview and those followed up by clinical record. In addition, there were no statistically significant differences in the primary outcome measure. There were some differences in the cost data, and therefore in the net benefit statistic in the repeat analysis. The probability that early intervention was cost-effective shifted down and to the right; however, the trajectory of the net benefit statistic remained the same. As previously alluded to, the literature suggests that patient reports of health service resource use are often higher than resource use taken from medical records, so this suggests that the cost data in the overall sample is an underestimate of the costs in the TAU group rather than an overestimate.

This study adds to the evidence base on the economic evaluation of early intervention in psychosis. Using patient-level data from a mix of urban and rural settings, the evaluation of an intervention taking place in a real-world setting with due consideration of the context in which the study took place provide additional insights into the effects of early intervention services. Previous research found that early intervention makes its cost savings by reducing in-patient admissions. Modelling studies have shown that the societal effect of early intervention is larger than its effect on the health sector through the effects early intervention has on employment and education.¹² This study has provided evidence using patient-level data that early intervention, delivered in a real-world setting, in a mental health system that has no youth oriented specialist early intervention service, can still provide a modest INB to the health sector even when the value of preventing a relapse requiring admission is unknown. We also found that early intervention has a large INB and is extremely likely to be cost-effective when a societal perspective is taken. As mental health interventions will often affect outcomes outside the health service, such as employment, housing and education, policy makers and service planners should be aware of this, and consider alternate sources of funding mental health interventions as benefits accrue beyond the health service.