Author's reply

In his letter about the article ‘Cost-effectiveness of early intervention in psychosis: systematic review’,¹ Andrew Amos defines the review as unsystematic, uncritical about the included literature and, ultimately, as an example of the use of spin to misrepresent the advantages of early intervention for psychosis (EIP) services.

As authors, we are pleased to see diverse opinions regards this work, which enriches the discussion and makes the topic more complex, as precisely analysed by Robert Rosenheck in his editorial.² However, there are some aspects in Amos’ letter that are not entirely correct or frankly misleading; therefore, we believe it is important to clarify.

First, this review adhered to a high-quality standard, following the recommended reporting guideline (PRISMA)³ and registering a protocol before starting the review. Information necessary for replicability is available to any reader in the main text and supplementary material. Our search strategy was comprehensive, including six databases, and two authors independently screened and applied previously stated eligibility criteria to reduce ‘cherry-picking’ of studies. Risk of bias assessments were conducted using widely validated instruments. This contrasts with the studies cited by Amos.^4,5 In both reviews he is the only author, in one of them only one database was searched and none of them have pre-registered protocols. Although the author recognised this limitation in a letter published in 2012,⁶ it seems this did not prevent him from applying the same method in 2014.

Regarding the included studies, Amos mentions that these were not critically analysed. We used three different instruments to appraise the risk of bias of the included studies. One was the widely applied Cochrane's risk of bias tool, to assess the effectiveness estimates, and two were specific tools used in economic evaluations (trial and model-based cost-effectiveness analyses).^7,8

Furthermore, we explicitly highlight the methodological deficiencies of the included studies in terms of internal validity and applicability to low-resource settings. In fact, we further specify that a meta-analysis would have been misleading considering the high heterogeneity of the studies (p. 389).

Amos mentioned that we did not highlight the risk of bias in Tsiachristas et al’s study,⁹ which is not a randomised controlled trial but an observational study. However, in this work propensity score matching was used to deal with confounding to account for the imbalanced samples. Although this technique does not eliminate other sources of confounding, it is a valid procedure to make inferences using observational data.¹⁰ Nevertheless, we still classified this study as at-high risk of bias, as it is clearly depicted in the supplementary material.

There are other aspects that we believe require clarification. Cost-minimisation studies are not the only type of economic evaluation with which we can affirm that an intervention is cost-saving. Besides, the limitations of such an approach have been largely stated and was the rationale for excluding such evaluations.¹¹

Likewise, it is not completely accurate to say that an intervention is not cost-effective if the treatment is not effective. Besides the obvious scenario where the new treatment is cheaper, there is also an option when the new treatment does not reach statistical significance and might be considered cost-effective. This is because costs and effects are measured with uncertainty, which is usually characterised using probabilistic sensitivity analyses, and it may be the case where a relevant proportion of these simulated samples lay above the threshold defined by a given country.^12,13

Leaving aside these technicalities, it is worth noting that Amos’ assertion about the ‘fact that the current consensus is that it is not possible to prevent transition to psychosis’ is questionable. The same reference cited in his letter is specific in stating that psychological interventions may reduce the risk of developing psychosis in people with clinical high risk by a half.¹⁴ It is true that this effect is not sustained at 2 years of follow-up, but this time window might be enough to make an intervention reach cost-effectiveness.

Besides, the interventions at this stage not only seek to prevent the first-episode of psychosis (FEP) but also engage young people with services, reduce comorbidities (including substance misuse disorders), decrease the duration of untreated psychosis and ameliorate the impact of the FEP by, for example, using less admissions to hospital and compulsory admissions.

Finally, there is a topic not covered in Amos’ letter but closely related to his strong accusation of considering this systematic review as an example of spin. This is about conflicts of interest and research allegiance of reviewers. This has been highlighted in other reviews of psychological therapies.¹⁵ In this regard, we can affirm that our review team was made up of health service researchers, health economists and epidemiologists with no financial or non-financial conflicts of interest. Likewise, only one study included was conducted by one of the authors of the review, who was not involved in rating the risk of bias of the studies. Furthermore, we were explicit about the fact that most of the studies were conducted by advocates of the EIP paradigm.

As authors, we welcome critical analysis and feedback for this and future work. We believe, nevertheless, that such criticisms should be stated in a constructive and collaborative manner with the focus on improving research and ultimately, patients’ well-being and quality of life.