Authors' reply

One of the founding principles of meta-analysis is to pool data from as many studies as possible.¹ Among other benefits this prevents studies being preselected for consideration on arbitrary grounds. It is difficult to imagine anything more arbitrary than restricting a meta-analysis of CBT for schizophrenia to studies that conform to some notional interpretation of the NICE guideline, as Peters seems to be suggesting, not to mention excluding any that were in Chinese.

Similarly, it would be wrong to exclude studies that used group CBT a priori. Here, though, it is entirely legitimate to examine this issue post hoc; that is, to ask whether use of group v. individual CBT significantly moderates effect size. Carrying out this analysis on our data reveals that the pooled effect sizes for both types of intervention were very similar in the meta-analysis of overall symptoms (effect size in 7 group studies –0.24 v. –0.23 in 24 individual studies; Q = 0.006; P = 0.94); for positive symptoms, group CBT had a non-significantly smaller effect size than individual CBT (effect size in 8 group studies –0.08 v. –0.25 in 23 individual studies; Q = 1.73; P = 0.19) (across both analyses, one study employed both group and individual CBT and three were rated as ‘unclear’). This might or might not be considered evidence that group CBT is less effective than individual CBT, but what it does not mean is that inclusion of group studies in our original meta-analyses somehow acted to dilute the pooled estimate - the effect sizes for studies using individual CBT are similar or lower to those we reported for all studies combined (effect sizes were –0.33 for overall symptoms and –0.25 for positive symptoms).

With regard to some of the other points raised by Peters, our diagnostic criteria were broad and similar to those used by NICE, Wykes et al and the Cochrane Collaboration. We recognised the possibility that Acceptance and Commitment Therapy might be different from regular CBT and presented an analysis in the article excluding two studies using this^2,3 and another where CBT took the form predominantly of coping skills enhancement;⁴ this did not materially affect the results. Peters expresses surprise over our decision to exclude studies that specifically targeted hallucinations from the meta-analysis of positive symptoms. As it happens, only three studies of hallucination-directed CBT also reported outcomes for positive symptoms. Adding the data from two of them^5,6 (data cannot be extracted from one study⁷) to the positive symptoms dataset makes no difference to the pooled effect size (–0.25; CI –0.36/–0.13).

Peters argues that there was too much heterogeneity among the results to obtain meaningful pooled estimates. In fact, the Cochrane Collaboration article she cites⁸ recommends (a) not pooling data using meta-analysis, (b) investigating heterogeneity using subgroup analysis or meta-regression or (c) using a random-effects model for meta-analysis, as this includes consideration of heterogeneity in the effect-size estimate. The authors also note that ‘even though a random-effects model helps to consider heterogeneity, it does not remove it - heterogeneity still needs to be considered in interpreting the results’. We used a random-effects model and examined heterogeneity.

We would like to reiterate that for those who wish to examine for themselves other points of the type raised by Peters, a detailed database of the studies we included is available online (http://www.cbtinschizophrenia.com/).