Electroconvulsive therapy for depression: a review of the quality of ECT versus sham ECT trials and meta-analyses

Background

Electroconvulsive therapy (ECT) is still being administered to approximately a million people annually. There have been no ECT versus simulated ECT (SECT) studies since 1985. The five meta-analyses of ECT versus SECT studies all claim that ECT is more effective than SECT for its primary target, severe depression. This review assesses the quality of those meta-analyses and of the 11 studies on which they are based.

Methods

The meta-analyses were evaluated primarily in terms of whether they considered the quality of the studies they included, but also in terms of whether they addressed efficacy beyond end of treatment. The methodological rigor of the 11 studies included by one or more of the meta-analyses was assessed using a 24-point Quality scale developed for this review.

Results

The five meta-analyses include between 1 and 7 of the 11 studies. The meta-analyses pay little or no attention to the multiple limitations of the studies they include. The 11 studies have a mean Quality score of 12.3 out of 24. Eight scored 13 or less. Only four studies describe their processes of randomization and testing the blinding. None convincingly demonstrate that they are double-blind. Five selectively report their findings. Only four report any ratings by patients. None assess Quality of Life. The studies are small, involving an average of 37 people. Four of the 11 found ECT significantly superior to SECT at the end of treatment, five found no significant difference and two found mixed results (including one where the psychiatrists reported a difference but patients did not). Only two higher Quality studies report follow-up data, one produced a near-zero effect size (.065) in the direction of ECT, and the other a small effect size (.299) in favor of SECT.

Conclusions

The quality of most SECT–ECT studies is so poor that the meta-analyses were wrong to conclude anything about efficacy, either during or beyond the treatment period. There is no evidence that ECT is effective for its target demographic – older women, or its target diagnostic group – severely depressed people, or for suicidal people, people who have unsuccessfully tried other treatments first, involuntary patients, or children and adolescents. Given the high risk of permanent memory loss and the small mortality risk, this longstanding failure to determine whether or not ECT works means that its use should be immediately suspended until a series of well designed, randomized, placebo-controlled studies have investigated whether there really are any significant benefits against which the proven significant risks can be weighed.