OCTET Study: flawed by type 2 error

The OCTET study overcame many legal and ethical difficulties in setting up a randomised controlled trial (RCT) of community treatment orders (CTOs). ¹ We welcome the acknowledgment of some of the limitations of the trial, but are surprised that claims are still being made that the study demonstrates that CTOs do not achieve their principle purpose of reducing relapse and readmission. ²

Imagine a hypothetical RCT comparing medication with placebo. The trial would be powered based on estimated effect size and its duration would be based on expected time for response. If, in this scenario, 25% of those in the placebo arm had inadvertently been given the active drug, and if the duration of the study had been only a third of that planned, it would be inconceivable that the investigators would claim a negative result proved the drug ineffective. Yet this is analogous to what has taken place with OCTET.

In OCTET, median length of compulsion in the community was 183 days in the CTO group v. 8 days in the Section 17 group. Although this seems to indicate that it was a trial of people who were largely either subject to long periods of community compulsion (CTO group) or only a few days of compulsion (Section 17 group), a more detailed examination brings this into question. Almost 25% of the Section 17 group were still subject to compulsion by the end of the study, and the mean length of compulsion in this group was 46 days. In the CTO group, only 50% were subject to compulsion by the end of the study, with a mean length under compulsion of 170 days. This has two main implications.

First, the difference in mean length of compulsion between the CTO group and the Section 17 group was only 125 days, or a little over 4 months. It is questionable whether this is sufficient time for any benefits of CTOs to become apparent, and presumably the initial intention had been to compare 12 months in each arm.

Second, in effect, a quarter of the control group were receiving the same type of intervention as the CTO group throughout the course of the study. Any possible benefit in the CTO group would have been offset by the same effects in a large number of control subjects, leading to a large reduction in the power of the study and to type 2 error. The sensitivity analysis does nothing to address this loss of power. We contend that given these problems, in conjunction with the broader issues of recruitment and selection, ³ it is not possible to claim that OCTET demonstrates CTOs to be ineffective.