A systematic review and meta-analysis of the effect of community treatment orders on aggression or criminal behaviour in people with a mental illness

Abstract

Aims

There has been concern about violent acts and other criminal behaviour by people with a possible history of mental health problems. We therefore assessed the effects of community treatment orders (CTOs) on self-, third-party-, and agency-reported criminal behaviour when compared to voluntary treatment.

Methods

A systematic search of PubMed/Medline, Embase, PsycINFO and criminal justice bibliographic databases for observational or randomised controlled trials (RCTs) comparing CTO cases with controls receiving voluntary psychiatric treatment. Relevant outcomes were reports of violence and aggression or contacts with the criminal justice system such as arrests and court appearances.

Results

Thirteen papers from 11 studies met inclusion criteria. Nine papers came from the United States and four from Australia. Two papers were of RCTs. Results for all outcomes were non-significant, the effect size declining as study design improved from non-randomised data on self-reported criminal behaviour, through third party criminal justice records and finally to RCTs. Similarly, there was no significant finding in the subgroup analysis of serious criminal behaviour.

Conclusions

On the limited available evidence, CTOs may not address aggression or criminal behaviour in people with mental illness. This is possibly because the risk of violence is increased by comorbid or nonclinical variables, which are beyond the scope of CTOs. These include substance use, a history of victimisation or maltreatment, and the wider environment. The management of risk should therefore focus on the whole person and their community through social and public health interventions, not solely legislative control.

Introduction

Community treatment orders (CTOs) allow compulsory treatment in the community for people with mental illness. They are an example of civil commitment, as opposed to forensic orders following criminal proceedings. CTO use in Australia and New Zealand is high by international standards although rates vary considerably within both countries (Light et al., 2017; O’Brien, 2014). Concerns have been raised regarding ethical and human rights implications of the CTOs (Brophy et al., 2021).

One of the other main criteria for CTO placement is the imminent danger of harm to self or others. Despite this, most of the relevant systematic reviews and meta-analyses on CTOs concentrate on health outcomes and do not consider violent acts or other criminal behaviour. There have been just three systematic reviews that have considered criminal behaviour or aggression in comparisons of CTO cases with voluntary controls, generally as a secondary outcome. The first was a Cochrane Systematic Review that was restricted to just three randomised controlled trials (RCTs) (Kisely et al., 2017). Because of the challenges of conducting RCTs in this area, it is unlikely that the participants were representative of people on CTOs, especially as dangerousness is generally an exclusion for entry into an RCT. The two other systematic reviews considered observational trials, the results of which may be more generalisable. However, one was published 17 years ago and the literature search in a second completed in 2019 (Churchill et al., 2007; Segal, 2020). As criminal behaviour was not the primary outcome, neither review attempted to meta-analyse the data nor extend their search to bibliographic databases specific to criminology.The two reviews came to very different conclusions. The earlier one concluded that CTOs were not associated with significant changes in criminal behaviour or aggression and therefore questioned their utility. By contrast, the more recent review reported significant reductions in both violence and crime in mirror image and controlled studies, concluding that these results were evidence of the benefits of CTOs. However, this review was limited by methodological concerns such as not following Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines (PRISMA) (Kisely et al., 2024; Moher et al., 2009). There may also have been further developments in the area. Given the continuing uncertainty, we therefore assessed the possible effects of CTOs on self-, third-party-, and agency-reported criminal behaviour when compared to voluntary treatment.

Method

Search strategy

The protocol for this systematic review was registered with PROSPERO (CRD42020155996). We followed PRISMA guidelines (Moher et al., 2009). Given the limitations of the most recent systematic review, we updated the search by Churchill et al. (2007) of PubMed/Medline, Embase and PsycINFO (2005 onwards). Following consultation with a professional librarian, we also searched Criminal Justice Abstracts, the Australian Criminology Database and ProQuest from inception onwards. All the searches were completed in June 2024 and restricted to peer-reviewed papers published in English. Table 1 shows our search terms that were finalised with the help of the same professional librarian. Ethical approval was not required for this systematic review, as all included primary data had previously been published.

Table 1. Search terms

Two of the authors (SK and CB) independently screened records and abstracts. A third reviewer (NG) was consulted in the case of any disagreement and consensus was achieved in all cases. The reference lists of selected retrieved papers were screened to identify additional studies that met inclusion criteria.

Inclusion criteria

We included any of the following study designs that compared people on CTOs for severe mental illness with contemporaneous controls receiving voluntary psychiatric treatment: RCTs, cohort, case control and cross-sectional studies.

Exclusion criteria

We excluded studies of inpatient treatment (including extended leave), forensic orders and compulsory treatment in the community for drug or alcohol related disorders, as well as those that did not have controls receiving voluntary psychiatric treatment.

Outcomes

We extracted data for the following outcomes: 1) violence or aggression measured, where possible, with standardised instruments; 2) contacts with the criminal justice system such as arrests or court appearances following CTO placement. Outcomes could be self-, third-party-, or agency-reported criminal behaviour. We focused on outcomes at one year as this is the most common endpoint in the literature and the impact of an intervention on health service beyond one year is difficult to ascertain (Kisely et al., 2017). However, we also assessed for the presence of these events up to 10 years from entry into the study.

Study quality

All studies identified for inclusion were cohort studies and RCTs. Two of the authors (SK and CB) independently assessed quality using the appropriate Joanna Briggs Institute tool for each. Both cover the following areas: selection of the study groups in terms of case definition, representativeness and similarity of controls; comparability of the groups such as the use of matching or multivariate techniques, and measurement of exposure and outcomes in a valid and reliable way. This included consideration of allocation concealment and blinding in the case of RCTs. The version for cohort studies has 11 items and the one for RCTs, thirteen. In the case of RCTs and quasi-experimental studies, we only appraised the primary outcome as it related to the aim of our review.

Similarly, we used the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework to assess the overall certainty of evidence from very low to high. Outcomes from RCTs were initially graded as high and those from observational studies as low. These were then graded up or down depending on study quality, inconsistency, indirectness, imprecision and publication bias.

Analysis

Where data were available for two or more studies, they were combined in a meta-analysis giving preference to data that had been adjusted for potential confounders. We used log odds ratios (ORs) to pool dichotomous and continuous data using inverse variance. We used an I ² statistic value of greater than 50% as an indicator of significant heterogeneity. We explored any heterogeneity further through sensitivity analyses of the effect of omitting each study in turn. The random effects model was used for all the analyses because we could not definitively exclude between-study variation even in the absence of statistical heterogeneity. We ran separate analyses for criminal justice contacts and reports of violence or aggression. We did not combine analyses from non-randomised designs and RCTs. If recorded, we undertook subgroup analyses of criminal behaviour coded as being severe. Finally, we undertook sensitivity analyses of the effect of excluding studies of lower quality such as those that only reported unadjusted or selectively chosen results.

Results

We found 3,929 citations of interest in the updated search. Of these, 23 full-text papers were potentially relevant and assessed for eligibility (Fig. 1). Eight papers met inclusion criteria (Gilbert et al., 2010; Link et al., 2011; Ogilvie and Kisely, 2022; Phelan et al., 2010; Pollack et al., 2005; Segal, 2019; Segal et al., 2023; Swanson et al., 2001). Reasons for exclusion were that records were unpublished theses or were not of a relevant design, setting and outcome (Fig. 1). Adding these studies to the previous search meant that there were 13 papers from 11 studies in total as different aspects of one study were reported in two papers (Fig. 1 and Table 2).

Figure 1. PRISMA diagram.

Table 2. Included studies

CTO = Community Treatment Order, OR = Odds Ratio, SEIFA = Socio-Economic Indexes for Areas, CBA = Controlled before and after, RCT = Randomised Controlled Trial

Of the 13 papers, nine were from the United States (Gilbert et al., 2010; Hiday and Scheid-Cook, 1987, 1989; Link et al., 2011; Phelan et al., 2010; Pollack et al., 2005; Steadman et al., 2001; Swanson et al., 2000, 2001) and four were from Australia (Ogilvie and Kisely, 2022; Power, 1992; Segal, 2019; Segal et al., 2023). Five studies had follow-up periods of up to 12 months, otherwise follow-up ranged between 36 months and 12.4 years or was unspecified (Table 2). Details for one study (Power, 1992) were only available from a subsequent systematic review (Churchill et al., 2007).

Three papers were from two RCTs (Steadman et al., 2001; Swanson et al., 2000, 2001). In the case of the RCT from North Carolina (Swanson et al., 2000, 2001), randomised results were supplemented by several post-hoc analyses. In the first, they analysed a non-random sample who underwent more than 180 days of CTO placement (Swanson et al., 2000). However, analysis of a group that has not been randomly assigned to treatment of less or more than 180 days may reflect a bias when the order was selectively extended when it appeared to be of benefit. In the second, a non-random group of individuals who were judged too violent to be included in the RCT were incorporated into a series of multiple staged, stepwise logistic regression models (Swanson et al., 2001). Notably, usable data on arrests were only available for a subgroup of participants with a prior history of multiple admissions combined with prior arrests and/or violent behaviour, not the whole sample (Swanson et al., 2001).

Overall, study quality was no more than moderate (Table 2, Supplementary Material 1). All but two studies adjusted for potential confounders through matching, adjustment or randomisation (Hiday and Scheid-Cook, 1987; Segal et al., 2023).

Outcomes

All but two studies used externally recorded criminal justice contacts such as arrests or court appearances as measured by administrative databases (Table 2). The two remaining studies used standardised measures (Phelan et al., 2010; Power, 1992). In a further two studies, externally recorded criminal justice data were supplemented by information from participants, families, staff or hospital records (Hiday and Scheid-Cook, 1987; Swanson et al., 2000). Three studies considered serious criminal behaviour (Link et al., 2011; Segal, 2019; Segal et al., 2023).

Table 3 displays the results indicating either improvement, decline, or no change, for people on CTOs compared to controls in terms of violence reported by participants or others, as well as criminal justice contacts, with subgroups for serious and minor criminal behaviour. Unadjusted results showed increases in criminal justice contacts while randomised data uniformly showed no significant difference. The results for controlled trials with adjusted analyses either showed a significant decrease or no difference.

Table 3. Studies by outcome

Unadj = Unadjusted; Adj = Adjusted; RCT = Randomised Controlled Trial

Figure 2 shows the results of the meta-analysis. There were no significant differences for any of the outcomes between CTO cases and voluntary controls, the effect size declining as study design improved from non-randomised data on self-reported criminal behaviour, through third-party criminal justice records and finally to RCTs. Similarly, there was no significant findings in the subgroup analysis of serious criminal behaviour (OR = 1.01; 95% CI = 0.52–1.97; p = 0.97; I ² = 98%; k = 3).

Figure 2. Aggression or criminal justice contacts.

In terms of sensitivity analyses, there was no change in the results when we excluded the study where details on aggression were derived from secondary sources (OR = 0.54; 95% CI = 0.17–1.75; p = 0.30; I ² = 80%; k = 3), or when analyses were restricted to studies that presented adjusted results for informant reported outcomes (OR = 0.49; 95% CI = 0.22–1.05; p = 0.07; I ² = 65%; k = 3) and criminal justice contacts (OR = 0.78; 95% CI = 0.53–1.15; p = 0.21; I ² = 58%; k = 6). There was also no change in the results when we excluded the study that only had usable data from selected subgroups of participants rather than the whole sample (OR = 0.94; 95% CI = 0.62–1.42; p = 0.76; I ² = 95%; k = 7).

Heterogeneity, publication bias and certainty of evidence

All the above results showed significant heterogeneity apart from those of criminal justice contacts as reported in RCTs and the subgroup analysis of adjusted informant reported outcomes. We further explored this by excluding each study in turn in every analysis but this did not result in any I ² value of less than 50%. However, removal of one study from the controlled trials of criminal justice contacts did reduce the I ² value from 94% to 56% (Segal et al., 2023). We were unable to analyse for the effects of publication bias as none of the analyses had 10 or more studies. As most of the included studies were observational, both outcomes (violence/aggression and criminal justice contacts) were initially rated as being of low certainty. This was downgraded to being very low given high heterogeneity (inconsistency), and because some of the studies had low numbers (imprecision) or other outcomes as the primary focus (indirectness).

Discussion

This is the first such meta-analysis of the effect of the CTOs on aggression and criminal behaviour. We only included studies with contemporaneous controls as mirror image designs are subject to regression to the mean and may overestimate any effects (Eccles et al., 2003). For instance, in the most comprehensive meta-analysis of the effect of CTOs on health outcomes, before-and-after studies showed changes in health service use, that were no longer apparent in RCTs or observational studies with contemporaneous comparisons (Barnett et al., 2018). Mirror image studies may also be affected by secular trends or sudden changes in practice or policy that are unrelated to CTOs. We identified 13 papers from 11 studies, of which two were RCTs. There were no significant differences for any of the outcomes between CTO cases and voluntary controls, the effect size declining as study design improved from non-randomised data on self-reported criminal behaviour, through third party criminal justice records and finally to RCTs. Similarly, there was no significant finding in the subgroup analysis of serious criminal behaviour.

Both CTOs and forensic orders have similarities in balancing patient autonomy with reducing symptoms and the potential harm to self and others. However, the use of CTOs is much higher than that of forensic orders and, as a result, it is important to assess any potential benefits in terms of criminal behaviour (Gill et al., 2020). This is also important because of the human rights concerns regarding the use of CTOs. The United Nations Convention on the Rights of Persons with Disabilities (CRPD), which Australia and New Zealand have ratified, requires the States Parties to ensure that people with disabilities enjoy legal capacity on an equal basis with others (article 12) (United Nations, 2006). It encourages supported, rather than substitute decision-making and requires that any measures related to the exercise of legal capacity must be proportional and tailored to the person’s circumstances, apply for the shortest time possible and subject to appropriate and effective safeguards. Article 14 of the CRPD states that the existence of disability shall not justify a deprivation of liberty, and any deprivation of liberty must be on an equal basis with others. While some have interpreted the CRPD as prohibiting involuntary psychiatric treatment under all circumstances, a balanced and realistic interpretation requires that involuntary psychiatric treatment must be used only as the last resort, with strong safeguards and for the shortest possible time to promote safety and the right to health, where less restrictive interventions cannot achieve that outcome (Gill and Sartorius, 2024). This raises ethical concerns on long term restriction of autonomy and liberty of individuals in the community through legislative mechanisms like the CTOs and requires empirical analysis of the evidence of their effectiveness.

Qualitative research illuminates the implications that CTOs have on human rights. In their meta-synthesis of stakeholder perspectives – including those of service users, relatives, mental health professionals and psychiatrists – Goulet et al. (2020) highlight that many recipients of CTOs perceive them to be a mechanism of medication compliance, and a means of ensuring community mental healthcare contact. While many see the value of CTOs in improving treatment adherence, and protecting recipients and others from harm, others (including service providers) raise concerns of outright legal abuse and coercion (Goulet et al., 2020). That is, some believe that the legal criteria for enforcing CTOs are too rigid, and greater flexibility would enhance the human rights of recipients. Significant variations in CTO exist across the jurisdictions in which they are used, reinforcing concerns about the ethical justification of CTOs, particularly if recipients did not break the law in the first instance (Rugkåsa et al., 2016).

Although some people with severe mental illness may be at greater risk of aggression, it is often comorbid substance or alcohol use and conduct or personality disorder that are the major drivers of either violence or contacts with the criminal justice system (Swanson et al., 2008; Witt et al., 2013). Using CTOs to improve adherence to the treatment of positive symptoms of psychosis does have a role but other contributors merit consideration (Swanson et al., 2006). These include homelessness and a history of criminal behaviour, violence, victimisation and sexual or physical child maltreatment, as well as a parental history of criminal involvement or alcohol use (Witt et al., 2013). Wider social factors may also be relevant such as exposure to violence in the surrounding environment (Swanson et al., 2002). There are similar findings when analyses are restricted to severe violence (Witt et al., 2013). This therefore points to the need to also address these factors even though they are less easily modifiable through direct clinical intervention.

Further research should therefore consider the role of interventions that address previous victimisation and maltreatment, comorbid alcohol or substance use and the wider social context such as homelessness. The human rights framework adopted by the CRPD requires the ratifying countries to promote the rights to habitation and rehabilitation, education, employment, health, adequate standard of living and social inclusion of persons with disabilities (Gill, 2019). Most mental health laws, however, continue to focus on civil commitment through legislative mechanisms such as CTOs rather than these economic, social and cultural rights (McSherry, 2014).

Simply using legislation to enforce treatment is unlikely to achieve the twin goals of promotion of community safety and protecting the human rights of the individuals with mental illness. Promotion of the economic, social and cultural rights of people with mental illness, through provision of housing, access to health services and other social goods is more likely to pave the way for protection of civil and political rights and lead to a safer and just society. This is especially relevant when there is no evidence that legislative mechanisms like CTOs lead to a reduction in aggression or criminal behaviour, as found by this meta-analysis. Alternative approaches therefore should give greater emphasis to addressing social determinants rather than solely focussing on individuals (Kirkbride et al., 2024). One example from the United States was a community building initiative for young people that reduced substance use, violence, delinquency, and behavioural issues (Kuklinski et al., 2015).

Limitations

All but two of the included studies were observational, and many were of administrative health data. These may be subject to recording bias and lack information on social aspects of disability. Cases and controls may also have differed in ways for which it was not possible to match or adjust. It is therefore possible that the CTO cases were more seriously ill, or at greater risk of aggression and other criminal behaviour than the controls. The reported levels of aggressive or criminal behaviour may therefore still represent a reduction from what might have been the case if these individuals had not been on a CTO. On a related issue, there was no information on relevant comorbidities such as antisocial personality disorder. The focus of this review was on 12-month outcomes as this was the timeframe considered by most of the included studies. This may have under-estimated any benefits that might have arisen from actual CTO placement if participants had been on voluntary treatment for part of the relevant follow-up period.

Many of the current studies relied on data originally collected for other purposes, not specifically for studying the relationship between mental illness and violence. For instance, findings of higher levels of criminal justice contact in people with mental illness may reflect that they are at an increased risk of being monitored and arrested, rather than of committing offences. The lack of evidence of any effect may also have been due to the limited quality of the available data as highlighted by the very low rating for the certainty of evidence using the GRADE framework. More studies are therefore indicated with violence as the primary outcome and sufficient power to identify a range of relevant risk factors. Where possible, outcomes should be from external sources such as criminal justice records.

The majority of our meta-analyses showed a high degree of heterogeneity. Although we tried to accommodate this with random effects models, our results should still be viewed with caution. Finally, we were unable to test for publication bias as none of the outcomes had 10 or more studies.

Conclusion

On the limited available evidence, there remains uncertainty regarding the role of CTOs in addressing aggression or criminal behaviour in people with mental illness. Given that the risk of violence is increased by comorbid or nonclinical variables, including victimisation, maltreatment and the wider environment, management should also focus on the whole person and their community through social and public health interventions, not just legislative control.