Violence risk assessments are commonplace in psychiatric settings where psychiatrists are expected to conduct such assessments with attention to deemed best practice for predicting future aggression. Current approaches have focused on structured clinical risk assessments, such as the Historical, Clinical and Risk Management - 20 (HCR-20), Reference Webster, Douglas, Eaves and Hart1 and actuarial risk assessments, such as the Violence Risk Appraisal Guide (VRAG). Reference Harris, Rice and Quinsey2 In recent years there has been a move away from actuarial risk assessments on the grounds that they do not predict individual but rather group risk, Reference Hart, Michie and Cooke3 although more recently it has been argued that violence risk instruments are essentially interchangeable. Reference Singh, Grann and Fazel4 The majority of research has focused on community-based follow-up of discharged patients, with studies beginning to raise questions over the predictive accuracy of risk instruments when applied to patients presenting with personality disorder and/or psychopathy. Reference Coid, Ullrich and Kallis5
Our understanding of how well violence risk instruments can predict aggression occurring in psychiatric settings is limited, with research failing to assist forensic psychiatric services that manage the care of longer-term patients. Previous research has applied extremely limited follow-up periods (i.e. ranging from 24 h to 12 months), Reference Chu, Daffern and Ogloff6–Reference Arbach-Lucioni, Andrés-Pueyo, Pomarol-Clotet and Gomar-Soñes9 failed to report sensitivity with area under the curve (AUC) values Reference Chu, Daffern and Ogloff6–Reference Vojt, Thomson and Marshall10 or to control for psychiatric diagnosis. Reference Chu, Daffern and Ogloff6 Nevertheless, good predictive accuracy with both the VRAG and the HCR-20 has been reported for patients with intellectual disabilities, Reference Fitzgerald, Gray, Alexander, Bagshaw, Chesterman and Huckle8 whereas other studies suggest moderate levels of predictive accuracy of the HCR-20 over a 12-month period. Reference Arbach-Lucioni, Andrés-Pueyo, Pomarol-Clotet and Gomar-Soñes9 Findings are not consistent, with poorer predictive validity found for those presenting with mental illness and psychopathic disorder Reference Fitzgerald, Gray, Alexander, Bagshaw, Chesterman and Huckle8 and poor accuracy for personality disorder, including those comorbid with schizophrenia. Reference O'Shea, Picchioni, Mason, Sugarman and Dickens7
We aimed to test the predictive accuracy of two instruments developed to assess risk of future aggression that are widely applied in clinical practice. The study does not claim to test predictions but rather to generate these for future studies. In addition, the value of these instruments to assessing aggression risk in hospital across an extended period of time (i.e. up to 4 years) has not been previously determined, certainly not for detained patients with personality disorder. Clinicians need to be confident in their application of risk instruments considering the high-stake decisions they have to make concerning placement, level of care required to keep patients and staff safe, and increasing mandatory requirements for the use of such risk instruments.
Method
Participants were adult male psychiatric patients detained in a high secure hospital in northern England housing patients with mental illness and/or personality disorder. The average age at the time of follow-up was 54.9 years (s.d.=10.1; range 37–81). The sample was primarily White. Those convicted of a sex offence were excluded, with focus on general violence only. Only those detained on the personality disorder unit were included, resulting in 96 patients, all of whom had completed the HCR-20 Reference Webster, Douglas, Eaves and Hart1 version 2, with 75 of these also having completed the VRAG. Reference Harris, Rice and Quinsey2 All had a diagnosis of personality disorder, with 25.3% also presenting with a definite diagnosis of major mental disorder at any time point in the past (i.e. historically, before data collection), and only 7.4% deemed to have a major mental disorder at the time of initial data collection. Regressions indicated that a history of, or current evidence of, a major mental disorder was not predictive of hospital violence in this sample at the 12-month (past diagnosis, t=0.08, P<0.94; current diagnosis, t=1.62, P<0.11) or 4-year follow-up (past diagnosis, t=0.19, P<0.85; current diagnosis, t=1.10, P<0.28).
Each patient's care team, including the responsible clinician, completed the HCR-20. A single identified member of the care team completed the VRAG. Staff were trained by one of the authors of the HCR-20. Participants were followed up at 12 months and 4 years at the hospital. Incidents of aggression were recorded using the hospital clinical recording system. This allowed for incidents to be recorded by staff in accordance with the type of aggression displayed. The study captured physical aggression, verbal aggression and threatening behaviour. Self-injurious behaviour was also recorded as a measure of self-directed aggression, again, using the staff reporting system. The HCR-20 is applied as an actuarial instrument for the purpose of the current study. The application of both instruments to self-injurious behaviour and verbal aggression is novel since neither was originally designed to predict these.
Analysis was completed using SPSS for AUC and the regression analysis. MedCalc was used to compute sensitivity, specificity, positive predictor values (PPVs) and positive likelihood ratio (PLR).
Results
Table 1 shows the predictive accuracy of the two instruments' totals (HCR-20 and VRAG) using AUC, also reporting sensitivity, Reference Lobo, Jiménez-Valverde and Real11 PPV and PLR. Cut-offs for PPV were identified using sensitivity and specificity values. PLR values are included to accommodate for the lack of information on aggression prevalence in high secure settings, which arguably can lead to misleading interpretations of PPV since this is sensitive to prevalence rates. Both PPV and PLR values should consequently be accounted for in interpretation of the tables. Total scores were used to calculate AUC. The violence risk assessment literature generally considers AUC values of 0.8–0.9 as high, with some noting 0.6–0.8 as moderate. 12 AUC values are interpreted more strictly beyond this literature base, with values of 0.60–0.69 considered poor, 0.70–0.79 fair, 0.80–0.89 good and 0.90+ excellent. There is thus some noted differences in how AUC values are applied across studies and the current study recognises this.
Table 1 Predictive accuracy of HCR-20 total and VRAG total across 12 months and 4 years a
| Behaviour | 12 months | 4 years | ||
|---|---|---|---|---|
| HCR-20 | VRAG | HCR-20 | VRAG | |
| Self-harm | ||||
| AUC
(CI) Sig/s.e. (CI sig*) Sensitivity, % (95% CI) PPV, % (95% CI) PLR (95% CI) |
NS (0.39–0.76) | 0.76 0.007/0.06 (0.64–0.89) 28.6 (14.6–46) 100 (69.1–100) (n/a) |
NS (0.46–0.74) | 0.72 0.005/0.06 (0.61–0.84) 51.4 (33.9–68.6) 100 (81.5–100) (n/a) |
| Physical aggression against patients | ||||
| AUC
(CI) Sig/s.e. (CI sig*) Sensitivity, % (95% CI) PPV, % (95% CI) PLR (95% CI) |
NS (0.50–0.79) | NS (0.41–0.87) | 0.64 0.05/0.07 (0.50–0.79) 27.8 (16.5–41.6) 78.9 (54.4–93.9) 2.92 (1.04–8.14) |
0.66 0.04/0.08 (0.51–0.81) 34.3 (19.1–52.2) 66.7 (40.9–86.7) 2.29 (0.96–5.45) |
| Physical aggression against staff | ||||
| AUC
(CI) Sig/s.e. (CI sig*) Sensitivity, % (95% CI) PPV, % (95% CI) PLR (95% CI) |
NS (0.54–0.92) | NS (0.58–0.98) | 0.68 0.04/0.07 (0.53–0.82) 27.8 (16.5–41.6) 78.9 (54.4–93.9) 2.92 (1.04–8.14) |
0.69 0.03/0.08 (0.54–0.84) 34.3 (19.1–52.2) 85.7 (57.2–98.2) 6.86 (1.65–28.6) |
| Verbal aggression against patients | ||||
| AUC
(CI) Sig/s.e. (CI sig*) Sensitivity, % (95% CI) PPV, % (95% CI) PLR (95% CI) |
NS (0.47–0.79) | NS (0.43–0.77) | 0.64 0.03/0.06 (0.52–0.76) 40.7 (27.6–54.9) 70.9 (51.9–85.8) 1.90 (0.98–3.69) |
0.72 0.002/0.06 (0.60–0.84) 57.1 (39.3–73.7) 80 (59.3–93.2) 4.57 (1.92–10.90) |
| Verbal aggression against staff | ||||
| AUC Sig/s.e. (CI sig*) Sensitivity, % (95% CI) PPV, % (95% CI) PLR (95% CI) |
0.65 0.02/0.06 (0.54–0.77) 42.6 (29.2–56.8) 76.7 (57.7–90.1) 2.56 (1.21–5.38) |
0.67 0.02/0.07 (0.53–0.81) 42.9 (26.3–60.6) 68.1 (45.1–86.1) 2.45 (1.13–5.31) |
0.72 0.0001/0.06 (0.61–0.84) 48.1 (34.3–62.2) 76.5 (58.8–89.2) 2.53 (1.28–5.00) |
0.71 0.003/0.06 (0.58–0.83) 57.1 (39.3–73.7) 71.4 (51.3–86.8) 2.86 (1.44–5.66) |
| Threatening patients, AUC (95% CI) | NS (0.27–0.78) | NS (0.28–0.53) | NS (0.38–0.73) | NS (0.50–0.82) |
| Threatening staff, AUC (95% CI) | NS (0.54–0.92) | NS (0.32–0.95) | NS (0.43–0.77) | NS (0.38–0.82) |
| Total aggression (not self-harm) | ||||
| AUC Sig/s.e. (CI sig*) Sensitivity, % (95% CI) PPV, % (95% CI) PLR (95% CI) |
0.63 0.03/0.06 (0.52–0.74) 46.3 (32.6–60.4) 67.6 (50.2–81.9) 1.62 (0.93–2.83) |
0.64 0.04/0.07 (0.51–0.77) 51.4 (33.9–68.6) 66.7 (46.0–83.5) 2.29 (1.18–4.42) |
0.69 0.001/0.05 (0.58–0.80) 61.1 (46.9–74.1) 71.7 (56.5–84.0) 1.97 (1.20–3.25) |
0.77 0.0001/0.05 (0.67–0.88) 77.1 (59.9–89.6) 75 (57.8–87.9) 3.43 (1.88–6.26) |
HCR-20, Historical, Clinical and Risk Management - 20; VRAG, Violence Risk Appraisal Guide; AUC, area under the curve; PPV, positive predictor value; PLR, positive likelihood ratio; NS, not significant; sig, significance.
a (HCR-20 n=96 at 12 months and n=92 at 4 years; VRAG n=75 at 12 months and n=75 at 4 years
* Asymptomatic significance.
If less stringent violence risk assessment interpretations of AUC values are applied, Table 1 demonstrates moderate predictive validity, with AUC values closer to good (i.e. over 0.7) for the VRAG, notably for patient self-harm at 12 months and 4 years, verbal aggression towards patients at 4 years, verbal aggression against staff at 4 years and for total aggression (not including self-harm) at 4 years. VRAG performed reasonably well in relation to physical aggression towards staff at 4 years, with this producing the largest PLR, followed by verbal aggression against patients and total aggression at 4 years. The HCR-20 total produced only one AUC that was over 0.7 (staff verbal aggression at 4 years), although total aggression at 4 years produced an AUC of 0.69 (although the PLR was minimal). Both the HCR-20 and the VRAG appeared to perform better at 4 years than at 12 months. Neither discriminated between patients displaying other threatening behaviour.
Considering Table 2 and using the AUC interpretations preferred in the risk assessment literature, only the historical components of the HCR-20 produced moderate AUCs and only in relation to self-harm, verbal aggression (staff and patient) and total aggression at 4 years. The clinical and risk management component also demonstrated moderate AUC values at 4 years for staff verbal aggression. These AUC values would, however, be considered poor if more stringent AUC cut-offs were applied. The only exception was the HCR-20 risk management component for ‘other threats’ to staff at 12 months, which produced an AUC of 0.83. This was, however, based on a small number of patients reported to demonstrate such behaviour (n=6).
Table 2 Predictive accuracy of HCR-20 historical, clinical and risk management scales across 12 months and 4 years (HCR-20 n=96 at 12 months and n=92 at 4 years)
| Behaviour | 12 months | 4 years | ||||
|---|---|---|---|---|---|---|
| HCR-20 Historical |
HCR-20 Clinical |
HCR-20 Risk |
HCR-20 Historical |
HCR-20 Clinical |
HCR-20 Risk |
|
| Patient self-harm | ||||||
| AUC
(CI) Sig/s.e. (CI sig*) Sensitivity, % (95% CI) PPV, % (95% CI) PLR (95% CI) |
NS (0.46–0.77) | NS (0.31–0.68) | NS (0.35–0.70) | 0.67 0.02/0.06 (0.55–0.80) 26.4 (15.3–40.3) 77.8 (52.4–93.6) 2.84 (1.01–8.00) |
NS (0.35–0.63) | NS (0.36–0.66) |
| Patient physical aggression, AUC (CI) | NS (0.46–0.77) | NS (0.49–0.75) | NS (0.38–0.74) | NS (0.47–0.72) | NS (0.49–0.78) | NS (0.45–0.74) |
| Staff physical aggression, AUC (CI) | NS (0.53–0.83) | NS (0.39–0.96) | NS (0.52–0.86) | NS (0.52–0.76) | NS (0.49–0.78) | NS (0.42–0.76) |
| Patient verbal aggression | ||||||
| AUC
(CI) Sig/s.e. (CI sig*) Sensitivity, % (95% CI) PPV, % (95% CI) PLR (95% CI) |
NS (0.47–0.79) | NS (0.46–0.76) | NS (0.33–0.65) | 0.65 0.01/0.06 (0.54–0.77) 41.5 (28.1–55.9) 70.9 (51.9–85.8) 1.98 (1.02–3.85) |
NS (0.44–0.68) | NS (0.43–0.68) |
| Staff verbal aggression | ||||||
| AUC
(CI) Sig/s.e. (CI sig*) Sensitivity, % (95% CI) PPV, % (95% CI) PLR (95% CI) |
NS (0.50–0.73) | NS (0.47–0.70) | 0.62 0.05/0.06 (0.50–0.74) 37.3 (25.0–50.8) 73.3 (54.1–87.7) 1.72 (0.86–3.46) |
0.67 0.008/0.06 (0.56–0.77) 47.2 (33.3–61.4) 73.5 (55.6–87.1) 2.25 (1.18–4.30) |
0.66 0.009/0.06 (0.55–0.77) 45.4 (31.9–54.4) 73.5 (55.6–87.1) 2.07 (1.09–3.95) |
0.68 0.004/0.06 (0.56–0.80) 38.9 (26.5–52.5) 67.67 (49.5–82.6) 1.31 (0.73–2.36) |
| Threatening patients, AUC (CI) | NS (0.46–0.76) | NS (0.19–0.78) | NS (0.19–0.64) | NS (0.39–0.70) | NS (0.34–0.69) | NS (0.37–0.70) |
| Threatening staff | ||||||
| AUC
(CI) Sig/s.e. (CI sig*) Sensitivity, % (95% CI) PPV, % (95% CI) PLR (95% CI) |
NS (0.33–0.76) | NS (0.41–0.89) | 0.83 0.007/0.05 (0.72–0.94) 10.2 (3.8–20.8) 100 (54.1–100) (n/a) |
NS (0.39–0.70) | NS (0.41–0.77) | NS (0.40–0.78) |
| Total aggression | ||||||
| AUC
(CI) Sig/s.e. (CI sig*) Sensitivity, % (95% CI) PPV, % (95% CI) PLR (95% CI) |
NS (0.50–0.73) | NS (0.45–0.68) | NS (0.46–0.69) | 0.69 0.002/0.05 (0.58–0.79) 60.4 (46–73.5) 69.6 (54.2–82.3) 1.85 (1.14–3.00) |
NS (0.49–0.71) | NS (0.49–0.72) |
HCR-20, Historical, Clinical and Risk Management - 20; AUC, area under the curve; PPV, positive predictor value; PLR, positive likelihood ratio; NS, not significant; sig, significance.
* Asymptomatic significance.
Discussion
Our findings have implications for the use of the HCR-20 and VRAG in predicting aggression occurring within psychiatric hospitals. In keeping with research in community follow-up, Reference Coid, Ullrich and Kallis5 it would appear that these instruments are not performing markedly well across aggression types with those with a personality disorder detained in conditions of secure psychiatric care. Indeed, PLRs generally ranged from minimal to small, with the only exception a moderate value in relation to the VRAG and its prediction of physical aggression against staff at 4 years, with verbal aggression against patients at 4 years closely following. The findings are broadly consistent with other research examining aggression occurring within hospitals, although previous research has used considerably shorter follow-up periods. Reference O'Shea, Picchioni, Mason, Sugarman and Dickens7,Reference Arbach-Lucioni, Andrés-Pueyo, Pomarol-Clotet and Gomar-Soñes9 Nevertheless, there is evidence for lower predictive accuracy with participants with a personality disorder, particularly in relation to physical violence, Reference O'Shea, Picchioni, Mason, Sugarman and Dickens7 although the VRAG does seem to have some utility, particularly in relation to physical aggression against staff and with predicting patient self-harm.
Overall, it was the historical component of the HCR-20 that was presenting with some potential to accurately discriminate, even though this was limited to the 4-year follow-up and not producing PLRs beyond small values. The marginal results for the historical components as a whole were perhaps supportive of the more favourable results indicated by the actuarial risk instrument, the VRAG. The VRAG does not include dynamic risk factors but a wider range of historical factors. It suggests that the more dynamic elements of risk assessment (HCR-20) are not contributing to aggression outcomes with this very specific population, and certainly not when longer time frames are being applied. Our results indicated that the more historical and static VRAG was a better predictor than the HCR-20. Interestingly, the VRAG also demonstrated moderate (closer to good) discriminatory potential in identifying patients likely to self-harm across both time points (12 months and 4 years); it also performed moderately to good in terms of predicting verbal aggression towards patients and staff at 4 years, and total aggression at 4 years.
Our results do perhaps suggest some degree of caution in the application of risk assessment instruments to patient groups characterised by enduring challenges in personality functioning. The difficulty for clinicians is that such difficulties are associated with an increased risk for aggression and yet the risk instruments more commonly applied do not seem to be discriminating beyond at least moderate with this patient group across the longer term. Our findings also suggest that actuarial assessments cannot be considered completely without merit Reference Hart, Michie and Cooke3 when focus is on hospital-based aggression.
The environment is also an important consideration. Although placement in a secure psychiatric setting is arguably a protective factor against the expression of overt aggression, Reference Chu, Daffern and Ogloff6 nothing is reliably known about how these risk instruments are converted, if at all, into clinical practice in the longer term to manage potential aggression risk. Reference Vojt, Thomson and Marshall10 The current study did not, for example, identify any means of reliably assessing the content or quality of risk management plans put in place following these assessments; such plans are varied and can be expected to change over time. This is an obvious limitation for a long-term study. Examining this in future research, however, would be valuable and perhaps future revisions of risk assessment tools could consider a rating centred on the quality of risk plans put in place following such assessments, to what extent they were implemented and how such strategies could be evaluated effectively over time. Research has not comprehensively addressed these issues; the limited research to date has focused on very brief time periods (e.g. 24 h follow-up Reference Ogloff and Daffern13 ) and not used all components of structured risk assessments (e.g. only considering the clinical component of the HCR-20). Reference Daffern, Howells, Hamilton, Mannion, Howard and Lilly14 Thus it would be a valuable direction for future research to pursue in more detail over longer time periods.
Psychiatrists who are required to complete violence risk assessments need, however, to be mindful of the debates and associated potential limitations in using risk instruments with patients with a primary diagnosis of personality disorder in high secure psychiatric populations. Further research needs to expand on these issues by going beyond what the current study was able to provide and examining the impact of medication, incorporating neuroimaging variables and comorbid conditions of potential interest such as epilepsy. Indeed, we recognise that the current sample is a highly specialised one, namely a high secure sample with long-standing issues relating to aggression; this does make it particularly challenging for a generally validated risk assessment tool to predict aggression owing to the specifics of this population. The lack of generalisability outside of high secure settings is certainly acknowledged. There is also a need to expand the current research by considering the role of personality clusters in determining aggression risk. Although controlling for those individuals presenting with more than one cluster may be challenging among a sample where more than one personality disorder is common, it would still remain a valuable avenue to explore. Future research could examine the specific nature of individual personality disorder traits and how they associate with aggression in more detail. It could also extend to considering the role of personality functioning, as promoted by DSM-5, 15 and where functioning challenges may impact on aggression risk. This would represent a novel and valuable area of study to consider.
Open access
eLetters
No eLetters have been published for this article.