Risk assessment: ‘numbers' and ‘values’

Risk assessment has two components, which I shall term ‘numbers’ and ‘values’. ‘Numbers’ refer to the estimation of the likelihood that an adverse event will occur in a stated period of time. The methods are mathematical and statistical. ‘Values’ refer to the processes of attaching a value to the risk and deciding what should be done about it. Benefits are weighed against costs in what is largely a moral enterprise. Maden (2003, this issue) asks ‘why all the fuss?’ about standardised risk assessment. My fuss is largely about the ‘values’, not so much about the ‘numbers’.

Numbers

Maden makes many useful points about risk estimation. He argues for a scientific approach to establishing best instruments - structured, standardised, replicable and tested on appropriate populations. He points out that transparent assessments are preferable to vague, clinical judgements. The development of such a risk assessment instrument is a major task, as its precision in each population needs to be separately tested. Maden is rather equivocal, however, about the scope for improved accuracy. At one point, he argues against producing yet another measure - later, he suggests we examine the effects of incorporating ‘dynamic’ factors.

The ‘base rate’ problem is also correctly identified. Maden points out that risk assessment is of little value when the base rate of violence in the population being tested is low (as it is, for example, in general community patients). However, many still fail to appreciate the astonishing effect of base rate on the accuracy of prediction. I illustrate this in Table 1, where the ‘positive predictive value’ of three tests is shown (that is, the proportion of patients the test predicts will be violent who actually turn out to be violent). For example, using the average precision of the 24 predictive studies assessed by Buchanan and Leese (2001), if 5% of the population of interest is violent, the test will be wrong 92 times out of 100. An immense effort went into the MacArthur study, which prospectively examined a host of risk factors for violence following discharge from hospital of over 900 patients (Monahan et al, 2001), and I believe it is unlikely that better accuracy can be achieved. After all, unlike engineers who must predict the release of toxic waste from an industrial plant, mental health workers are trying to predict an act committed by a person, an agent with intention, who is engaged in ongoing myriad and complex interactions with others. Note that settling for a reduced ‘sensitivity’ means that fewer violent persons will be detected. For a relatively simple way of understanding the mathematics, see Szmukler (2001).

Table 1. How the positive predictive value varies with the base rate of violence in the population of interest

	Positive predictive value (PPV)
Base	Sensitivity 0.52	Sensitivity 0.75	Sensitivity 0.27
rate	Specificity 0.681	Specificity 0.752	Specificity 0.983
1%	0.02	0.03	0.12
2%	0.03	0.06	0.22
5%	0.08	0.14	0.42
10%	0.15	0.25	0.60
15%	0.22	0.35	0.70
20%	0.29	0.43	0.77
25%	0.35	0.50	0.82
30%	0.41	0.56	0.85

Values

The action to be taken following estimation of the likelihood of the risk is fundamentally determined by a question of values, often with political dimensions. For example, what false positive rate is acceptable? Who should decide? Of special relevance to the prediction of violence in the mentally disordered are the potentially serious consequences of being wrongly classed as dangerous. There are also other more general costs. Risk assessment might emphasise control and containment at the expense of treatment; divert resources towards those assumed risky and away from the majority of those with mental illness; reinforce stereotypes of mentally ill people as dangerous; and deter people who would benefit from mental health services, including risky persons, from seeking help for fear of coercive treatments.

But there are still more troubling aspects. Risk management decisions weigh up the costs and benefits to society in general, or to particular social groups, of particular courses of action aimed at controlling risk. In the case of mental disorders, we are weighing the benefits to society as a whole against costs that largely fall on a small segment of the population - those with mental illnesses. Given society's long history of prejudice against mentally ill individuals, the threat to this socially-excluded group is very worrying.

Even more fundamental is an issue that reveals a discrimination against people with mental disorder that is rarely challenged. The ethical principle in question is that of ‘fairness’ or ‘justice’. The principle of justice, defined by Aristotle more than 2000 years ago, is that ‘equals should be treated equally and unequals unequally’. The circles in Figure 1 will help us to understand these problems.

Figure 1. The relationships between mental disorder and risk of violence in the population ‘a’=area of overlap between mental disorder and perceived as dangerous

Within the large circle, representing the entire population, are two overlapping circles. One defines those who are perceived as dangerous and the other defines those who suffer from a mental disorder. At present, it is only those within the area ‘a’ where danger and mental disorder overlap, who are eligible for preventive detention or other restriction (under the Mental Health Act 1983) because they are judged to present an unacceptable risk to others. Unlike the rest of us, this is so even if they have not yet committed an offence. Can we justify the preventive detention of people with mental disorders on the grounds of their risk to others, but not the remainder of persons, equally dangerous, shown in the cross-hatched ‘dangerous’ circle?

The answer is no. The common conjunction in mental health legislation of two grounds for the involuntary treatment of mentally ill persons - first, for their ‘health or safety’ or second, for the ‘protection of others’ - confuses two distinct justifications. The first involves a health interest for the patient; he/she is to benefit from treatment he/she is unable to recognise a need for. The second involves a public protection interest, in which the health interests of the patient are secondary (or even non-existent). This turns on the question of risk, not treatment. Unless all of us are equally liable to preventive detention for posing the same level of risk to others, irrespective of whether we are mentally disordered or not, we discriminate against those who have the designation of mental disorder. This designation is, of course, elastic. Society will always seek to include within this rubric those whom it fears or wishes to exclude, so they can be detained for long periods without the protections of the full criminal justice process. Some will argue that the ‘treatability’ of those with mental disorders is a justification. This cannot be the case. Our secure hospitals are filled with large numbers of apparently dangerous mentally ill people who have proved resistant to treatment. Nor is there any reason not to believe that non-mentally disordered, dangerous persons may be just as likely, if not more likely, to respond to ‘psychosocial treatment’ programmes to reduce their risk of violence (e.g. structured groups for those who habitually drink and drive, interventions based on exposure to victims, etc.).

Discrimination can be avoided in two ways: first, by making legislation generic so that all persons are liable to preventive detention on the basis of having exceeded a particular threshold in the risk they pose to others; that is, all those in the ‘dangerous’ circle in Fig. 1, not just those in the segment ‘a’; or second, not allowing preventive detention for anyone.

In relation to risk assessment, the argument for fairness has special relevance. If persons with mental disorder should not be singled out as liable for preventive detention on the basis of risks to others, it follows that risk assessment to predict who will be violent should not be restricted to this group of persons (i.e. to only those in the ‘mental disorder’ circle in Fig. 1). If we wish to reduce violence in our community and if we are to use risk assessment to help, then all of us (i.e. everyone in the large outer circle in Fig. 1) should be equally liable to be assessed when there is some kind of ‘trigger’ event indicating that such testing is appropriate. How this might be implemented is unclear. People subject to a risk assessment might be all those who have been involved in a violent incident of any kind (for example, persons with injuries seen by general practitioners, accident and emergency departments, units dealing with trauma, or by police), all those who misuse substances (including alcohol), all those who have been involved in a road traffic accident, those who have been the subject of accusations of threatening or aggressive behaviour, say by neighbours or in the workplace, and so on. Risk assessments would thus become the business of many agencies - health professionals (not just mental health) in any setting, the police and social agencies (including local authority services, homeless persons units, employment offices). ‘Neighbourhood watch’ schemes could assume a whole new dimension.

I doubt that many of us would support such an approach, even if the ‘numbers’ were able to lead to accurate prediction. Yet we accept such procedures in the case of those with mental disorder. This double standard exposes the extent to which we discriminate against those with mental illness, devaluing their rights compared with ours.

Conclusions

One conclusion is unlikely to be contentious. Risk factors for violence are derived from population-level analyses and lead to conclusions framed in terms of probabilities. As a result, useful population-level interventions may be suggested. For example, if patients with comorbid psychosis and substance abuse are more likely to be violent than average patients, services for this group of patients might be the subject of special investment.

Next, it follows from a capacity-based approach to involuntary treatment, as argued elsewhere, that patients who lack capacity to make treatment decisions for themselves and for whom treatment is in their ‘best (health) interests’, can be treated against their will - including those who are a risk to others (Szmukler & Holloway, 2000). I am not suggesting that people with mental illnesses who are potentially violent should not be treated, but that treatment should be based on a health interest.

Next, fairness demands equal treatment of those with mental illnesses in relation to preventive detention. I suggest that the only persons who should enter the frame for such consideration are those (mentally disordered and not mentally disordered) who have committed a serious offence, and that it is for the courts to determine whether a longer than normal sentence is warranted or whether psychiatric treatment is appropriate. In its determinations, the courts will be advised by those with relevant expertise. The ‘numbers’ will be best for this group because base rates of violence will be relatively high. Experts like Professor Maden will continue to play an important role.

Finally, it should be insisted that what is not supportable, as the argument above has demonstrated, is the demand for a risk assessment on all patients engaged with mental health services as is currently the policy in most, if not all, mental health services in this country. This is clearly discriminatory.