Outcome of clinical risk management in the Gloucester rehabilitation service

Abstract

Aims and Method

To assess the impact of a clinical risk assessment and management policy introduced in a tertiary rehabilitation service, in response to UK government statute. The study assessed the effect of a formal risk management approach for high-risk cases over a year, in terms of changes in risk-associated behaviour and perceived levels of risk, and in achievement of clinical risk management goals.

Results

Initially 35 of 107 patients were rated as high risk, mostly of neglect or of harm to others, and were subjected to formal risk management. At follow-up only 16 patients were still rated as high risk, most reduction occurring in risk of neglect. Half of the 112 risk management goals established were fully achieved, 58 partially and 18 not achieved. Total Social Behaviour Schedule ratings were significantly lower at follow-up.

Clinical Implications

Risk management appeared to be successful in terms of reducing overall levels of risk, the achievement of risk management goals and the reduction of risk-associated behaviours. The lack of a control group and blindness limits the validity of these findings. However, organised risk management procedures may help to support and promote good general clinical management.

The Royal College of Psychiatrists (1996) has advised that clinical risk management is an ‘integral part of psychiatric practice’, and that people presenting an increased risk of harm should be given the highest priority for resource allocation. It has been statutory procedure for policies in risk assessment and management to be in place in all mental health provider units since 1994. However, there is a lack of empirical evidence regarding the effect of the introduction of risk assessment in clinical practice. While it may be expected that the explicit management of risk might be effective and lead to risk reduction, there is a lack of evidence to support this contention. A literature search failed to identify any published outcome research in this area, and in literature cited to support the Green Paper for proposed changes to the Mental Health Act (1983) the Department of Health did not describe any directly relevant research (Department of Health, 2001).

This study aims to provide preliminary data regarding the impact of clinical risk management, both in terms of the achievement of the clinical goals established within the risk management plans, and also the effect on risk-associated behaviour, and change in overall levels of risk.

Method

Service context

At the time of this study, the Gloucester rehabilitation service was treating approximately 160 patients with severe mental illness in the city of Gloucester. The community rehabilitation team worked extensively in a large supported lodgings scheme. There were 12 rehabilitation in-patients and a well-developed hostel system.

In response to a homicide inquiry in 1997, the Severn NHS Trust developed a clinical risk assessment and management policy, supported by a training programme. The rehabilitation service started a programme of clinical risk assessment through its annual clinical review process from autumn 1997.

Risk assessment and management

All staff in the rehabilitation service participated in the Trust training in risk assessment and management, largely through joint team training. The process of initial clinical risk assessment and risk management planning was completed in Care Programme Approach (CPA) multi-disciplinary review meetings. Prior to this the keyworker (usually a community psychiatric nurse, but in some cases a social worker) had carried out a full evaluation of risk based on case note information and (as advised in the training programme) identifying actuarial indicators and contemporary factors relevant to the risk of neglect, harm to self, or harm to others. At the end of the review, a discussion within the multi-disciplinary team determined whether each patient posed a mild, moderate or severe risk of harm to self or others, or self-neglect. This judgement was qualitative, but related specifically to consensus reached in the Trust training programme, which was completed as a team exercise.

For patients considered to be at high risk in any category, a formal risk management plan was developed, broken explicitly into management goals, which were to be achieved over the following year. This followed a model of clinical outcome monitoring previously described (Macpherson et al, 1999). All patients were also rated by keyworkers with reference to 1 month prior to the review meeting on ratings of: hostile social contacts; personal appearance and hygiene; inappropriate sexual behaviour; destructive behaviour; and self-harming behaviour; these items were taken directly from the Social Behaviour Schedule (SBS; Wykes & Sturt, 1986). Ratings on each domain range from 0 (no problem) up to 4, each increment having a clearly described level and frequency of abnormal behaviour.

At the follow-up CPA review meeting 1 year later, each patient was reassessed following the same risk assessment process and using the same SBS ratings, and within the review meeting it was determined whether risk management goals had been ‘fully achieved’, ‘partially achieved’ or ‘not achieved’ as described previously by Macpherson et al (1999). Finally, the keyworker was asked to rate whether risk management had been beneficial in each case on a five-point Likert scale: ‘disagree strongly’, ‘disagree a little’, ‘unsure’, ‘agree a little’ or ‘agree strongly’.

Data analysis

Data were entered into the Statistical Package for the Social Sciences, version 10 for Windows. Non-parametric Wilcoxon matched-pairs signed ranks test and Spearman's correlation coefficients were used to assess change in risk ratings, and the relationship between perceived change in risk and risk goal achievement respectively. Analysis of change in the rating of risk between the two assessments was made by kappa correlation coefficients.

Results

Risk ratings

Ratings were completed for 107 patients who were subjected to initial risk management and assessed at 12 months' follow-up. The project aimed to include the entire service case-load, but a number of patients were discharged or moved from the locality over the year of the study. The characteristics of the rehabilitation and study population are shown in Table 1.

Table 1. Clinical and demographic characteristics of the rehabilitation service population and the high-risk study population

	High-risk study population (n=35) n (%)	Rehabilitation population (n=72) n (%)
Primary ICD—10 diagnosis1
Schizophrenia/schizoaffective disorder	28 (80)	45 (63)
Affective disorder	2 (6)	11 (15)
Organic disorder	3 (9)	3 (4)
Personality disorder	2 (6)	8 (11)
Obsessive—compulsive disorder	0 (0)	5 (7)
Legal status
Informal	31 (89)	67 (93)
Section 37/41	2 (6)	2 (3)
Section 3/37	1 (3)	1 (1)
Section 25	1 (3)	2 (3)
Gender
Male	24 (69)	49 (7)
Female	11 (31)	23 (3)
Accommodation
Supported lodgings	14 (40)	33 (46)
Hostel/residential home	10 (29)	16 (22)
Ward	5 (14)	1 (1)
Independent/group home	6 (17)	22 (31)
Age in years
Mean (range, s.d.)	44.4 (24-68, s.d. 12.3)	45.1 (23-69, s.d. 11.9)

Thirty-five patients (33%) were rated as high risk in at least one area at the first assessment, and these patients represent the study group under consideration. Their clinical characteristics are shown separately in Table 1, and do not differ substantially from the population on the rehabilitation register. The remaining 72 patients were not subjected to formal risk management plans, as they were not initially rated as high risk. However, they were followed up in the same manner and, at review, three patients were (at that point) assessed as being at high risk, all of self-neglect. At this point a further three patients whose clinical management had been taken over by the rehabilitation service were also assessed as being at high risk of harm, two of them of harm to others and one of neglect. Ratings of risk at initial assessment and at 12 months' follow-up are presented in Table 2. At base-line, two patients were rated as being at high risk of both neglect and harm to others. It can be seen that risk ratings changed over the 12-month follow-up period, and κ correlations for risk ratings between the two points were as follows: risk to self, κ=0.55; risk to others, κ=0.68; risk of neglect, κ=0.28. Overall, there was a reduction in perceived level of risk between the two assessments. The team rated overall risk as ‘decreased’ in 24 cases, ‘unchanged’ in 7 cases and ‘increased’ in 4 cases.

Table 2. Multi-disciplinary risk ratings in study groups at baseline and 12-month follow-up

	Baseline	Follow-up
Risk of neglect
Mild	13	16
Moderate	8	15
Severe	14	4
Risk of harm to others
Mild	16	15
Moderate	4	9
Severe	15	11
Risk of harm to self
Mild	28	28
Moderate	4	6
Severe	3	1

Risk management goals

In total, 112 risk management goals were developed for the 35 high-risk patients (mean 3.2 per patient, range 1-5). The most common goals were managing compliance, in 29 patients (83%), and developing structured activities/day care, in 23 patients (66%).

At follow-up patients had fully achieved 56 goals (mean 1.6 per patient, range 0-4) and partially achieved 38 goals (mean 1.1, range 0-2). There were 18 non-achieved goals (mean 0.5, range 0-3). Compliance goals were achieved fully in 19, partially in 6 and not at all in 4 cases. Goals to establish or improve day care were achieved fully in 15, partially in 5 and not at all in 3 cases. Team ratings of change in risk goals (as ‘increased’, ‘decreased’ or ‘the same’) were weakly correlated with the total number of goals achieved, but not significantly correlated with the number of goals that had been set but not achieved (r=-0.41, P=0.02; r=0.31, P=0.22, respectively).

Behaviour ratings

Total behaviour ratings across the five domains of the SBS reduced significantly over the study period, from mean 3.1 (s.d. 2.6, range 0-9) at initial assessment to mean 2.2 (s.d. 2.8, range 0-11) at follow-up (Z=-2.3, P=0.02).

Discussion

The main findings of this study were: risk as assessed by staff in a high-risk rehabilitation population reduced over a 12-month period. Ratings of high risk of neglect were particularly likely to decrease. Staff tended to perceive that risk had reduced at follow-up. Ratings of challenging behaviour on five items of the SBS showed a significant reduction between the two assessments. A wide range of goals aimed to reduce risk were established, of which roughly half were fully achieved and one-third partially achieved. Goals in the area of treatment compliance and the development of structured day care/activities were generally achieved, at least partially. Staff perception of reduced risk was weakly related to the achievement of goals, but unrelated to the level of non-achievement of goals.

The finding that ratings of risk of neglect tended to change more than other risk areas over time is interesting. It could be argued that this area of risk is more nebulous and difficult to define. While numerous rating scales to predict dangerousness and suicidality have been devised, the area of self-neglect remains under-researched, and the need for a greater evidence base in this area emerged repeatedly within the Trust training programme. A recurrent issue raised by staff was the complex assessment of a patient whose neglect is substantial but is not causing concern to the individual or apparently to the wider society, but is a legitimate concern for health professionals. It is important to acknowledge that a small group of patients rated as low risk at initial assessment was considered at follow-up to be at high risk (in all cases, of neglect). This emphasises the need for active engagement and follow-up in complex cases of severe mental illness, as advocated in the contemporary model of assertive outreach (Burns & Guest, 1999), and it is interesting that the community rehabilitation team described has developed subsequently into a formal assertive outreach team.

The central findings of this study will, we hope, be welcomed, at a time when there is public concern about the ability of mental health services to deal effectively with risk. We have shown that in a standard clinical setting, rather than a major research centre, the adoption of a programme of clinical risk management was associated with the reduction in ratings of risk, reduced perception of risk among staff, and lower ratings of challenging behaviour. It has also been demonstrated that it is possible to integrate methods of rating and clinically managing risk into a standard health care setting, in a way that has helped to focus work on the key clinical areas, and can be demonstrated to have been effective: our findings can be seen to add to the evidence that the longer a patient remains under supervised care, the lower the risks of future violence and reoffending (Lamb et al, 1988; Bloom & Williams, 1994).

The development of a simple system of setting goals, with subsequent rating of their achievement, has in our view been essential to this piece of work. While most of the goals were in areas that were obviously linked to risk, for example targeting compliance, others reflected an awareness that areas that might not obviously link with risk (such as family work, drug changes targeting side-effects, art therapy concerned with childhood experiences) were all considered to be legitimate foci for risk management. In our experience, the danger of therapeutic nihilism resulting from the allocation of a ‘high-risk’ label (Kaliski, 1997) to a case does not appear to have been realised. The clinical approach reported in this paper was of holistic clinical management, based on comprehensive assessment of need. It may be relevant that within this service training has a high status in all disciplines, and team-based training in risk management was considered to have been a successful process. Research suggests that at present there are surprisingly low levels of training in risk management within most UK Trusts (Davies et al, 2001).

It is important to consider the limitations inherent within this study's methodology. The lack of a control group, without a specific risk management approach, means that change cannot necessarily be attributed to the clinical method described, and may have resulted from more general factors associated with the work of the team. We agree with Snowden et al (1999) that effective risk management does not require specific skills, but needs a sound, comprehensive, clinical approach. It is also important to note the evidence (reviewed by Buckley et al, 1995) of the value of clozapine in managing aggression. The service described in this paper operates a large clozapine clinic. The population in the study reported here was small, and from a rehabilitation service, which might have led to bias in favour of more chronic, long-term cases, including a number of forensic cases (the regional forensic service having no community provision in the county). A further limitation is the possibility that improvements noted in the study might be due to a temporal effect, in conditions such as affective disorders which are likely to change over time.

Petch (2001) has expressed concerns about clinical risk management, based on perceptions of the process as politically driven, lacking an evidence base, and a means of attributing responsibility to clinical services for problems such as massive levels of substance misuse, which are not being addressed by society. Risk management raises questions about which societal problems should properly be managed within the remit of psychiatry. We have sympathy for these views, but believe that our findings offer some encouragement, particularly if we consider the target of risk management to be the reduction of challenging behaviour such as violence and neglect. If simple audit methods such as this can be completed in standard clinical settings, we may learn more incrementally about the requirements for successful risk management.

A final methodological concern relates to prediction, in an area of practice where serious untoward events are thankfully rare (Taylor & Gunn, 1999). During the year of this study there was no homicide or suicide in the population studied. Kennedy (2001) has argued that for every homicide predicted accurately using standard methods, there would be 111 false positives, and we agree with him that in order to achieve a meaningful and practical approach within standard clinical settings, a focus on risk of violence rather than homicide is the only feasible approach, and that it is ‘worth the effort’. While our results indicate a reduction of perceived risk and challenging behaviour, an actual reduction in suicide or homicide risk cannot be inferred. Indeed, the service described believes strongly in the need for therapeutic risk-taking, as advocated by Davis (1997), in order to effectively support the patients we treat. The development of organised risk management systems might help to support and promote sound and defensible practice which could be reviewed appropriately in the event of untoward events.

In her review of schizophrenia and the risk of violence, Taylor (1995) concluded that patients with psychosis rarely reoffend on return to the community, and do better than patients who are non-psychotic. She argued that ‘it ought to be an attainable goal that secondary prevention of serious violence by people with psychosis could be almost complete.’ This study has presented findings that suggest that risk management can be effective in a number of ways. There is a need for larger-scale formal research to consider further the components necessary for effective risk management in different populations.

Declaration of interest

None.