Improving the quality of medical reports to mental health review tribunals

Abstract

AIMS AND METHOD

We aimed to determine, using clinical audit, the effect of implementing national guidelines on the quality of responsible medical officers' (RMOs’) reports to the mental health review tribunal (MHRT). We blindly assessed the quality of 50 consecutive reports concerning patients detained under Sections 3 and 37. Twenty-five reports were written before guidelines were circulated; a further 25 were written following the distribution of guidelines and a checklist with every request for a report.

RESULTS

The quality of reports, as measured by our checklist, significantly improved following the circulation of guidelines.

CLINICAL IMPLICATIONS

Increasing the awareness of guidelines by widespread circulation and the audit process is an effective way of improving the quality of RMOs' reports to the MHRT.

Since the inception of the Mental Health Act (MHA) 1983, the number of applications to the mental health review tribunal (MHRT) by detained patients has risen (Blumenthal & Wessely, 1994). Because tribunals require the responsible medical officer (RMO) to submit a detailed clinical report for each application, report writing has become an increasingly important (and time-consuming) part of everyday psychiatric practice.

Previous literature regarding the quality of reports has, for the most part, offered expert opinion as to which points should specifically be included by RMO authors (Woolf, 1991; Langley, 1993). Audit has demonstrated that the majority of reports do not address the basic criteria for detention required by the MHA (Ismail et al, 1998) but that improvement might follow the formulation of local guidelines (Davison & Perez de Albeniz, 1997). The previous studies have been limited by the absence of agreed national criteria, on which to base standards, and by unblinded research methodologies that may have allowed the introduction of bias during appraisal of report quality.

Recently, and for the first time, comprehensive report-writing guidelines have been circulated by the MHRT (Regional Chairmen of the MHRT for England and Wales, 2000) after consultation with the Royal College of Psychiatrists. These provide guidance as to what the MHRT finds useful in RMOs' reports. We aimed to determine, using clinical audit, the effect of implementing the national guidelines on the quality of RMOs' reports to the MHRT. We hypothesised that the introduction and circulation of guidelines in our NHS trust would be associated with an improvement in the quality of reports.

Method

Setting

Sheffield is a large city in the north of England with a population of 600 000. Adult mental health services are arranged into four geographic sectors, each served by a community mental health team, day services and a psychiatric in-patient ward. The other in-patient services cover old age psychiatry, forensic psychiatry, rehabilitation, substance misuse, psychiatry of learning disability and child psychiatry. The total number of detentions under the MHA in Sheffield during the year April 2000 to March 2001 was 679, with 280 under substantive ‘treatment’ sections. One hundred and fifty patients were detained under Section 3 between April 2000 and October 2000, with a further 99 detained under Section 3 during the period November 2000 to March 2001. In the 12-month period of April 2000 to March 2001 there were 31 patients detained under Section 37 (with 27 of these additionally restricted by Section 41). The 27 restricted patients either had a Section in place prior to April 2000 or were placed on a Section 37 during the period of April 2000 to October 2000 (a restricted Section 37 is not renewed by the RMO). The four unrestricted Section 37 patients were all placed on the Section, or had the Section renewed, during the April 2000 to October 2000 period. One hundred and six of those detained under Sections 3 or 37 appealed to the MHRT, with 62 tribunals actually taking place, the others being cancelled owing to the patient withdrawing the application or being re-graded to informal by the RMO before the tribunal date.

The Study

The audit took place between July 2000 and February 2001. At the outset, we derived a checklist of 18 criteria from the MHRT guidelines. Criteria were chosen following discussion with RMOs at the trust's medical audit meeting. We chose criteria that could be applied to any report, regardless of the individual clinical circumstances. Reports were included in the audit if they related to a patient detained under Section 3 or 37. Patients detained on Section 2 and other Sections were excluded (because the short notice for these tribunals can lead to the MHRT accepting an oral report from the RMO).

During a pilot phase, interrater reliability was checked. Two raters (P.E. and M.D.H.) independently assessed five randomly selected reports using the checklist. Then the pre-guidelines sample of 25 (22 Section 3; three Section 37) consecutive RMOs' reports between 1 June 2000 and 31 October 2000 was collected.

The intervention stage commenced on 1 November 2000. MHRT guidelines were circulated with every request for a report. Our checklist (effectively a simplified version of the guidelines) was also circulated with report requests. A post-guidelines sample of 25 (22 Section 3; three Section 37) consecutive reports was collected between 1 December 2000 and 27 February 2001.

Each of the 50 reports were then randomly allocated to one of the two assessors. Assessors were blind to the date of the report (the date was concealed in such a way that it was not possible to tell from which sample each report was drawn, but it was possible to check that the author had dated the report). Reports were assessed with the checklist of 18 criteria, and each awarded an overall quality score out of 18.

Results

Interrater reliability, in the pilot phase, was good (mean quality scores, P.E. 8.8 (s.d. 3.7) v. M.D.H. 9.6 (s.d. 2.9), Pearson correlation coefficient R=0.99, P<0.001).

The 25 pre-guideline reports were compared with the 25 post-guideline reports. Since the data were normally distributed, we used an independent sample t-test to test the null hypothesis that there would be no significant improvement in the mean quality score following the circulation of guidelines. The mean quality score (out of a possible total of 18) was 11.32 pre-guidelines compared with 14.00 post-guidelines (mean difference 2.68; 95% Cl 0.87-4.49; t=2.97; d.f.=48; P<0.005).

The number of reports satisfying each of the individual criteria, before and after the circulation of guidelines, is shown in Table 1. We used Fisher's exact test to test the null hypothesis that the number of reports satisfying each criterion would not increase following the distribution of guidelines.

Table 1. Number of reports satisfying each criterion before and after the circulation of the mental health review tribunal guidelines

The RMO's report should state	Pre-guidelines (n=25)	Post-guidelines (n=25)	Fisher's exact test
Patient's name	25	25	—
Report date	25	24	—
Section of MHA	22	21	—
Date of Section	12	15	NS
Name of RMO and author's name (if not the same person)	15	23	P=0.009
Name of keyworker	7	17	P=0.005
MHA category of mental disorder	4	7	NS
Diagnosis (ICD—10)	15	23	P=0.009
Duration of illness	19	23	NS
Why in-patient treatment needed	15	21	NS
Why informal treatment not possible	13	20	P=0.036
Why detained (health/safety/protecting others)	16	17	NS
Progress on the ward	24	24	—
Current medication	19	20	NS
Non-pharmacological aspects of treatment	11	15	NS
Patient's attitude to treatment	18	19	NS
Outstanding risk factors	16	18	NS
Effects of immediate discharge	8	18	P=0.005

Discussion

As predicted, the quality of reports improved following our intervention. The specific areas of improvement (seen in Table 1) are of particular interest.

The increased frequency with which RMOs' names appeared in reports resulted from an improvement on the part of junior doctor authors, who had previously tended to state their own name without explicitly identifying the responsible consultant. The other main areas of improvement may reflect increased awareness of the Care Programme Approach (inclusion of keyworker's name), themes related to risk management (effect of immediate discharge and reason why informal treatment not possible) and the benefits of diagnostic precision (use of the ICD—10 system; World Health Organization, 1992).

The overall change in quality score was significant but not large (2.68 out of 18; 15% improvement) and it is therefore important to identify those areas where there was room for further improvement. RMOs rarely stated the category of mental disorder under which patients were detained. We propose that this occurred because the classification was not perceived as having clinical significance. It does, however, have considerable legal significance — especially with regard to the issue of whether treatment will bring benefit or prevent deterioration.

Drug treatments were well-reported, but non-pharmacological therapies were not. This may have been because RMOs saw pharmacotherapy as the primary treatment modality or because patients were not receiving non-pharmacological treatments. We think that the latter is unlikely; non-drug treatment is invariably a part of the overall management plan (e.g. occupational therapy) but might not always be conceptualised, by doctors, as treatment per se.

The accurate dating of the Section (we accepted date of detention or expiry) also failed to improve following the circulation of guidelines. RMOs might have assumed that the tribunal would be familiar with such details, and hence omit them.

Our method was designed to maximise the reliability and validity of the results. However, we report the findings of an audit cycle, not a controlled trial of an intervention. We cannot ‘prove’ that the intervention was responsible for the improvement seen, although it is unlikely that it was not. Any confounding factors, such as differences in the populations of authors or differences in the clinical characteristics of detained patients seeking appeal, would tend to be evenly distributed between the before and after phases. It is noteworthy that each of our sampling periods included the junior doctors' rotational date.

Junior doctors wrote a significant proportion of the reports (under RMO supervision and with counter-signature). It is possible that, as the guidelines were circulated to RMOs, they may not always have been passed on to juniors. This means that the effects of the intervention may have been underestimated and would have been larger had RMOs directly compiled all the reports.

Accepting that the improvement seen was significant, but not universal, we consider that, in our NHS trust, the publication of guidelines by the MHRT has been a success. Feedback from RMOs has indicated a secondary benefit — writing reports is quicker and easier when authors know what is expected of them. The local challenge is now to continue the audit process with a specific focus on those areas that did not improve following the first cycle. This is of particular importance because the preparation of an adequate report for the MHRT is not only a legal requirement but also an ethical duty of the doctor to his/her detained patient.