Local initiatives in clinical effectiveness

With the arrival of clinical governance, psychiatrists working for the National Health Service (NHS) can no longer work in isolation, and commitment to both clinical effectiveness and continuing professional development (CPD) is expected and likely to become mandatory. Clinical governance gives clinical effectiveness a high priority within NHS organisations, both at primary and secondary care levels, together with clearer lines of accountability.

Clinical effectiveness, the NHS and the doctor

Clinical effectiveness is a rubric for various activities with the common aim of improving the outcome for the patient. It is a wider concept than clinical audit, its predecessor, and has now been embraced within clinical governance. The importance to the NHS is highlighted by the establishment of a number of quality control groups at a national level, including NICE (National Institute for Clinical Effectiveness), National Service Frameworks, Health Improvement Programmes and the Commission for Health Improvement, alongside White Papers such as A First Class Service (Department of Health, 1998) and those on clinical and performance indicators (NHS Executive, 1999a , b ). Clinical effectiveness is here to stay.

It is not only the UK NHS psychiatrist who now needs to work within clinical effectiveness. A rise in consumerism, greater availability of information and research findings, together with a changing medico-legal picture, all contribute to a worldwide move towards such practice. Different terms may be used in different places, but the emphasis on improving the quality of clinical practice is the same.

The principles that embrace clinical effectiveness are similar to those of the wider concept of clinical governance, well documented in general (Scally & Donaldson, 1998), in psychiatry (Oyebode et al, 1999) and specifically with regard to psychiatric CPD (Wattis & McGinnis, 1999). This article will focus on the role of the psychiatrist within clinical effectiveness and particularly on a number of examples of initiatives with which I can claim personal experience if not success.

It has, for several years, been the duty of every doctor to take part in regular and systematic clinical audit (General Medical Council, 1995), and from 1999 the chief executive of each trust is responsible for ensuring clinical as well as financial quality. Despite an emphasis on multi-disciplinary working in psychiatry, there is little doubt that doctors are the main focus of clinical governance. External inquiries into homicide often focus on the consultant psychiatrist, and experience in other areas of medicine, such as the Bristol case (Keogh et al, 1998), highlights this.

It is necessary within any organisation to have a vision or strategy for the development of clinical effectiveness.

Developing a strategy

Organisational

Developing a clinical effectiveness strategy is not to be left to hospital or service managers. Within appropriate organisations, the psychiatrist should be involved in the development of such a strategy, to ensure that there is sufficient and appropriate clinical emphasis and to ensure that it is not a ‘tick the box’ exercise. The presence of a clinical effectiveness strategy assists the whole process of improving practice. A strategy should define:

1. (a) aims/desired outcomes;

2. (b) how topics for scrutiny will be selected;

3. (c) who to involve;

4. (d) how to facilitate improvement;

5. (e) interaction with information strategy; and

6. (f) evaluation of individual projects and the whole process.

It is not enough to consider only within-organisation clinical effectiveness. To improve practice and outcomes in psychiatry, it is important to involve other partners and agencies, such as primary care and social services. Hence, it becomes shared or ‘joined-up’ effectiveness. If we consider a person with schizophrenia and how to improve outcome, the need for joined-up effectiveness rapidly becomes apparent.

We must also recognise that different people, and different agencies, view the desired outcome differently. The psychiatrist's aim of clinical remission may differ from the NHS's aim of avoidance of hospital readmission, which may differ from the user's desired outcome of employment or adequate intimate social relationships. Furthermore, clinically effective treatments are not necessarily better-received treatments. For example, counselling may be less efficacious but more favourably received than other treatments.

The key players in the facilitation of clinically effective practice are suggested by referral to the cycle of ‘inform – change – monitor’ (NHS Executive, 1996). It will usually be necessary to organise a core group of people to coordinate such activity – and this group will be accountable to the trust board, perhaps via a clinical governance subgroup/committee. The trust clinical effectiveness group is likely to consist of representatives from:

1. • directorate/speciality

2. • postgraduate education

3. • information technology

4. • research and development

5. • risk management

6. • clinical audit/effectiveness

7. • general practice

8. • purchasers/health authority.

The trust-level clinical effectiveness group plays the role of promoting clinical effectiveness throughout the organisation. Clearly, the structure and role of this group will vary between different organisations, particularly between general and mental health trusts. Clinical governance means that clinical effectiveness groups or committees must be endowed with the power to make things happen, not just to cogitate and reflect.

Such a group could be expected to produce an annual report on progress with clinical effectiveness, particularly with positive improvements/changes in practice. The report might focus on progress with implementation of nationally and locally accepted clinical guidelines. Increasingly, reports consider outcomes, both in terms of patient care and of the project itself. An estimation of the cost of each project is useful, even if it can be measured only approximately, for example, by the number of hours of professional time consumed.

Personal

The psychiatrist may wish to consider his or her own development plan with regard to clinical effectiveness. As with continuing professional development, this is a topic which could be addressed in an annual appraisal. The personal strategy can be considered within the same themes as the organisational strategy, as illustrated in Box 1.

Box 1. A psychiatrist's personal clinical effectiveness strategy

Aims/outcomes

1. • To be knowledgeable about evidence-based practice

2. • To promote evidence-based practice throughout teams in which involved, particularly among junior medical staff

3. • To partake in evidence-based ward rounds and journal clubs

Whom to involve

1. • Psychiatrist

2. • Secretary

3. • Junior medical staff

4. • Clinical audit/effectiveness department

Facilitate improvement

1. • Attend evidence-based practice course, e.g. Oxford course in evidence-based psychiatry

2. • Obtain training in computer skills

Information technology

1. • Acquire computerised access at point of patient care

Evaluation

1. • Is evidence-based practice an underpinning principle of my practice?

2. • Has it lead to any improvement in care delivered/patient outcome?

3. • At what cost?

Local clinical effectiveness initiatives

Although not necessarily generalisable, it may be helpful to use some examples to suggest means by which clinical effectiveness can be put into practice. I will briefly describe the mental health service provided at Airedale NHS Trust, as this will assist in understanding the success and failure of certain projects.

The service

Airedale NHS Trust is integrated in that it provides acute general hospital, community (such as health visitors, speech therapists, chiropodists) and mental health services to a population of 190 000 covering a large geographical area of mixed urban and rural nature across West and North Yorkshire. Forty-three adult and forty-five elderly beds at the District General Hospital support a sectorised psychiatric service with community mental health teams (CMHTs). Drug and alcohol, rehabilitation and child and adolescent services are provided.

The mental health medical staffing consists of:

1. • four adult consultant psychiatrists

2. • two old age consultant psychiatrists

3. • 1.5 child and adolescent consultant psychiatrists

4. • seven senior house officers (SHOs) (4-5 GP trainees, 2-3 career psychiatrists)

5. • five staff-grade psychiatrists

6. • 1-2 specialist registrars (intermittent)

7. • 2-3 medical students from the University of Leeds.

The Trust as a whole has 60 consultants across the usual medical and surgical specialities.

Against this background, I have been involved in leading mental health clinical audit since 1993 (the group evolving to clinical effectiveness in 1995) and, in a separate role, leading clinical effectiveness for the whole Trust since 1996.

Using research to promote clinical effectiveness

Local research can act as a stimulant, not only to the researchers, but also to other staff in terms of keeping up to date with trends in practice. At a time when evidence-based practice was beginning to flourish, we performed a relatively simple study of the proportion of interventions in psychiatric practice that were evidence-based. This was perhaps more difficult in 1995, as it was before the Cochrane and other databases/publications contained expert analyses of the evidence base. Nevertheless, we found that by a semi-systematic search of available literature 60% of decisions relating to treatments (psychological, activity-based or pharmacological) used in both in- and out-patient settings were indeed based upon evidence from a systematic review or at least one randomised control trial (Summers & Kehoe, 1996b ). Subsequent presentation and publication of the study assisted in local promotion of clinical effectiveness. For example, other junior medical staff saw that commitment to clinical effectiveness could lead to personal (e.g. publication in a journal/improvement in curriculum vitae) as well as organisational gain.

Developing evidence-based practice

Training

An integral part of the local trust's clinical effectiveness is for staff to be trained in the principles and delivery of such practice. Several of our consultant psychiatrists have attended Oxford University's evidence-based psychiatry course (run by the Centre for Evidence-Based Mental Health) and other staff have attended local courses.

Along similar lines, a large number of staff from the mental health unit have attended ‘critical appraisal skills workshops’, some under the umbrella of CASP (Critical Appraisal Skills Programme), and we now run a programme of locally based critical appraisal skills workshops, on an in-house basis. Such workshops promote the general ethos of evidence-based practice, although one may argue about the necessity for all individuals to have well-developed critical appraisal skills when there are now (and increasingly so) a number of sources where evidence has already been expertly appraised (e.g. the Cochrane database, Best Evidence, Evidence-Based Mental Health).

Evidence-based journal clubs

We have attempted to run evidence-based journal clubs over the past 2–3 years, with mixed success. Much depends on the needs and enthusiasm of trainees, which are likely to vary within any one group, as well as from one group to the next. This is particularly so in a small group of mixed general practice and psychiatry trainees. Nevertheless, we have encouraged, each 6 months, trainees to contribute evidence-based questions, which they (or their colleagues) then attempt to answer by reference to available evidence following a broadly systematic approach. The journal club organiser (one of the consultant psychiatrists) usually sets the topics for the first weekly sessions each 6 months.

Example of topics of journal club presentations

1. • What is the most effective treatment for obsessive-compulsive disorder?

2. • Is family therapy effective in treatment of schizophrenia?

3. • Does psychomotor retardation suggest a diagnosis of severe depression?

Trainees are taught how to search systematically for evidence, helped by the intelligence officer/librarian.

Trainees present their findings to the rest of the group (usually from six to eight trainees, three or four consultants, two specialist registrars and medical students) and a summarised version and the ‘take-home message’, written with appropriate references, is kept in an ‘evidence-based folder’ in the department. The latter is along the lines of the ‘evidence-based prescriptions’ described in the useful pocket book Evidence-Based Medicine (Sackett et al, 1999).

It has often proved difficult to ask ‘answerable’ questions, and the thoroughness of trainees' searches has varied markedly. Each 6 months necessitates several discussions of principles of evidence-based practice. It is clear that the consultants need to be re-enthused, and a broad base of evidence-based enthusiasm is required to sustain such repetition.

On the positive side, we have recently been more interactive between ward rounds, case conferences and journal clubs. We have developed a simplified system (i.e. a big red book!) in which we record questions and proposals for evidence-based appraisal, to be addressed at journal clubs, arising from case conferences and ward rounds. Several descriptions of more formalised evidence-based journal clubs exist elsewhere (Gilbody, 1996; Coombe et al, 1999).

Evidence-based ward rounds

Using the model outlined for use in general acute medical rounds in Oxford, one of our psychiatrists has introduced a similar ‘real-time’ computer search system to assist in decision-making (Ellis et al, 1995). One of the secretaries was trained to assist in the searching process. During the ward round, questions regarding diagnosis and treatment are put into a searchable format, a computerised search with access to databases (Cochrane, Medline) is undertaken and information is fed back to the team. A comparison is being undertaken by the team to determine whether such ‘real-time searching’ changes decision-making any more than reference to up-to-date textbooks. It has proven difficult to sustain this model for more than several months.

A less ambitious approach, practised by at least one of the other consultant teams, is to run the weekly ward using teaching of evidence-based practice as an underpinning principle. Thus, when questions of diagnosis, treatment or prognosis arise, an evidence search is undertaken by one of the team (medical student, SHO, sometimes a student or junior nurse) to bring back an answer in time for the following ward round, or sooner. This model is sustainable and proves useful as a means not only of answering clinical questions but also of learning search skills and promoting evidence-based practice.

Postgraduate library/information technology

As part of the overall clinical effectiveness strategy, it is necessary to involve the postgraduate training organiser (or tutor or equivalent), the librarian and the information technology department. To assist in clinically effective practice, information needs to be available at the point of patient care. In mental health, this is particularly difficult as most patient care occurs outside of a hospital. We have yet to establish access to information databases for our CMHTs, although this should be achievable in the not too distant future. Such access is available to some clinical areas within the hospital (e.g. the psychiatric wards and some clinic areas) and the on-site hospital library, and the postgraduate centre has several terminals with access to databases and the Internet.

One of the librarians has been employed partly as an ‘intelligence officer’ to assist in training individuals to perform evidence searches and produce a regular bulletin on publication and events relating to the development of clinical effectiveness.

Lay participation and working as a clinical effectiveness group

User participation in the development of psychiatric services has become increasingly widespread over the past 5-10 years in the UK. A number of models have been described, including our own (Summers & Kehoe, 1996a ; Judd, 1997). Such participation has been strongly encouraged at a national level (Department of Health, 1999).

It is difficult to determine the benefits of such lay participation, but it appears instinctively correct to involve past and present users as well as representation from organisations such as Mind and the Manic Depression Fellowship. Such people often approach the process of treatment and outcome from a different perspective.

At Airedale, we have a mental health clinical effectiveness group which evolved from the clinical audit group. This consists of medical, nursing, social work, occupational therapy, mental health management and psychology staff from within the Trust and from social services. There are also representatives from the two main purchasing health authorities and user representatives from organised groups or recruited via the Advocacy Service and ‘user councils’. The Airedale clinical effectiveness group promotes clinical effectiveness throughout the various mental health specialities and audits progress with particular projects, usually by a small project group approach. Most projects have had user representation. The group receives top-down information (e.g. effective health care bulletins, national performance indicators, White Papers) and decides on appropriate action. The implementation of recommendations is assisted by the Directorate Management Group.

The clinical effectiveness group has an ongoing action plan with specific target dates for each project. Some projects may be proactive, arising from local concerns or from discussions within the mental health unit, whereas others may be reactive (e.g. following a Mental Health Act Commission visit or from findings of serious untoward incident reviews). An example of the action plan is shown in Table 1.

Recent/ongoing projects include:

1. • introduction of Health of the Nation Outcome Scales (HoNOS) for all full Care Programme Approach (CPA) cases

2. • audit of CPA standards using national audit tool

3. • audit of use of Section 5(2) of the Mental Health Act 1983

4. • audit of complaints

5. • mental health needs assessment project

6. • review of serious untoward incidents and corrective actions

7. • depression treatment in primary care

8. • review of services for people with schizophrenia using Clinical Advisory Standards Group

9. • patient information review.

These projects have varied in their intensity and duration; some are ongoing, some have been repeated several times and improvements in practice have been demonstrated.

Lay participation has been particularly fruitful in the improvement of written information on illnesses and clinical services, ensuring legibility as well as appropriateness of content.

Mental health needs assessment/collaboration

This is perhaps our most resource-intensive (i.e. most costly) project. The importance of needs assessment was highlighted in the NHS and Community Care Act 1990, and subsequent guidance has suggested roles for health authorities and trusts (Department of Health, 1993). We have performed prevalence studies of severe mental illness in our psychiatric sectors using a joint approach by the Trust, Health Authority, local social services, probation services, local GPs (now a primary care group) and the Nuffield Institute for Health Studies. All working-age adults with a history of manic depression, schizophrenia or recurrent hospital admission have been identified for each practice. Each general practice has obtained a ‘disease-register’ of severe mental illness cases, including those in contact with specialist mental health services and those only in current contact with primary care. The data are aggregated on an anonymous basis at an area level. We are currently assessing individuals' needs using the Camberwell Assessment of Needs alongside other health and social information. General practices are using the disease-registers to a greater or lesser degree. Some have made little use of it, others use the register as a basis for communication and clinical reviews by the link-CMHT worker, of which there is one for each GP practice from the local CMHT. In one of the sectors, the GPs intend to use such disease-registers as a means of delivering clinical guidelines for the primary care management of schizophrenia derived from the Sainsbury guidelines (Sainsbury Centre for Mental Health, 1998).

The project is ongoing but has required a full-time community psychiatric nurse to perform the individual needs assessment and a research assistant for data collection. We hope to incorporate individual needs assessment and the updating of the disease-register into everyday practice. This is now made easier by a computerised CPA database, but the database does not include those cases out of contact with specialist mental health services (approximately 30% in our area). The information has already proven useful in planning local services, and the results of the individual needs assessment should highlight local gaps in the service, both from user and from health professional perspectives.

The mental health needs assessment project has also helped to develop working relationships between the various health and social agencies.

Using outcomes scales

In the development of clinically effective practice, it is clearly necessary to measure outcomes. Several years ago, we audited our use of rating scales within the mental health directorate and found more than 50 to be in use. In general, the staff were not trained in the interpretation of scores, or they were using invalidated scales with doubtful clinical implications. We have subsequently stopped using most of these and have implemented the widespread use of a single measure, HoNOS, initially in all those people subject to ‘full CPA’. This again has been costly in implementation, using a cascade system of training staff (James & Kehoe, 1999). A recent audit found that almost 100% of full CPA cases had regular HoNOS scoring performed by their keyworker alongside CPA reviews. This has been achieved over the past two years, but its longer-term use will only be likely if direct benefits are realised, either for the individual concerned (for example, by shaping their care) or for the keyworkers. In the future, it may be the case that HoNOS scores form part of a national ‘minimum data set’ or national ‘clinical indicators’.

Evaluation of clinical effectiveness

It is always worthwhile to stand back and consider whether investment in such projects is worthwhile. Although specific monies are allocated to clinical audit/effectiveness, these are usually spent on infrastructure or support staff. Beyond this, much has depended on the enthusiasm and good will of clinical staff in terms of time commitment. Clinical governance will further encourage psychiatrists to be personally involved in clinical effectiveness initiatives - but there are no suggestions as to where the time will come from!

Each initiative or project can be evaluated by considering the following:

1. • topic

2. • objectives

3. • conclusions

4. • recommendation

5. • changes implemented

6. • costs (direct and professional - hours).

Conclusion

Clinical effectiveness provides a means whereby the quality of care delivered can be improved, hopefully giving better clinical outcomes. It is a wide concept which envelops clinical guidelines, evidence-based practice, audit, risk management and further education/professional development. Psychiatrists must be involved in such development, and some of the examples provided here may hopefully stimulate you to at least reflect upon your own commitment to clinical effectiveness at a personal and organisational level.

Multiple choice questions

1. 1. Clinical effectivenesss is of increasing influence because of:

   1. a the major impact of clinical audit

   2. b a worldwide increase in consumerism

   3. c greater availability of research findings

   4. d increased understanding of new statistical methods

   5. e a changing medico-legal picture.

2. 2. Evidence-based psychiatry studies have found that:

   1. a less than 20% of clinical decisions are based on evidence

   2. b ward rounds with real-time computerised information searches are more effective than traditional rounds

   3. c participation in a critical appraisal skills workshop is essential before referring to the Cochrane database

   4. d information on the latest treatment techniques should be accessible only in the hospital library

   5. e it is possible to run evidence-based journal clubs.

3. 3. Lay participation:

   1. a in psychiatric services is decreasing

   2. b should be discouraged because of conflict with evidence-based medicine

   3. c can assist in the process of clinical effectiveness

   4. d assists in the development of legible information leaflets

   5. e demonstrably improves the outcome of clinical effectiveness initiatives.

Table 1 Clinical effectiveness action plan

Topic/objective	Progress to date	Outstanding work	Target	Lead
Involvement of carers in CPA	Questionnaire produced ethics approval	Survey carers	December 1999	Mental health manager
				Making Space representative
Post-suicide audit	Protocol established	Discuss with Gps	January 2000	Practice development nurse
	Being followed in all cases
	Annual review completed
Electroconvulsive therapy (ECT): guidelines and outcome	Audit cycle completed x 1	Include user feedback	November 1999	Consultant psychiatrist, ECT nurse, advocacy worker
	New machine acquired
	New guidelines established

MCQ answers

1		2		3
a	F	a	F	a	F
b	T	b	F	b	F
c	T	c	F	c	T
d	F	d	F	d	T
e	T	e	T	e	F