3 - Research designs for the evaluation of services

Summary

Introduction

The intention of the chapter is to show that research highly relevant for the evaluative comparison of alternative patterns of service can be successfully carried out even though strictly controlled designs are rarely applicable. Such research would be based firstly on the routine application of methods of clinical audit, secondly on routine monitoring using high-quality data recorded during ordinary clinical practice, and finally on the sampling frames provided by the resulting epidemiologically based mental health information systems. A range of more closely comparable and controlled experiments, which in the past would have required intensive data collection and long-term analysis and write-up, could be carried out at relatively little extra cost by using such systems either directly or as sampling frames.

A combined epidemiological and clinical strategy for evaluating the effectiveness and efficiency of mental health services is outlined. Examples are taken mainly from the United Kingdom where the National Health Service, in spite of the creation of an internal market, still has the opportunity to implement a strategy for mental health informatics that meets top-down managerial needs for information by using bottom-up clinical recording.

The three overlapping methods of evaluation (clinical audit, monitoring and partially controlled experiments) are considered, first separately and then in combination. Recent uncoordinated evaluative studies of acute and medium-term hospital care are considered. They illustrate current designs and methods, which provide useful but limited conclusions.

In summary, the planning of services could be both improved and speeded up by the adoption of an audited recording system based on a minimum of routinely collected clinical data collected once only by the clinicians themselves. Such systems would also support more practically applicable evaluative research.

Designs for evaluative studies

Knowledge of three broad and overlapping kinds provides a basis for planning and evaluating services. The first is recognition of the characteristics, epidemiology and causes (biological, psychological, social) of mental disorders and any associated social disablement.

The more knowledge of this kind accumulates, the easier it becomes to acquire the second, which is concerned with effective methods of primary, secondary and tertiary prevention.