The case against ‘the evidence’: A different perspective on evidence-based medicine

Abstract

Background

An evidenced-based approach to psychiatry is playing an increasingly prominent role in treatment decision-making for individual patients and for populations. Many doctors are now critical of the emphasis being placed on ‘the evidence’ and concerned that clinical practice will become more constrained.

Aims

To demonstrate that evidence-based medicine is not new, sources of evidence are limited and psychosocial aspects of medicine are neglected in this process.

Method

Some of the literature is reviewed. Ideas and arguments are synthesised into a critical commentary.

Results

These are considered under four headings: evidence-based medicine is not new; what evidence is acceptable; the doctor as therapist; and the emergence of a new utilitarian orthodoxy.

Conclusions

It is agreed that a degree of professional consensus is necessary. However, too great an emphasis on evidence-based medicine oversimplifies the complex and interpersonal nature of clinical care.

Clinical effectiveness, evidence-based medicine (EBM) and related terms were the politically correct medical slogans of the 1990s. For many they are ‘buzz-words’ conveying a modern progressive approach and in some circles it is unwise to express scepticism. Evidence-based medicine is being embraced by all specialities and there has been a strong signal that psychiatry is joining the movement by the introduction in 1998 of a psychiatric journal dedicated to evidence-based practice. Although the ideas are not new and the basic assumptions of EBM are sound, there are major reservations about how its tenets are being promoted. The only evidence deemed acceptable in decisions about treatment is that derived from randomised controlled trials (RCTs). The value and role of the doctor as therapist is being undermined, the psychological and social aspects of medicine are neglected by EBM and the movement is in danger of creating a new utilitarian orthodoxy. These criticisms are considered in turn.

EVIDENCE-BASED MEDICINE IS NOT NEW

The launch of the EBM initiative gave a clear impression that it was a new development to base patient treatment on hard up-to-date scientific facts. Clearly there was also a strong political element so that the Department of Health could be seen to be supporting an important new policy initiative.

Hard evidence, as defined by the criteria of the EBM movement, has existed for many years and psychiatrists have acted on the findings of several major RCTs since the Second World War. It is worth recalling three of these.

Insulin coma therapy became the treatment of choice for schizophrenia after its introduction in 1933 at Vienna (Sakel, 1938). The evidence of its value was never fully convincing, but after an RCT (Ackner & Oldham, 1962) showed that it conferred no benefit over barbiturates for inducing coma it was soon abandoned.

In the early 1960s the Medical Research Council undertook a landmark study of the treatment of depressive illness (Medical Research Council, 1965). This large RCT compared the efficacy of electroconvulsive therapy, imipramine, phenelzine and placebo. It clearly established the efficacy of imipramine, which has been used since as the gold standard for comparing the efficacy of subsequent drugs with antidepressant potential. This study, carried out 40 years ago, would have complied with current EBM standards and it is worth noting that a certain Professor A. L. Cochrane was a member of the committee running the trial.

Also in the early 1960s, The National Institute of Mental Health (NIMH) carried out a large RCT of the treatment of schizophrenia (NIMH-PSC Collaborative Study Group, 1964). Fluphenazine and thioridazine were compared with chlorpromazine and placebo. It showed the efficacy of drug treatment in acute schizophrenia and showed that there was no evidence for the view that chlorpromazine was more effective in patients requiring sedation or that the piperazine derivative fluphenazine was more effective in withdrawn patients. It also cast doubt on the view that extrapyramidal dysfunction was necessary for improvement to occur.

There is little doubt that psychiatrists acted on the results of these trials. Their broad conclusions provided the foundations for the modern treatment of depressive illness and acute schizophrenia.

WHAT EVIDENCE IS ACCEPTABLE?

Evidence-based medicine is described by Sackett et al (1996) as “the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients” and they go on to speak of needing to balance clinical expertise with such evidence and that neither alone is sufficient. It is encouraging that some of the founders of the EBM movement call for it to be a part of informing good clinical practice. Unfortunately they did not attempt to define or categorise the elusive quality of clinical competence (Greenhalgh, 1999) and the necessary balance was endangered further by the evangelism of some practitioners. The emphasis on hard ‘scientific’ data tends to devalue sophisticated clinical expertise largely derived from experience and the detailed study of individual patients.

Limitations of the evidence

Evidence deemed acceptable by the EBM movement as that on which treatment must be based is essentially information derived from RCTs or meta-analyses. This narrow approach diminishes consideration of other types of evidence available from naturalistic enquiry, case material and experiential sources and, for some doctors, is too blinkered a view.

Many clinicians regret the demise of the psychiatric case report. We feel that this significantly impoverishes the literature and further increases the tilt towards publications of an evidence-based nature and away from those considering individual patients. In 1999 this journal was encouraged to publish case reports for two reasons.

“The first is that the case is capable of generalising some fresh conjecture that is then amenable to empirical testing. The second is that the case embodies an effective refutation of a particular hypothesis.” (Farmer, 1999)

For clinicians these are not the only values: in addition, case studies can be a particularly helpful way of stimulating new clinical and psychodynamic insights and of illustrating nuances of treatment.

In clinical practice we are concerned with not missing cases and with not misdiagnosing non-cases as cases. In research the most important consideration is to have a specific and defined sample. Most psychiatric data are continuous, whereas EBM treatment data are categorical.

There may be no evidence. “It is a short step from without substantial evidence to without substantial value” (Bradley & Field, 1995). However, the absence of evidence of effectiveness is not the same as absence of effectiveness. Not all therapies are studied to the same extent.

Limitations of RCTs

Because the conclusions of RCTs are frequently deemed to be the only acceptable form of evidence, some observations about their design are pertinent. Most trials have an upper age limit of 64 years for inclusion. In many trials, women of child-bearing age or pregnant are excluded. Patients with mixed diagnoses and comorbidity are also excluded. Drop-outs are a major problem in RCTs. People may not cooperate for a variety of reasons: side-effects are often cited, nevertheless personality and social factors must play important roles. Randomised controlled trials rarely give information about these important issues. It often appears that those who are motivated to cooperate in RCTs are not a typical cross-section of our patients.

Many patients have multiple diagnoses and problems. Diagnosis alone is a poor predictor of treatment outcome. Personality characteristics and social circumstances influence therapeutic response. Efficacy in RCTs is no guarantee of effectiveness in the field, and effectiveness in the field is no guarantee of effectiveness in the individual patient. Efficacy and effectiveness may be conflated, to the confusion of clinicians, researchers and policy-makers (Chiesa & Fonagy, 1999; Wells, 1999).

Randomised controlled trials only provide us with information about groups, not individuals. A drug treatment may be helpful for a patient in the context of an individually tailored community care package or with psychological support, but not otherwise. There may be little or no hard evidence on which the treatment regime for a particular patient can be based. “Current emphasis on EBM and clinical effectiveness has meant that the prescribing behaviour of clinicians, including psychiatrists, is high on the research agendas” (Buston et al, 1998). However, other factors must be taken into consideration.

Limitation of meta-analyses and systematic reviews

Because the results from meta-analysis seem very precise and convincing and are beginning to have an impact on practice, clinicians must be reminded of the “file drawer problem” (Rosenthal, 1979). For any given research area, one cannot tell how many studies have been conducted but never reported. Considerable uncertainty exists. If the studies that show non-significant results were known, could these negate completely the conclusions of the meta-analysis? Easterbrook et al (1991), in a careful retrospective study of 487 research projects in Oxford, confirmed the presence of a publication bias in a cohort of clinical research studies and suggested that conclusions based only on a review of published data should be interpreted cautiously. They emphasised the need to identify the results of unpublished as well as published studies.

Caution needs to be exercised in reading systematic reviews, particularly if the amount of information found is small; few trials and small numbers lessen the reliability of a review. It is rare to find a systematic review devoted to adverse effects (Anonymous, 1999).

Role of qualitative research

Databases for EBM are founded on metaanalyses of quantitative research and therefore miss information that can be gained from a naturalistic enquiry: from qualitative research where the prime goal is not to enumerate (Buston et al, 1998). Qualitative research methods provide an in-depth examination of a small number of patients rather than a limited examination of large numbers. They are also able to investigate researchers' interpretations and meanings. Necessarily, in hypothesis-driven quantitative research the variables are reduced and information is honed down at the start — the entire complexity of the setting in which the subjects/patients find themselves is missing.

Qualitative research places great emphasis on examining why a particular treatment did or did not work and generating new ideas from these facts. A recent series (Mays & Pope, 2000; Meyer, 2000; Pope et al, 2000) described the application and usefulness of qualitative research methods in different settings and outlined how this alternative to quantitative research is coming to be accepted increasingly in health care research. This approach has the capacity to explore what meanings the symptom, the consultation and the treatment may have for the patient.

Assessing the evidence

Critical appraisal is clearly important. Some journal clubs have successfully introduced it (Warner & King, 1997; Geddes, 1998). However, one can be in danger of not seeing the wood for the trees. Simple reiteration of the paper is clearly not satisfactory but journal clubs where the critically appraised parts of the paper are exhaustively discussed risk losing a lively discussion of the subject of the paper — perhaps even including “interesting patients I have seen”. This type of discussion may be dismissed as mere anecdote but perhaps the role of narrative as explanation is as important for the doctor as for the patient.

Narrative of illness provides a framework for approaching a patient's problems holistically and may uncover diagnostic and therapeutic options (Greenhalgh, 1999). In addition to emphasising the interpretive (the discernment of meaning) component of taking a history, it offers a method for addressing existential qualities such as inner hurt, despair, hope, grief and moral pain, which frequently accompany and may even constitute people's illness. A recent series of articles (Elwyn & Gwyn, 1999; Greenhalgh, 1999; Greenhalgh & Hurwitz, 1999; Jones, 1999; Launer, 1999) made a very strong plea that the lost tradition of narrative should be revived in the teaching and practice of medicine.

A considerable amount of time now is being spent on critical appraisal exercises. Psychiatric trainees might spend some of it better by devoting more time to studying individual patients in depth and with a critical attitude to acquiring basic psychotherapeutic skills. Busy practising doctors often do not have the time or the expertise regularly to carry out accurate and useful critical appraisal. Most rely on expert abstracting journals such asJournal Watch to do this for them. The French pharmacological journalPrescrire has taken this a step further. A quick glance at the cartoon icon beside each review tells readers whether the drug is a major breakthrough (little man joyfully jumping up and down), a ‘me too’ drug (little man dropping a capsule into a box full of others) or simply ineffective (little man booting a capsule out of the window). This interesting innovation puts regular critical appraisal into context for busy clinicians.

THE DOCTOR AS THERAPIST

The clinical effectiveness initiative does not take into account the contribution to the outcome of the doctor's personality and attitude. The personal qualities of the physician have an important bearing on the outcome for the patient. This needs to be emphasised to psychiatrists and to the medical profession as a whole. For psychotherapy, outcome depends largely on the qualities of empathy, warmth and genuineness in the therapist, qualities that must affect any doctor/patient interaction. In addition, other patient factors are important. A recent follow-up study of breast cancer (Watson et al, 1999) showed a reduced survival in early breast cancer patients with depression and attitudes of hopelessness, thus emphasising the need to treat the whole patient rather than to focus entirely on organic aspects of the disease.

Naturally the importance of the doctor as the therapist or ‘the therapy’ has changed over time and varies from culture to culture. As specific treatments, both surgical and medical, have been developed the role of doctor as therapist may have diminished. Nevertheless, the doctor-patient relationship remains a potent factor and is particularly important in chronic ill health, terminal illness, rehabilitation and any disorder that may be influenced psychologically. Even in illnesses where there is a specific treatment, the full potential of the placebo response should be realised because it reflects the positive psychological need to overcome illness.

In the last decade of the 20th century a curious paradox developed. Doctors are now being urged at one and the same time to take users' wishes for treatment into account and to follow the edicts and restrictions of EBM. As the medical profession is embracing EBM, more and more patients are seeking out alternative and mostly unevaluated therapies. A recent survey revealed that the number of people using non-conventional treatments had doubled in the previous 6 years (Gregoriadis, 1999). One in five Britons had turned to complementary or alternative therapies, revealing that osteopathy, acupuncture, reflexology, aromatherapy and yoga have been gaining in popularity. The survey also found that 80% of people believe that such treatments will become more popular. The medical profession needs to understand the reasons for this trend. A number of suggestions have been put forward in explanation, including: the belief that conventional therapies do not work: less willingness to put up with side-effects; a desire to spend longer with a therapist than is possible with their general practitioner, where the average consultation lasts 8 min; and the demand for a more holistic approach to problems. Conventional medicine, by overemphasising its technical and scientific aspects, risks losing the art and humanity of its practice. It will be regrettable if two medical streams become established separately: an orthodox school linked with EBM and a range of alternative therapies.

It is essential for the National Health Service to provide a technically successful medical service that is packaged in a personal, empathic and holistic way. This approach was outlined clearly at the beginning of the 20th century by William Osler (Bliss, 1999), who understood that medicine has to be rooted in science but that doctors also must be healers.

AN EMERGING NEW UTILITARIAN ORTHODOXY?

How would Archie Cochrane view the emerging scene? His contributions are impressive, particularly to the development of epidemiology as a medical science, but would he be happy about all the activities linked with his name? He was a freethinking iconoclastic individual (Ford & Godber, 1998) with a healthy cynicism, who would not accept dogma (WKCM, 1988). He brought an open sceptical approach to medical problems and we think that he would be saddened to find that his name now embodies a new rigid medical orthodoxy while the real impact of his many achievements might be overlooked.

Archie Cochrane was not a clinician — a fact that must be kept in mind — and he may not have had the special experience of seeing a patient respond dramatically to treatment. Clinicians see this at first hand and have to balance these insights with evidence from accepted EBM sources. This is currently a problem in old age psychiatry. The EBM data on donepezil show a marginal improvement in patients with mild to moderate dementia. On the other hand, there are good examples of individual patients showing an impressive improvement (Dening & Lawton, 1998; Manchip & Morrison, 1999). Evidence of this nature must be recognised and not dismissed as anecdotal.

This emerging orthodoxy is leading to problems in three areas: the macro-economic level of purchasing health; the individual transaction between the doctor and patient; and the inhibiting influence of political correctness.

Information from EBM is being used to encourage minimalistic purchasing in the name of science. This pseudoscience may lead to rationing and the non-purchase of care that clinical judgement says is useful. A number of purchasers have not allowed the prescription of donepezil, rivastigmine and galantamine, citing ‘the evidence’ for denying patients this therapeutic possibility, rather than allowing a 3-month trial to discover those individuals who will benefit. The National Institute for Clinical Excellence (2001) recently has seen ‘the evidence’ in a different light and recommended guidelines for the use of anti-dementia drugs, regardless of the patients' address.

The medical profession is a broad church and often there are different ways of treating patients. In the EBM culture it is difficult to imagine how inspired new ideas can be developed and how new treatments can be initiated. The history of medical progress provides many examples of therapeutic developments as a result of the successful treatment of one patient (e.g. the first surgical operation for mitral stenosis).

Unfortunately the EBM approach is also a covert assertion that only factors that can be measured are recognised as important. This perspective reinforces an enduring weakness in British medicine: a dismissive attitude towards the importance of psychological and social factors in the causation and treatment of disease in the individual. We are convinced that scientific medical practice must be underpinned by the need to understand and respond empathically to the illness in accord with the patient's experiential perspective. This view is confirmed by the launch of a new journal Medical Humanities (Evans & Greaves, 1999). This will provide a forum for inquiry and serious discussion of the need to engage the humanities in medical education and professional development. It will explore how the scientific understanding of disease can be integrated with the human understanding of the experience.

DISCUSSION

Evidence-based medicine is not a new concept. There is an impressive tradition in the UK, extending over 50 years, of assessing the efficacy of new drugs through the use of RCTs. Psychiatrists are familiar with the seminal work of the Medical Research Council on the efficacy of anti-depressants, the work of the NIMH on phenothiazines, and similar trials. A degree of professional consensus is necessary so that treatment is dependent neither on luck nor on postcode. However, the proponents of EBM oversimplify the complex nature of clinical care. The needs of each patient are different, requiring individual professional assessment, and are often multiple, requiring application of skills of different disciplines. The social, economic and biological contexts of each patient result in varying outcomes. It is questionable whether one can ever have simple causality in human relationships. Let us hope that the day will not come when, on seeing a patient, the doctor's first thought is to turn to the Cochrane Library.

Our views are in accord with the criticisms that have been expressed already (Black, 1998; Kelly, 1999; Laugharne, 1999). Treating patients is a complex process. Evidence-based medicine must be only one element in the complexity of clinical decision-making. Patients need to be managed in a holistic way, in the context of their culture, unique psychological make-up and their relationship with their physician. Kafka's aphorism is as valid today as in 1917: “Prescribing is so easy, understanding people so hard” (Kafka, 1917).

Clinical Implications and Limitations

CLINICAL IMPLICATIONS

1. • Meta-analyses and systematic reviews do not take into account unpublished, possibly non-significant results.

2. • Personal qualities of the physician have an important bearing on outcome for the patient.

3. • Sophisticated clinical expertise with regard to an individual patient needs to be balanced with an evidence base derived from a group. Too much emphasis on a narrow range of acceptable evidence oversimplifies the complex nature of clinical care.

LIMITATIONS

1. • Professional consensus is necessary.

2. • One cannot argue against the use of evidence per se. Further evidence could be gathered by measuring aspects of the doctor—patient relationship.

3. • Evidence-based medicine could be used by patients in their decision-making, if carefully presented.