There is a deep mistrust of psychiatry fostered by reports suggesting that psychotropic drug efficacy is very small. Kirsch et al concluded that antidepressants should only be used in severely ill patients; Reference Kirsch, Deacon, Huedo-Medina, Scoboria, Moore and Johnson1 the efficacy of cholinesterase inhibitors in Alzheimer's disease and of lithium prophylaxis in bipolar disorder has been questioned; Reference Kaduszkiewicz, Zimmermann, Beck-Bornholdt and van den Bussche2,Reference Moncrieff3 and we found a smaller antipsychotic drug–placebo difference in schizophrenia than we intuitively expected. Reference Leucht, Arbter, Engel, Kissling and Davis4 These reviews inspired an article in The New Yorker summarising them, Reference Menand5 and fuelled a vocal antipsychiatry movement. Reference Breggin6,Reference Kirsch7 Psychiatrists, patients, caregivers and the press are unsettled by these findings and some may think that psychiatric medication is not worth the bother. But is this small efficacy really true, and what about other medical interventions? As medicine is becoming highly specialised, few psychiatrists are familiar with the evidence of general medicine and psychiatric drugs. In this context we reviewed the efficacy of psychiatric pharmacotherapy in the perspective of standard medical drugs, making this paper the first attempt to provide a panoramic overview of major drugs. It is not possible to compare qualitatively different outcomes in qualitatively different diseases, but one can compare the percentages of patients helped with a drug or placebo, keeping in mind the differences in outcome for the mere purpose of perspective. We hasten to add a warning not to be overly concrete and to interpret this review as a qualitative perspective and not as a comparison. Therefore we discuss major factors that need to be taken into account in the interpretation of clinical trials and systematic reviews.
Method
Identification of diseases of interest and search strategy
We reviewed textbooks, Reference Herold8,Reference Möller, Laux and Deister9 identified common diseases by consensus (S.L., S.H. and J.M.D.) based on frequency, importance and available treatment, and consulted national and international guidelines to identify primary treatments. We hand-searched the Cochrane Library, and searched Medline combining MeSH terms for the medical and psychiatric disorders with the MeSH term for meta-analysis (no time or language limit, last search May 2009) and references of included reports for systematic reviews of randomised controlled trials that applied meta-analysis and compared monotherapy of these treatments with placebo.
We first excluded meta-analyses of studies of subgroups (e.g. elderly people) and chose reviews of classes of drugs rather than single drugs (e.g. any antipsychotic, rather than only haloperidol) if available, based on the assumption that the original reviewers had made an appropriate decision to pool the drugs. We then chose the most recent reviews, because even if methodologically better an older review would have certainly been out of date. This was a conservative decision, because old meta-analyses in psychiatry usually had higher effect sizes (see Discussion and online Table DS1). The rare exceptions were slightly older meta-analyses that reported the indices necessary for our analysis more completely. These usually were Cochrane reviews which were preferred in case of doubt, because they use similar methodology and always fully report the data. To corroborate these decisions we always compared different reviews for consistency of results and contacted authors in the rare case that the results were discrepant. (These additional reviews are quoted in the footnotes of the tables in the online data supplement.) The quality of the included systematic reviews was evaluated with the AMSTAR score (range of possible values 0–11). Reference Shea, Hamel, Wells, Bouter, Kristjansson and Grimshaw10 Only primary efficacy outcomes in the areas of interest according to the treatment guidelines were extracted.
Statistical analysis
For continuous outcomes we extracted effect sizes and their 95% confidence intervals, presented both as differences in original units (mean difference) and as standardised mean differences (SMD). Mean differences were calculated according to the general formula (mean group A)—(mean group B), e.g. 75 kg in the drug group minus 70 kg in the placebo group gives a mean difference in body weight of 5 kg. Standardised mean differences (SMDs) provide a difference in standard deviation units (mean group A—mean group B) / standard deviation, e.g. (75—70) / 10 = 0.50, using the values from the previous example.
For dichotomous outcomes we presented the percentage of participants improved in the drug and placebo groups, the absolute risk/response difference (ARD; % responder drug – % responder placebo); the relative risk reduction (RRR; 1 – (% risk drug / % risk placebo) or relative response (RR) ratio (% responder drug / % responder placebo); and the number needed to treat (NNT), with their 95% confidence intervals. We also presented the P value, the number of studies and participants included and the average study duration (see online Table DS2 for a detailed description of these parameters).
Where our five standard parameters (mean difference, SMD, ARD, RRR, RR, NNT) were not reported in the studies, we transformed the existing data, or re-calculated meta-analyses by entering single study results using Review Manager version 5.0 or Comprehensive Meta-analysis version 2 for Windows. 11,Reference Borenstein, Hedges, Higgins and Rothstein12 S.H. ran the searches, S.H. and S.L. selected the reports. S.H. extracted the data, S.L. independently verified them, disagreements were resolved by J.M.D. and W.K., and M.D. rated the AMSTAR score.
Results
The Medline searches yielded 6175 abstracts and we hand-searched 1830 titles of Cochrane reviews – see online Figs DS1–24 for Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) diagrams of the selection process. Reference Liberati, Altman, Tetzlaff, Mulrow, Gotzsche and Ioannidis13 We included 94 meta-analyses of 48 drugs in 20 medical diseases (median AMSTAR score 9.0, 95% CI 8.2–9.2) and 33 meta-analyses of 16 drugs in 8 psychiatric disorders (median AMSTAR score 8.0, 95% CI 6.9–8.9). In the text we systematically present the raw numbers (for dichotomous outcomes the percentage responders in the placebo and drug groups; for continuous outcomes the average mean difference) and the average effect size (ARD and RRR/RR for dichotomous outcomes, SMD for continuous outcomes). Tables 1 and 2 present only some examples. Online Tables DS3 and DS4 present a comprehensive list including number of studies/participants, numbers needed to treat, P values and confidence intervals for each outcome and each intervention. A positive sign means that a drug either increased a positive outcome (e.g. response) or reduced a negative outcome (e.g. relapse). All the effect sizes in online Tables DS3 and DS4 are presented in Fig. 1 to give the overall gestalt. For this purpose, effect sizes for dichotomous outcomes (ARD, RR/RRR) were converted to SMDs in Comprehensive Meta-analysis 2. Reference Borenstein, Hedges, Higgins and Rothstein12,Reference Chinn14 This figure corresponds to online Fig. DS25 which indicates which dot relates to which study or outcome. Figures DS26 and DS28 present the same gestalt for relative and absolute risk/responder differences.
Medical disorders
In Tables 1 and DS3 the data are presented in an abbreviated ‘participants – intervention – comparator – outcome’ (PICO) format (the comparator is always placebo or no treatment).
Hypertension: antihypertensives for reduction of blood pressure, prevention of cardiovascular events and mortality
Several drug classes yielded similar results (Table DS3). Combining all agents, blood pressure was reduced by 9.4 mmHg systolic and 5.5 mmHg diastolic in the short term (SMDs 0.54 and 0.56 respectively). Reference Law, Morris, Jordan and Wald15 In the long term all drug classes significantly reduced cardiovascular events, e.g. angiotensin-converting enzyme (ACE) inhibitors reduced such events from 18% to 14% (ARD 4%, RRR 22%). 16 A significant reduction of mortality has not been shown for all of them (Table DS3).
Fig. 1 Summary of effect sizes.
All effect sizes in online Tables DS3 and DS4 are presented except for duplicates (e.g. effect size on dichotomous response and continuous reduction of severity in schizophrenia). Online Fig. DS25 identifies which dot corresponds to which result, and Figs DS26–29 present the results of dichotomous outcomes as relative and absolute risk/responder differences. Data on older meta-analyses from Table DS1 are not included. Effect sizes of dichotomous outcomes were converted to standardised mean differences expressed as Hedges’ g. Effect sizes of general medicine medication are presented on the left-hand side (median 0.37, mean 0.45, 95% CI 0.37–0.53) and psychiatric drugs on the right-hand side (median 0.41, mean 0.49, 95% CI 0.41–0.57).
TABLE 1 Examples of the efficacy of general medicine drugs v. placebo (full version is given in online Table DS3)
| Participants with outcome % | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Study/AMSTARFootnote a | Therapy | No. of studies | n | Duration, mean | Outcome (units) | Placebo | Drug | ARDFootnote b % | RRR/RR % | MD | SMDFootnote c |
| Hypertension | |||||||||||
| Effects on blood pressure | |||||||||||
| Law et al 15/5 | Any antihypertensive | 94 | 17 641 | 8 wk | RR systolic (mmHg) | 9.4Footnote *** | 0.56Footnote *** | ||||
| 94 | 17 641 | 8 wk | RR diastolic (mmHg) | 5.5Footnote *** | 0.54Footnote *** | ||||||
| Long-term effects on cardiovascular events and mortality | |||||||||||
| BPLTTC16/5 | ACE inhibitors | 5 | 18 229 | 3.9 yr | Cardiovascular events | 18.1 | 14.1 | 4 NI | 22Footnote *** | 0.16 | |
| Acute stroke | |||||||||||
| Wardlaw17/9 | Thrombolysis | 22 | 6 283 | 12-26 wk | Death/dependency | 55.8 | 50.9 | 5Footnote ** | 9Footnote ** | 0.11 | |
| Prevention of cardiovascular disease and stroke | |||||||||||
| Baigent et al 21/5 | Aspirin (primary prev.) | 6 | 95 000 | 5.8 yr | Cardiovascular events | 0.57/yr | 0.51/yr | 0.07/yr | 12/yrFootnote *** | 0.06 | |
| Baigent et al 21/5 | Aspirin (secondary prev.) | 16 | 17 000 | NI | Cardiovascular events | 8.2/yr | 6.7/yr | 1.5/yr | 19/yrFootnote *** | 0.12 | |
| Law et al 22/5 | Statins | 164 | ̃38 000 | 2–6 wk | LDL cholesterol (mmol/l) | 1.54Footnote *** | NI | ||||
| Baigent et al 23/6 | Statins | 14 | 90 056 | 5.0 yr | Cardiovascular events | 17.8 | 14.1 | 4Footnote *** | 21Footnote *** | 0.15 | |
| Chronic heart failure | |||||||||||
| Flather et al 98/6 | ACE inhibitors long term | 5 | 12 763 | 2.9 yr | Mortality | 26.8 | 23.0 | 4Footnote *** | 15Footnote *** | 0.11 | |
| Rheumatoid arthritis | |||||||||||
| Suarez-Almazor et al 99/10 | Methotrexate | 5 | 218 | 412 wk | Tender joints | NE | 0.86Footnote *** | ||||
| Migraine | |||||||||||
| McCrory & Gray31/8 | Sumatriptan | 8 | 2 221 | 2 h | Pain-free | 8.5 | 29.5 | 20Footnote *** | 220Footnote *** | 0.41 | |
| Linde & Rossnagel33/9 | Propanolol | 4 | 205 | 13 wk | Response | 30.9 | 52.3 | 35Footnote * | 80Footnote * | 0.49 | |
| Asthma | |||||||||||
| Sin et al 36/7 | Corticosteroids | 19 | 3 271 | >12 wk | FEV1 (l) | 0.33Footnote *** | 0.56Footnote *** | ||||
| 11 | 8 999 | >12 wk | Exacerbations | NI | NI | NI | 54Footnote * | NE | |||
| Chronic obstructive pulmonary disease | |||||||||||
| Yang et al 42/10 | Inhaled corticosteroids | 3 | 952 | 8–24 wk | FEV1 (l) | 0.10Footnote *** | 0.36Footnote *** | ||||
| 4 | 2 063 | >26 wk | Exacerbations | 0.26Footnote *** | 0.20Footnote *** | ||||||
| Diabetes | |||||||||||
| Saenz et al 44/11 | Metformin | 12 | 1 587 | 21.5 wk | Fasting glucose (mmol/l) | 1.84Footnote *** | 0.87Footnote *** | ||||
| 1 | 753 | 10.7 yr | Mortality | 21.7 | 14.6 | 7Footnote ** | 32Footnote ** | 0.27 | |||
| Chronic hepatitis C | |||||||||||
| Myers et al 46/10 | Interferon | 8 | 409 | 426 wk | Virological response | 1.0 | 38.3 | 35Footnote *** | 1070Footnote *** | 2.27 | |
| Reflux oesophagitis | |||||||||||
| Moayyedi et al 48/9 | Proton pump inhibitors | 5 | 645 | 8 wk | Remission | 28.3 | 83.2 | 58Footnote *** | 256Footnote *** | 1.39 | |
| Ulcerative colitis | |||||||||||
| Sutherland & Macdonald50/9 | 5-ASA | 4 | 892 | 8 wk | Remission | 10.0 | 19.9 | 8Footnote *** | 70Footnote ** | 0.44 | |
| Multiple sclerosis | |||||||||||
| Rice et al 53/10 | Interferon | 3 | 919 | 2.0 yr | Exacerbation | 69.5 | 55.2 | 14Footnote *** | 19Footnote *** | 0.34 | |
| Breast cancer | |||||||||||
| EBCTCG56/4 | Polychemotherapy | 60 | 28 764 | 15.0 yr | Mortality age <50 | 42.4 | 32.4 | 10Footnote *** | 24Footnote *** | 0.24 | |
| Non-small cell lung cancer | |||||||||||
| Bria et al 57/8 | Adjuvant chemotherapy | 21 | 7 408 | 4.5 yr | Mortality | NI | NI | 3 NI | 9Footnote * | NE | |
| Antibiotics for various diseases | |||||||||||
| Glasziou et al 60/10 | Otitis media | 10 | 2 791 | 2–7 d | With pain | 22.2 | 16 | 6.2 NI | 28Footnote *** | 0.22 | |
| Falagas et al 61/8 | Cystitis | 4 | 1 062 | 3–17 d | Cure | 25.7 | 61.8 | 36.1 NI | 139Footnote *** | 0.85 | |
ACE, angiotensin-converting enzyme; ARD, absolute response or risk difference; ASA, acetylsalicylic acid; BPLTTC, Blood Pressure Lowering Treatment Trialists' Collaboration; d, day; EBCTCG, Early Breast Cancer Trialists' Collaborative Group; FEV1, forced expiratory volume in 1 s; h, hours; LDL, low-density lipoprotein; MD, mean difference; NE, not estimable; NI, not indicated; NNT, number needed to treat; NS, not significant; prev., prevention; RRR/RR, response ratio/relative risk reductions; SMD, standardised mean difference; wk, week; yr, year.
a. AMSTAR quality score (range of possible values 0–11).
b. Positive values always mean superiority of drug.
c. Italics indicate mean estimated values.
Footnote * P<0.05
Footnote ** P<0.01
Footnote *** P<0.001. Results on absolute and relative risk/responder differences do not always exactly match with the formulae presented in the manuscript due to weighting processes in meta-analyses.
Acute ischaemic stroke: thrombolysis, aspirin and heparin for prevention of death or dependency
Thrombolysis reduced death or dependency from 56% to 51% (ARD 5%, RRR 9%), Reference Wardlaw, Murray, Berge and Del Zoppo17 but when administered after 4.5 h mortality is increased by haemorrhages. Reference Hacke, Kaste, Bluhmki, Brozman, Davalos and Guidetti18 Aspirin reduced death or dependency from 46% to 45%, Reference Sandercock, Counsell, Gubitz and Tseng19 whereas heparin was ineffective. Reference Sandercock, Counsell and Kamal20
Cardiovascular disease: aspirin for primary and secondary prevention of cardiovascular events and mortality
In secondary prevention, low-dose aspirin reduced serious cardiovascular events per year from 8.2% to 6.7% (ARD 1.5%, RRR 19%) and vascular mortality per year from 4.1% to 3.7% (ARD 0.29%, RRR 9%, P = 0.05). Reference Baigent, Blackwell, Collins, Emberson, Godwin and Peto21 In primary prevention, aspirin reduced the number of cardiovascular events per year from 0.57% to 0.51%, but there was no effect on mortality because the reduction of occlusive events was balanced by an increase in major bleeds (mortality per year: placebo 0.19%, drug 0.19%). Reference Baigent, Blackwell, Collins, Emberson, Godwin and Peto21
Hypercholesterolaemia: statins for reduction of cholesterol levels and prevention of cardiovascular disease and mortality
In the short term, statins reduced low-density lipoprotein (LDL) cholesterol by 1.54 mmol/l or 31%. Reference Law, Wald and Rudnicka22 In the long term, cardiovascular events were reduced from 18% to 14% (primary and secondary prevention combined, ARD 4%, RRR 21%) and 5-year mortality from 9.7% to 8.5%. Reference Baigent, Keech, Kearney, Blackwell, Buck and Pollicino23
Chronic heart failure: various drugs for reduction of mortality
Angiotensin-converting enzyme inhibitors, angiotensin receptor blockers, beta-blockers and diuretics respectively reduced mortality rates from 27% to 23% (ARD 4%, RRR 15%), from 18% to 11%, from 13% to 8% and from 12% to 3%. Reference Garg and Yusuf24–Reference Faris, Flather, Purcell, Poole-Wilson and Coats27 Digitalis significantly reduced hospital admission (from 33% to 25%) but not mortality. Reference Hood, Dans, Guyatt, Jaeschke and McMurray28
Rheumatoid arthritis: antirheumatic drugs for the reduction of tender joints
Various immunosuppressants, corticosteroids and other agents reduced the number of tender joints with reasonably good SMDs between 0.33 and 1.33 (raw values for mean differences were not presented; Table DS3). Reference Suarez-Almazor, Belseck, Shea, Homik, Wells and Tugwell29,Reference Blumenauer, Judd, Wells, Burls, Cranney and Hochberg30
Acute migraine: effects of sumatriptan and aspirin on the number of patients pain-free after 2 h
Sumatriptan increased the percentage of patients pain-free after 2 h from 9% to 30% (ARD 20%, RR 220%) Reference McCrory and Gray31 and intravenous aspirin increased it from 15% to 27%. Reference Lampl, Voelker and Diener32
Prophylaxis of migraine: effects of propanolol and anticonvulsants on responder rates and on the number of migraine attacks
Fifty-two per cent responded to propanolol prophylaxis and 31% to placebo (ARD 35%, RR 80%). Reference Linde and Rossnagel33 Patients had approximately one migraine attack less (SMD 0.47). Reference Linde and Rossnagel33 The results of anticonvulsants were similar. Reference Mulleners and Chronicle34
Chronic asthma: effects of inhaled corticosteroids and beta-2-agonists on forced expiratory volume and on asthma exacerbations
The first-line drugs for chronic, severe asthma are inhaled corticosteroids and beta-2-agonists (short-acting as needed, long-acting in patients with refractory disease). 35 Inhaled corticosteroids increased forced expiratory volume in 1 s (FEV1) by 330 ml (SMD 0.56). Reference Sin, Man, Sharpe, Gan and Man36 The addition of long-acting beta-2-agonists improved FEV1 by 190 ml (SMD 0.35), Reference Ni, Greenstone, Danish, Magdolinos, Masse and Zhang37 but the reduction of asthma exacerbations found by some meta-analyses is controversial, Reference Sin, Man, Sharpe, Gan and Man36,Reference Bateman, Nelson, Bousquet, Kral, Sutton and Ortega38 because another meta-analysis found more severe exacerbations. Reference Salpeter, Buckley, Ormiston and Salpeter39
Chronic obstructive pulmonary disease: effects of various agents on FEV1 and on disease exacerbations
Guidelines recommend anticholinergics, beta-2-agonists and inhaled corticosteroids. Reference Vogelmeier, Buhl, Criée, Gillissen, Kardos and Köhler40 The anticholinergic tiotropium improved FEV1 by 200 ml (SMD 0.99). Reference Barr, Bourbeau, Camargo and Ram41 It reduced exacerbations from 31% to 23% (ARD 5%, RRR 17%). Reference Barr, Bourbeau, Camargo and Ram41 Inhaled corticosteroids improved FEVl by 100 ml (SMD 0.36) and the number of exacerbations per patient and year by 0.26 (SMD 0.20). Reference Yang, Fong, Sim, Black and Lasserson42 The data on long-acting beta-2-agonists are equivocal. They reduced exacerbations (e.g. Salpeter et al), Reference Salpeter, Buckley and Salpeter43 but one systematic review found them to increase respiratory deaths. Reference Salpeter, Buckley and Salpeter43
Type 2 diabetes: various antidiabetics for reduction of HbA1c and mortality
Metformin reduced HbA1c by 1% (SMD 0.97) and α-glucosidase inhibitors reduced it by 0.8% (SMD 0.64). Reference Saenz, Fernandez-Esteban, Mataix, Ausejo, Roque and Moher44,Reference Van de Laar, Lucassen, Akkermans, Van de Lisdonk, Rutten and van Weel45 In the long term, metformin reduced the death rate from 22% to 15% (ARD 7%, RRR 32%), Reference Saenz, Fernandez-Esteban, Mataix, Ausejo, Roque and Moher44 but α-glucosidase inhibitors have not been shown to change the death rate. Reference Van de Laar, Lucassen, Akkermans, Van de Lisdonk, Rutten and van Weel45
Hepatitis C: effects of interferon and ribavirin on virological response/morbidity and mortality
Interferon increased the number of participants with no detectable virus at treatment end (virological response) from 1% to 38% (ARD 35%, RR 1070%). Reference Myers, Regimbeau, Thevenot, Leroy, Mathurin and Opolon46 Ribavirin was only efficacious in combination with interferon. Reference Brok, Gluud and Gluud47
Reflux oesophagitis: proton pump inhibitors for clinical remission and relapse prevention
Proton pump inhibitors are highly effective in acute treatment (response: placebo 28%, drug 83%, ARD 58%, RR 256%), Reference Moayyedi, Santana, Khan, Preston and Donnellan48 and also in maintenance treatment (relapse: placebo 75%, drug 36%). Reference Donnellan, Sharma, Preston and Moayyedi49
Ulcerative colitis: 5-aminosalicylic acid for clinical remission and maintenance of remission
Five-aminosalicylic acid (5-ASA) increased remission from 10% with placebo to 20% (ARD 8%, RR 70%), Reference Sutherland and Macdonald50 and maintenance of remission from 37% to 53% (ARD 18%, RR 50%). Reference Sutherland and Macdonald51
Multiple sclerosis: corticosteroids for treatment of acute episodes and interferon for prevention of exacerbations
Acute treatment with corticosteroids increased the proportion of responders from 28% with placebo to 68% (ARD 41%, RR 140%). Reference Filippini, Brusaferri, Sibley, Citterio, Ciucci and Midgard52 In the first 2 years, prevention with interferon beta reduced exacerbations from 70% to 55% (ARD 14%, RRR 19%). Reference Rice, Incorvaia, Munari, Ebers, Polman and D'Amico53
Parkinson's disease: effects of levodopa on disease symptoms
There was no systematic review of the standard treatments levodopa or dopamine agonists with data compared with placebo. We parenthetically note that the National Institute for Health and Clinical Excellence (NICE) guideline based its recommendation on a pivotal 42-week trial in which levodopa produced 7 points more improvement in the Unified Parkinson's Disease Rating Scale total score than placebo (SMD 0.93), 54 but also a 7% stronger decline of striatal dopamine transporter density (SMD —0.44), suggesting a possible acceleration of nigrostriatal dopamine nerve terminal loss. Reference Fahn, Oakes, Shoulson, Kieburtz, Rudolph and Lang55
Breast and lung cancer: polychemotherapy for reduction of mortality
Breast cancer is the most frequent neoplasm in women and lung cancer is the leading cause of cancer death. Polychemotherapy reduced the 15-year breast cancer mortality in younger women (<50 years) from 42% to 32% (ARD 10%, RRR 24%) but in older women only from 50% to 47%. 56 Tamoxifen added to polychemotherapy reduced the 15-year mortality in oestrogen receptor-positive patients from 35% to 26%. 56 In the study by Bria et al, adjuvant chemotherapy led to a small reduction of 5-year lung-cancer mortality (ARD 3%, RRR 9%), Reference Bria, Gralla, Raftopoulos, Cuppone, Milella and Sperduti57 confirming a landmark previous meta-analysis. 58
Infectious diseases: antibiotics for rhinosinusitis, otitis media, uncomplicated cystitis and prophylaxis of wound infection after surgery
The effects of antibiotics depend on the infection. We did not find meta-analyses on severe infections such as pneumonia or on antivirals (monotherapy v. placebo) for HIV. A meta-analysis concluded against their general use in rhinosinusitis owing to small effect size (response: placebo 57%, drug 64%, ARD 7%, RRR 13%). Reference Young, De Sutter, Merenstein, van Essen, Kaiser and Varonen59 The use of antibiotics in otitis media is debated, as within 2–7 days 78% of patients recovered spontaneously compared with 84% taking antibiotics (ARD 6%, RR 28%). Reference Glasziou, Del Mar, Sanders and Hayem60 In contrast, the efficacy in uncomplicated cystitis (response: placebo 26%, drug 62%) and for the prophylaxis of wound infections after major operations (infections: placebo 39%, antibiotics 10%) was clear. Reference Falagas, Kotsantis, Vouloumanou and Rafailidis61,Reference Nelson, Glenny and Song62
Psychiatric disorders
Full data are given in Table DS4; examples are summarised in Table 2.
Schizophrenia: antipsychotics for reduction of overall symptoms and relapse prevention
In acute treatment, second-generation antipsychotics increased the percentage responding from 24% with placebo to 41% (ARD 18%, RR 70%), and reduced the Brief Psychiatric Rating Scale/Positive and Negative Syndrome Scale total score by 9/10 points (SMD 0.51). Reference Leucht, Arbter, Engel, Kissling and Davis4 Antipsychotic maintenance treatment reduced relapse rates from 57% to 22% within approximately 10 months (ARD 38%, RRR 65%). Reference Leucht, Komossa, Heres, Kissling and Davis63
Bipolar disorder: mood stabilisers for acute mania, antidepressants for depression and mood stabilisers for relapse prevention
Various antimania agents increased the percentage responding from approximately 30% with placebo to approximately 50% within 3 weeks (response to lithium 52% v. placebo 34%, ARD 17%, RR 50%; Reference Storosum, Wohlfarth, Schene, Elferink, van Zwieten and van den Brink64 response to valproate 47% v. placebo 21%, ARD 27%, RR 150%; Reference Smith, Cornelius, Warnock, Tacchi and Taylor65 response to carbamazepine 51% v. placebo 26%, ARD 25%, RRR 100%; Reference Smith, Cornelius, Warnock, Tacchi and Taylor65 response to antipsychotics 50% v. placebo 31%, ARD 20%, RR 60%). Reference Scherk, Pajonk and Leucht66 In bipolar depression, antidepressants increased the response rate from 34% to 58% (ARD 34%, RR 130%). Reference Gijsman, Geddes, Rendell, Nolen and Goodwin67 In maintenance treatment, lithium reduced relapse rates from 81% to 36% (ARD 53%, RRR 51%), Reference Davis, Janicak and Hogan68 or from 61% to 40% after excluding studies in which lithium was suddenly discontinued (ARD 24%, RRR 35%). Reference Geddes, Burgess, Hawton, Jamison and Goodwin69
Major depressive disorder: antidepressants for acute depression and relapse prevention
The absolute responder differences in recent meta-analyses of various selective serotonin reuptake inhibitors (SSRIs) (or tricyclic antidepressants used as an active comparator in SSRI v. placebo studies) Reference Storosum, Elferink, van Zwieten, van den Brink, Gersons and van Strik70 v. placebo in major depressive disorder were 10–15% (Table DS4). For example, paroxetine increased the percentage responding from 42% to 53% (ARD 10%, RR 20%) and reduced the Hamilton Rating Scale for Depression score by 3 points (SMD 0.32). Reference Barbui, Furukawa and Cipriani71 These studies were currently primarily conducted in out-patients with less severe disorder (e.g. 90% of the sample were out-patients in the meta-analysis by Barbui et al). Reference Barbui, Furukawa and Cipriani71
Maintenance treatment with any antidepressant reduced the relapse rate from 41% with placebo to 18% (ARD 23%, RRR 58%), Reference Geddes, Carney, Davies, Furukawa, Kupfer and Frank72 consistent with another meta-analysis restricted to new antidepressants (placebo 48% v. drug 26%, ARD 22%, RRR 44%). Reference Hansen, Gartlehner, Lohr, Gaynes and Carey73
Obsessive–compulsive disorder: effects of SSRIs on responder rates and overall symptoms
Selective serotonin reuptake inhibitors increased the proportion of patients responding in the acute phase from 23% to 43% (ARD 20%, RRR 84%). Reference Soomro, Altman, Rajagopal and Oakley-Browne74 These drugs reduced the mean Yale–Brown Obsessive Compulsive Scale score by 3.2 points (SMD 0.44). Reference Soomro, Altman, Rajagopal and Oakley-Browne74
Panic disorder: tricyclic antidepressants, SSRIs and benzodiazepines for anxiety symptoms
The mean SMDs (raw differences in rating scale scores or responder rates were not indicated) of tricyclic antidepressants, SSRIs and benzodiazepines in acute treatment were 0.40–0.41. Reference Mitte75
Alzheimer's disease: cholinesterase inhibitors for prevention of cognitive decline
Within 6 months, cholinesterase inhibitors increased the percentage of participants unchanged or improved from 17% to 24% (ARD 7%, RRR 43%). Reference Birks76 The cognitive subscore of the Alzheimer's Disease Assessment Scale was better by 2 points (SMD 0.41). Reference Birks76
Attention-deficit hyperactivity disorder: effects of various drugs on symptoms
Methylphenidate (SMD 0.78), amphetamines (SMD 1.00) and atomoxetine (SMD 0.64) showed robust effect sizes in overall reduction of attention-deficit hyperactivity disorder symptoms (raw differences in rating scale scores or responder rates were not indicated). Reference Schachter, Pham, King, Langford and Moher77–Reference Cheng, Chen, Ko and Ng79
Discussion
Any comparison of treatments for different diseases can only be qualitative in nature and therefore Fig. 1 is no more than a way to place psychiatric drugs in the perspective of general medicine medication. Some general medical drugs have very high effect sizes, but those obtained by psychiatric drugs are in the same range as most general medical pharmacotherapeutics. This said, the increment of improvement by a drug must be viewed in the context of the seriousness of the disease, the suffering induced, the outcome in question, societal values and the natural course including the duration of the disease. In the following paragraphs we discuss a number of these issues which readers should take into account in interpreting the results.
TABLE 2 Examples of efficacy of psychiatric drugs v. placebo (full version is given in online Table DS4
| Participants with outcome % | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Study/AMSTARFootnote a | Therapy | No. of studies | n | Weeks, mean | Outcome (units) | Placebo | Drug | ARDFootnote b % | RRR/RR % | MD | SMDFootnote b |
| Schizophrenia | |||||||||||
| Acute treatment | |||||||||||
| Leucht et al 4/10 | SGAs | 28 | 4498 | 9 | Responders | 23.7 | 40.6 | 18Footnote *** | 70Footnote *** | 0.43 | |
| 35 | 5568 | 10 | PANSS/BPRS | NI | 0.51Footnote *** | ||||||
| Leucht et al 4/10 | HaloperidolFootnote d | 10 | 1440 | 6 | Responders | 19.5 | 29.3 | 12%Footnote *** | 60%Footnote *** | 0.30 | |
| 11 | 1540 | 6 | PANSS/BPRS | NI | 0.53Footnote *** | ||||||
| Maintenance treatment | |||||||||||
| Leucht et al 63/10 | Antipsychotics | 62 | 6392 | 42 | Relapse | 57.0 | 22.0 | 38%Footnote *** | 65Footnote *** | 0.92 | |
| Bipolar affective disorder | |||||||||||
| Acute manic episode | |||||||||||
| Storosum et al 64/6 | Lithium | 6 | 811 | 3 | Responders | 34.0 | 52.0 | 17Footnote *** | 50Footnote ** | 0.41 | |
| 7 | 1165 | 3 | YMRS/MRS | NI | 0.40Footnote *** | ||||||
| Smith et al 65/19 Footnote e | Valproate | 2 | 182 | 3 | Responders | 21.1 | 47.1 | 27%Footnote *** | 150%Footnote * | 0.66 | |
| 4 | 782 | 3 | YMRS/MRS | NI | 0.40Footnote *** | ||||||
| Smith et al 65/9 Footnote e | Carbamazepine | 2 | 443 | 3 | Responders | 25.5 | 51.1 | 25%Footnote *** | 100%Footnote *** | 0.61 | |
| 2 | 331 | 3 | YMRS | 6.6Footnote *** | 0.53Footnote *** | ||||||
| Scherk et al 66/10 | SGAs and haloperidol | 12 | 2939 | 3 | Responders | 30.8 | 49.9 | 20Footnote *** | 60Footnote *** | 0.44 | |
| 12 | 2939 | 3 | YMRS/MRS/MS | 4.7Footnote c Footnote *** | 0.45Footnote *** | ||||||
| Depressive episode | |||||||||||
| Gijsman et al 67/9 | Antidepressants | 4 | 662 | 7 | Responders | 34.1 | 57.7 | 34Footnote *** | 130Footnote ** | 0.53 | |
| Maintenance therapy | |||||||||||
| Davis et al Reference Davis, Janicak and Hogan68 | Lithium | 9 | 421 | NI | Any relapse | 81.4 | 36.2 | 53Footnote *** | 51Footnote *** | 1.12 | |
| Geddes et al 69/8 | Lithium | 5 | 770 | 73 | Any relapse | 61.0 | 40.0 | 24Footnote ** | 35Footnote ** | 0.47 | |
| Macritchie et al 100/10 | Valproate | 1 | 281 | 52 | Any relapse | 38.3 | 24.1 | 14%Footnote * | 37%Footnote * | 0.37 | |
| Major depressive disorder | |||||||||||
| Acute episode | |||||||||||
| Barbui et al 71/8 | Paroxetine | 22 | 5112 | 7.5 | Responders | 42.4 | 53.2 | 10Footnote *** | 20Footnote *** | 0.24 | |
| 34 | 5764 | 7.5 | HRSD | 2.62Footnote *** | 0.31Footnote *** | ||||||
| Storosum et al 70/4 | TCAs (new AD studies) | 32 | 4314 | 6 | Responders | 31.0 | 46.0 | 15Footnote *** | 50Footnote *** | 0.35 | |
| 6 HRSD | 2.65Footnote *** | 0.33Footnote *** | |||||||||
| Bech et al 101/5 | Fluoxetine | 16 | 2761 | 6 | Responders | 24.2 | 37.8 | 13.6%Footnote *** | 65%Footnote *** | 0.35 | |
| 7 | NI | 6 | HRSD | NI | 0.30 | ||||||
| Kirsch et al 1/7 | New ADs | 35 | 5133 | 6 | HRSD | 1.80Footnote *** | 0.32Footnote *** | ||||
| Maintenance treatment | |||||||||||
| Geddes et al 72/9 | Antidepressants | 35 | 5032 | 63 | Relapse | 41.0 | 18.0 | 23 NI | 58Footnote *** | 0.64 | |
| Hansen et al 100/10 | New ADs | 11 | 3326 | 68 | Recurrence | 48.0 | 26.0 | 22 NI | 44Footnote *** | 0.53 | |
| Davis et al Reference Davis, Janicak and Hogan68 | Lithium | 9 | 227 | NI | Relapse | 75.0 | 36.0 | 39 NI | 53Footnote *** | 0.92 | |
| Obsessive–compulsive disorder | |||||||||||
| Soomro et al 74/11 | SSRIs | 17 | 3097 | 10 | Y–BOCS | 3.21Footnote *** | 0.44Footnote *** | ||||
| 13 | 2709 | 10 | Responders | 22.6 | 43.3 | 20Footnote *** | 84Footnote *** | 0.53 | |||
| Panic disorder | |||||||||||
| Mitte75/4 | TCAs, SSRIs, BZDs | 17–25 | NI | 8 | Anxiety | NI | 0.41Footnote * | ||||
| Dementia | |||||||||||
| Birks76/8 | Cholinesterase inhibitors | 10 | 4236 | 26 | ADAS-Cog | 2.38Footnote *** | 0.41Footnote *** | ||||
| 9 | 3118 | 28 | MMSE | 1.33Footnote *** | 0.39Footnote *** | ||||||
| 8 | 3402 | 26 | Not worse | 16.8 | 24.4 | 7Footnote *** | 43Footnote *** | 0.26 | |||
| Attention-deficit hyperactivity disorder | |||||||||||
| Schachter et al 77/9 | Methylphenidate | 22 | 963 | 3.3 | HyperactivityFootnote f | NI | 0.78Footnote *** | ||||
AD, antidepressant; ADAS-Cog, Alzheimer's Disease Assessment Scale Cognitive Subscale; ARD, absolute response or risk difference; BPRS, Brief Psychiatric Rating Scale; BZD, benzodiazepine; HRSD, Hamilton Rating Scale for Depression; MD, mean difference; MMSE, Mini-Mental State Examination; MRS, Mania Rating Scale; MS, Mania Scale; NI, not indicated; NNT, number needed to treat; PANSS, Positive and Negative Syndrome Scale; RRR/RR, response ratio/relative risk reduction; SMD, standardised mean difference; SGA, second-generation antipsychotic; SSRI, selective serotonin reuptake inhibitor; TCA, tricyclic antidepressant; Y–BOCS, Yale–Brown Obsessive Compulsive Scale; YMRS, Young Mania Rating Scale.
a. AMSTAR quality score (range of possible values 0–11).
b. Italics indicate mean estimated values.
c. YMRS only.
d. In studies on SGAs.
e. Updated and supplemented by our own searches.
f. Teacher rated.
* P<0.05
** P<0.01
*** P<0.001. Results on absolute and relative risk/responder differences do not always exactly match with the formulae presented in the manuscript due to weighting processes in meta-analyses.
Outcomes
Psychiatry is often criticised for using rating scales which are subjective and considered ‘soft’ outcomes, whereas many medical treatments prevent ‘hard’ outcomes such as death or major events (stroke, heart attack, etc.). High blood pressure or cholesterol levels per se do not lead to suffering, therefore they should not be the primary outcome, rather their long-term consequences. Sometimes an intermediate outcome is improved but mortality increases; for example, in a large multicentre effectiveness trial for asthma (n = 26 000), long-acting beta-2-agonists increased respiratory-related deaths. Reference Nelson, Weiss, Bleecker, Yancey and Dorinsky80 In diabetes, aggressive glycaemic control reduced glucose levels compared with standard care, but increased mortality rates (n = 10 251). 81
Other drugs reduce the symptoms and suffering originating directly from the disease such as oesophagitis or migraine, but their pathophysiological disease processes do not progress to death. Psychiatric drugs fall in this category. Therefore, reduction of disease severity (e.g. degree of delusions and hallucinations in schizophrenia) and prevention of future episodes are primary outcomes, and it is not entirely appropriate to criticise psychiatry for using ‘soft’ outcomes. This said, there is considerable room for improvement in psychiatric outcome measures, Reference Leucht, Davis, Engel, Kane and Wagenpfeil82 and death or suicide should be always reported. The example of lithium shows that some psychiatric drugs may reduce suicide rates. Reference Baldessarini, Tondo and Hennen83,Reference Cipriani, Pretty, Hawton and Geddes84
Placebo effects
Readers may be surprised that many effect sizes in both areas were not larger. The median of all effect sizes was 0.40, similar to that found in another analysis of Cochrane reviews (0.32). Reference Furukawa, Watanabe, Omori, Montori and Guyatt85 In this context there is a general misconception that with placebo all patients will have a poor outcome, but many patients will recover spontaneously owing to the natural course of the disorder (for example, a manic episode will remit by itself) and placebo effects.
Effect sizes for dichotomous and continuous outcomes
For dichotomous outcomes both relative and absolute risk reductions should be considered. There is substantial evidence showing that clinicians tend to overestimate treatment effects presented as relative risk reductions. Reference Covey86 For example, statins reduced cardiovascular events from approximately 18% to approximately 14%. Reference Baigent, Keech, Kearney, Blackwell, Buck and Pollicino23 The relative risk reduction of 22% ((1—(0.14/0.18))×100) is more impressive than the absolute risk difference of 4% (14% – 18% = |–4%|). On the other hand, if the risk in the placebo group is low, the maximally possible absolute risk reduction must be lower than the base rate (here 18%), making the relative risk reduction more important.
In continuous outcomes the standardised mean difference (Cohen's d, Hedges’ g, etc.) is necessary when different instruments are used to measure the same concept (e.g. two depression scales) or if the original unit is difficult to interpret intuitively (e.g. the score of an unknown rating scale). As the SMD is relative to the pooled standard deviation, large variability will reduce it. In psychiatry this often occurs with rating scales in somewhat ill-defined, ‘variable’ diseases such as depression, whereas in general medicine the measure may be a highly accurate laboratory test (e.g. serum cholesterol concentration) in a well-defined disease entity. Cohen's rule that an SMD of 0.2 is a small effect size, 0.5 medium and 0.8 a large effect size is often used, but Cohen hastened to say that the interpretation depends on the context; Reference Cohen87 a small SMD for a fatal disease is more important than a large SMD for a transitory rash. In the future, quality-adjusted life years (QALYs) could be a uniform measure for comparisons across treatments, but these are not yet available for all drugs and we did not find this outcome in the meta-analyses. In addition, there is much debate about the validity of QALYs (see, for example, studies by Schlander Reference Schlander88 and Griebsch et al Reference Griebsch, Coast and Brown89 ).
Sample size
Meta-analyses in somatic medicine sometimes include impressively large patient numbers, e.g. 95 000 participants in studies of the primary prevention of cardiovascular events with aspirin. Reference Baigent, Blackwell, Collins, Emberson, Godwin and Peto21 Aspirin reduced the risk of a cardiovascular event from 0.57% per year to 0.51% per year. Angiotensin-converting enzyme inhibitors for hypertension reduced 5-year mortality from 10.4% to 9.2% in 18 229 participants. 16 In such situations, large sample sizes are needed for two reasons: first, the aspirin v. placebo difference was 0.07% event and the ACE inhibitors v. placebo difference was 1.2% events, requiring large sample sizes for statistical significance; second, the base rate (equivalent to the risk in the placebo group) was very low (e.g. 0.57% per year without aspirin), limiting the drug effect to a maximum 0.57% per year. Nevertheless, for mortality even a small difference can be clinically meaningful. In psychiatry the difference in percentages of those responding to drug or placebo is usually higher and it has been shown that here meta-analyses with at least 1000 participants are robust. Reference Trikalinos, Churchill, Ferri, Leucht, Tuunainen and Wahlbeck90
Drug effects could accumulate over time
The mean duration of the studies included in a meta-analysis should always be considered. For example, treated or not, few patients with hypertension will die in the course of a year. Thus, to obtain a large difference in mortality, studies of many years’ duration would be necessary, but such studies are almost impossible to conduct for many reasons. Therefore, shorter studies are performed which show only small differences. Although only very long-term studies could prove this, it is likely that the reduction of mortality accumulates over time. In this context, many psychiatric drugs not only improve the acute episode but also prevent further episodes. Patients with severe recurrent depression might have 20 episodes in their lifetime, which could be reduced by medication to 10. Reference Geddes, Carney, Davies, Furukawa, Kupfer and Frank72
Has drug efficacy decreased over the decades?
To be systematic we generally chose the most up-to-date systematic reviews, but there is an impression that earlier meta-analyses in psychiatry yielded higher effect sizes (see online Table DS1 for some examples). In the first 103 double-blind studies in depression, summarised in 1993, approximately two-thirds responded to tricyclic antidepressants or monoamine esterase inhibitors compared with a third responding to placebo. Reference Davis, Wang and Janicak91 The large National Institute of Mental Health schizophrenia trial, published in 1964, reported that 69% responded to antipsychotics and 24% to placebo (NNT 2, effect size 1.31). Reference Cole92 In the first large obsessive–compulsive disorder trial, published in 1991, half the sample responded to clomipramine and only 5% to placebo. 93 Recent meta-analyses found much smaller effect sizes for both the new SSRIs and clomipramine. Reference Ackerman and Greenland94 The reasons for decreasing effect sizes are not entirely understood. The early trials were often small and single-centre, and methodology less well developed (blinding, scales, external auditing, statistical methods). There may also have been more publication bias, as efforts to control it have expanded only in the past two decades. Modern trials are often large, multicentre studies but have other problems such as the impossibility of recruiting severely ill patients with truly acute disorders because of ethical concerns, the availability of effective medication leaving few drug-naive patients, and the phenomenon of symptomatic volunteers answering an advertisement for free medication and thereby increasing placebo response. Reference Walsh, Seidman, Sysko and Gould95 It is possible that there are similar temporal trends in general medicine and the phenomenon needs thorough examination.
Limitations
We made a considerable effort to be systematic, but for the reasons stated below we could not meet all criteria of a systematic review. We did not examine a single drug but put different medications in perspective, for which an established methodology does not exist.
First, we could not present a complete collection, but we chose common diseases by consensus based on frequency, importance and available treatment. It would be difficult to operationalise the selection. For example, there are diseases that are frequent but not severe (an extreme example is the common cold). Others are extremely severe but rare (e.g. certain cancers). The selection was made a priori, and once chosen all diseases and drugs were presented. We feel that the selection is representative and that the major diseases of the industrialised world are included; nevertheless, the selection process may have introduced bias.
Second, in the selection among reviews, we emphasised up-to-dateness and full presentation, but we compared the results of different meta-analyses on the same topic which were usually consistent. Third, a review of reviews is observational by nature: our unit of analysis was published meta-analyses, which does not exist for all drugs/indications, and the included reports differed in the exact methods, publication dates, inclusion criteria, etc. Fourth, many meta-analyses did not present the data in a consistent manner, resulting in a major challenge for us. We made substantial efforts to present the results in a consistent way by back-calculating indices, but stringent following of the PRISMA statement would facilitate future attempts. Reference Liberati, Altman, Tetzlaff, Mulrow, Gotzsche and Ioannidis13
Fifth, we did not address side-effects. These are a serious problem of many psychotropic drugs, although improvements have been made. For example, SSRIs have much less serious toxicity than tricyclic antidepressants. General medicine drugs also have important side-effects, for example death induced by bleeding from thrombolysis or aspirin or cancer chemotherapies. It would have been simply impossible to describe side-effects as well and to balance them with efficacy, because there are many subjective judgement calls. Finally, publication bias is a major problem for meta-analyses. For example, Turner et al (see Table DS4) showed that the inclusion of unpublished antidepressant trials reduced the effect size. Reference Turner, Matthews, Linardatos, Tell and Rosenthal96 Publication bias exists in general medicine as well (see, for example, Rising et al), Reference Rising, Bacchetti and Bero97 and we are not aware of evidence comparing its degree in different fields.
There are many reasons why doctors, patients and caregivers are and should be critical about psychotropic drug treatment, such as unclear disease aetiology, lack of diagnostic tests, commercial conflict of interest, unclear mechanism of drug action and side-effects. Moreover, some people think that psychiatric disorders are purely psychological conditions that should be treated exclusively with psychotherapy. However, the efficacy of psychotropic drugs is supported by randomised controlled trials. In this context we have put psychiatric drugs in the perspective of general medicine medication.
Acknowledgements
We thank Drs Malcom Law, Toshi Furukawa, Corrado Barbui and Shelley Salpeter for replying to our requests about their studies.
eLetters
No eLetters have been published for this article.