Hostname: page-component-745bb68f8f-f46jp Total loading time: 0 Render date: 2025-01-24T05:01:18.921Z Has data issue: false hasContentIssue false
  • English
  • Français

Assessing the Quality of Randomized Controlled Trials: Current Issues and Future Directions

Published online by Cambridge University Press:  10 March 2009

David Moher ,
Alejandro R. Jadad  and
Peter Tugwell
David Moher
Affiliation:
University of Ottawa
Alejandro R. Jadad
Affiliation:
McMaster University
Peter Tugwell
Affiliation:
University of Ottawa
Get access Rights & Permissions [Opens in a new window]

Abstract

Assessing the quality of randomized controlled trials is a relatively new and important development. Three approaches have been developed: component, checklist, and scale assessment. Component approaches evaluate selected aspects of trials, such as masking. Checklists and scales involve lists of items thought to be integral to study quality. Scales, unlike the other methods, provide a summary numeric score of quality, which can be formally incorporated into a systematic review. Most scales to date have not been developed with sufficient rigor, however. Empirical evidence indicates that differences in scale development can lead to important differences in quality assessment. Several methods for including quality scores in systematic reviews have been proposed, but since little empirical evidence supports any given method, results must be interpreted cautiously. Future efforts may be best focused on gathering more empirical evidence to identify trial characteristics directly related to bias in the estimates of intervention effects and on improving the way in which trials are reported.

Type
Special Section: The Quality of the Medical Evidence: Is It Good Enough?
Information
International Journal of Technology Assessment in Health Care , Volume 12 , Issue 2 , Spring 1996 , pp. 195 - 208
Copyright
Copyright © Cambridge University Press 1996

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

1.Ad Hoc Working Group for Critical Appraisal of the Medical Literature. A proposal for more informative abstracts of clinical articles. Annals of Internal Medicine, 1987, 106, 598–604.CrossRefGoogle Scholar
2.Andrew, E.Method for assessment of the reporting standard of clinical trials with Roentgen contrast media. Acta Radiologica Diagnosis, 1984, 25, 55–58.CrossRefGoogle ScholarPubMed
3.Andrew, E., Eide, H., Fuglerud, P., et al. Publications on clinical trials with x-ray contrast media: Differences in quality between journals and decades. European Journal of Radiology, 1990, 10, 92–97.CrossRefGoogle ScholarPubMed
4.Badgley, R. F.An assessment of research methods reported in 103 scientific articles from two Canadian medical journals. Canadian Medical Association Journal, 1961, 85, 246–50.Google ScholarPubMed
5.Beckerman, H., de Bie, R. A., Bouter, L. M., et al. The efficacy of laser therapy for musculoskeletal and skin disorders. In Beckerman, H. & Bouter, L. (eds.), Effectiviteit van fysiotherapie: Een literatuuronderzoek. Maastricht: Rijksuniversiteit Limburg, 1990, xx–xx.Google Scholar
6.Bland, J. M., Jones, D. R., Bennett, S., et al. Is the clinical trial evidence about new drugs statistically adequate? British Journal of Clinical Pharmacology, 1985, 19, 155–60.CrossRefGoogle ScholarPubMed
7.Brown, S. A.Measurement of quality of primary studies for meta-analysis. Nursing Research, 1991, 40, 352–55.CrossRefGoogle ScholarPubMed
8.Brown, A. S.Meta-analysis of diabetes patient education research: Variations in intervention effects across studies. Research in Nursing & Health, 1992, 15, 409–19.CrossRefGoogle ScholarPubMed
9.Chalmers, I.Preparing, maintaining, and disseminating systematic review of the effects of health care. Annals of the New York Academy of Sciences, 1993, 703, 156–65.CrossRefGoogle Scholar
10.Chalmers, I., Adams, M., Dickersin, K., et al. A cohort study of summary reports of controlled trials. Journal of the American Medical Association, 1990, 263, 1401–05.CrossRefGoogle ScholarPubMed
11.Chalmers, I., Hetherington, J., Elbourne, D., et al. Materials and methods used in synthesizing evidence to evaluate the effects of care during pregnancy and childbirth. In Chalmers, I., Enkin, M., & Keirse, M. J.n.c. (eds.), Effective care in pregnancy and childbirth. Oxford: Oxford University Press, 1989, xx–xx.Google Scholar
12.Chalmers, T. C., Adams, M., Dickersin, K., et al. A method for assessing the quality of randomized control trial. Journal of the American Medical Association, 1981, 2, 31–49.Google ScholarPubMed
13.Chalmers, T. C., Celano, P., Sacks, H. S., & Smith, H.Bias in treatment assignment in controlled clinical trials. New England Journal of Medicine, 1983, 309, 1358–61.CrossRefGoogle ScholarPubMed
14.Chalmers, T. C., & Lau, J.Meta-analytic stimulus for changes in clinical trails. Statistical Methods in Medical Research, 1993, 2, 161–72.Google Scholar
15.Cho, M. K., & Bero, L. A.Instruments for reassessing the quality of drug studies published in the medical literature. Journal of the American Medical Association, 1994, 272, 101–04.CrossRefGoogle Scholar
16.Colditz, G. A., Miller, J. N., & Mosteller, F.How study design affects outcomes in comparison of therapy. Medical Statistics in Medicine, 1989, 8, 441–54.CrossRefGoogle ScholarPubMed
17.DerSimonian, R., Charette, L. J., McPeek, B., & Mosteller, F.Reporting on methods in clinical trials. New England Journal of Medicine, 1982, 306, 1332–37.CrossRefGoogle ScholarPubMed
18.Detsky, A. S., Naylor, C. D., O’Rourke, K., et al. Incorporating variations in the quality of individual randomized trials into meta-analysis. Journal of Clinical Epidemiology, 1992, 45, 225–65.CrossRefGoogle ScholarPubMed
19.Evans, M., & Pollack, A. V.A score system for evaluating random control clinical trials of prophylaxis of abdominal surgical wound infection. British Journal of Surgery, 1985, 72, 256–60.CrossRefGoogle ScholarPubMed
20.Feinstein, A. R.Clinicmetrics. New Haven, CT: Yale University Press, 1987.CrossRefGoogle Scholar
21.Fleiss, J. L., & Gross, A. J.Meta-analysis in epidemiology, with special reference to studies of the association between exposure to environmental tobacco smoke and lung cancer: A critique. Journal of Clinical Epidemiology, 1991, 44, 127–39.CrossRefGoogle Scholar
22.Gardner, M. J., Machin, D., & Campbell, M. J. Use of checklists in assessing the statistical content of medical studies. In Statistics with confidence: Confidence intervals and statistical guidelines. London: BMJ Publications, 1989, 101–08.Google Scholar
23.Goodman, S. N., Berlin, J., Fletcher, R. H., & Fletcher, S. W.Manuscript quality before and after peer review and editing. Annals of Internal Medicine, 1994, 121, 11–21.CrossRefGoogle ScholarPubMed
24.Gøtzsche, P.Methodology and overt and hidden bias: Reports of 196 double-blind trials of nonsteroidal antiinflammatory drugs in rheumatoid arthritis. Controlled Clinical Trials, 1989, 10, 31–56 (erratum:356).CrossRefGoogle ScholarPubMed
25.Grant, A.Reporting controlled trials. British Journal of Obstetrics and Gynaecology, 1989, 96, 397–400.CrossRefGoogle ScholarPubMed
26.Grégoire, G., Derderian, F., & Le Lorier, J.Selecting the language of the publications included in a meta-analysis: Is there a tower of babel bias? Journal of Clinical Epidemiology, 1995, 48, 158–63.CrossRefGoogle Scholar
27.Haynes, R. B., Mulrow, C. D., Huth, E. J., et al. More informative abstracts revisited. Annals of Internal Medicine, 1990, 113 69–76.CrossRefGoogle ScholarPubMed
28.Imperiale, T. F., & McCulloughm, A. J.Do corticosteroids reduce mortality from alcoholic hepatitis? A meta-analysis of the randomised trials. Annals of Internal Medicine, 1990, 113, 299–307.CrossRefGoogle Scholar
29.Jadad-Bechara, A. R., Moore, R. A., & Carrol, D.Assessing the quality of reports of randomized clinical trials: Is blinding necessary? Controlled Clinical Trials, 1996, 17, 1–12.CrossRefGoogle Scholar
30.Jenicek, M.Meta-analysis in medicine: Where we are and where we want to go. Journal of Clinical Epidemiology, 1989, 42, 35–44.CrossRefGoogle Scholar
31.Kleijnen, J., Knipschild, P., & ter Riet, G.Clinical trials of homeopathy. British Medical Journal, 1991, 302, 316–23.CrossRefGoogle ScholarPubMed
32.Koes, B. W., Assendelft, W. J. J., van der Heijden, G. J. M. G., et al. Spinal manipulation and mobilisation for back and neck pain: A blinded review. British Medical Journal, 1991, 303, 1298–303.CrossRefGoogle ScholarPubMed
33.Levine, J.Trial assessment procedure scale (TAPS). Bethesda, MD: Department of Health and Human Services, National Institute of Mental Health, 1980.Google Scholar
34.Lionel, N. D. W., & Herxheimer, A.Assessing reports of therapeutic trials. British Medical Journal, 1970, 3, 637–40.CrossRefGoogle ScholarPubMed
35.Mahon, W. A., & Daniel, E. E.A method for the assessment of reports of drug trials. Canadian Medical Association Journal, 1964, 90, 565–69.Google ScholarPubMed
36.McDowell, I., & Newell, C.Measuring health: A guide to rating scales and questionnaires. New York: Oxford University Press, 1987.Google Scholar
37.Meinert, C. L.Clinical trials, design, conduct, and analysis. New York: Oxford University Press, 1986.CrossRefGoogle Scholar
38.Moher, D., Jadad, A. R., Nichol, G., et al. Assessing the quality of randomized controlled trials: An annotated bibliography of scales and checklists. Controlled Clinical Trials, in press.Google Scholar
39.Nurmohame, M. T., Rosendaal, F. R., Buller, H. R., et al. Low-molecular-weight heparin versus standard heparin in general and orthopaedic surgery: A meta-analysis. Lancet, 1992, 340, 152–56.CrossRefGoogle Scholar
40.Onghenia, P., & Van Houdenhove, B.Antidepressants-induced analgesia in chronic nonmalignant pain: A meta-analysis of 39 placebo-controlled studies. Pain, 1992, 49, 205–19.CrossRefGoogle Scholar
41.Pocock, S. J.Clinical trials: A practical approach. Chichester, UK: John Wiley & Sons, 1983.Google Scholar
42.Pocock, S. J., Hughes, M. D., & Lee, R. J.Statistical problems in the reporting of clinical trials: A survey of three medical journals. New England Journal of Medicine, 1987, 317, 426–32.CrossRefGoogle ScholarPubMed
43.Powe, N. R., Kinnison, M. L., & Steinberg, E. P.Quality assessment of randomized controlled trials of contrast media. Radiology, 1989, 170, 377–80.CrossRefGoogle ScholarPubMed
44.Poynard, T.Evaluation de la qualité méthodologique des essais therapeutiques randomisés. La Presse Medicate, 1988, 17, 315–18.Google Scholar
45.Reisch, J. S., Tyson, J. E., & Mize, S. G.Aid to the evaluation of therapeutic studies. Pediatrics, 1989, 84, 815–27.CrossRefGoogle Scholar
46.Sandercock, P. A. G., van den Belt, A. G. M., Lindley, R. I., & Slattery, J.Antithrombotic therapy in acute ischaemic stroke: An overview of the completed randomized trials. Journal of Neurology, Neurosurgery, and Psychiatry, 1993, 56, 17–25.CrossRefGoogle Scholar
47.Schulz, K. F., Chalmers, I., Hayes, R. J., & Altman, D. G.Failure to conceal intervention allocation schedules in trials influenced estimates of treatment effects. Controlled Clinical Trials, 1994, 15, 63S.CrossRefGoogle Scholar
48.Smith, K., Cook, D., Guyatt, G. H., et al. Respiratory muscle training in chronic airflow limitation: A meta-analysis. American Review of Respiratory Disease, 1992, 145, 533–39.CrossRefGoogle ScholarPubMed
49.Spitzer, W. O., Lawrence, V., Dales, R., et al. Links between passive smoking and disease: A best evidence synthesis. Clinical and Investigative Medicine, 1990, 13, 17–42.Google ScholarPubMed
50.Standards of Reporting Trials Group. A proposal for structured reporting of randomized controlled trials. Journal of the American Medical Association, 1994, 272, 1926–31.CrossRefGoogle Scholar
51.Steiner, D. L., & Norman, G. R.Health measurement scales: A practical guide to their development and use. Oxford: Oxford University Press, 1989.Google Scholar
52.Streptomycin in Tuberculosis Trials Committee. Streptomycin treatment of pulmonary tuberculosis: A Medical Research Council investigation. British Medical Journal, 1948, 11, 769–82.Google Scholar
53.Ter Riet, G., Kleijnen, J., & Knipschild, P.Acupuncture and chronic pain: A criteria-based meta-analysis. Journal of Clinical Epidemiology, 1990, 43, 1191–99.CrossRefGoogle ScholarPubMed
54.Thomson, M. E., & Kramer, M. S.Methodologic standards for controlled clinical trials of early contact and maternal-infant behavior. Pediatrics, 1984, 73, 294–200.CrossRefGoogle ScholarPubMed
55.Weintraub, M.How to critically assess clinical drug trials. Drug Therapy, 1982, 12, 131–48.Google Scholar