Hostname: page-component-78c5997874-mlc7c Total loading time: 0 Render date: 2024-11-15T11:15:42.911Z Has data issue: false hasContentIssue false
  • English
  • Français

CLINICAL HETEROGENEITY IN SYSTEMATIC REVIEWS AND HEALTH TECHNOLOGY ASSESSMENTS: SYNTHESIS OF GUIDANCE DOCUMENTS AND THE LITERATURE

Published online by Cambridge University Press:  23 January 2012

Gerald Gartlehner ,
Suzanne L. West ,
Alyssa J. Mansfield ,
Charles Poole ,
Elizabeth Tant ,
Linda J. Lux  and
Kathleen N. Lohr
Gerald Gartlehner
Affiliation:
Danube University, Krems; RTI [email protected]
Suzanne L. West
Affiliation:
RTI International
Alyssa J. Mansfield
Affiliation:
RTI International
Charles Poole
Affiliation:
University of North Carolina at Chapel Hill
Elizabeth Tant
Affiliation:
RTI International
Linda J. Lux
Affiliation:
RTI International
Kathleen N. Lohr
Affiliation:
RTI International
Get access Rights & Permissions [Opens in a new window]

Abstract

Objectives: The aim of this study was to synthesize best practices for addressing clinical heterogeneity in systematic reviews and health technology assessments (HTAs).

Methods: We abstracted information from guidance documents and methods manuals made available by international organizations that develop systematic reviews and HTAs. We searched PubMed® to identify studies on clinical heterogeneity and subgroup analysis. Two authors independently abstracted and assessed relevant information.

Results: Methods manuals offer various definitions of clinical heterogeneity. In essence, clinical heterogeneity is considered variability in study population characteristics, interventions, and outcomes across studies. It can lead to effect-measure modification or statistical heterogeneity, which is defined as variability in estimated treatment effects beyond what would be expected by random error alone. Clinical and statistical heterogeneity are closely intertwined but they do not have a one-to-one relationship. The presence of statistical heterogeneity does not necessarily indicate that clinical heterogeneity is the causal factor. Methodological heterogeneity, biases, and random error can also cause statistical heterogeneity, alone or in combination with clinical heterogeneity.

Conclusions: Identifying potential modifiers of treatment effects (i.e., effect-measure modifiers) is important for researchers conducting systematic reviews and HTAs. Recognizing clinical heterogeneity and clarifying its implications helps decision makers to identify patients and patient populations who benefit the most, who benefit the least, and who are at greatest risk of experiencing adverse outcomes from a particular intervention.

Keywords

Clinical heterogeneityComparative effectivenessEffect measure modificationHealth technology assessmentSystematic review
Type
METHODS
Information
International Journal of Technology Assessment in Health Care , Volume 28 , Issue 1 , January 2012 , pp. 36 - 43
Copyright
Copyright © Cambridge University Press 2012

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

1.Agency for Healthcare Research and Quality. Methods Reference Guide for Effectiveness and Comparative Effectiveness Reviews. 2007. http://effectivehealthcare.ahrq.gov/repFiles/2007_10DraftMethodsGuide.pdf (accessed August 31, 2010).Google Scholar
2.Berlin, JA. Invited commentary: Benefits of heterogeneity in meta-analysis of data from epidemiologic studies. Am J Epidemiol. 1995;142:383–387.CrossRefGoogle ScholarPubMed
3.Berlin, JA, Santanna, J, Schmid, CH, Szczech, LA, Feldman, HI. Individual patient- versus group-level data meta-regressions for the investigation of treatment effect modifiers: Ecological bias rears its ugly head. Stat Med. 2002;21:371–387.CrossRefGoogle ScholarPubMed
4.Borenstein, M, Hedges, LV, Higgins, JPT, Rothstein, HR. Introduction to meta-analysis. New York: John Wiley and Sons, Ltd.; 2009.CrossRefGoogle Scholar
5.Centre for Reviews and Dissemination. Centre for Reviews and Disssemination's (CRD) systematic reviews: Guidance for undertaking reviews in health care. 2008. http://www.york.ac.uk/inst/crd/pdf/Systematic_Reviews.pdf (accessed May 24, 2010).Google Scholar
6.Cochran, W. The combination of estimates from different experiments. Biometrics. 1954;10:101–121.CrossRefGoogle Scholar
7.Colditz, GA, Burdick, E, Mosteller, F. Heterogeneity in meta-analysis of data from epidemiologic studies: A commentary. Am J Epidemiol. 1995;142:371–382.CrossRefGoogle ScholarPubMed
8.Dias, S, Welton, NJ, Caldwell, DM, Ades, AE. Checking consistency in mixed treatment comparison meta-analysis. Stat Med. 29:932–944.CrossRefGoogle Scholar
9.Drug Effectiveness Review Project. Review methods and report production procedures. 2008. http://www.ohsu.edu/xd/research/centers-institutes/evidence-based-policy-center/derp/documents/methods.cfm (accessed August 31, 2010).Google Scholar
10.European Network for Health Technology Assessment. HTA core model for medical and surgical interventions 1.0R. 2008. http://www.eunethta.net/upload/WP4/Final%20Deliverables/HTA%20Core%20Model%20for%20Medical%20and%20Surgical%20Interventions%201%200r.pdf (accessed August 30, 2010).Google Scholar
11.Gartlehner, G, Hansen, RA, Thieda, P, et al. Comparative effectiveness of second-generation antidepressants in the pharmacologic treatment of adult depression. Comparative Effectiveness Review No. 7. Rockville, MD: Agency for Healthcare Research and Quality. January 2007. http://www.effectivehealthcare.ahrq.gov/index.cfm/search-for-guides-reviews-and-reports/?pageaction=displayproduct&productid=61 (accessed November 16, 2011).Google Scholar
12.Greenland, S, O'Rourke, K. Meta-analysis. In: Rothman, KJ, Greenland, S, Lash, TL eds. Modern Epidemiology. Philadelphia, PA: Lippincott Williams & Wilkins; 2008:652–682.Google Scholar
13.Higgins, JP, Thompson, SG. Quantifying heterogeneity in a meta-analysis. Stat Med. 2002;21:1539–1558.CrossRefGoogle ScholarPubMed
14.Higgins, JP, Thompson, SG, Deeks, JJ, Altman, DG. Measuring inconsistency in meta-analyses. BMJ. 2003;327:557–560.CrossRefGoogle ScholarPubMed
15.Higgins, JPT, Green, S, eds. Cochrane Handbook for Systematic Reviews of Interventions 5.0.2 [updated September 2009]. 2009. www.cochrane-handbook.org.CrossRefGoogle Scholar
16.Ioannidis, JP, Patsopoulos, NA, Evangelou, E. Uncertainty in heterogeneity estimates in meta-analyses. BMJ. 2007;335:914–916.CrossRefGoogle ScholarPubMed
17.Kennedy, SH, Fulton, KA, Bagby, RM, et al. Sexual function during bupropion or paroxetine treatment of major depressive disorder. Can J Psychiatry. 2006;51:234–242.CrossRefGoogle ScholarPubMed
18.Kravitz, RL, Duan, N, Braslow, J. Evidence-based medicine, heterogeneity of treatment effects, and the trouble with averages. Milbank Q. 2004;82:661–687.CrossRefGoogle ScholarPubMed
19.Layton, D, Pearce, GL, Shakir, SA. Safety profile of tolterodine as used in general practice in England: Results of prescription-event monitoring. Drug Saf. 2001;24:703–713.CrossRefGoogle ScholarPubMed
20.Morton, SC, Adam, JL, Suttorp, MJ, Shekelle, PG. Meta-regression approaches: What, why, when, and how? Technical Review 8 prepared by Southern California-RAND-Evidence-based Practice Center, under Contract No 290-97-0001. AHRQ Publication No 04–033, Rockville, MD: Agency for Healthcare Research and Quality, 2004.Google Scholar
21.Petitti, DB. Approaches to heterogeneity in meta-analysis. Stat Med. 2001;20:3625–3633.CrossRefGoogle ScholarPubMed
22.Poole, C, Greenland, S. Random-effects meta-analyses are not always conservative. Am J Epidemiol. 1999;150:469–475.CrossRefGoogle Scholar
23.Shadish, WR, Cook, TD, Campbell, DT. Experimental and quasi-experimental designs for generalized causal inference. Florence, KY: Houghton Mifflin College; 2003.Google Scholar
24.Thompson, SG. Why sources of heterogeneity in meta-analysis should be investigated. BMJ. 1994;309:1351–1355.CrossRefGoogle ScholarPubMed
25.West, SL, Gartlehner, G, Mansfield, AJ et al. , Comparative effectiveness review methods: Clinical heterogeneity. Rockville, MD. Agency for Healthcare Research and Quality: RTI International-University of North Carolina Evidence-based Practice Center; 2010.Google Scholar
Supplementary material: File

Gartlehner Supplementary Table

Supplementary Table 1: Summary of common statistical approaches to test for and explore heterogeneity

Download Gartlehner Supplementary Table(File)
File 42.5 KB