Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-745bb68f8f-mzp66 Total loading time: 0 Render date: 2025-01-13T02:22:52.826Z Has data issue: false hasContentIssue false

27 - Meta-Analysis

from Part IV - Statistical Approaches

Published online by Cambridge University Press:  25 May 2023

By
Yuri Jadotte ,
Anne Moyer  and
Jessica Gurevitch
Edited by
Austin Lee Nichols  and
John Edlund
Austin Lee Nichols
Affiliation:
Central European University, Vienna
John Edlund
Affiliation:
Rochester Institute of Technology, New York
Get access

Summary

Meta-analysis is a form of data synthesis that statistically combines the results of primary research studies responding to a given question. It has become an indispensable tool for decision making and advancement of knowledge in a variety of disciplines. This chapter provides an overview of this method, beginning with a brief discussion of systematic reviews – the research methodology that undergirds meta-analysis. The chapter then explores specific components of this approach as it is most widely applied in the literature, including issues related to effect sizes, heterogeneity of study outcomes, scope of the analysis, and quality-control issues to consider when conducting a meta-analysis. A brief overview of new and emerging methods for the synthesis of primary research data is also provided, highlighting different forms of meta-analysis and different approaches for the synthesis of research data. Practical examples are provided as illustrations to clarify and reinforce the concepts presented in this chapter.

Keywords

Evidence SynthesisSystematic ReviewScientific GeneralityEvidence-Based DecisionsScoping ReviewResearch SynthesisEffect SizeVote Counting Procedure
Type
Chapter
Information
The Cambridge Handbook of Research Methods and Statistics for the Social and Behavioral Sciences
Volume 1: Building a Program of Research
 , pp. 583 - 605
Publisher: Cambridge University Press
Print publication year: 2023

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Adams, D. C., Gurevitch, J., & Rosenberg, M. S. (1997). Resampling tests for meta-analysis of ecological data. Ecology, 78(4), 1277–1283. https://doi.org/10.1890/0012-9658(1997)078[1277:RTFMAO]2.0.CO;2Google Scholar
Boedeker, P. & Henson, R. K. (2020). Evaluation of heterogeneity and heterogeneity interval estimators in random-effects meta-analysis of the standardized mean difference in education and psychology. Psychological Methods, 25(3), 346–364. https://doi.org/10.1037/met0000241CrossRefGoogle ScholarPubMed
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to Meta-Analysis. John Wiley & Sons.Google Scholar
Campbell, J. M., Klugar, M., Ding, S., Carmody, D. P., Hakonsen, S. J., Jadotte, Y. T., White, S., & Munn, Z. (2015). Diagnostic test accuracy: methods for systematic review and meta-analysis. International Journal of Evidence-Based Healthcare, 13(3), 154–162. https://doi.org/10.1097/xeb.0000000000000061 CrossRefGoogle Scholar
Card, N. A. (2016). Applied Meta-Analysis for Social Science Research (paperback edition). Guilford Press.Google Scholar
Cassey, P., Ewen, J. G., Blackburn, T. M., & Møller, A. P. (2004). A survey of publication bias within evolutionary ecology. Proceedings of the Royal Society of London. Series B: Biological Sciences, 271 (suppl 6), S451–S454.CrossRefGoogle ScholarPubMed
Cinar, O., Umbanhowar, J., Hoeksema, J. D., & Viechtbauer, W. (2021). Using information‐theoretic approaches for model selection in meta‐analysis. Research Synthesis Methods, 12(4), 537–556. https://doi.org/10.1002/jrsm.1489Google Scholar
Dickersin, K. (1990). The existence of publication bias and risk factors for its occurrence. JAMA, 263(10), 1385–1389.CrossRefGoogle ScholarPubMed
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. BMJ, 315(7109), 629–634.Google Scholar
Farooq, F., Mogayzel, P. J., Lanzkron, S., Haywood, C., & Strouse, J. J. (2020). Comparison of US federal and foundation funding of research for sickle cell disease and cystic fibrosis and factors associated with research productivity. JAMA Network Open, 3(3), e201737. https://doi.org/10.1001/jamanetworkopen.2020.1737Google Scholar
Forstmeier, W., Wagenmakers, E., & Parker, T. H. (2017). Detecting and avoiding likely false‐positive findings–a practical guide. Biological Reviews, 92(4), 1941–1968.CrossRefGoogle ScholarPubMed
Gurevitch, J. & Hedges, L. V. (1999). Statistical issues in ecological meta-analyses. Ecology, 80(4), 1142–1149. https://doi.org/10.1890/0012-9658(1999)080[1142:SIIEMA]2.0.CO;2Google Scholar
Gurevitch, J., Koricheva, J., Nakagawa, S., & Stewart, G. (2018). Meta-analysis and the science of research synthesis. Nature, 555(7695), 175–182. https://doi.org/10.1038/nature25753Google Scholar
Hamberg, K. (2008). Gender bias in medicine. Women’s Health, 4(3), 237–243.Google Scholar
Hedges, L. V. & Olkin, I. (1985). Statistical Methods for Meta-Analysis. Elsevier Science. http://qut.eblib.com.au/patron/FullRecord.aspx?p=1901162Google Scholar
Hedges, L. V. & Vevea, J. L. (1998). Fixed- and random-effects models in meta-analysis. Psychological Methods, 3(4), 486–504.Google Scholar
Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world? Behavioral and Brain Sciences, 33(2–3), 61–83. https://doi.org/10.1017/S0140525X0999152XGoogle Scholar
Higgins, J. P. T. & Thompson, S. G. (2002). Quantifying heterogeneity in a meta‐analysis. Statistics in Medicine, 21(11), 1539–1558.CrossRefGoogle ScholarPubMed
Hobbs, N. T. & Hilborn, R. (2006). Alternatives to statistical hypothesis testing in ecology: A guide to self teaching. Ecological Applications, 16(1), 5–19. https://doi.org/10.1890/04-0645CrossRefGoogle ScholarPubMed
Ioannidis, J. P. A. (2018). The proposal to lower p value thresholds to .005. JAMA, 319(14), 1429–1430. https://doi.org/10.1001/jama.2018.1536Google Scholar
Kambach, S., Bruelheide, H., Gerstner, K., et al. (2020). Consequences of multiple imputation of missing standard deviations and sample sizes in meta-analysis. Ecology and Evolution, 10(20), 11699–11712. https://doi.org/10.1002/ece3.6806Google Scholar
Kinlock, N. L. (2019). A meta-analysis of plant interaction networks reveals competitive hierarchies as well as facilitation and intransitivity. American Naturalist, 194(5), 640–653. https://doi.org/10.1086/705293Google Scholar
Lajeunesse, M. J. (2016). Facilitating systematic reviews, data extraction and meta-analysis with the metagear package for R. Methods in Ecology and Evolution, 7(3), 323–330. https://doi.org/10.1111/2041-210X.12472Google Scholar
Lane, A., Luminet, O., Nave, G., & Mikolajczak, M. (2016). Is there a publication bias in behavioural intranasal oxytocin research on humans? Opening the file drawer of one laboratory. Journal of Neuroendocrinology, 28(4). https://doi.org/10.1111/jne.12384.CrossRefGoogle Scholar
Lockwood, C., Munn, Z., & Porritt, K. (2015). Qualitative research synthesis: Methodological guidance for systematic reviewers utilizing meta-aggregation. International Journal of Evidence-Based Healthcare, 13(3), 179–187.CrossRefGoogle ScholarPubMed
Lowry, E., Rollinson, E. J., Laybourn, A. J., et al. (2013). Biological invasions: A field synopsis, systematic review, and database of the literature. Ecology and Evolution, 3(1), 182–196. https://doi.org/10.1002/ece3.431Google Scholar
Marshall, I. J. & Wallace, B. C. (2019). Toward systematic review automation: A practical guide to using machine learning tools in research synthesis. Systematic Reviews, 8(1), 1–10.Google Scholar
Munn, Z., Aromataris, E., Tufanaru, C., et al. (2019). The development of software to support multiple systematic review types: The Joanna Briggs Institute System for the Unified Management, Assessment and Review of Information (JBI SUMARI). International Journal of Evidence-Based Healthcare, 17(1), 36–43.Google Scholar
Nosek, B. A., Alter, G., Banks, G. C., et al. (2015). Promoting an open research culture. Science, 348(6242), 1422–1425. https://doi.org/10.1126/science.aab2374CrossRefGoogle ScholarPubMed
O’Dea, R. E., Lagisz, M., Jennions, M. D., et al. (2021). Preferred reporting items for systematic reviews and meta-analyses in ecology and evolutionary biology: A PRISMA extension. Biological Reviews, 96(5), 1695–1722. https://doi.org/10.1111/brv.12721CrossRefGoogle ScholarPubMed
Orwin, R. G. (1983). A fail-safe N for effect size in meta-analysis. Journal of Educational Statistics, 8(2), 157–159.Google Scholar
Peters, M. D., Godfrey, C. M., Khalil, H., et al. (2015). Guidance for conducting systematic scoping reviews. International Journal of Evidence-Based Healthcare, 13(3), 141–146. https://doi.org/10.1097/xeb.0000000000000050 Google Scholar
Rosenberg, M. S. (2005). The file‐drawer problem revisited: A general weighted method for calculating fail‐safe numbers in meta‐analysis. Evolution, 59(2), 464–468.Google Scholar
Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638.Google Scholar
Salgado, J. F. & Moscoso, S. (2019). Meta-analysis of the validity of general mental ability for five performance criteria: Hunter and Hunter (1984) revisited. Frontiers in Psychology, October 17. https://doi.org/10.3389/fpsyg.2019.02227Google Scholar
Schmid, C., Stewart, G., Rothstein, H., & Lajeunesse, M. (2013). Software for statistical meta-analysis. In Gurevitch, J. & Mengersen, K. (eds.), Handbook of Meta-Analysis in Ecology and Evolution (pp. 174–192). Princeton University Press.Google Scholar
Snilstveit, B., Vojtkova, M., Bhavsar, A., Stevenson, J., & Gaarder, M. (2016). Evidence & gap maps: A tool for promoting evidence informed policy and strategic research agendas. Journal of Clinical Epidemiology, 79, 120–129.Google Scholar

Save book to Kindle

To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×