Book contents
- The Cambridge Handbook of Research Methods in Clinical Psychology
- The Cambridge Handbook of Research Methods in Clinical Psychology
- Copyright page
- Contents
- Figures
- Tables
- Contributors
- Acknowledgments
- Part I Clinical Psychological Science
- Part II Observational Approaches
- Part III Experimental and Biological Approaches
- Part IV Developmental Psychopathology and Longitudinal Methods
- Part V Intervention Approaches
- Part VI Intensive Longitudinal Designs
- Part VII General Analytic Considerations
- 28 Reproducibility in Clinical Psychology
- 29 Meta-Analysis
- 30 Mediation, Moderation, and Conditional Process Analysis
- 31 Statistical Inference for Causal Effects in Clinical Psychology
- 32 Analyzing Nested Data
- 33 Missing Data Analyses
- 34 Machine Learning for Clinical Psychology and Clinical Neuroscience
- Index
- References
29 - Meta-Analysis
Integration of Empirical Findings through Quantitative Modeling
from Part VII - General Analytic Considerations
Published online by Cambridge University Press: 23 March 2020
Book contents
- The Cambridge Handbook of Research Methods in Clinical Psychology
- The Cambridge Handbook of Research Methods in Clinical Psychology
- Copyright page
- Contents
- Figures
- Tables
- Contributors
- Acknowledgments
- Part I Clinical Psychological Science
- Part II Observational Approaches
- Part III Experimental and Biological Approaches
- Part IV Developmental Psychopathology and Longitudinal Methods
- Part V Intervention Approaches
- Part VI Intensive Longitudinal Designs
- Part VII General Analytic Considerations
- 28 Reproducibility in Clinical Psychology
- 29 Meta-Analysis
- 30 Mediation, Moderation, and Conditional Process Analysis
- 31 Statistical Inference for Causal Effects in Clinical Psychology
- 32 Analyzing Nested Data
- 33 Missing Data Analyses
- 34 Machine Learning for Clinical Psychology and Clinical Neuroscience
- Index
- References
Summary
Meta-analysis is a well-established approach to integrating research findings, with a long history in the sciences and in psychology in particular. Its use in summarizing research findings has special significance given increasing concerns about scientific replicability, but it has other important uses as well, such as integrating information across studies to examine models that might otherwise be too difficult to study in a single sample. This chapter discusses different forms and purposes of meta-analyses, typical elements of meta-analyses, and basic statistical and analytic issues that arise, such as choice of meta-analytic model and different sources of variability and bias in estimates. The chapter closes with discussion of emerging issues in meta-analysis and directions for future research.
- Type
- Chapter
- Information
- Publisher: Cambridge University PressPrint publication year: 2020
References
Berkey, C. S., Hoaglin, D. C., Mosteller, F., & Colditz, G. A. (1995). A Random-Effects Regression Model for Meta-Analysis. Statistics in Medicine, 14(4), 395–411.Google Scholar
Bickel, P. J., & Blackwell, D. (1967). A Note on Bayes Estimates. Annals of Mathematical Statistics, 38, 1907–1911.Google Scholar
Blackwell, D., & Girshick, M. A. (1954). Theory of Games and Statistical Decisions. New York: John Wiley.Google Scholar
Bodnar, O., Link, A., Arendacká, B., Possolo, A., & Elster, C. (2017). Bayesian Estimation in Random Effects Meta-Analysis Using a Non-informative Prior. Statistics in Medicine, 36(2), 378–399.CrossRefGoogle ScholarPubMed
Carlsson, R., Schimmack, U., Williams, D. R., & Bürkner, P.-C. (2017). Bayes Factors from Pooled Data Are no Substitute for Bayesian Meta-Analysis: Commentary on Scheibehenne, Jamil, and Wagenmakers (2016). Psychological Science, 28(11), 1694–1697.Google Scholar
Carter, E. C., Kofler, L. M., Forster, D. E., & McCullough, M. E. (2015). A Series of Meta-Analytic Tests of the Depletion Effect: Self-Control Does Not Seem to Rely on a Limited Resource. Journal of Experimental Psychology: General, 144, 796–815.Google Scholar
Center for Open Science. (2018). Transparency and Openness Promotion Guidelines. Retrieved from https://cos.io/our-services/top-guidelines/Google Scholar
Cheung, M. (2015). Meta-Analysis: A Structural Equation Modeling Approach. Chichester: Wiley.Google Scholar
Clarke, B. S. (1999). Asymptotic Normality of the Posterior in Relative Entropy. IEEE Transactions on Information Theory, 45(1), 165–176.Google Scholar
Cochran, W. G. (1954). The Combination of Estimates from Different Experiments. Biometrics, 10(1), 101–129.Google Scholar
Copas, J. B. (1999). What Works? Selectivity Models and Meta-Analysis. Journal of the Royal Statistical Society: Series A (Statistics in Society), 162, 95–109.Google Scholar
Copas, J. B. (2013). A Likelihood-Based Sensitivity Analysis for Publication Bias in Meta-Analysis. Journal of the Royal Statistical Society: Series C (Applied Statistics), 62, 47–66.Google Scholar
Duval, S., & Tweedie, R. (2000). Trim and Fill: A Simple Funnel-Plot-Based Method of Testing and Adjusting for Publication Bias in Meta-Analysis. Biometrics, 56, 455–463.Google Scholar
Fleiss, J. L., Cohen, J., & Everitt, B. S. (1969). Large Sample Standard Errors of Kappa and Weighted Kappa. Psychological Bulletin, 72(5), 323–327.Google Scholar
Fritz, C. O., Morris, P. E., & Richler, J. J. (2012). Effect Size Estimates: Current Use, Calculations, and Interpretation. Journal of Experimental Psychology: General, 141(1), 2–18.Google Scholar
Furlow, C. F., & Beretvas, S. N. (2005). Meta-Analytic Methods of Pooling Correlation Matrices for Structural Equation Modeling under Different Patterns of Missing Data. Psychological Methods, 10(2), 227–254.CrossRefGoogle ScholarPubMed
Glass, G. V. (2015). Meta-Analysis at Middle Age: A Personal History. Research Synthesis Methods, 6(3), 221–231.Google Scholar
Greenland, S., & O’Rourke, K. (2001). On the Bias Produced by Quality Scores in Meta-Analysis, and a Hierarchical View of Proposed Solutions. Biostatistics, 2(4), 463–471.Google Scholar
Gustafson, P., & Clarke, B. (2004). Decomposing Posterior Variance. Journal of Statistical Planning and Inference, 119(2), 311–327.CrossRefGoogle Scholar
Hagger, M. S., Chatzisarantis, N. L. D., Alberts, H., Anggono, C. O., Batailler, C., Birt, A. R., … Zwienenberg, M. (2016). A Multilab Preregistered Replication of the ego-Depletion Effect. Perspectives on Psychological Science, 11(4), 546–573.Google Scholar
Hagger, M. S., Wood, C., Stiff, C., & Chatzisarantis, N. L. D. (2010). Ego Depletion and the Strength Model of Self-Control: A Meta-Analysis. Psychological Bulletin, 136, 495–525.Google Scholar
Hardy, R. J., & Thompson, S. G. (1996). A Likelihood Approach to Meta-Analysis with Random Effects. Statistics in Medicine, 15(6), 619–629.Google Scholar
Hedges, L. V. (1984). Estimation of Effect Size under Nonrandom Sampling: The Effects of Censoring Studies Yielding Statistically Insignificant Mean Differences. Journal of Educational and Behavioral Statistics, 9, 61–85.Google Scholar
Higgins, J. P. T., Altman, D. G., Gøtzsche, P. C., Jüni, P., Moher, D., Oxman, A. D., … Cochrane Statistical Methods Group (2011). The Cochrane Collaboration’s Tool for Assessing Risk of Bias in Randomized Trials. British Medical Journal, 343, d5928.Google Scholar
Higgins, J. P. T., Thompson, S. G., & Spiegelhalter, D. J. (2009). A Re-evaluation of Random-Effects Meta-Analysis. Journal of the Royal Statistical Society: Series A (Statistics in Society), 172(1), 137–159.Google Scholar
Huang-Pollock, C. L., Karalunas, S. L., Tam, H., & Moore, A. N. (2012). Evaluating Vigilance Deficits in ADHD: A Meta-Analysis of CPT Performance. Journal of Abnormal Psychology, 121(2), 360–371.Google Scholar
Ioannidis, J. P. A. (2005). Differentiating Biases from Genuine Heterogeneity: Distinguishing Artifactual from Substantive Effects. In Rothstein, H., Sutton, A. J., & Borestein, M. (Eds.), Publication Bias in Meta-Analysis: Prevention, Assessment and Adjustments (pp. 287–302). Chichester: Wiley.CrossRefGoogle Scholar
Ioannidis, J. P. A. (2008). Interpretation of Tests of Heterogeneity and Bias in Meta-Analysis. Journal of Evaluation in Clinical Practice, 14(5), 951–957.Google Scholar
Iyengar, S., & Greenhouse, J. B. (1988). Selection Models and the File Drawer Problem. Statistical Science, 3(1), 109–117.Google Scholar
Jin, Z.-C., Zhou, X.-H., & He, J. (2015). Statistical Methods for Dealing with Publication Bias in Meta-Analysis. Statistics in Medicine, 34(2), 343–360.Google Scholar
Jüni, P., Witschi, A., Bloch, R., & Egger, M. (1999). The Hazards of Scoring the Quality of Clinical Trials for Meta-Analysis. Journal of the American Medical Association, 282(11), 1054–1060.Google Scholar
Kass, R. E., Eden, U. T., & Brown, E. N. (2014). Analysis of Neural Data. New York: Springer.Google Scholar
Kraemer, H. C. (2005). A Simple Effect Size Indicator for Two-Group Comparisons? A Comment on Requivalent. Psychological Methods, 10(4), 413–419.Google Scholar
Kraemer, H. C., & Andrews, G. (1982). A Nonparametric Technique for Meta-Analysis Effect Size Calculation. Psychological Bulletin, 91(2), 404–412.Google Scholar
Krueger, R. F., & Markon, K. E. (2006). Reinterpreting Comorbidity: A Model-Based Approach to Understanding and Classifying Psychopathology. Annual Review of Clinical Psychology, 2(1), 111–133.Google Scholar
Lee, C. H., Cook, S., Lee, J. S., & Han, B. (2016). Comparison of Two Meta-Analysis Methods: Inverse-Variance-Weighted Average and Weighted Sum of z-Scores. Genomics & Informatics, 14(4), 173–180.Google Scholar
Li, S., Jiang, H., Yang, H., Chen, W., Peng, J., Sun, M., … Zeng, J. (2015). The Dilemma of Heterogeneity Tests in Meta-Analysis: A Challenge from a Simulation Study. PLOS ONE, 10(5), e0127538.Google Scholar
Lin, D. Y., & Zeng, D. (2010). On the Relative Efficiency of Using Summary Statistics versus Individual-Level Data in Meta-Analysis. Biometrika, 97(2), 321–332.CrossRefGoogle ScholarPubMed
Lumley, T. (2002). Network Meta-Analysis for Indirect Treatment Comparisons. Statistics in Medicine, 21(16), 2313–2324.Google Scholar
McShane, B. B., Böckenholt, U., & Hansen, K. T. (2016). Adjusting for Publication Bias in Meta-Analysis: An Evaluation of Selection Methods and Some Cautionary Notes. Perspectives on Psychological Science, 11(5), 730–749.Google Scholar
Munafo, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., … Ioannidis, J. P. A. (2017). A Manifesto for Reproducible Science. Nature Human Behaviour, 1(1), 0021.Google Scholar
Noorbaloochi, S., & Meeden, G. (1983). Unbiasedness as the Dual of Being Bayes. Journal of the American Statistical Association, 78, 619–623.Google Scholar
Park, S., Serpedin, E., & Qaraqe, K. (2013). Gaussian Assumption: The Least Favorable but the Most Useful. IEEE Signal Processing Magazine, 30(3), 183–186.CrossRefGoogle Scholar
Revelle, W., & Zinbarg, R. E. (2009). Coefficients Alpha, Beta, Omega, and the GLB: Comments on Sijtsma. Psychometrika, 74(1), 145–154.Google Scholar
Rosenberg, M. S. (2010). A Generalized Formula for Converting Chi-Square Tests to Effect Sizes for Meta-Analysis. PLoS ONE, 5(4), e10059.Google Scholar
Rosenthal, R., & Rubin, D. B. (2003). Requivalent: A Simple Effect Size Indicator. Psychological Methods, 8(4), 492–496.Google Scholar
Ruscio, J. (2008). A Probability-Based Measure of Effect Size: Robustness to Base Rates and Other Factors. Psychological Methods, 13(1), 19–30.Google Scholar
Scheibehenne, B., Jamil, T., & Wagenmakers, E.-J. (2016). Bayesian Evidence Synthesis Can Reconcile Seemingly Inconsistent Results: The Case of Hotel Towel Reuse. Psychological Science, 27(7), 1043–1046.Google Scholar
Schmidt, F. L., & Hunter, J. E. (2015). Methods of Meta-Analysis: Correcting Error and Bias in Research Findings. Thousand Oaks, CA: Sage.Google Scholar
Shadish, W. R. (2015). Introduction to the Special Issue on the Origins of Modern Meta-Analysis: Introduction to Special Issue. Research Synthesis Methods, 6(3), 219–220.CrossRefGoogle Scholar
Sharma, L., Markon, K. E., & Clark, L. A. (2014). Toward a Theory of Distinct Types of “Impulsive” Behaviors: A Meta-Analysis of Self-Report and Behavioral Measures. Psychological Bulletin, 140(2), 374–408.Google Scholar
Smith, T. C., Spiegelhalter, D. J., & Thomas, A. (1995). Bayesian Approaches to Random-Effects Meta-Analysis: A Comparative Study. Statistics in Medicine, 14(24), 2685–2699.Google Scholar
Stein, M. S., & Nossek, J. A. (2017). A Pessimistic Approximation for the Fisher Information Measure. IEEE Transactions on Signal Processing, 65(2), 386–396.Google Scholar
Stram, D. O. (1996). Meta-Analysis of Published Data Using a Linear Mixed-Effects Model. Biometrics, 52, 536–544.CrossRefGoogle ScholarPubMed
Switzer, F. S., Paese, P. W., & Drasgow, F. (1992). Bootstrap Estimates of Standard Errors in Validity Generalization. Journal of Applied Psychology, 77(2), 123–129.Google Scholar
Van Aert, R. C. M., Wicherts, J. M., & van Assen, M. A. L. M. (2016). Conducting Meta-Analyses Based on p Values: Reservations and Recommendations for Applying p-Uniform and p-Curve. Perspectives on Psychological Science, 11(5), 713–729.Google Scholar
Van Houwelingen, H. C., Zwinderman, K. H., & Stijnen, T. (1993). A Bivariate Approach to Meta-Analysis. Statistics in Medicine, 12(24), 2273–2284.Google Scholar
Veroniki, A. A., Jackson, D., Viechtbauer, W., Bender, R., Bowden, J., Knapp, G., … Salanti, G. (2016). Methods to Estimate the Between-Study Variance and Its Uncertainty in Meta-Analysis. Research Synthesis Methods, 7(1), 55–79.CrossRefGoogle ScholarPubMed
Viswesvaran, C., & Ones, D. S. (1995). Theory Testing: Combining Psychometric Meta-Analysis and Structural Equations Modeling. Personnel Psychology, 48(4), 865–885.Google Scholar
Xu, A., & Raginsky, M. (2017). Information-Theoretic Lower Bounds on Bayes Risk in Decentralized Estimation. IEEE Transactions on Information Theory, 63(3), 1580–1600.Google Scholar
Zwaan, R., Etz, A., Lucas, R., & Donnellan, M. (2018). Making Replication Mainstream. Behavioral and Brain Sciences, 41, e120.Google Scholar