13 - Meta-analysis and combination of information

Summary

The impact of a scientific report is a function both of the presentation and the originality and the quality of the results. To achieve maximal impact, new empirical results should be reported in a form that enables readers to combine them effectively with other relevant data, without burdening readers with having to redo all the analyses behind the new results. Meta-analysis is a broad term used for methods analysing a set of similar or related experiments jointly, for purposes of general comparison, exhibiting grander structure, spotting outliers and examining relevant factors for such, and so on. As such the vital concept is that of combining different sources of information, and often enough not based on the full sets of raw data but on suitable summary statistics for each experiment. This chapter examines some natural and effective approaches to such problems, involving construction of confidence distributions for the more relevant parameters. Illustrations include meta-analysis of certain death-after-surgery rates for British hospitals. Yet other applications involving meta-analysis methods developed here are offered in Chapter 14.

Introduction

Meta-analysis, the art and science of combining results from a set of independent studies, is big business in medicine, and is also an important tool in psychology, social sciences, ecology, physics and other natural sciences. The yearly number of medical meta-analyses has increased exponentially in recent years, and rounded 2000 by 2005 (Sutton and Higgins, 2008).

The first issue to resolve in a meta-analysis is to define the parameter, possibly a vector, to study. This might not be as easy as it sounds because related studies might vary with respect to definition and they will also usually vary in their experimental or observational setup and statistical methods. Along with the definition, the population of related potential studies is delineated. In rather rare cases is it clear that the value of the parameter is exactly the same in related studies. Then a fixed effects model may be appropriate. When the parameter varies across related studies, or when there are unobserved heterogeneities across studies, the statistical model to be used in the meta-analysis should include random effects.

The next problem is to determine the criterion and algorithm for selecting studies for the meta-analysis. Ideally the collection should be a random sample from the population of related potential studies.