A COMPARISON OF THE RESULTS OF CHECKED VERSUS UNCHECKED INDIVIDUAL PATIENT DATA META-ANALYSES

Abstract

Objectives: Systematic reviews and meta-analyses of individual patient data involve the central collection, validation, and reanalysis of raw data from randomized controlled trials. Checking individual patient data before its inclusion in a meta-analysis involves a number of different procedures that can be both time- and resource-intensive. We therefore aimed to assess the utility of data checking by investigating whether checks made any appreciable difference to the results of an individual patient data meta-analysis.

Methods: Data that were included in a meta-analysis of postoperative radiotherapy in non–small-cell lung cancer were used in a comparison of checked and unchecked data to investigate whether checking impacted the final results of the meta-analysis. Data “as received” were compared with fully checked and with followed-up data.

Results: Checking influenced the results, in this case mainly due to the exclusion of a single trial that failed to meet checking procedures. Checking data from most trials had only a small effect and did not materially alter the overall results of the meta-analysis.

Conclusions: Although data checking and cleaning is time-consuming, and for the majority of trials may make little difference to the analysis, such procedures provide a useful safeguard against rare occurrences of data with major problems. Checking may also lend additional confidence in the data set, which may be particularly important when using unpublished data that has not been subject to standard peer review.