237 Studies of epilepsy surgery outcomes are statistically underpowered.

Abstract

OBJECTIVES/GOALS: Low statistical power is a problem is many fields. We performed a systematic review to determine the median statistical power of studies of epilepsy surgery outcomes. METHODS/STUDY POPULATION: We performed a PubMed search for studies reporting epilepsy surgery outcomes for the years 1980-2000, focusing on studies using stereo-electroencephalography (SEEG). We extracted patient count data for comparisons of surgical outcome between groups, based on a prognostic factor. We defined a clinically meaningful difference the surgical outcome for MRI positive (66.9%) compared to MRI negative (45.5%) in the largest study in the series. The statistical power of a Chi-square test was computed as the percentage of simulated runs (10,000 repetitions) assuming this difference with a p-value less than 0.05. RESULTS/ANTICIPATED RESULTS: Based on 69 studies, the median sample size was 38 patients, and the median statistical power was 24%. This implies at least a 17% (0.5/[0.24+0.05)) chance a study with a significant result in false, assuming 1:1 pre-test odds. A 'typical’ SEEG study with 33 patients and 2:1 allocation had a median significant odds ratio of 6.5, which over-estimates the true odds ratio of 2.4. DISCUSSION/SIGNIFICANCE: Studies of epilepsy surgery outcomes using SEEG are statistically underpowered. This means true effects will be missed, the chance a study with a significant result is false will be inflated, and significant effects found will be over-estimated. Studies of surgery outcome need better statistical rigor if they are to reliably guide treatment.