W09-02 - Statistical Solutions for the Evaluation of Reliability In-study

Abstract

One principal characteristic of an outcome measure in a clinical trial is reliability. The reliability of a measurement is an important consideration in the choice of the outcome measure for a clinical trial and in the choice of measures used for eligibility and exclusion. Just as the power of a study is reported in a final publication, so also should the reliability of the outcome and eligibility measurements so as to allow the authors to better describe, and readers to better understand, the sources of imprecision in study results, and to improve the design of future trials. The precision of measurements will be discussed, as the lower the precision, the more subjects will be needed for the clinical trial, which can affect outcomes and increase cost. The speaker will address the main components of measurement error, including systematic bias (general learning or fatigue effects), and random error due to biological or mechanical variation (including administration of study efficacy measures). In a clinical trial, error components should be meaningfully quantified for the researcher and physician to relate the described error to judgments regarding analytical goals (rather than just the statistical significance of the reliability indicators). Researchers should be cautious in (1) concluding acceptable reliability even if a correlation is high; (2) extrapolating the results of to a new sample; and (3) comparing reliability results between studies.

Learning Objectives:

From this workshop, clinicians and statisticians in industry and academia will learn:

1. Understanding reliability to assess how precise are measurements.

2. To identify graphical, numerical methods to a range of models encountered in clinical trials.