W09-01 - Academic Perspectives on Reliability in Clinical Trials

Abstract

The purpose of this discussion is to explore academics’ views on the use of reliability as a vital element to clinical trials and research. The main research question guiding this investigation is “How do academics approach the understanding of reliability as a resource in their courses and research studies?” There have been several recent attempts to use more advanced statistical analysis to judge reliability in clinical trials (including, bayesian methods, item response analysis, generalizability theory), however these methods of reliability are rarely discussed in academia, and are not commonly utilized in clinical trials. The common texts used in teaching reliability are The Standards for Educational and Psychological Testing (AERA et al., 1985), and Cronbach's reliability (1990); both of which expound reliability as a necessary, although insufficient, condition for validity, and both of which primarily discuss reliability from an educational testing perspective rather than in clinical trials. Because power and other features depend heavily on the reliability of the measures employed, so also should the reliability coefficient of study measures be reported in clinical trials. This would prompt a deeper understanding by academics, authors and readers for potential factors that could explain negative results, or a discrepancy between results from different studies. This would also allow subsequent academic and clinical researchers to better explain the reliability of measures in the design of future studies.

Learning Objectives:

From this workshop, clinicians and statisticians in industry and academia will learn:

1. How academic origins influence perspectives on understanding reliability?

2. Recent attempts to use more advanced statistical analysis in clinical trials.

3. Concerns with the traditional view of reliability