Bootstrapping

Extract

Two questions confront data analysts: What's going on in the data? And how certain are the conclusions? One must deal with both questions to make rational decisions. John Tukey has called the two aspects of analysis exploratory and confirmatory. Consider, for example, a study in which patient data are analyzed for patterns that might identify causative agents for a particular disease. First, one should explore the data to reveal meaningful regularity (or perhaps irregularity might be a better way of saying it). For example, is there clustering in the geographic distribution of the patients that might indicate an environmental agent? After tentative pattern identification, confirmation is required to determine whether the patterns could simply be due to chance variation. That is, is the clustering strong enough to merit further detailed investigation or will the pattern disappear when further data are analyzed?

Classical statistical methods approach these questions through the foundation of probabilistic inference. I believe that it is no exaggeration to say that development of such methods during the 20th century are among the most important achievements in the history of science. They have contributed to the scientific method a systematic way of discerning what A.S.C. Ehrenberg calls “lawlike relationships” in empirical data. Their application has been decisive to progress in fields as diverse as nuclear physics, engineering design, genetics, agriculture, and paleontology, as well as medical science.