2 - Standard natural experiments

Summary

The title of this part of the book—“Discovering Natural Experiments”—suggests a first foundational issue for discussion. The random or as-if random assignment that characterizes natural experiments occurs as a feature of social and political processes—not in connection with a manipulation planned and carried out by an experimental researcher. This is what makes natural experiments observational studies, not true experiments.

For this reason, however, researchers face a major challenge in identifying situations in which natural experiments occur. Scholars often speak not of “creating” a natural experiment, but of “exploiting” or “leveraging” an opportunity for this kind of approach in the analysis of observational data. In an important sense, natural experiments are not so much designed as discovered.

How, then, does one uncover a natural experiment? As the survey in Part I of the book will suggest, new ideas for sources of natural experiments—such as close elections or weather shocks—seem to arise in unpredictable ways. Moreover, their successful use in one context does not guarantee their applicability to other substantive problems. The discovery of natural experiments is thus as much art as science: there appears to be no algorithm for the generation of convincing natural experiments, and analysts are challenged to think carefully about whether sources of natural experiments discovered in one context are applicable to other settings.

Yet, the best way to recognize the potential for using a natural experiment productively is often through exposure to examples. This can generate ideas for new research, as existing approaches are modified to suit novel contexts and questions, and it can also lead researchers to recognize new sources of natural experiments. Part I of the book therefore surveys and discusses in detail existing research, as a way to broach the central topic of how to discover natural experiments.