34 - Analyzing the Downstream Effects of Randomized Experiments

Summary

The work in this volume provides profound evidence of the value of randomized experimentation in political science research. Laboratory and field experiments can open up new fields for exploration, shed light on old debates, and answer questions previously believed to be intractable. Although acknowledgment of the value of experimentation in political science is becoming more commonplace, significant criticisms remain. The oft-repeated shortcomings of experimental research tend to center on the practical and ethical limitations of randomized interventions. I begin this chapter by detailing some of these criticisms and then explore one means of extending the value of randomized interventions beyond their original intent to ameliorate some of these same perceived limitations.

One of the most prominent critiques of this genre is that randomized experiments tend to be overly narrow in scope in terms of time frame and subject matter, as well as high in cost. Although short-term experiments may incur costs similar to observational research, they often focus on a single or just a few variations in an independent variable, seemingly limiting their applicability to a breadth of topics that a survey could cover. The high cost associated with long-term data collection and the necessity of maintaining contact with the subjects involved impedes the likelihood of gathering information on long-term outcomes. There are also few incentives to conduct interventions in which the impacts may only be determined years down the road.