In this book we concentrate on two prominent approaches to causality that underpin almost all of the work in political science estimating causal relationships: the Rubin Causal Model (RCM) and the Formal Theory Approach (FTA). In this and the next two chapters, we focus on the RCM model. In Chapter 6 we discuss FTA. RCM is an approach that has its genesis in the statistical literature and early studies of field experiments, whereas FTA comes primarily from the econometric literature and is the approach used by experimental economists in many laboratory experiments (although statisticians such as Pearl [2000] are also advocates of an approach that is similar to FTA).

We begin our presentation of RCM by defining some basic variables and terms used in the approach.

Variables in Modeling the Effects of a Cause

The Treatment Variable

We are interested in the effect of information on how voters choose in the election studied or target population; that is, information is our possible causal variable. To simplify our analysis, we think of being informed as a binary variable (later we discuss analyses where this assumption is relaxed). We can think of there being two states of the world, one in which an individual i is informed about the choices before them in the election and the other in which the individual i is uninformed.

Control in Experiments

Controlling Observables in Experiments

We begin our analysis of the Rubin Causal Model (RCM)-based approaches to estimating the effects of a cause with a review of those that work through the control of observable variables that can make it difficult to estimate causal effects. Specifically, using the notation of the previous chapter, there are two types of observable variables that can cause problems for the estimation of the effects of a cause, Zi and Xi. Recall that Yi is a function of Xi and Ti is a function of Zi. That is, Xi represents the other observable variables that affect our dependent variable besides the treatment variable and Zi represents the set of observable variables that affect the treatment variable. Moreover, these variables may overlap and we define Wi = Zi ∪ Xi.

In experiments researchers deal with these observable variables in two ways – through random assignment and through the ability to manipulate these variables as they do with treatment variables. In the next chapter we show how such random assignment sidesteps both observable and unobservable variables that can interfere with measuring the causal effect of the treatment.

But experimenters also can manipulate some of the observable variables that might have an effect on treatments or directly on voting behavior and thereby reduce their effects. For instance, one observable variable that might affect the treatment variable is the mechanism by which a voter learns the information.

Placing Experimental Research in Context

In typical discussions of estimating causality, social scientists who come from a statistics perspective often begin with a review of the experimental approach in an idealized setting that rarely exists, argue that the experimental approach as idealized is not feasible in social science, and then go on to discuss how causality is measured in observational data. For example, Winship and Morgan (1999) begin their otherwise excellent review of the literature in social science on measuring the effects of causes with the statement (p. 660), “sociologists, economists, and political scientists must rely on what is now known as observational data – data that have been generated by something other than a randomized experiment – typically surveys, censuses, or administrative records.” This tendency to bracket off measuring causality in experimental social science from measuring causality in observational data presumes that experiments are “either-or propositions”: a researcher can either conduct an “ideal” experiment, which we argue in this book would not be ideal for many questions in which political scientists are interested, or work with observational data.

Most of experimental social science is not the hypothesized ideal or classical experiment, usually with good reason. The bracketing off prevents a discussion of how causality is measured in experiments as they exist in social science and a realistic comparison of those methods to research with observational data.

What Is a Formal Model?

We turn in this chapter to the Formal Theory Approach (FTA) to causality. The key difference between FTA and the Rubin Causal Model (RCM) is that a formal model serves as the basis for the causal relationships studied. To understand what we mean by FTA, it is useful to define what we mean by a formal model. We define a formal model as a set of precise abstract assumptions or axioms about the data generating process (DGP) presented in symbolic terms that are solved to derive predictions about that process. These predictions are of two types: point predictions and relationship predictions. Point predictions are precise predictions about the values of the variables in the model when the model is in equilibrium, whereas relationship predictions are predictions about how we may expect two variables in the model to be related. Defining what is meant by whether the model is in equilibrium can vary with the model as well; different formal models rely on different equilibrium concepts, which is something that we investigate later in Section 6.5.4. Some of these relationship predictions may be predicted to be “causal” in that changes in one variable “cause” changes in the other variable.

Definition 6.1 (FormalModel): A set of precise abstract assumptions or axioms about the DGP presented in symbolic terms that are solved to derive predictions about the DGP.

Definition 6.2 (Point Prediction of a Formal Model): A precise prediction from a formal model about the values of the variables in the model when the model is in equilibrium.

In the previous chapters we have examined both experimental and nonexperimental research, taking a largely common perspective. Although we have mentioned some of the differences between experimental work and nonexperimental analysis and some of the different types of experimental research, we have generally focused on commonalities rather than distinctions. Yet, the differences in approaches can be important and many are controversial. In this part of the book we turn to these differences. Usually the controversies have to do with arguments about the validity, robustness, or generality of particular experimental designs. Thus, before we turn to the specific differences, we begin with a review of the concept of validity in research. Then we turn to particular and sometimes controversial issues in experimentation such as the location of an experiment (whether lab or field), the subjects recruited (whether subjects are students), and how the subjects are motivated (whether financial incentives are used).

Validity of Experimental Research

Suppose that we have conducted some empirical research with either experimental or nonexperimental data. We ideally want a research design that will provide us with valid results that are true for the population we are analyzing and robust results that generalize beyond our target population (see Definition 2.2). So, for example, before we begin to study the effect of information on voting – the effect of changes in Ti on Yi – we would like to come up with an experimental design that will give us results that meet these criteria.

The Promise of Collaboration

In May 2009, a group of experimental political scientists from all three heritages – political psychology, political economics, and statistics – met at Northwestern University. The meeting will soon lead to a new Handbook of Experimental Political Science with essays by the various attendees. We see this handbook as an important companion to our book. This meeting, like two previous ones at New York University (NYU) in February 2008 and February 2009 (the first two meetings of the now Annual NYU Center of Experimental Social Sciences Experimental Political Science Conference), are the first conferences of which we are aware (our collective memories reach back over 20 years) in which such a broad range of experimentalists in political science have interacted. Typically experimentalists in political science have interacted mainly with those in their respective subfields within the experimental community or with non–political scientists who take similar focuses.

We believe the movement of more interaction between experimentalists in political science across heritages has the potential of adding significantly to our knowledge of political behavior and institutions. The possible collaborations are many. Consider the experimental literature on voting, one of the most studied aspects of political behavior. Although the purpose of this book is not a substantive review of experimental literature on particular topics, we have presented a number of examples of experiments on voting to illustrate various methodological issues.

Constructivism – the principal theoretical revolution in the study of ethnic identities over the past thirty years – has established that individuals have multiple ethnic options with a choice of which one to activate in any given context, and that the ethnic identities they activate can change over time, often endogenously to political and economic outcomes. The implication for our data collection efforts is that they must make a distinction between ethnic “structure” (the set of potential ethnic identities that characterizes a population) and ethnic “practice” (the set of identities actually activated by that population), must accommodate the possibility of the multiplicity of identities in both structure and practice, and must be sensitive to context and time in collecting these data. But our cross-national datasets on ethnic groups, and the measures constructed on the basis of these datasets, are resolutely primordialist – they do not distinguish between structure and practice, do not accommodate the possibility of multiplicity, and are not sensitive to time and context. The lack of theoretically justified data hampers the production of high-quality research on the origin and effect of ethnic diversity and on related concepts. Although there is now a steadily increasing number of studies on the role of ethnicity in determining economic growth, the consolidation of democracy, the distribution of public goods, the study of large-scale violence, and the shape of party systems, the poor quality of the data on which these studies are based means that we cannot be confident of their results.

INTRODUCTION

The idea that a person's “race” is not the indelible mark of one's bloodline, phenotype, or some other biologistic notion no longer raises eyebrows, but it remains for the most part politely ignored by quantitatively oriented, survey-based social science researchers. Although we acknowledge that race, like ethnicity, is a social construct marked by fluidity, multiplicity, and contingency, we continue to measure racial and ethnic identities as fixed, categorical variables. This gap between theory and practice may be consequential if indeed it is the case that a fluid, continuous reality is measured in fixed, categorical terms. In this chapter, a new approach to measuring ethnoracial self-identification is presented, one that aims to define the parameters of this gap by explicitly taking variations in racial or ethnic self-identification as the object of inquiry.

Specifically, the chapter compares the standard approach to ethnoracial self-identification in surveys against a new measurement approach in which respondents are given “identity points” to allocate across a relevant set of social identity categories. This identity point allocation scheme allows respondents, as they see fit, to identify with as many racial or ethnic groups and to weight the strength of their self-identification across these groups. This new approach not only allows us to test for the sensitivity of ethnoracial self-identification to a question format but, in doing so, also presents a visibly different portrait of racial or ethnic self-identification than is found in most social surveys.

CONTENT ANALYSIS INTRODUCED

Content analysis, simply put, is the quantitative investigation of message characteristics. Most definitions are a bit more specific than this, often delineated by the degree to which a scientific method is assumed. The following definition is employed here:

In content analysis, as in all quantitative investigations, the quality of a measure is dependent on several factors. First, there must be a clear conceptualization of the construct of interest, for it is the congruence between conceptualization and operationalization (measurement) that constitutes basic internal validity (Babbie 1998).

First, We Conceptualize

With the construct of collective identity, conceptualization can be problematic, in that theoretic approaches abound. Abdelal et al. (2006: 695) refer to the “definitional anarchy” of identity research and, as Bruland and Horowitz (2003: 1) note, “the existence of identity as a universal but largely implicit concept makes it difficult to isolate and understand its use.”

The concept of racial identity – and, particularly in the United States, black racial identity – is ubiquitous in scholarly work on race in the social sciences and the humanities as well as in everyday political discourse. Several social scientists, however, have generally questioned the utility of the concept of “identity,” racial or otherwise, for social analysis. Scholars such as Rogers Brubaker and Frederick Cooper argue that the concept is used to capture a variety of confusing and sometimes contradictory clusters of meanings. They would have social scientists abandon the term for a range of more precise terms. Some philosophers have also questioned the analytical, moral, and political utility of both the general concept of identity and the more specific concept of black identity. Scholars such as Tommie Shelby, for example, call for the abandonment of the notion of a “collective black identity” in order to place the project of building philosophically liberal and politically progressive black solidarity on stronger philosophical, political, and moral grounds.