Introduction

A significant part of what we know about the social and political world comes from case studies. Case studies famously contributed, for instance, to uncovering the tendency towards oligarchy in political parties, the inner working of the exercise of power in democracies, the dynamics of international crises, the logics of authority and control in organizations, the interplay between values and institutions in the Indian caste system, the sources of success and failure of deterrence, and the causes of social revolutions (Michels 1911; Dahl 1961; Crozier 1964; Dumont 1970; Allison 1971; George and Smoke 1974; Skocpol 1979). Beyond these classical and influential works, the case study research tradition remains popular as researchers explore the political development of imperial Germany in comparative perspective, the causes and characteristics of nuclear accidents, the 1986 Challenger launch disaster, the evolution of institutions, the role of reputational claims in foreign policy decision-making or the genesis of the welfare state (Esping-Andersen 1990; Sagan 1993; Mercer 1996; Vaughan 1996; Berman 2001; Thelen 2004; for more examples, see Feagin, Orum and Sjoberg 1991; George and Bennett 2005: 287–325; Gerring 2007: 2–5). In international relations, case studies have made a central contribution to both the international security and the international political economy subfields (Snyder 1989; Kacowicz 2004; Odell 2004).

The genesis of this book lies in the early 2000s, at the European University Institute (EUI), where a number of PhD researchers started to complain about the neglect of ‘qualitative methods’. As only a minority of the faculty worked principally with quantitative methods, we had assumed that the rest were qualitative in the way that Molière's M. Jourdain was a speaker of prose. A series of discussions and debates revealed that in most cases they were talking about something else, a specific form of epistemology rather than a method, and one whose meaning was being continually stretched across the discipline. While it was difficult to tie down exactly what was meant by ‘qualitative’, it seemed to be defined in opposition to ‘positivist’, another description that most professors found difficult to accept for themselves and which was also subject to considerable stretching.

The EUI was not alone here, for this was merely the latest expression of a Manicheanism in which social scientists seem to be driven to define themselves into opposing camps. The fact that we could never find a shared name or vocabulary for the two approaches suggested that the question was altogether more complicated. It was also apparent that most of the issues at stake were not new but echoed debates in philosophy, sociology and political science going back to classical times.

Quantification is one way of employing the scientific method to discover things about the world. In the social sciences we are trying to discover things about the social world, but the approach we use can still be regarded as scientific. The scientific approach attempts to abstract from the nuances and details of a story the salient features that can be built up into a theoretical statement (or statements) expected to hold true of any situation that can be defined in terms of the same abstractions. If such a theoretical statement does not hold true in some specific situation, this is presumed to be either because the theory was wrong or because it was not sufficiently elaborated. Elaborating social theories to bring in additional features of the world, found necessary for a full explanation, is an important feature of the scientific approach; but for elaboration to progress very far we need to employ quantitative analysis, as this chapter will try to show.

The transition from case studies to quantitative analysis is largely a matter of the number of cases. If you have one case, no causal inferences can be made. If you have two cases, you can rule out something as a necessary condition for something else. If you have three cases you can rule out two things, or you can start to make quantitative statements (for example, something might be found to pertain two-thirds of the time).

Paradigms in the social sciences

Partisans articulate their positions with passion and intensity, yet the nature of what divides them is hard to pin down. At times we hear of a stand-off between ‘qualitative’ scholars, who make use of archival research, ethnology, textual criticism, and discourse analysis; and ‘quantitative’ scholars, who deploy mathematics, game theory, and statistics. Scholars in the former tradition supposedly disdain the new, hypernumerate, approaches to political science as opaque and overly abstract, while scholars of the latter stripe deride the ‘old’ ways of studying politics as impression istic and lacking in rigor. At other times the schism is portrayed as being about the proper aspiration of the discipline – between those who believe that a scientific explanation of political life is possible, that we can derive something akin to physical laws of human behavior, and those who believe it is not … at still other times the rivals are portrayed as ‘rational choice theorists,’ whose work is animated by the assumption that individuals are rational maximizers of self-interest (often economics, sometimes not), and those who allow for a richer range of human motivations (Shapiro, Smith and Masoud 2004a: 1).

This quotation from the introduction to a recent volume on Problems and Methods in the Study of Politics addresses a core methodological issue for the social sciences in general: how many approaches/methods are available for students in the discipline?

Introduction

Normative theory and empirical research have become separate branches of social science. Yet, as I will suggest, empirical research can be guided by normative theory; and normative theory can be improved by empirical research. This is not always well done. Empirical researchers resort too often to ad hoc normative assumptions. On the other side, some theorists studying social problems still rely on hypothetical arguments in spite of available empirical evidence, while others interpret empirical research naïvely without the necessary critical knowledge (Favell and Modood 2003).

We should not, moreover, erase the difference altogether. Normative problems can never be fully resolved through analytical explanation or hermeneutical interpretation, nor can deep disagreement within normative theory be overcome by testing the empirical presuppositions. The goal of a unified political theory is not merely illusory but profoundly misunderstands the nature of this enterprise. Normative political theory mirrors political disagreements among citizens that cannot be resolved through conceptual analysis nor by inference from empirical evidence, but only through politics itself. The contribution of political theory to political debates is not to settle disputes but to clarify arguments and to highlight the values involved in political choices. Such theory should be supported by social science research to specify the realworld conditions and consequences of the choices that its normative propositions advocate.

What is multilevel regression modeling?

Consider an educational study with data from students in many schools, predicting in each school the students' grades y on a standardized test given their scores on a pre-test x and other information. A separate regression model can be fit within each school, and the parameters from these schools can themselves be modeled as depending on school characteristics (such as the socioeconomic status of the school's neighborhood, whether the school is public or private, and so on). The student-level regression and the school-level regression here are the two levels of a multilevel model.

In this example, a multilevel model can be expressed in (at least) three equivalent ways as a student-level regression:

• A model in which the coefficients vary by school (thus, instead of a model such as y = α + βx + error, we have y = αj + βjx + error, where the subscripts j index schools),

• A model with more than one variance component (student-level and school-level variation),

• A regression with many predictors, including an indicator variable for each school in the data.

More generally, we consider a multilevel model to be a regression (a linear or generalized linear model) in which the parameters—the regression coefficients—are given a probability model. This second-level model has parameters of its own—the hyperparameters of the model—which are also estimated from data.

The two key parts of a multilevel model are varying coefficients, and a model for those varying coefficients (which can itself include group-level predictors).

Statistical graphics are sometimes summarized as “exploratory data analysis” or “presentation” or “data display.” But these only capture part of the story. Graphs are a way to communicate graphical and spatial information to ourselves and others. Long before worrying about how to convince others, you first have to understand what's happening yourself.

Why to graph

Going back through the dozens of examples in this book, what are our motivations for graphing data and fitted models? Ultimately, the goal is communication (to self or others). More immediately, graphs are comparisons (to zero, to other graphs, to horizontal lines, and so forth). We “read” a graph both by pulling out the expected (for example, the slope of a fitted regression line, the comparisons of a series of confidence intervals to zero and each other) and the unexpected.

In our experience, the unexpected is usually not an “outlier” or aberrant point but rather a systematic pattern in some part of the data. For example, consider the binned residual plots in Section 5.6 for the well-switching models. There was an unexpectedly low rate of switching from wells that were just barely over the dangerous level for arsenic, possibly suggesting that people were moderating their decisions when in this ambiguous zone, or that there was other information not included in the model that could explain these decisions.

Analysis of variance (ANOVA) refers to a specific set of methods for data analysis and to a way of summarizing multilevel models:

• As a tool for data analysis, ANOVA is typically used to learn the relative importance of different sources of variation in a dataset. For example, Figure 13.8 displays success rates of pilots at a flight simulator under five different treatments at eight different airports. How much of the variation in the data is explained by treatments, how much by airports, and how much remains after these factors have been included in a linear model?

• If a multilevel model has already been fit, it can be summarized by the variation in each of its batches of coefficients. For example, in the radon modeling in Chapter 12, how much variation in radon levels is explained by floor of measurement and how much by geographical variation? Or, in the analysis of public opinion by state in Section 14.1, how much of the variation is explained by demographic factors (sex, age, ethnicity, education), and how much by states and regions? These “analysis of variance” questions can be of interest even for models that are primarily intended for prediction, or for estimating particular regression coefficients.

The sections of this chapter address the different roles of ANOVA in multilevel data analysis. We begin in Section 22.1 with a brief review of the goals and methods of classical analysis of variance, outlining how they fit into our general multilevel modeling approach.

Linear regression is a method that summarizes how the average values of a numerical outcome variable vary over subpopulations defined by linear functions of predictors. Introductory statistics and regression texts often focus on how regression can be used to represent relationships between variables, rather than as a comparison of average outcomes. By focusing on regression as a comparison of averages, we are being explicit about its limitations for defining these relationships causally, an issue to which we return in Chapter 9. Regression can be used to predict an outcome given a linear function of these predictors, and regression coefficients can be thought of as comparisons across predicted values or as comparisons among averages in the data.

One predictor

We begin by understanding the coefficients without worrying about issues of estimation and uncertainty. We shall fit a series of regressions predicting cognitive test scores of three- and four-year-old children given characteristics of their mothers, using data from a survey of adult American women and their children (a subsample from the National Longitudinal Survey of Youth).

We start by modeling the children's test scores given an indicator for whether the mother graduated from high school (coded as 1) or not (coded as 0).

Getting started with R, Bugs, and a text editor

Follow the instructions at www.stat.columbia.edu/∼gelman/arm/software/ to download, install, and set up R and Bugs on your Windows computer. The webpage is occasionally updated as the software improves, so we recommend checking back occasionally. R, OpenBugs, and WinBugs have online help with more information available at www.r-project.org, www.math.helsinki.fi/openbugs/, and www.mrc-bsu.cam.ac.uk/bugs/.

Set up a working directory on your computer for your R work. Every time you enter R, your working directory will automatically be set, and the necessary functions will be loaded in.

Configuring your computer display for efficient data analysis

We recommend working with three nonoverlapping open windows, as pictured in Figure C.1: an R console, the R graphics window, and a text editor (ideally a program such as Emacs or WinEdt that allows split windows, or the script window in the Windows version of R). When programming in Bugs, the text editor will have two windows open: a file (for example, project. R) with R commands, and a file (for example, project.bug) with the Bugs model. It is simplest to type commands into the text file with R commands and then cut and paste them into the R console. This is preferable to typing in the R console directly because copying and altering the commands is easier in the text editor. To run Bugs, there is no need to open a Bugs window; R will do this automatically when the function bugs() is called (assuming you have set up your computer as just described, which includes loading the R2WinBUGS package in R).

Multilevel modeling can be thought of in two equivalent ways:

• We can think of a generalization of linear regression, where intercepts, and possibly slopes, are allowed to vary by group. For example, starting with a regression model with one predictor, yi = α + βxi + ∈i, we can generalize to the varyingintercept model, yi = αj[i] + βxi + ∈i, and the varying-intercept, varying-slope model, yi = αj[i] + βj[i]xi + ∈i (see Figure 11.1 on page 238).

• Equivalently, we can think of multilevel modeling as a regression that includes a categorical input variable representing group membership. From this perspective, the group index is a factor with J levels, corresponding to J predictors in the regression model (or 2J if they are interacted with a predictor x in a varying-intercept, varying-slope model; or 3J if they are interacted with two predictors X(1), X(2); and so forth).

In either case, J−1 linear predictors are added to the model (or, to put it another way, the constant term in the regression is replaced by J separate intercept terms). The crucial multilevel modeling step is that these J coefficients are then themselves given a model (most simply, a common distribution for the J parameters αj or, more generally, a regression model for the αj's given group-level predictors). The group-level model is estimated simultaneously with the data-level regression of y.

We next explain how to fit multilevel models in Bugs, as called from R. We illustrate with several examples and discuss some general issues in model fitting and tricks that can help us estimate multilevel models using less computer time. We also present the basics of Bayesian inference (as a generalization of the least squares and maximum likelihood methods used for classical regression), which is the approach used in problems such as multilevel models with potentially large numbers of parameters.

Appendix C discusses some software that is available to quickly and approximately fit multilevel models. We recommend using Bugs for its flexibility in modeling; however, these simpler approaches can be useful to get started, explore models quickly, and check results.

Introduction

Generalized linear modeling is a framework for statistical analysis that includes linear and logistic regression as special cases. Linear regression directly predicts continuous data y from a linear predictor Xβ = β0 + X1β1 + ⋯ + Xkβk. Logistic regression predicts Pr(y = 1) for binary data from a linear predictor with an inverselogit transformation. A generalized linear model involves:

1. A data vector y = (y1, …, yn)

2. Predictors X and coefficients β, forming a linear predictor Xβ

3. A link function g, yielding a vector of transformed data ŷ = g−1(Xβ) that are used to model the data

4. A data distribution, p(y|ŷ)

5. Possibly other parameters, such as variances, overdispersions, and cutpoints, involved in the predictors, link function, and data distribution.

The options in a generalized linear model are the transformation g and the data distribution p.

• In linear regression, the transformation is the identity (that is, g(u) ≡ u) and the data distribution is normal, with standard deviation σ estimated from data.

• In logistic regression, the transformation is the inverse-logit, g−1(u) = logit−1(u) (see Figure 5.2a on page 80) and the data distribution is defined by the probability for binary data: Pr(y = 1) = ŷ.

We now go through the steps of understanding and working with multilevel regressions, including designing studies, summarizing inferences, checking the fit of models to data, and imputing missing data.