Introduction

This book is about regression models and variants like path models, simultaneous-equation models, logits and probits. Regression models can be used for different purposes:

(i) to summarize data,

(ii) to predict the future,

(iii) to predict the results of interventions.

The third–causal inference–is the most interesting and the most slippery. It will be our focus. For background, this section covers some basic principles of study design.

Causal inferences are made from observational studies, natural experiments, and randomized controlled experiments. When using observational (non-experimental) data to make causal inferences, the key problem is confounding. Sometimes this problem is handled by subdividing the study population (stratification, also called cross-tabulation), and sometimes by modeling. These strategies have various strengths and weaknesses, which need to be explored.

In medicine and social science, causal inferences are most solid when based on randomized controlled experiments, where investigators assign subjects at random–by the toss of a coin–to a treatment group or to a control group. Up to random error, the coin balances the two groups with respect to all relevant factors other than treatment. Differences between the treatment group and the control group are therefore due to treatment. That is why causation is relatively easy to infer from experimental data. However, experiments tend to be expensive, and may be impossible for ethical or practical reasons. Then statisticians turn to observational studies.

Stratification

A path model is a graphical way to represent a regression equation or several linked regression equations. These models, developed by the geneticist Sewell Wright, are often used to make causal inferences. We will look at a couple of examples and then explain the logic, which involves response schedules and the idea of stability under interventions.

Blau and Duncan (1967) are thinking about the stratification process in the United States. According to Marxist scholars of the time, the US is a highly stratified society. Status is determined by family background, and transmitted through the school system. Blau and Duncan have data in their chapter 2, showing that family background variables do influence status–but the system is far from deterministic. The US has a permeable social structure, with many opportunities to succeed or fail. Blau and Duncan go on to develop the path model shown in figure 1 on the next page, in order to answer questions like these:

The five variables in the diagram are son's occupation, son's first job, son's education, father's occupation, and father's education. Data come from a special supplement to the March 1962 Current Population Survey. The respondents are the sons (age 20–64), who answer questions about current job, first job, and parents.

This book is primarily intended for advanced undergraduates or beginning graduate students in statistics. It should also be of interest to many students and professionals in the social and health sciences. Although written as a textbook, it can be read on its own. The focus is on applications of linear models, including generalized least squares, two-stage least squares, probits and logits. The bootstrap is explained as a technique for estimating bias and computing standard errors.

The contents of the book can fairly be described as what you have to know in order to start reading empirical papers that use statistical models. The emphasis throughout is on the connection–or lack of connection–between the models and the real phenomena. Much of the discussion is organized around published studies; key papers are reprinted here for ease of reference. Some may find the tone of the discussion too skeptical. If you are among them, I would make an unusual request: suspend belief until you finish reading the book. (Suspension of disbelief is all too easily obtained, but that is a topic for another day.)

The first chapter contrasts observational studies with experiments, and introduces regression as a technique that may help to adjust for confounding in observational studies. There is a chapter that explains the regression line, and another chapter with a quick review of matrix algebra. (At Berkeley, half the statistics majors need these chapters.) The going would be much easier with students who knew such material.

Chapter 1 Observational Studies and Experiments

Exercise Set A

1. In table 1, there were 837 deaths from other causes in the total treatment group (screened plus refused) and 879 in the control group. Not much different.

Comments. (i) Groups are the same size, so we can look at numbers or rates. (ii) The difference in number of deaths is relatively small, and not statistically significant.

1. This comparison is biased. The control group includes women who would have accepted screening if they had been asked, and are therefore comparable to women in the screening group. But the control group also includes women who would have refused screening. The latter are poorer, less well educated, less at risk from breast cancer. (A comparison that includes only the subjects who follow the investigators' treatment plans is called “per protocol analysis,” and is generally biased.)

2. Natural experiment. The fact that the Lambeth Company moved its pipe (i) sets up the comparison with Southwark & Vauxhall (table 2) and (ii) makes it harder to explain the difference in death rates between the Lambeth customers and the Southwark & Vauxhall customers on the basis of some difference between the two groups–other than the water. For instance, people were generally not choosing between the two water companies on the basis of how the water tasted. If they had been, self selection and confounding would be bigger issues.[…]

Introduction

This chapter explains simultaneous-equation models, and how to estimate them using instrumental variables (or two-stage least squares). These techniques are needed to avoid simultaneity bias (aka endogeneity bias). The lead example will be hypothetical supply and demand equations for butter in the state of Wisconsin. The source of endogeneity bias will be explained, and so will methods for working around this problem.

Then we discuss two real examples—(i) the way education and fertility influence each other, and (ii) the effect of school choice on social capital. These examples indicate how social scientists use two-stage least squares to handle (i) reciprocal causation and (ii) self-selection of subjects into the sample. (In the social sciences, two-stage least squares is often seen as the solution to problems of statistical inference.) At the end of the chapter there is a literature review, which puts modeling issues into a broader perspective.

We turn now to butter. Supply and demand need some preliminary discussion. For an economist, butter supply is not a single quantity but a relationship between quantity and price. The supply curve shows the quantity of butter that farmers would bring to market at different prices. In the left Figure 1. Supply and demand. The vertical axis shows quantity; the horizontal axis, price.

hand panel of figure 1, price is on the horizontal axis and quantity on the vertical. (Economists usually do it the other way around.)

Notice that the supply curve slopes up.

Introduction

This chapter is about the regression line. The regression line is important on its own (to statisticians), and it will help us with multiple regression in chapter 4. The first example is a scatter diagram showing the heights of 1078 fathers and their sons (figure 1). Each pair of fathers and sons becomes a dot on the diagram. The height of the father is plotted on the x-axis; the height of his son, on the y-axis. The left hand vertical strip (inside the chimney) shows the families where the father is 64 inches tall to the nearest inch; the right hand vertical strip, families where the father is 72 inches tall. Many other strips could be drawn too. The regression line (solid) approximates the average height of the sons, given the heights of their fathers. This line goes through the centers of all the vertical strips. The regression line is flatter than the SD line, which is dashed. “SD” is shorthand for “standard deviation”; definitions come next.

The regression line

We have n subjects indexed by i = 1, …, n, and two data variables x and y. A data variable stores a value for each subject in a study. Thus, xi is the value of x for subject i, and yi is the value of y.

Overview

In this book I show how to use a simple spatial model of voting as a tool to analyze parliamentary roll call data. Each legislator is represented by one point, and each roll call is represented by two points – one for Yea and one for Nay. On every roll call each legislator votes for the closer outcome point, at least probabilistically. These points form a spatial map that summarizes the roll calls. In this sense a spatial map is much like a road map. A spreadsheet that tabulates all the distances between pairs of sizable cities in the U.S. contains the same information as the corresponding map of the U.S., but the spreadsheet gives you no idea what the U.S. looks like. Much like a road map, a spatial map formed from roll calls gives us a way of visualizing the political world of a legislature. The closeness of two legislators on the map shows how similar their voting records are, and the distribution of legislators shows what the dimensions of the space are.

The number of dimensions needed to represent the points is usually small, because legislators typically decide how to vote on the basis of their positions on a small number of underlying evaluative, or basic, dimensions. For example, in recent U.S. Congresses, we can easily predict how a “liberal” or a “conservative” will vote on most issues. These basic dimensions structure the roll call votes and are captured by the spatial maps.