This paper examines the construction and use of data sets in political science. We focus on three interrelated questions: How might we assess data quality? What factors shape data quality? and How can these factors be addressed to improve data quality? We first outline some problems with existing data set quality, including issues of validity, coverage, and accuracy, and we discuss some ways of identifying problems as well as some consequences of data quality problems. The core of the paper addresses the second question by analyzing the incentives and capabilities facing four key actors in a data supply chain: respondents, data collection agencies (including state bureaucracies and private organizations), international organizations, and finally, academic scholars. We conclude by making some suggestions for improving the use and construction of data sets.

It is a capital mistake, Watson, to theorise before you have all the evidence. It biases the judgment.

—Sherlock Holmes in “A Study in Scarlet”

Statistics make officials, and officials make statistics.”

—Chinese proverb

This special issue of Political Analysis hopefully marks a watershed in how we as a discipline think about modeling time-series—cross-section (TSCS) data. By that I mean moving beyond thinking of TSCS data as only presenting a series of technical estimation problems that are seen as violations of the Gauss-Markov assumptions, problems that can be treated by better estimation methods. The articles in this special issue point to ways of moving forward in thinking of TSCS data as providing opportunities to model intrinsic and important features of the data (while also paying attention to the various statistical issues).

Scholars of political behavior increasingly embed experimental designs in opinion surveys by randomly assigning respondents alternative versions of questionnaire items. Such experiments have major advantages: they are simple to implement and they dodge some of the difficulties of making inferences from conventional survey data. But survey experiments are no panacea. We identify problems of inference associated with typical uses of survey experiments in political science and highlight a range of difficulties, some of which have straightforward solutions within the survey-experimental approach and some of which can be dealt with only by exercising greater caution in interpreting findings and bringing to bear alternative strategies of research.

This paper pursues a number of theoretical explorations and conjectures pertaining to the uncovered set in spatial voting games. It was stimulated by the article “The Uncovered Set and the Limits of Legislative Action” by W. T. Bianco, I. Jeliazkov, and I. Sened (2004, Political Analysis 12:256—78) that employed a grid-search computational algorithm for estimating the size, shape, and location of the uncovered set, and it has been greatly facilitated by access to the CyberSenate spatial voting software being developed by Joseph Godfrey. I bring to light theoretical considerations that account for important features of the Bianco, Jeliazkov, and Sened results (e.g., the straight-line boundaries of uncovered sets displayed in some of their figures, the “unexpectedly large” uncovered sets displayed in other figures, and the apparent sensitivity of the location of uncovered sets to small shifts in the relative sizes of party caucuses) and present theoretical insights of more general relevance to spatial voting theory.

In 1995, Beck and Katz (B&K) instructed the profession on “What to do (and not to do) with time-series, cross-section data,” and almost instantly their prescriptions became the new orthodoxy for practitioners. Our assessment of the intellectual aftermath of this paper, however, does not inspire confidence in the conclusions reached during the past decade. The 195 papers we reviewed show a widespread failure to diagnose and treat common problems of time-series, cross-section (TSCS) data analysis. To show the importance of the consequences of the B&K assumptions, we replicate eight papers in prominent journals and find that simple alternative specifications often lead to drastically different conclusions. Finally, we summarize many of the statistical issues relative to TSCS data and show that there is a lot more to do with TSCS data than many researchers have apparently assumed.

Limited dependent variable (LDV) data are common in political science, and political methodologists have given much good advice on dealing with them. We review some methods for LDV “change point problems” and demonstrate the use of Bayesian approaches for count, binary, and duration-type data. Our applications are drawn from American politics, Comparative politics, and International Political Economy. We discuss the tradeoffs both philosophically and computationally. We conclude with possibilities for multiple change point work.

Although published works rarely include causal estimates from more than a few model specifications, authors usually choose the presented estimates from numerous trial runs readers never see. Given the often large variation in estimates across choices of control variables, functional forms, and other modeling assumptions, how can researchers ensure that the few estimates presented are accurate or representative? How do readers know that publications are not merely demonstrations that it is possible to find a specification that fits the author's favorite hypothesis? And how do we evaluate or even define statistical properties like unbiasedness or mean squared error when no unique model or estimator even exists? Matching methods, which offer the promise of causal inference with fewer assumptions, constitute one possible way forward, but crucial results in this fast-growing methodological literature are often grossly misinterpreted. We explain how to avoid these misinterpretations and propose a unified approach that makes it possible for researchers to preprocess data with matching (such as with the easy-to-use software we offer) and then to apply the best parametric techniques they would have used anyway. This procedure makes parametric models produce more accurate and considerably less model-dependent causal inferences.

We introduce the conditional frailty model, an event history model that separates and accounts for both event dependence and heterogeneity in repeated events processes. Event dependence and heterogeneity create within-subject correlation in event times thereby violating the assumptions of standard event history models. Simulations show the advantage of the conditional frailty model. Specifically they demonstrate the model's ability to disentangle the sources of within-subject correlation as well as the gains in both efficiency and bias of the model when compared to the widely used alternatives, which often produce conflicting conclusions. Two substantive political science problems illustrate the usefulness and interpretation of the model: state policy adoption and terrorist attacks.

When respondents use the ordinal response categories of standard survey questions in different ways, the validity of analyses based on the resulting data can be biased. Anchoring vignettes is a survey design technique, introduced by King et al. (2004, Enhancing the validity and cross-cultural comparability of measurement in survey research. American Political Science Review 94 [February]: 191–205), intended to correct for some of these problems. We develop new methods both for evaluating and choosing anchoring vignettes and for analyzing the resulting data. With surveys on a diverse range of topics in a range of countries, we illustrate how our proposed methods can improve the ability of anchoring vignettes to extract information from survey data, as well as saving in survey administration costs.

How should one analyze data when the underlying models being tested are statistically intractable? In this article, we offer a simulation approach that involves creating sets of artificial data with fully known generating models that can be meaningfully compared to real data. The strategy depends on constructing simulations that are well matched to the data against which they will be compared. Our particular concern is to consider concurrently how voters place parties on issue scales and how they evaluate parties based on issues. We reconsider the Lewis and King (2000) analysis of issue voting in Norway. The simulation findings resolve the ambiguity that Lewis and King report, as voters appear to assimilate and contrast party placements and to evaluate parties directionally. The simulations also provide a strong caveat against the use of individually perceived party placements in analyses of issue voting.

This paper suggests a three-stage procedure for the estimation of time-invariant and rarely changing variables in panel data models with unit effects. The first stage of the proposed estimator runs a fixed-effects model to obtain the unit effects, the second stage breaks down the unit effects into a part explained by the time-invariant and/or rarely changing variables and an error term, and the third stage reestimates the first stage by pooled OLS (with or without autocorrelation correction and with or without panel-corrected SEs) including the time-invariant variables plus the error term of stage 2, which then accounts for the unexplained part of the unit effects. We use Monte Carlo simulations to compare the finite sample properties of our estimator to the finite sample properties of competing estimators. In doing so, we demonstrate that our proposed technique provides the most reliable estimates under a wide variety of specifications common to real world data.

Although political scientists have begun to investigate the properties of Internet surveys, much remains to be learned about the utility of the Internet mode for conducting major survey research projects such as national election studies. This paper addresses this topic by presenting the results of an extensive survey comparison experiment conducted as part of the 2005 British Election Study. Analyses show statistically significant, but generally small, differences in distributions of key explanatory variables in models of turnout and party choice. Estimating model parameters reveals that there are few statistically significant differences between coefficients generated using the in-person and Internet data, and the relative explanatory power of rival models is virtually identical for the two types of data. In general, the in-person and Internet data tell very similar stories about what matters for turnout and party preference in Britain. Determining if similar findings obtain in other countries should have high priority on the research agenda for national election studies.

Would holding elections by mail increase voter turnout? Many electoral reform advocates predict that mail ballot elections will boost participation, basing their prediction on the high turnout rate among absentee voters and on the rise in voter turnout after Oregon switched to voting by mail. However, selection problems inherent to studies of absentee voters and Oregon give us important reasons to doubt whether their results would extend to more general applications of voting by mail. In this paper, we isolate the effects of voting in mail ballot elections by taking advantage of a natural experiment in which voters are assigned in a nearly random process to cast their ballots by mail. We use matching methods to ensure that, in our analysis, the demographic characteristics of these voters mirror those of polling-place voters who take part in the same elections. Drawing on data from a large sample of California counties in two general elections, we find that voting by mail does not deliver on the promise of greater participation in general elections. In fact, voters who are assigned to vote by mail turn out at lower rates than those who are sent to a polling place. Analysis of a sample of local special elections, by contrast, indicates that voting by mail can increase turnout in these otherwise low-participation contests.

In this study, we propose a model of individual voter behavior that can be applied to aggregate data at the district (or precinct) levels while accounting for differences in political preferences across districts and across voters within each district. Our model produces a mapping of the competing candidates and electoral districts on a latent “issues” space that describes how political preferences in each district deviate from the average voter and how each candidate caters to average voter preferences within each district. We formulate our model as a random-coefficients nested logit model in which the voter first evaluates the candidates to decide whether or not to cast his or her vote, and then chooses the candidate who provides him or her with the highest value. Because we allow the random coefficient to vary not only across districts but also across unobservable voters within each district, the model avoids the Independence of Irrelevant Alternatives Assumption both across districts and within each district, thereby accounting for the cannibalization of votes among similar candidates within and across voting districts. We illustrate our proposed model by calibrating it to the actual voting data from the first stage of a two-stage state governor election in the Brazilian state of Santa Catarina, and then using the estimates to predict the final outcome of the second stage.

In this paper, we demonstrate the econometric consequences of different specification and estimation choices in the analysis of spatially interdependent data and show how to calculate and present spatial effect estimates substantively. We consider four common estimators—nonspatial OLS, spatial OLS, spatial 2SLS, and spatial ML. We examine analytically the respective omitted-variable and simultaneity biases of nonspatial OLS and spatial OLS in the simplest case and then evaluate the performance of all four estimators in bias, efficiency, and SE accuracy terms under more realistic conditions using Monte Carlo experiments. We provide empirical illustration, showing how to calculate and present spatial effect estimates effectively, using data on European governments' active labor market expenditures. Our main conclusions are that spatial OLS, despite its simultaneity, performs acceptably under low-to-moderate interdependence strength and reasonable sample dimensions. Spatial 2SLS or spatial ML may be advised for other conditions, but, unless interdependence is truly absent or minuscule, any of the spatial estimators unambiguously, and often dramatically, dominates on all three criteria the nonspatial OLS commonly used currently in empirical work in political science.

Poole's (2000, Non-parametric unfolding of binary choice data. Political Analysis 8:211–37) nonparametric Optimal Classification procedure for binary data produces misleading rank orderings when applied to the modern House of Commons. With simulations and qualitative evidence, we show that the problem arises from the government-versus-opposition nature of British (Westminster) parliamentary politics and the strategic voting that is entailed therein. We suggest that political scientists think seriously about strategic voting in legislatures when interpreting results from such techniques.

Since the inception of the American National Election Study (ANES) in the 1940s, data have been collected via face-to-face interviewing in the homes of members of area probability samples of American adults, the same gold-standard approach used by the U.S. Census Bureau, other federal agencies, and some nongovernment researchers for many of the most high-profile surveys conducted today. This paper explores whether comparable findings about voters and elections would be obtained by a different, considerably less expensive method: Internet data collection from nonprobability samples of volunteer respondents. Comparisons of the 2000 and 2004 ANES data (collected via face-to-face interviewing with national probability samples) with simultaneous Internet surveys of volunteer samples yielded many differences in the distributions of variables and in the associations between variables (even controlling for differences between the samples in reported interest in politics). Accuracy was higher for the face-to-face/probability sample data than for the Internet/volunteer sample data in 88% of the possible comparisons. This suggests that researchers interested in assuring the accuracy of their findings in describing populations should rely on face-to-face surveys of probability samples rather than Internet samples of volunteer respondents.

The analysis of time-series cross-sectional (TSCS) data has become increasingly popular in political science. Meanwhile, political scientists are also becoming more interested in the use of multilevel models (MLM). However, little work exists to understand the benefits of multilevel modeling when applied to TSCS data. We employ Monte Carlo simulations to benchmark the performance of a Bayesian multilevel model for TSCS data. We find that the MLM performs as well or better than other common estimators for such data. Most importantly, the MLM is more general and offers researchers additional advantages.

Numerous studies demonstrate that the highly educated are more likely to vote. The literature has not determined, however, why this is the case. The panel structure of the Current Population Survey allows us to exploit exogenous life cycle differences across otherwise similar cohorts, so that the marginal effect of education can be isolated from unobserved correlated factors. We find that an additional year of schooling has very little impact on voter turnout.