In the study of social processes, the presence of unobserved heterogeneity is a regular concern. It should be particularly worrisome for the statistical analysis of networks, given the complex dependencies that shape network formation combined with the restrictive assumptions of related models. In this paper, we demonstrate the importance of explicitly accounting for unobserved heterogeneity in exponential random graph models (ERGM) with a Monte Carlo analysis and two applications that have played an important role in the networks literature. Overall, these analyses show that failing to account for unobserved heterogeneity can have a significant impact on inferences about network formation. The proposed frailty extension to the ERGM (FERGM) generally outperforms the ERGM in these cases, and does so by relatively large margins. Moreover, our novel multilevel estimation strategy has the advantage of avoiding the problem of degeneration that plagues the standard MCMC-MLE approach.

Does treatment mode matter in studies of the effects of candidate race or ethnicity on voting decisions? The assumption implicit in most such work is that such treatment mode differences are either small and/or theoretically well understood, so that the choice of how to signal the race of a candidate is largely one of convenience. But this assumption remains untested. Using a nationally representative sample of white voting-age citizens and a modified conjoint design, we evaluate whether signaling candidate ethnicity with ethnic labels and names results in different effects than signaling candidate ethnicity with ethnically identifiable photos and names. Our results provide strong evidence that treatment-mode effects are substantively large and statistically significant. Further, these treatment-mode effects are not consistent with extant theoretical accounts. These results highlight the need for additional theoretical and empirical work on race/ethnicity treatment-mode effects.

The item count technique (ICT-MLE) regression model for survey list experiments depends on assumptions about responses at the extremes (choosing no or all items on the list). Existing list experiment best practices aim to minimize strategic misrepresentation in ways that virtually guarantee that a tiny number of respondents appear in the extrema. Under such conditions both the “no liars” identification assumption and the computational strategy used to estimate the ICT-MLE become difficult to sustain. I report the results of Monte Carlo experiments examining the sensitivity of the ICT-MLE and simple difference-in-means estimators to survey design choices and small amounts of non-strategic respondent error. I show that, compared to the difference in means, the performance of the ICT-MLE depends on list design. Both estimators are sensitive to measurement error, but the problems are more severe for the ICT-MLE as a direct consequence of the no liars assumption. These problems become extreme as the number of treatment-group respondents choosing all the items on the list decreases. I document that such problems can arise in real-world applications, provide guidance for applied work, and suggest directions for further research.

Measuring the causal impact of state behavior on outcomes is one of the biggest methodological challenges in the field of political science, for two reasons: behavior is generally endogenous, and the threat of unobserved variables that confound the relationship between behavior and outcomes is pervasive. Matching methods, widely considered to be the state of the art in causal inference in political science, are generally ill-suited to inference in the presence of unobserved confounders. Heckman-style multiple-equation models offer a solution to this problem; however, they rely on functional-form assumptions that can produce substantial bias in estimates of average treatment effects. We describe a category of models, flexible joint likelihood models, that account for both features of the data while avoiding reliance on rigid functional-form assumptions. We then assess these models’ performance in a series of neutral simulations, in which they produce substantial (55% to ${>}$90%) reduction in bias relative to competing models. Finally, we demonstrate their utility in a reanalysis of Simmons’ (2000) classic study of the impact of Article VIII commitment on compliance with the IMF’s currency-restriction regime.

In Qualitative Comparative Analysis (QCA), empirical researchers use the consistency value as one, if not sole, criterion to decide whether an association between a term and an outcome is consistent with a set-relational claim. Braumoeller (2015) points out that the consistency value is unsuitable for this purpose. We need to know the probability of obtaining it under the null hypothesis of no systematic relation. He introduces permutation testing for estimating the $p$ value of a consistency score as a safeguard against false positives. In this paper, I introduce permutation-based power estimation as a safeguard against false-negative conclusions. Low power might lead to the false exclusion of truth table rows from the minimization procedure and the generation and interpretation of invalid solutions. For a variety of constellations between an alternative and null hypothesis and numbers of cases, simulations demonstrate that power estimates can range from 1 to 0. Ex post power analysis for 63 truth table analyses shows that even under the most favorable constellation of parameters, about half of them can be considered low-powered. This points to the value of estimating power and calculating the required number of cases before the truth table analysis.

Duration analyses in political science often model nonproportional hazards through interactions with analysis time. To facilitate their interpretation, methodologists have proposed methods to visualize time-varying coefficients or hazard ratios. While these techniques are a useful, initial postestimation step, I argue that they are insufficient to identify the overall impact of a time-varying effect and may lead to faulty inference when a coefficient changes its sign. I show how even significant changes of a coefficient’s sign do not imply that the overall effect is reversed over time. In order to enable a correct interpretation of time-varying effects in this context, researchers should visualize their results with survivor functions. I outline how survivor functions are calculated for models with time-varying effects and demonstrate the need for such a nuanced interpretation using the prominent finding of a time-varying effect of mediation on interstate conflict. The reanalysis of the data using the proposed visualization methods indicates that the conclusions of earlier mediation research are misleading. The example highlights how survivor functions are an essential tool to clarify the ambiguity inherent in time-varying coefficients in event history models.

In recent years, political and social scientists have made increasing use of conjoint survey designs to study decision-making. Here, we study a consequential question which researchers confront when implementing conjoint designs: How many choice tasks can respondents perform before survey satisficing degrades response quality? To answer the question, we run a set of experiments where respondents are asked to complete as many as 30 conjoint tasks. Experiments conducted through Amazon’s Mechanical Turk and Survey Sampling International demonstrate the surprising robustness of conjoint designs, as there are detectable but quite limited increases in survey satisficing as the number of tasks increases. Our evidence suggests that in similar study contexts researchers can assign dozens of tasks without substantial declines in response quality.

Measuring the polarization of legislators and parties is a key step in understanding how politics develops over time. But in parliamentary systems—where ideological positions estimated from roll calls may not be informative—producing valid estimates is extremely challenging. We suggest a new measurement strategy that makes innovative use of the “accuracy” of machine classifiers, i.e., the number of correct predictions made as a proportion of all predictions. In our case, the “labels” are the party identifications of the members of parliament, predicted from their speeches along with some information on debate subjects. Intuitively, when the learner is able to discriminate members in the two main Westminster parties well, we claim we are in a period of “high” polarization. By contrast, when the classifier has low accuracy—and makes a relatively large number of mistakes in terms of allocating members to parties based on the data—we argue parliament is in an era of “low” polarization. This approach is fast and substantively valid, and we demonstrate its merits with simulations, and by comparing the estimates from 78 years of House of Commons speeches with qualitative and quantitative historical accounts of the same. As a headline finding, we note that contemporary British politics is approximately as polarized as it was in the mid-1960s—that is, in the middle of the “postwar consensus”. More broadly, we show that the technical performance of supervised learning algorithms can be directly informative about substantive matters in social science.