Accurately estimating risk preferences is of critical importance when evaluating data from many economic experiments or strategic interactions. I use a simulation model to conduct power analyses over two lottery batteries designed to classify individual subjects as being best explained by one of a number of alternative specifications of risk preference models. I propose a case in which there are only two possible alternatives for classification and find that the statistical methods used to classify subjects result in type I and type II errors at rates far beyond traditionally acceptable levels. These results suggest that subjects in experiments must make significantly more choices, or that traditional lottery pair batteries need to be substantially redesigned to make accurate inferences about the risk preference models that characterize a subject’s choices.

The current publication system in economics has encouraged the inflation of positive results in empirical papers. Registered Reports, also called Pre-Results Reviews, are a new submission format for empirical work that takes pre-registration one step further. In Registered Reports, researchers write their papers before running the study and commit to a detailed data collection process and analysis plan. After a first-stage review, a journal can give an In-Principle-Acceptance guaranteeing that the paper will be published if the authors carry out their data collection and analysis as pre-specified. We here propose a practical guide to Registered Reports for empirical economists. We illustrate the major problems that Registered Reports address (p-hacking, HARKing, forking, and publication bias), and present practical guidelines on how to write and review Registered Reports (e.g., the data-analysis plan, power analysis, and correction for multiple-hypothesis testing), with R and STATA codes. We provide specific examples for experimental economics, and show how research design can be improved to maximize statistical power. Last, we discuss some tools that authors, editors, and referees can use to evaluate Registered Reports (checklist, study-design table, and quality assessment).

While effect size estimates, post hoc power estimates, and a priori sample size determination are becoming a routine part of univariate analyses involving measured variables (e.g., ANOVA), such measures and methods have not been articulated for analyses involving latent means. The current article presents standardized effect size measures for latent mean differences inferred from both structured means modeling and MIMIC approaches to hypothesis testing about differences among means on a single latent construct. These measures are then related to post hoc power analysis, a priori sample size determination, and a relevant measure of construct reliability.

Challenge studies are often implemented for assessing whether a subject is sensitive to a certain agent or allergen. In particular, researchers test groups of subjects to determine if there really exists a causal relationship between some agent of interest and a response. To answer such a question, we need to detect the presence of the phenomenon in just one individual. Typically, however, there are a large number of unknown risk factors associated with the response and a potentially small population prevalence. Hence, standard statistical techniques, by averaging the treatment effect across the group, may miss a significant response of a single individual and lead to inconclusive results. We develop an alternative approach based on union-intersection testing that will allow a practitioner to correctly examine observations on an individual apart from the other subjects and test the hypothesis of interest: Does the phenomenon exist in the population? More specifically, we show how this technique adjusts for the multiple number of tests encountered when analyzing data for each individual subject separately. Furthermore, we demonstrate power calculations for the determination of sample size prior to performing the study. The performance of the union-intersection approach in comparison to linear models and semiparametric techniques is considered through sample size calculations and simulations. The union-intersection testing methodology out performs the Kolmogorov tests. However, the nested linear model performs as well if not better than the union-intersection tests. To illustrate the ideas presented in the paper, we provide an application in which we analyze psychological data collected by way of a challenge study design.

The Wald, likelihood ratio, score, and the recently proposed gradient statistics can be used to assess a broad range of hypotheses in item response theory models, for instance, to check the overall model fit or to detect differential item functioning. We introduce new methods for power analysis and sample size planning that can be applied when marginal maximum likelihood estimation is used. This allows the application to a variety of IRT models, which are commonly used in practice, e.g., in large-scale educational assessments. An analytical method utilizes the asymptotic distributions of the statistics under alternative hypotheses. We also provide a sampling-based approach for applications where the analytical approach is computationally infeasible. This can be the case with 20 or more items, since the computational load increases exponentially with the number of items. We performed extensive simulation studies in three practically relevant settings, i.e., testing a Rasch model against a 2PL model, testing for differential item functioning, and testing a partial credit model against a generalized partial credit model. The observed distributions of the test statistics and the power of the tests agreed well with the predictions by the proposed methods in sufficiently large samples. We provide an openly accessible R package that implements the methods for user-supplied hypotheses.

Primates are one of the most threatened groups of mammals. Understanding their patterns of population occurrence and abundance, especially in response to threats, is critical for informing conservation action. The crested black macaque Macaca nigra is the only Critically Endangered species of Sulawesi's seven endemic macaques. Little is known about its distribution or its response to deforestation and hunting. We conducted a camera-trap survey across the entire species range using an occupancy-based analytical approach to (1) establish the first range-wide baseline of occurrence, (2) investigate how environmental and anthropogenic factors influence occurrence, (3) identify priority conservation subpopulations, and (4) test the efficacy of the sampling and analytical protocol for temporal monitoring of M. nigra using occupancy as the state variable. From 9,753 camera-trap days, M. nigra was detected on 473 days at 77 of the 111 camera locations. Species occupancy was 0.66 and highest inside protected areas and closed canopy forest. We identified eight distinct subpopulations, based on distribution and forest fragment size. To inform future monitoring, we used a power analysis to determine if our effort would allow us to detect inter-annual occupancy declines of 10%, and found that 90 camera locations surveyed for 3 months (8,100 camera days) across three consecutive seasons is the effort required to detect such change with 80% certainty. Our study underscores the importance of well-managed protected areas and intact forests for the long-term survival of the crested black macaque, and tests the effectiveness of camera traps to monitor primates at the landscape scale.

This paper probes the attempt to use power analysis to link three domains: ontology, explanation, and the strategy of political actors. It shows how Katzenstein and Seybert develop an open social ontology that serves as the backdrop for explanations that need to be causal but indeterminate, and a cautious political practice. It exposes tensions in two important links. First, the open ontology becomes retranslated as an explanatory cause. Second, the call for being cautious because of ‘unknown unknowns’ may just as well invite strategies of doubling down in control power.

Identifying optimal sampling designs for detecting population-level declines is critical for optimizing expenditures by research and monitoring programmes. The Tropical Ecology Assessment and Monitoring (TEAM) network is the most extensive tropical camera-trap monitoring programme, but the effectiveness of its sampling protocol has not been rigorously assessed. Here, we assess the power and sensitivity of the programme's camera-trap monitoring protocol for detecting occupancy changes in unmarked populations using the freely available application PowerSensor!. We found that the protocol is well suited to detect moderate (≥ 5%) population changes within 3–4 years for relatively common species that have medium to high detection probabilities (i.e. p > 0.2). The TEAM protocol cannot, however, detect typical changes in rare and evasive species, a category into which many tropical species and many species of conservation concern fall. Additional research is needed to build occupancy models for detecting change in rare and elusive species when individuals are unmarked.

Field experiments were conducted in Chile and western Canada to measure short-distance (0 to 100 m) outcrossing from transgenic safflower (Carthamus tinctorius L.) intended for plant molecular farming to non-transgenic commodity safflower of the same variety. The transgenic safflower used as the pollen source was transformed with a construct for seed-specific expression of a high-value protein and constitutive expression of a gene conferring resistance to the broad-spectrum herbicide glufosinate. Progeny of non-transgenic plants grown in plots adjacent to the transgenic pollen source were screened for glufosinate resistance to measure outcrossing frequency. Outcrossing frequency differed among locations: values closest to the transgenic pollen source (0 to 3 m) ranged from 0.48 to 1.67% and rapidly declined to between 0.0024 to 0.03% at distances of 50 to 100 m. At each location, outcrossing frequency was spatially heterogeneous, indicating insects or wind moved pollen asymmetrically. A power analysis assuming a binomial distribution and a range of alpha values (type 1 error) was conducted to estimate an upper and lower confidence interval for the probable transgenic seed frequency in each sample. This facilitated interpretation when large numbers of seeds were screened from the outcrossing experiments and no transgenic seeds were found. This study should aid regulators and the plant molecular farming industry in developing confinement strategies to mitigate pollen mediated gene flow from transgenic to non-transgenic safflower.