Measurement error undermines the accuracy of dietary intake data. The 24-hour dietary recall (24HR) is the standard data collection method in nutrition surveillance. Several neurocognitive processes underpin the act of recall, and individuals differ in their performance of these processes. This study aimed to investigate whether variation in neurocognitive processes, measured using four cognitive tasks, was associated with variation in measurement error of 24HRs. Participants (n 139) completed the Trail Making Test, the Wisconsin Card Sorting Test, the Visual Digit Span, and the Vividness of Visual Imagery questionnaire. During a controlled feeding study, participants completed three technology-assisted 24HR; the Automated Self-Administered Dietary Assessment Tool (ASA24), Intake24, and an Interviewer-Administered Image-Assisted 24HR (IA-24HR) one week apart. Percentage error between reported and true energy intakes was calculated. Using linear regression, the association between cognitive task scores and absolute percentage error in estimated energy intake was assessed. Longer time spent completing the Trail Making Test, an indicator of visual attention and executive functioning, was associated with greater error in energy intake estimation using ASA24 (B 0.13, 95% CI 0.04, 0.21) and Intake24 (B 0.10, 95% CI 0.02, 0.19). Regression models explained 13.6% (ASA24) and 15.8% (Intake24) of the variance in energy estimation error. No cognitive task scores were associated with error using IA-24HR. This study demonstrates that variation between individuals in neurocognitive processes explains some of the variation in 24HR error. Further investigation into the role of neurocognitive processes in 24HR and their role in the reliability of dietary intake data is warranted.

Social scientists often use ranking questions to study people’s opinions and preferences. However, little is understood about the general nature of measurement errors in such questions, let alone their statistical consequences and what researchers can do about them. We introduce a statistical framework to improve ranking data analysis by addressing measurement errors in ranking questions. First, we characterize measurement errors from random responses—arbitrary and meaningless responses based on a wide range of random patterns. We then quantify bias due to random responses, show that the bias may change our conclusion in any direction, and clarify why item order randomization alone does not solve the statistical issue. Next, we introduce our methodology based on two key design-based considerations: item order randomization and the addition of an “anchor” ranking question with known correct answers. They allow researchers to (1) learn about the direction of the bias and (2) estimate the proportion of random responses, enabling our bias-corrected estimators. We illustrate our methods by studying the relative importance of people’s partisan identity compared to their racial, gender, and religious identities in American politics. We find that about 30% of respondents offered random responses and that these responses may affect our substantive conclusions.

Past work on closed-ended survey responses demonstrates that inferring stable political attitudes requires separating signal from noise in “top of the head” answers to researchers’ questions. We outline a corresponding theory of the open-ended response, in which respondents make narrow, stand-in statements to convey more abstract, general attitudes. We then present a method designed to infer those attitudes. Our approach leverages co-variation with words used relatively frequently across respondents to infer what else they could have said without substantively changing what they meant—linking narrow themes to each other through associations with contextually prevalent words. This reflects the intuition that a respondent may use different specific statements at different points in time to convey similar meaning. We validate this approach using panel data in which respondents answer the same open-ended questions (concerning healthcare policy, most important problems, and evaluations of political parties) at multiple points in time, showing that our method’s output consistently exhibits higher within-subject correlations than hand-coding of narrow response categories, topic modeling, and large language model output. Finally, we show how large language models can be used to complement—but not, at present, substitute—our “implied word” method.

We propose a two-step procedure to estimate structural equation models (SEMs). In a first step, the latent variable is replaced by its conditional expectation given the observed data. This conditional expectation is estimated using a James–Stein type shrinkage estimator. The second step consists of regressing the dependent variables on this shrinkage estimator. In addition to linear SEMs, we also derive shrinkage estimators to estimate polynomials. We empirically demonstrate the feasibility of the proposed method via simulation and contrast the proposed estimator with ML and MIIV estimators under a limited number of simulation scenarios. We illustrate the method on a case study.

When latent variables are used as outcomes in regression analysis, a common approach that is used to solve the ignored measurement error issue is to take a multilevel perspective on item response modeling (IRT). Although recent computational advancement allows efficient and accurate estimation of multilevel IRT models, we argue that a two-stage divide-and-conquer strategy still has its unique advantages. Within the two-stage framework, three methods that take into account heteroscedastic measurement errors of the dependent variable in stage II analysis are introduced; they are the closed-form marginal MLE, the expectation maximization algorithm, and the moment estimation method. They are compared to the naïve two-stage estimation and the one-stage MCMC estimation. A simulation study is conducted to compare the five methods in terms of model parameter recovery and their standard error estimation. The pros and cons of each method are also discussed to provide guidelines for practitioners. Finally, a real data example is given to illustrate the applications of various methods using the National Educational Longitudinal Survey data (NELS 88).

An asymptotic expression for the reliability of a linearly equated test is developed using normal theory. The reliability is expressed as the product of two terms, the reliability of the test before equating, and an adjustment term. This adjustment term is a function of the sample sizes used to estimate the linear equating transformation. The results of a simulation study indicate close agreement between the theoretical and simulated reliability values for samples greater than 200. Findings demonstrate that samples as small as 300 can be used in linear equating without an appreciable decrease in reliability.

This article considers the application of the simulation-extrapolation (SIMEX) method for measurement error correction when the error variance is a function of the latent variable being measured. Heteroskedasticity of this form arises in educational and psychological applications with ability estimates from item response theory models. We conclude that there is no simple solution for applying SIMEX that generally will yield consistent estimators in this setting. However, we demonstrate that several approximate SIMEX methods can provide useful estimators, leading to recommendations for analysts dealing with this form of error in settings where SIMEX may be the most practical option.

The achievement level is a variable measured with error, that can be estimated by means of the Rasch model. Teacher grades also measure the achievement level but they are expressed on a different scale. This paper proposes a method for combining these two scores to obtain a synthetic measure of the achievement level based on the theory developed for regression with covariate measurement error. In particular, the focus is on ordinal scaled grades, using the SIMEX method for measurement error correction. The result is a measure comparable across subjects with smaller measurement error variance. An empirical application illustrates the method.

This article discusses alternative procedures to the standard F-test for ANCOVA in case the covariate is measured with error. Both a functional and a structural relationship approach are described. Examples of both types of analysis are given for the simple two-group design. Several cases are discussed and special attention is given to issues of model identifiability. An approximate statistical test based on the functional relationship approach is described. On the basis of Monte Carlo simulation results it is concluded that this testing procedure is to be preferred to the conventional F-test of the ANCOVA null hypothesis. It is shown how the standard null hypothesis may be tested in a structural relationship approach. It is concluded that some knowledge of the reliability of the covariate is necessary in order to obtain meaningful results.

It is shown that measurement error in predictor variables can be modeled using item response theory (IRT). The predictor variables, that may be defined at any level of an hierarchical regression model, are treated as latent variables. The normal ogive model is used to describe the relation between the latent variables and dichotomous observed variables, which may be responses to tests or questionnaires. It will be shown that the multilevel model with measurement error in the observed predictor variables can be estimated in a Bayesian framework using Gibbs sampling. In this article, handling measurement error via the normal ogive model is compared with alternative approaches using the classical true score model. Examples using real data are given.

Covariate-adjusted treatment effects are commonly estimated in non-randomized studies. It has been shown that measurement error in covariates can bias treatment effect estimates when not appropriately accounted for. So far, these delineations primarily assumed a true data generating model that included just one single covariate. It is, however, more plausible that the true model consists of more than one covariate. We evaluate when a further covariate may reduce bias due to measurement error in another covariate and in which cases it is not recommended to include a further covariate. We analytically derive the amount of bias related to the fallible covariate’s reliability and systematically disentangle bias compensation and amplification due to an additional covariate. With a fallible covariate, it is not always beneficial to include an additional covariate for adjustment, as the additional covariate can extensively increase the bias. The mechanisms for an increased bias due to an additional covariate can be complex, even in a simple setting of just two covariates. A high reliability of the fallible covariate or a high correlation between the covariates cannot in general prevent from substantial bias. We show distorting effects of a fallible covariate in an empirical example and discuss adjustment for latent covariates as a possible solution.

Item calibration is an essential issue in modern item response theory based psychological or educational testing. Due to the popularity of computerized adaptive testing, methods to efficiently calibrate new items have become more important than that in the time when paper and pencil test administration is the norm. There are many calibration processes being proposed and discussed from both theoretical and practical perspectives. Among them, the online calibration may be one of the most cost effective processes. In this paper, under a variable length computerized adaptive testing scenario, we integrate the methods of adaptive design, sequential estimation, and measurement error models to solve online item calibration problems. The proposed sequential estimate of item parameters is shown to be strongly consistent and asymptotically normally distributed with a prechosen accuracy. Numerical results show that the proposed method is very promising in terms of both estimation accuracy and efficiency. The results of using calibrated items to estimate the latent trait levels are also reported.

This paper proposes a structural analysis for generalized linear models when some explanatory variables are measured with error and the measurement error variance is a function of the true variables. The focus is on latent variables investigated on the basis of questionnaires and estimated using item response theory models. Latent variable estimates are then treated as observed measures of the true variables. This leads to a two-stage estimation procedure which constitutes an alternative to a joint model for the outcome variable and the responses given to the questionnaire. Simulation studies explore the effect of ignoring the true error structure and the performance of the proposed method. Two illustrative examples concern achievement data of university students. Particular attention is given to the Rasch model.

The ability of bisection procedures to specify the form of the psychophysical scale depends upon the precision of the technique. It is demonstrated that the precision of bisection techniques is a function of the stimulus interval bisected. Consequently, the choice of stimuli in a bisection experiment may predispose the ability of the experiment to distinguish between alternative psychophysical scales. The testing of interval scale properties of derived scales and the assessment of context effects in bisection experiments was also discussed.

Asymptotic expansions of the maximum likelihood estimator (MLE) and weighted likelihood estimator (WLE) of an examinee’s ability are derived while item parameter estimators are treated as covariates measured with error. The asymptotic formulae present the amount of bias of the ability estimators due to the uncertainty of item parameter estimators. A numerical example is presented to illustrate how to apply the formulae to evaluate the impact of uncertainty about item parameters on ability estimation and the appropriateness of estimating ability using the regular MLE or WLE method.

This paper is concerned with combining observed scores from sections of tests. It is demonstrated that in the presence of population information a linear combination of true scores can be estimated more efficiently than by the same linear combination of the observed scores. Three criteria for optimality are discussed, but they yield the same solution which can be described and motivated as a multivariate shrinkage estimator.

In survey research it is not uncommon to ask questions of the following type: “How many times did you undertake action a in reference period T of length τ?.” The relationship is established between τ and the correlation of the number of reported actions with some background variable. To this end it is assumed that the process of actions satisfies a renewal model with individual heterogeneity. Also a model has to be formulated for possible recall effects. Applications are given in the field of medical consumption.

A type of data layout that may be considered as an extension of the two-way random effects analysis of variance is characterized and modeled based on group invariance. The data layout seems to be suitable for several scenarios in psychometrics, including the one in which multiple measurements are taken on each of a set of variables, and the measurements can be divided into exchangeable subsets. The algebraic structure of the model is studied, which leads to results that are applicable to such problems as estimating correlation matrix corrected for attenuation and testing symmetry hypotheses.

In psychometrics, the canonical use of conditional likelihoods is for the Rasch model in measurement. Whilst not disputing the utility of conditional likelihoods in measurement, we examine a broader class of problems in psychometrics that can be addressed via conditional likelihoods. Specifically, we consider cluster-level endogeneity where the standard assumption that observed explanatory variables are independent from latent variables is violated. Here, “cluster” refers to the entity characterized by latent variables or random effects, such as individuals in measurement models or schools in multilevel models and “unit” refers to the elementary entity such as an item in measurement. Cluster-level endogeneity problems can arise in a number of settings, including unobserved confounding of causal effects, measurement error, retrospective sampling, informative cluster sizes, missing data, and heteroskedasticity. Severely inconsistent estimation can result if these challenges are ignored.