There are potentially multiple sources that make it difficult to compare the typical survey measure of the left-right self-placement cross-nationally. We focus on differential item functioning (DIF) due to the different use of response scales when the left-right is framed as an aggregate dimension of policies. We also examine whether and to what extent ordinary citizens’ use of the scale is cross-nationally comparable. Our goal is twofold. First, we assess the cross-national comparability of the left-right self-placement scale using the anchoring vignette method used in nine European countries. Second, we propose a measure that quantifies the extent of DIF at the country level. Our original survey and other benchmark studies suggest that the size of cross-national DIF (CN-DIF) in citizens' use of a left-right scale is relatively small when the left-right concept is considered in policy terms and when a comparison is made between Western European countries.

The Defining Issues Test (DIT) has been widely used in psychological experiments to assess one’s developmental level of moral reasoning in terms of postconventional reasoning. However, there have been concerns regarding whether the tool is biased across people with different genders and political and religious views. To address the limitations, in the present study, I tested the validity of the brief version of the test, that is, the behavioral DIT, in terms of the measurement invariance and differential item functioning (DIF). I could not find any significant non-invariance at the test level or any item demonstrating practically significant DIF at the item level. The findings indicate that neither the test nor any of its items showed a significant bias toward any particular group. As a result, the collected validity evidence supports the use of test scores across different groups, enabling researchers who intend to examine participants’ moral reasoning development across heterogeneous groups to draw conclusions based on the scores.

A common approach when studying the quality of representation involves comparing the latent preferences of voters and legislators, commonly obtained by fitting an item response theory (IRT) model to a common set of stimuli. Despite being exposed to the same stimuli, voters and legislators may not share a common understanding of how these stimuli map onto their latent preferences, leading to differential item functioning (DIF) and incomparability of estimates. We explore the presence of DIF and incomparability of latent preferences obtained through IRT models by reanalyzing an influential survey dataset, where survey respondents expressed their preferences on roll call votes that U.S. legislators had previously voted on. To do so, we propose defining a Dirichlet process prior over item response functions in standard IRT models. In contrast to typical multistep approaches to detecting DIF, our strategy allows researchers to fit a single model, automatically identifying incomparable subgroups with different mappings from latent traits onto observed responses. We find that although there is a group of voters whose estimated positions can be safely compared to those of legislators, a sizeable share of surveyed voters understand stimuli in fundamentally different ways. Ignoring these issues can lead to incorrect conclusions about the quality of representation.

Many large survey courses rely on multiple professors or teaching assistants to judge student responses to open-ended questions. Even following best practices, students with similar levels of conceptual understanding can receive widely varying assessments from different graders. We detail how this can occur and argue that it is an example of differential item functioning (or interpersonal incomparability), where graders interpret the same possible grading range differently. Using both actual assessment data from a large survey course in Comparative Politics and simulation methods, we show that the bias can be corrected by a small number of “bridging” observations across graders. We conclude by offering best practices for fair assessment in large survey courses.

Reliable and valid assessment of sports nutrition knowledge can inform athlete nutrition education to address knowledge gaps. This study aimed to test the reliability and validity of an electronically administered sports nutrition knowledge tool – Platform to Evaluate Athlete Knowledge of Sports Nutrition Questionnaire (PEAKS-NQ). A 94-item PEAKS-NQ was piloted to 149 developmental athletes (DA) in New Zealand, with a subset invited to complete the PEAKS-NQ again to assess reliability. Reliability was evaluated using sign test, intraclass correlation and Cronbach’s α. Accredited sports dietitians (ASD; n 255) completed the PEAKS-NQ to establish construct validity via known-groups methodology and provided relevance scores to determine the scale content validity index (S-CVI). Rasch analysis was conducted to identify potentially problematic items and test reliability. Score differences between DA and ASD were analysed using independent t or non-parametric tests. DA (n 88) were 17·8 (sd 1·4) years, 61·4 % female and mostly in high school (94·3 %). ASD (n 45) were 37·8 (sd 7·6) years, 82·2 % female, with >5 years of dietetic experience (59·1 %). ASD scored higher than DA in all sections and overall (91·5 (sd 3·4) v. 67·1 (sd 10·5) %) (P < 0·001). There were no differences between retests (n 18; P = 0·14). Cronbach’s α was 0·86. S-CVI indicated good content validity (0·88). Rasch analysis resulted in a fifty-item PEAKS-NQ with high item (0·91) and person (0·92) reliability. The PEAKS-NQ is reliable and valid for assessing sports nutrition knowledge which could assist practitioners effectively tailor and evaluate nutrition education.

We examined the item properties of the Two Peas Questionnaire (TPQ) among a sample of same-sex twin pairs from the Washington State Twin Registry. With the exception of the ‘two peas’ item, three of the mistakenness items showed differential item functioning. Results showed that the monozygotic (MZ) and dizygotic (DZ) pairs may differ in their responses on these items, even among those with similar latent traits of similarity and confusability. Upon comparing three classification methods to determine the zygosity of same-sex twins, the overall classification accuracy rate was over 90% using the unit-weighted pair zygosity sum score, providing an efficient and sufficiently accurate zygosity classification. Given the inherent nature of twin-pair similarity, the TPQ is more accurate in the identification of MZ than DZ pairs. We conclude that the TPQ is a generally accurate, but by no means infallible, method of determining zygosity in twins who have not been genotyped.

Recent work has pioneered the use of expert surveys to estimate cross-national party positions in a common ideological space. In this paper, we report findings from an original dataset designed to evaluate bridging strategies between European and American party placements. Specifically, we compare the use of “anchoring vignettes” (fictional party platforms) with an alternative approach that asks comparativist scholars who live in the US (whom we call transatlantic or TA experts) to place parties and parties in their country of expertise on a series of issues scales. The results provide an optimistic assessment of the ability of TA experts to serve as valid bridges across the Atlantic. The resulting cross-comparable estimates of party positions show instances of both convergence and divergence between American and European party systems, including parallels between systems on the cross-cutting issue of international economic integration.

Objective: Relative to the vast literature that employs measures of decision-making (DM), rigorous examination of their psychometric properties is sparse. This study aimed to determine whether three measures of DM assess the same construct, and to measure invariance of this construct across relevant covariates. Method: Participants were 372 adolescents at risk of escalation in cannabis use. DM was assessed via four indices from the Cups Task, Game of Dice Task (GDT), and Iowa Gambling Task (IGT). We used confirmatory factor analysis to assess unidimensionality of the DM construct, and moderated nonlinear factor analysis (MNLFA) to examine its measurement invariance. Results: The unidimensional model of DM demonstrated good fit. MNLFA results revealed that sex influenced mean DM scores, such that boys had lower risk-taking behaviors. There was evidence of differential item functioning (DIF), such that IQ and age moderated the IGT intercept and GDT factor loading, respectively. Significant effects were retained in the final model, which produced participant-specific DM factor scores. These scores showed moderate stability over time. Conclusions: Indices from three DM tasks loaded significantly onto a single factor, suggesting that these DM tasks assess a single underlying construct. We suggest that this construct represents the ability to make optimal choices that maximize rewards in the presence of risk. Our final DM factor accounts for DIF caused by covariates, making it comparable across adolescents with different characteristics. (JINS, 2019, 25, 661–667)

The limitations for self-reporting of dietary patterns are widely recognised as a major vulnerability of FFQ and the dietary screeners/scales derived from FFQ. Such instruments can yield inconsistent results to produce questionable interpretations. The present article discusses the value of psychometric approaches and standards in addressing these drawbacks for instruments used to estimate dietary habits and nutrient intake. We argue that a FFQ or screener that treats diet as a ‘latent construct’ can be optimised for both internal consistency and the value of the research results. Latent constructs, a foundation for item response theory (IRT)-based scales (e.g. Patient Reported Outcomes Measurement Information System) are typically introduced in the design stage of an instrument to elicit critical factors that cannot be observed or measured directly. We propose an iterative approach that uses such modelling to refine FFQ and similar instruments. To that end, we illustrate the benefits of psychometric modelling by using items and data from a sample of 12 370 Soldiers who completed the 2012 US Army Global Assessment Tool (GAT). We used factor analysis to build the scale incorporating five out of eleven survey items. An IRT-driven assessment of response category properties indicates likely problems in the ordering or wording of several response categories. Group comparisons, examined with differential item functioning (DIF), provided evidence of scale validity across each Army sub-population (sex, service component and officer status). Such an approach holds promise for future FFQ.

Objectives: We examined performance on the Boston Naming Test (BNT) in older and younger adults who were monolingual English or French speakers, or bilingual speakers of English and French (n=215). Methods: Monolingual participants completed the task in their native language, and bilingual participants completed the task in English, French, and bilingual (either-language) administrations. Results: Overall, younger and older monolingual French speakers performed worse than other groups; bilingual participants performed worst in the French administration and approximately two-thirds of bilingual participants performed better when responses were accepted in either language. Surprisingly, however, a subset of bilinguals performed worse when responses were accepted in either language as compared to their maximum score achieved in either English or French. This either-language disadvantage does not appear to be associated with the degree of balanced bilingualism, but instead appears to be related to overall naming abilities. Differential item analysis comparing language groups and the different administrations identified several items that displayed uniform and/or non-uniform differential item functioning (DIF). Conclusions: The BNT does not elicit equivalent performance in English and French, even when assessing naming performance in monolingual French speakers using the French version of the test. Scores were lower in French overall, and several items exhibited DIF. We recommend caution in interpreting performance on these items in bilingual speakers. Finally, not all bilinguals benefit from an either-language administration of the BNT. (JINS, 2015, 21, 350–363)

To date, the statistical software designed for assessing differential item functioning (DIF) with Mantel-Haenszel procedures has employed the following statistics: the Mantel-Haenszel chi-square statistic, the generalized Mantel-Haenszel test and the Mantel test. These statistics permit detecting DIF in dichotomous and polytomous items, although they limit the analysis to two groups. On the contrary, this article describes a new approach (and the related software) that, using the generalized Mantel-Haenszel statistic proposed by Landis, Heyman, and Koch (1978), permits DIF assessment in multiple groups, both for dichotomous and polytomous items. The program is free of charge and is available in the following languages: Spanish, English and Portuguese.

Background: Old age respondents may differ systemically in their responses to measures of psychological distress over and above their actual latent distress levels when compared to younger respondents. The current study aimed to investigate the potential for age-related bias(es) in the Kessler 6 Psychological Distress Scale (K6) items.

Methods: Data from the 2007 Australian National Survey of Mental Health and Wellbeing were analyzed using Item Response Theory to detect the presence of item bias in each of the K6 items. The potential for item bias was assessed by systematically comparing respondents classed as young (16–34 years), middle aged (35–64 years), and old aged (65–85 years). The significance and magnitude of the item bias between the age groups was assessed using the log-likelihood ratio method of differential item functioning.

Results: After statistical adjustment, there were no biases of significant magnitude influencing the endorsement of K6 items between young and middle-aged respondents or between middle-aged and old age respondents. There was a bias of significant magnitude present in the endorsement of the K6 item addressing levels of fatigue between young and old age respondents.

Conclusions: Despite the identification of significant item bias in the endorsement of K6 items between the age groups, the magnitude and influence of the bias on total K6 scores is likely to have little influence on the overall interpretation of group data when comparing psychological distress across the lifespan. Researchers should be cautious, however, when examining individual levels of fatigue related to psychological distress in older individuals.