It is well-known that the representations of the Thurstonian Case III and Case V models for paired comparison data are not unique. Similarly, when analyzing ranking data, other equivalent covariance structures can substitute for those given by Thurstone in these cases. That is, we may more broadly define the family of covariance structures satisfying Case III and Case V conditions. This paper introduces the notion of equivalence classes which defines a more meaningful partition of the covariance structures of the Thurstonian ranking models. In addition, the equivalence classes of Case V and Case III are completely characterized.

This paper assesses the psychometric value of allowing test-takers choice in standardized testing. New theoretical results examine the conditions where allowing choice improves score precision. A hierarchical framework is presented for jointly modeling the accuracy of cognitive responses and item choices. The statistical methodology is disseminated in the ‘cIRT’ R package. An ‘answer two, choose one’ (A2C1) test administration design is introduced to avoid challenges associated with nonignorable missing data. Experimental results suggest that the A2C1 design and payout structure encouraged subjects to choose items consistent with their cognitive trait levels. Substantively, the experimental data suggest that item choices yielded comparable information and discrimination ability as cognitive items. Given there are no clear guidelines for writing more or less discriminating items, one practical implication is that choice can serve as a mechanism to improve score precision.

This paper presents two probabilistic models based on the logistic and the normal distribution for the analysis of dependencies in individual paired comparison judgments. It is argued that a core assumption of latent class choice models, independence of individual decisions, may not be well-suited for the analysis of paired comparison data. Instead, the analysis and interpretation of paired comparison data may be much simplified by allowing for within-person dependencies that result from repeated evaluations of the same options in different pairs. Moreover, by relating dependencies among the individual-level responses to (in)consistencies in the judgmental process, we show that the proposed graded paired comparison models reduce to ranking models under certain conditions. Three applications are presented to illustrate the approach.

The recent development of probabilistic Thurstonian choice models for analyzing preference data has been motivated by the need to describe both inter- and intra-individual difference, the multidimensional nature of choice objects, and the effects of similarity and comparability among choice objects. A common feature of these models is that they focus on a single preference judgment. It is customary, however, to ask subjects not only for an overall preference judgment but also for additional paired comparison responses regarding specific attributes. This paper proposes a generalization of Thurstonian probabilistic choice models for analyzing both multiple preference responses and their relationships. The approach is illustrated by modeling data from two multivariate preference experiments.