Hostname: page-component-5f745c7db-nzk4m Total loading time: 0 Render date: 2025-01-06T06:08:52.012Z Has data issue: true hasContentIssue false
  • English
  • Français

A New Explanation and Proof of the Paradoxical Scoring Results in Multidimensional Item Response Models

Published online by Cambridge University Press:  01 January 2025

Pascal Jordan  and
Martin Spiess
Pascal Jordan*
Affiliation:
University of Hamburg
Martin Spiess
Affiliation:
University of Hamburg
*
Correspondence should be made to Pascal Jordan, University of Hamburg, Von-Melle-Park 5, 20146 Hamburg, Germany. Email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

In multidimensional item response models, paradoxical scoring effects can arise, wherein correct answers are penalized and incorrect answers are rewarded. For the most prominent class of IRT models, the class of linearly compensatory models, a general derivation of paradoxical scoring effects based on the geometry of item discrimination vectors is given, which furthermore corrects an error in an established theorem on paradoxical results. This approach highlights the very counterintuitive way in which item discrimination parameters (and also factor loadings) have to be interpreted in terms of their influence on the latent ability estimate. It is proven that, despite the error in the original proof, the key result concerning the existence of paradoxical effects remains true—although the actual relation to the item parameters is shown to be a more complicated function than previous results suggested. The new proof enables further insights into the actual mathematical causation of the paradox and generalizes the findings within the class of linearly compensatory models.

Keywords

paradoxical resulttest fairnesscompensatory modelmultidimensional item response theoryperson parameter estimationhyperplanes
Type
Original Paper
Information
Psychometrika , Volume 83 , Issue 4 , December 2018 , pp. 831 - 846
Copyright
Copyright © 2017 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Dontchev, A. L., &Rockafellar, R. T.(2009).Implicit functions and solution mappings: A view from variational analysis.Berlin:SpringerGoogle Scholar
Finkelman, M. D.,Hooker, G., &Wang, Z.(2010).Prevalence and magnitude of paradoxical results in multidimensional item response theory.Journal of Educational and Behavioral Statistics,35 (6),744–761.Google Scholar
Hooker, G.(2010).On separable tests, correlated priors, and paradoxical results in multidimensional item response theory.Psychometrika,75 (4),694–707.CrossRefGoogle Scholar
Hooker, G.,Finkelman, M.,&Schwartzman, A.(2009).Paradoxical results in multidimensional item response theory.Psychometrika,74 (3),419–442.Google Scholar
Jordan, P., &Spiess, M.(2012).Generalizations of paradoxical results in multidimensional item response theory.Psychometrika,77,127–152.CrossRefGoogle Scholar
Karlin, S., &Rinott, Y.(1980).Classes of orderings of measures and related correlation inequalities II. Multivariate reverse rule distributions.Journal of Multivariate Analysis,10 (4),499–516.CrossRefGoogle Scholar
Lax, P.(2007).Linear algebra,New York:WileyGoogle Scholar
Mardia, K. V.,Kent, J. T., &Bibby, J. M.(1979).Multivariate analysis,London:Academic PressGoogle Scholar
Puntanen, S.,Styan, G. P., &Isotalo, J.(2011).Matrix tricks for linear statistical models: Our personal top twenty,New York:SpringerCrossRefGoogle Scholar
Reckase, Mark D., &Luo, Xin(2015).A Paradox by Another Name Is Good Estimation.Millsap, R.,Bolt, D.,van der Ark, L., &Wang, W. C.,Quantitative Psychology Research.Springer Proceedings in Mathematics & Statistics(89).Cham:SpringerCrossRefGoogle Scholar
Reckase, M.(2009).Multidimensional item response theory,New York:SpringerGoogle Scholar
Rockafellar, R. T.(1970).Convex analysis,Princeton:Princeton University PressCrossRefGoogle Scholar
Samejima, F.(1974).Normal ogive model on the continuous response level in the multidimensional latent space.Psychometrika,39 (1),111–121.CrossRefGoogle Scholar
van der Linden, W. J.(2012).On compensation in multidimensional response modeling.Psychometrika,77,21–30.CrossRefGoogle Scholar
van Rijn, P. W., &Rijmen, F.(2012).A note on explaining away and paradoxical results in multidimensional item response theory (ETS No. RR-12-13),Princeton:Educational Testing ServiceGoogle Scholar