Hostname: page-component-f554764f5-68cz6 Total loading time: 0 Render date: 2025-04-21T00:07:48.741Z Has data issue: false hasContentIssue false
  • English
  • Français

Efficient Corrections for Standardized Person-Fit Statistics

Published online by Cambridge University Press:  27 December 2024

Kylie Gorney Open the ORCID record for Kylie Gorney [Opens in a new window] ,
Sandip Sinharay  and
Carol Eckerly
Kylie Gorney*
Affiliation:
Michigan State University
Sandip Sinharay
Affiliation:
Educational Testing Service
Carol Eckerly
Affiliation:
Educational Testing Service
*
Correspondence should be made to Kylie Gorney, Department of Counseling, Educational Psychology, and Special Education, Michigan State University, 460 Erickson Hall, 620 Farm Lane, East Lansing, MI 48824, USA. Email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Many popular person-fit statistics belong to the class of standardized person-fit statistics, T, and are assumed to have a standard normal null distribution. However, in practice, this assumption is incorrect since T is computed using (a) an estimated ability parameter and (b) a finite number of items. Snijders (Psychometrika 66(3):331–342, 2001) developed mean and variance corrections for T to account for the use of an estimated ability parameter. Bedrick (Psychometrika 62(2):191–199, 1997) and Molenaar and Hoijtink (Psychometrika 55(1):75–106, 1990) developed skewness corrections for T to account for the use of a finite number of items. In this paper, we combine these two lines of research and propose three new corrections for T that simultaneously account for the use of an estimated ability parameter and the use of a finite number of items. The new corrections are efficient in that they only require the analysis of the original data set and do not require the simulation or analysis of any additional data sets. We conducted a detailed simulation study and found that the new corrections are able to control the Type I error rate while also maintaining reasonable levels of power. A real data example is also included.

Keywords

Person fititem response theoryaberrant behavior
Type
Theory & Methods
Information
Psychometrika , Volume 89 , Issue 2 , June 2024 , pp. 569 - 591
Copyright
Copyright © 2024 The Author(s), under exclusive licence to The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Albers, C. J., Meijer, R. R., Tendeiro, J. N.. (2016). Derivation and applicability of asymptotic results for multiple subtests person-fit statistics. Applied Psychological Measurement, 40(4), 274–288.CrossRefGoogle ScholarPubMed
Bedrick, E. J.. (1997). Approximating the conditional distribution of person fit indexes for checking the Rasch model. Psychometrika, 62(2), 191–199.CrossRefGoogle Scholar
Cheng, Y., Yuan, K.-H.. (2010). The impact of fallible item parameter estimates on latent trait recovery. Psychometrika, 75(2), 280–291.CrossRefGoogle ScholarPubMed
Cizek, G. J., & Wollack, J. A. (Eds.). (2017). Handbook of quantitative methods for detecting cheating on tests. Routledge.Google Scholar
de la Torre, J., Deng, W.. (2008). Improving person-fit assessment by correcting the ability estimate and its reference distribution. Journal of Educational Measurement, 45(2), 159–177.CrossRefGoogle Scholar
Drasgow, F., Levine, M. V., Williams, E. A.. (1985). Appropriateness measurement with polychotomous item response models and standardized indices. British Journal of Mathematical and Statistical Psychology, 38(1), 67–86.CrossRefGoogle Scholar
Glas, C. A. W., Meijer, R. R.. (2003). A Bayesian approach to person fit analysis in item response theory models. Applied Psychological Measurement, 27(3), 217–233.CrossRefGoogle Scholar
Gorney, K., Sinharay, S., Liu, X.. (2024). Using item scores and response times in person-fit assessment. British Journal of Mathematical and Statistical Psychology, 77(1), 151–168.CrossRefGoogle ScholarPubMed
Gorney, K., Wollack, J. A.. (2023). Using item scores and distractors in person-fit assessment. Journal of Educational Measurement, 60(1), 3–27.CrossRefGoogle Scholar
Hong, M., Lin, L., Cheng, Y.. (2021). Asymptotically corrected person fit statistics for multidimensional constructs with simple structure and mixed item types. Psychometrika, 86(2), 464–488.CrossRefGoogle ScholarPubMed
Li, M. F., Olejnik, S.. (1997). The power of Rasch person-fit statistics in detecting unusual response patterns. Applied Psychological Measurement, 21(3), 215–231.CrossRefGoogle Scholar
Magis, D., Béland, S., Raîche, G.. (2014). Snijders’s correction of the infit and outfit indices with estimated ability level: An analysis with the Rasch model. Journal of Applied Measurement, 15(1), 82–93.Google Scholar
Magis, D., Raîche, G., Béland, S.. (2012). A didactic presentation of Snijders’s lz∗\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l_z^*$$\end{document} index of person fit with emphasis on response model selection and ability estimation. Journal of Educational and Behavioral Statistics, 37(1), 57–81.CrossRefGoogle Scholar
Molenaar, I. W., Hoijtink, H.. (1990). The many null distributions of person fit indices. Psychometrika, 55(1), 75–106.CrossRefGoogle Scholar
Nering, M. L.. (1997). The distribution of indexes of person fit within the computerized adaptive testing environment. Applied Psychological Measurement, 21(2), 115–127.CrossRefGoogle Scholar
Noonan, B. W., Boss, M. W., Gessaroli, M. E.. (1992). The effect of test length and IRT model on the distribution and stability of three appropriateness indexes. Applied Psychological Measurement, 16(4), 345–352.CrossRefGoogle Scholar
Reise, S. P.. (1995). Scoring method and the detection of person misfit in a personality assessment context. Applied Psychological Measurement, 19(3), 213–229.CrossRefGoogle Scholar
Santos, K. C. P., de la Torre, J., von Davier, M.. (2020). Adjusting person fit index for skewness in cognitive diagnosis modeling. Journal of Classification, 37(2), 399–420.CrossRefGoogle Scholar
Sinharay, S. (2016a). Assessment of person fit using resampling-based approaches. Journal of Educational Measurement, 53(1), 63–85.CrossRefGoogle Scholar
Sinharay, S. (2016b). Asymptotic corrections of standardized extended caution indices. Applied Psychological Measurement, 40(6), 418–433.CrossRefGoogle Scholar
Sinharay, S. (2016c). Asymptotically correct standardization of person-fit statistics beyond dichotomous items. Psychometrika, 81(4), 992–1013.CrossRefGoogle Scholar
Sinharay, S. (2016d). The choice of the ability estimate with asymptotically correct standardized person-fit statistics. British Journal of Mathematical and Statistical Psychology, 69(2), 175–193.CrossRefGoogle Scholar
Snijders, T. A. B.. (2001). Asymptotic null distribution of person fit statistics with estimated person parameter. Psychometrika, 66(3), 331–342.CrossRefGoogle Scholar
Tatsuoka, K. K.. (1984). Caution indices based on item response theory. Psychometrika, 49(1), 95–110.CrossRefGoogle Scholar
van Krimpen-Stoop, E. M. L. A., Meijer, R. R.. (1999). The null distribution of person-fit statistics for conventional and adaptive tests. Applied Psychological Measurement, 23(4), 327–345.CrossRefGoogle Scholar
van Krimpen-Stoop, E. M. L. A., Meijer, R. R.. (2002). Detection of person misfit in computerized adaptive testing with polytomous items. Applied Psychological Measurement, 26(2), 164–180.CrossRefGoogle Scholar
von Davier, M., Molenaar, I. W.. (2003). A person-fit index for polytomous Rasch models, latent class models, and their mixture generalizations. Psychometrika, 68(2), 213–228.CrossRefGoogle Scholar
Warm, T. A.. (1989). Weighted likelihood estimation of ability in item response theory. Psychometrika, 54(3), 427–450.CrossRefGoogle Scholar