Hostname: page-component-745bb68f8f-g4j75 Total loading time: 0 Render date: 2025-01-08T12:00:29.423Z Has data issue: false hasContentIssue false
  • English
  • Français

Asymmetric Item Characteristic Curves and Item Complexity: Insights from Simulation and Real Data Analyses

Published online by Cambridge University Press:  01 January 2025

Sora Lee Open the ORCID record for Sora Lee [Opens in a new window]  and
Daniel M. Bolt
Sora Lee*
Affiliation:
University of Wisconsin
Daniel M. Bolt
Affiliation:
University of Wisconsin
*
Correspondence should be made to Sora Lee, University of Wisconsin, Madison, WI, USA. Email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

While item complexity is often considered as an item feature in test development, it is much less frequently attended to in the psychometric modeling of test items. Prior work suggests that item complexity may manifest through asymmetry in item characteristics curves (ICCs; Samejima in Psychometrika 65:319–335, 2000). In the current paper, we study the potential for asymmetric IRT models to inform empirically about underlying item complexity, and thus the potential value of asymmetric models as tools for item validation. Both simulation and real data studies are presented. Some psychometric consequences of ignoring asymmetry, as well as potential strategies for more effective estimation of asymmetry, are considered in discussion.

Keywords

item response theoryasymmetric item characteristic curvesitem complexityitem validation
Type
Original Paper
Information
Psychometrika , Volume 83 , Issue 2 , June 2018 , pp. 453 - 475
Copyright
Copyright © 2017 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Electronic supplementary material The online version of this article (doi:10.1007/s11336-017-9586-5) contains supplementary material, which is available to authorized users.

References

Baker, F. B., & Kim, S. H., (Eds.). (2004). Item response theory: Parameter estimation techniques. CRC Press.CrossRefGoogle Scholar
Bazán, J.L., Branco, M.D., Bolfarine, H., (2006). A skew item response model, Bayesian Analysis, 1(4), 861–892.CrossRefGoogle Scholar
Bolfarine, H., Bazán, J.L., (2010). Bayesian estimation of the logistic positive exponent IRT model, Journal of Educational and Behavioral Statistics, 35, 693–713.CrossRefGoogle Scholar
Bolt, D.M., Deng, S., Lee, S., (2014). IRT model misspecification and measurement of growth in vertical scaling, Journal of Educational Measurement, 51(2), 141–162.CrossRefGoogle Scholar
Bolt, D.M., Kim, J-S, Blanton, M., Knuth, E., (2016). Applications of item response theory in mathematics education research. In Izsák, A., Remillard, J.T., Templin, J.(Eds.), Psychometric methods in mathematics education: Opportunities, challenges, and interdisciplinary collaborations. Journal for research in mathematics education monograph series. Reston, VA:National Council of Teachers of Mathematics.Google Scholar
Bolt, D.M., Lall, V.F., (2003). Estimation of compensatory and noncompensatory multidimensional item response models using Markov chain Monte Carlo, Applied Psychological Measurement, 27(6), 395–414.CrossRefGoogle Scholar
De Ayala, R.J., (2013). The theory and practice of item response theory. New York:Guilford Publications.Google Scholar
De Boeck, P., Wilson, M., (2004). Explanatory item response models: A generalized linear and nonlinear approach. New York:Springer.CrossRefGoogle Scholar
Embretson, S., (1984). A general latent trait model for response processes, Psychometrika, 49(2), 175–186.CrossRefGoogle Scholar
Falk, C.F., Cai, L., (2016). Semi-parametric item response functions in the context of guessing, Journal of Educational Measurement, 53, 229–247.CrossRefGoogle Scholar
Hess, K., (2006). Exploring cognitive demand in instruction and assessment. Dover, NH: National Center for the Improvement of Educational Assessment. Retrieved from http://www.nciea.org/publications/DOK_ApplyingWebb_KH08.pdf.Google Scholar
Jannarone, R.J., (1986). Conjunctive item response theory kernels, Psychometrika, 51, 357–373.CrossRefGoogle Scholar
Junker, B.W., Sijtsma, K., (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory, Applied Psychological Measurement, 25(3), 258–272.CrossRefGoogle Scholar
Lee, S., (2015). A comparison of methods for recovery of asymmetric item characteristic curves in item response theory. Unpublished masters thesis. University of Wisconsin, Madison.Google Scholar
Maris, E., (1995). Psychometric latent response models, Psychometrika, 60(4), 523–547.CrossRefGoogle Scholar
McDonald, R. P., (1967). Nonlinear factor analysis (Psychometric Monograph No. 15). Richmond, VA: Psychometric Corporation. Retrieved from http://www.psychometrika.org/journal/online/MN15.pdf.Google Scholar
Molenaar, D., (2014). Heteroscedastic latent trait models for dichotomous data, Psychometrika, 80(3), 625–644.CrossRefGoogle ScholarPubMed
Molenaar, D., Dolan, C.V., de Boeck, P., (2012). The heteroscedastic graded response model with a skewed latent trait: Testing statistical and substantive hypotheses related to skewed item category functions, Psychometrika, 77(3), 455–478.CrossRefGoogle ScholarPubMed
Neidorf, T. S., Binkley, M., Gattis, K., & Nohara, D., (2006). Comparing mathematics content in the National Assessment of Educational Progress (NAEP), Trends in International Mathematics and Science Study (TIMSS), and Program for International Student Assessment (PISA) 2003 assessments. Technical Report, Institute of Education Sciences, NCES 2006-029.Google Scholar
Ramsay, J. O. (2000). TESTGRAF: A computer program for nonparametric analysis of testing data. Unpublished manuscript, McGill University. ftp://ego.psych.mcgill.ca/pub/ramsay/testgraf.Google Scholar
Reckase, M.D., (1985). The difficulty of test items that measure more than one ability, Applied Psychological Measurement, 9, 401–412.CrossRefGoogle Scholar
Rizopoulos, D., (2006). ltm: An R package for latent variable modeling and item response theory analyses, Journal of Statistical Software, 17(5), 1–25.CrossRefGoogle Scholar
Samejima, F., (1995). Acceleration model in the heterogeneous case of the general graded response model, Psychometrika, 60(4), 549–572.CrossRefGoogle Scholar
Samejima, F., (2000). Logistic positive exponent family of models: Virtue of asymmetric item characteristic curves, Psychometrika, 65, 319–335.CrossRefGoogle Scholar
San Martín, E., Del Pino, G., De Boeck, P., (2006). IRT models for ability-based guessing, Applied Psychological Measurement, 30(3), 183–203.CrossRefGoogle Scholar
Sympson, J. B., (1978). A model for testing with multidimensional items. In Weiss, D. J. (Ed.), Proceedings of the 1977 Computerized Adaptive Testing Conference (pp. 82–98). Minneapolis: University of Minnesota, Department of Psychology, Psychometric Methods Program.Google Scholar
Tuerlinckx, F., De Boeck, P., (2001). The effect of ignoring item interactions on the estimated discrimination parameters in item response theory, Psychological Methods, 6(2), 181–195.CrossRefGoogle ScholarPubMed
van der Linden, W.J., (2007). A hierarchical framework for modeling speed and accuracy on test items, Psychometrika, 72(3), 287–308.CrossRefGoogle Scholar
Webb, N.L., (2002). Depth-of-knowledge levels for four content areas. Madison:University of Wisconsin Center for Educational Research.Google Scholar
Supplementary material: File

Lee and Bolt supplementary material

Lee and Bolt supplementary material 1
Download Lee and Bolt supplementary material(File)
File 31.2 KB
Supplementary material: File

Lee and Bolt supplementary material

Lee and Bolt supplementary material 2
Download Lee and Bolt supplementary material(File)
File 96.3 KB
Supplementary material: File

Lee and Bolt supplementary material

Lee and Bolt supplementary material 3
Download Lee and Bolt supplementary material(File)
File 16.1 KB