Hostname: page-component-745bb68f8f-v2bm5 Total loading time: 0 Render date: 2025-01-07T18:56:16.788Z Has data issue: false hasContentIssue false
  • English
  • Français

Mapping Unobserved Item–Respondent Interactions: A Latent Space Item Response Model with Interaction Map

Published online by Cambridge University Press:  01 January 2025

Minjeong Jeon Open the ORCID record for Minjeong Jeon [Opens in a new window] ,
Ick Hoon Jin ,
Michael Schweinberger  and
Samuel Baugh
Minjeong Jeon*
Affiliation:
University of California, Los Angeles
Ick Hoon Jin
Affiliation:
Yonsei University
Michael Schweinberger
Affiliation:
Rice University
Samuel Baugh
Affiliation:
University of California, Los Angeles
*
Correspondence should be made to Minjeong Jeon, University of California, Los Angeles, CA90095, USA. Email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Classic item response models assume that all items with the same difficulty have the same response probability among all respondents with the same ability. These assumptions, however, may very well be violated in practice, and it is not straightforward to assess whether these assumptions are violated, because neither the abilities of respondents nor the difficulties of items are observed. An example is an educational assessment where unobserved heterogeneity is present, arising from unobserved variables such as cultural background and upbringing of students, the quality of mentorship and other forms of emotional and professional support received by students, and other unobserved variables that may affect response probabilities. To address such violations of assumptions, we introduce a novel latent space model which assumes that both items and respondents are embedded in an unobserved metric space, with the probability of a correct response decreasing as a function of the distance between the respondent’s and the item’s position in the latent space. The resulting latent space approach provides an interaction map that represents interactions of respondents and items, and helps derive insightful diagnostic information on items as well as respondents. In practice, such interaction maps enable teachers to detect students from underrepresented groups who need more support than other students. We provide empirical evidence to demonstrate the usefulness of the proposed latent space approach, along with simulation results.

Keywords

item response datalatent space modelnetwork modelbipartite networkinteractionsinteraction map
Type
Theory and Methods
Information
Psychometrika , Volume 86 , Issue 2 , June 2021 , pp. 378 - 403
Copyright
Copyright © 2021 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Minjeong Jeon, Ick Hoon Jin, and Michael Schweinberger are co-first authors with equal contribution.

Supplementary Information The online version supplementary material available at https://doi.org/10.1007/s11336-021-09762-5.

References

Agarwal, D., & Chen, B.-C. (2009). Regression-based latent factor models. Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 19-28.CrossRefGoogle Scholar
Bates, D., Mächler, M, Bolker, B, Walker, S(2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67 (1), 1–48CrossRefGoogle Scholar
Bernaards, C.A., Jennrich, R.I.(2005). Gradient projection algorithms and software for arbitrary rotation criteria in factor analysis. Educational and Psychological Measurement, 65, 676–696CrossRefGoogle Scholar
Borsboom, D(2008). Psychometric perspectives on diagnostic systems. Journal of Clinical Psychology, 64 (9), 1089–1108CrossRefGoogle ScholarPubMed
Chalmers, R.P.(2012). mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48 (6), 1–29CrossRefGoogle Scholar
Chen, W.-H., Thissen, D.(1997). Local dependence indexes for item pairs using item response theory. Journal of Educational and Behavioral Statistics, 22, 265–289CrossRefGoogle Scholar
Draney, K.(2007). The Saltus model applied to proportional reasoning data. Journal of Applied Measurement, 8, 438–455Google ScholarPubMed
Epskamp, S., Borsboom, D., Fried, E.I.(2018). Estimating psychological networks and their accuracy: A tutorial paper. Behavior Research Methods, 50, 195–212CrossRefGoogle ScholarPubMed
Fox, J.P., Glas, C.A.(2001). Bayesian estimation of a multilevel IRT model using Gibbs sampling. Psychometrika, 66, 271–288CrossRefGoogle Scholar
Furr, D. C., Lee, S.-Y., Lee, J.-H., & Rabe-Hesketh, S. (2016). Two-parameter logistic item response model - STAN. https://mc-stan.org/users/documentation/case-studies/tutorialtwopl.html.Google Scholar
Gelman, A., Rubin, D.B.(1992). Inference from iterative simulation using multiple sequences. Iterative simulation using multiple sequences. Statistical Science, 7, 457–472CrossRefGoogle Scholar
Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT Press.Google Scholar
Gower, J.C.(1975). Generalized procrustes analysis. Psychometrika, 40, 33–51CrossRefGoogle Scholar
Hoff, P. (2005). Bilinear mixed-effects models for dyadic data. Journal of the American Statistical Association, 286–295.CrossRefGoogle Scholar
Hoff, P. (2020). Additive and multiplicative effects network models. Statistical Science (to appear).Google Scholar
Hoff, P., Raftery, A., Handcock, M.S.(2002). Latent space approaches to social network analysis. Journal of the American Statistical Association, 97, 1090–1098CrossRefGoogle Scholar
Ishwaran, H., Rao, J.S.(2005). Spike and slab variable selection: Frequentist and Bayesian strategies. The Annals of Statistics, 33, 730–773CrossRefGoogle Scholar
Jennrich, R.I.(2002). A simple general method for oblique rotation. Psychometrika, 67, 7–19CrossRefGoogle Scholar
Jin, I.H., Jeon, M.(2019). A doubly latent space joint model for local item and person dependence in the analysis of item response data. Psychometrika, 84, 236–260CrossRefGoogle ScholarPubMed
Jin, I. H., Jeon, M., Schweinberger, M., & Lin, L. (2018). Hierarchical network item response modeling for discovering differences between innovation and regular school systems in Korea. Available at. arxiv.org/abs/1810.07876.Google Scholar
Lauritzen, S. (1996). Graphical models, Oxford: Oxford University PressCrossRefGoogle Scholar
Markovits, H., Fleury, M-L, Quinn, S., Venet, M.(1998). The development of conditional reasoning and the structure of semantic memory. Child Development, 69, 742–755CrossRefGoogle ScholarPubMed
Marsman, M., Borsboom, D., Kruis, J., Epskamp, S., van Bork, R., Waldorp, L.J., Maris, GKJ(2018). An introduction to network psychometrics: Relating ising network models to item response theory models. Multivariate Behavioral Research, 53, 15–35CrossRefGoogle ScholarPubMed
McCullagh, P., Nelder, J.A. (1983). Generalized linear models, London: Chapman & HallCrossRefGoogle Scholar
Pearl, J. (1988). Probabilistic reasoning in intelligent systems: Networks of plausible inference, San Francisco: Morgan KaufmannGoogle Scholar
Rasch, G. (1961). On general laws and meaning of measurement in psychology. In Proceedings of the fourth Berkeley symposium on mathematical statistics and probability (volume 4) (pp. 321–333).Google Scholar
Revelle, W. (2019). psych: Procedures for psychological, psychometric, and personality research [Computer software manual]. Evanston, Illinois. Retrieved from https://CRAN.R-project.org/package=psych(Rpackageversion1.9.12).Google Scholar
Rost, J.(1990). Rasch models in latent classes: An integration of two approaches to item analysis. Applied Psychological Measurement, 14, 271–282CrossRefGoogle Scholar
Schweinberger, M., Snijders, TABStolzenberg, R.M.(2003). Settings in social networks: A measurement model. Sociological methodology, Boston & Oxford: Basil Blackwell 307–341Google Scholar
Sewell, D.K., Chen, Y.(2015). Latent space models for dynamic networks. Journal of the American Statistical Association, 110, 1646–1657CrossRefGoogle Scholar
Skrondal, A., Rabe-Hesketh, S. (2004). Generalized latent variable modeling: Multilevel, longitudinal, and structural equation models, Boca Raton, FL: Chapman & Hall/CRCCrossRefGoogle Scholar
Smith, A.L., Asta, D.M., Calder, C.A.(2019). The geometry of continuous latent space models for network data. Statistical Science, 34, 428–453CrossRefGoogle ScholarPubMed
Social and community planning research. (1987). British social attitude, the 1987 report. Aldershot: Gower Publishing.Google Scholar
Spiel, C., Gluck, J.Hartig, J., Klieme, E., Leutner, D.(2008). A model based test of competence profile and competence level in deductive reasoning. Assessment of competencies in educational contexts: State of the art and future prospects, Gottingen: Hogrefe 41–60Google Scholar
Spiel, C., Gluck, J., Gossler, H.(2001). Stability and change of unidimensionality: The sample case of deductive reasoning. Journal of Adolescent Research, 16, 150–168CrossRefGoogle Scholar
Wainer, H., Kiely, G.L.(1987). Item clusters and computerized adaptive testing: A case for testlets. Journal of Educational Measurement, 24, 185–201CrossRefGoogle Scholar
Wasserman, S., Faust, K. (1994). Social network analysis: Methods and applications, Cambridge: Cambridge University PressCrossRefGoogle Scholar
Wilson, M., Adams, R.J.(1995). Rasch models for item bundles. Psychometrika, 60, 181–198CrossRefGoogle Scholar
Supplementary material: File

Jeon et al. supplementary material

Appendix A-G
Download Jeon et al. supplementary material(File)
File 669.8 KB