Hostname: page-component-745bb68f8f-b95js Total loading time: 0 Render date: 2025-01-07T17:39:07.214Z Has data issue: false hasContentIssue false
  • English
  • Français

A Two-Tier Full-Information Item Factor Analysis Model with Applications

Published online by Cambridge University Press:  01 January 2025

Li Cai
Li Cai*
Affiliation:
University of California, Los Angeles
*
Requests for reprints should be sent to Li Cai, University of California, Education and Psychology, Los Angeles, CA, 90095-1521, USA. E-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Motivated by Gibbons et al.’s (Appl. Psychol. Meas. 31:4–19, 2007) full-information maximum marginal likelihood item bifactor analysis for polytomous data, and Rijmen, Vansteelandt, and De Boeck’s (Psychometrika 73:167–182, 2008) work on constructing computationally efficient estimation algorithms for latent variable models, a two-tier item factor analysis model is developed in this research. The modeling framework subsumes standard multidimensional IRT models, bifactor IRT models, and testlet response theory models as special cases. Features of the model lead to a reduction in the dimensionality of the latent variable space, and consequently significant computational savings. An EM algorithm for full-information maximum marginal likelihood estimation is developed. Simulations and real data demonstrations confirm the accuracy and efficiency of the proposed methods. Three real data sets from a large-scale educational assessment, a longitudinal public health survey, and a scale development study measuring patient reported quality of life outcomes are analyzed as illustrations of the model’s broad range of applicability.

Keywords

multidimensional item response theoryfactor analysisbifactor modeltestlet response modelpatient reported outcomeslatent variable modeling
Type
Original Paper
Information
Psychometrika , Volume 75 , Issue 4 , December 2010 , pp. 581 - 612
Copyright
Copyright © 2010 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Part of this research is made possible by a pre-doctoral advanced quantitative methodology training grant (R305B080016) from the Institute of Education Sciences, a statistical methodology grant from the Institute of Education Sciences (R305D100039), and a grant from the National Institute on Drug Abuse (R01DA026943). The author is enormously grateful to Drs. Darren DeWalt, Susan Ennett, Robert Gibbons, Anthony Lehman, and David Thissen for their permission to use the data sets for numerical illustrations. Data collection for the Context project was supported by a grant from the National Institute on Drug Abuse (R01DA13459). The development of the Pediatric Asthma Impact Scale was funded by National Institute of Arthritis And Musculoskeletal and Skin Diseases (1U01AR052181-01). The development of IRTPRO was supported by the National Cancer Institute in the form of an SBIR contract (#HHSN-2612007-00013C) awarded to Scientific Software International. The views expressed in this paper are the author’s alone and do not reflect the views and policies of the funding agencies or grantees.

References

Adams, R., Wu, M. (2002). PISA 2000 technical report, Paris: Organization for Economic Cooperation and Development.Google Scholar
Baker, F.B., Kim, S.H. (2004). Item response theory: Parameter estimation techniques, (2nd ed.). New York: Marcel Dekker.CrossRefGoogle Scholar
Bartholomew, D.J., Tzamourani, P. (1999). The goodness-of-fit of latent trait models in attitude measurement. Sociological Methods and Research, 27, 525–546.CrossRefGoogle Scholar
Bishop, Y.M.M., Fienberg, S.E., Holland, P.W. (1975). Discrete multivariate analysis: theory and practice, Cambridge: MIT Press.Google Scholar
Bock, R.D., Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46, 443–459.CrossRefGoogle Scholar
Bock, R.D., Gibbons, R., Muraki, E. (1988). Full-information item factor analysis. Applied Psychological Measurement, 12, 261–280.CrossRefGoogle Scholar
Bock, R.D., Lieberman, M. (1970). Fitting a response model for n dichotomously scored items. Psychometrika, 35, 179–197.CrossRefGoogle Scholar
Bollen, K.A. (1989). Structural equations with latent variables, New York: Wiley.CrossRefGoogle Scholar
Bolt, D. (2005). Limited and full information estimation of item response theory models. In Maydeu-Olivares, A., McArdle, J.J. (Eds.), Contemporary psychometrics (pp. 27–71). Mahwah: Earlbaum.Google Scholar
Bradlow, E.T., Wainer, H., Wang, X. (1999). A Bayesian random effects model for testlets. Psychometrika, 64, 153–168.CrossRefGoogle Scholar
Braeken, J., Tuerlinckx, F., De Boeck, P. (2007). Copula functions for residual dependency. Psychometrika, 72, 393–411.CrossRefGoogle Scholar
Cai, L. (2008a). A Metropolis-Hastings Robbins-Monro algorithm for maximum likelihood nonlinear latent structure analysis with a comprehensive measurement model. Unpublished doctoral dissertation, Department of Psychology, University of North Carolina.Google Scholar
Cai, L. (2008). SEM of another flavour: two new applications of the supplemented EM algorithm. British Journal of Mathematical and Statistical Psychology, 61, 309–329.CrossRefGoogle ScholarPubMed
Cai, L. (2010). High-dimensional exploratory item factor analysis by a Metropolis-Hastings Robbins-Monro algorithm. Psychometrika, 7, 33–57.CrossRefGoogle Scholar
Cai, L. (in press). Metropolis-Hastings Robbins-Monro algorithm for confirmatory item factor analysis. Journal of Educational and Behavioral Statistics.Google Scholar
Cai, L., du Toit, S.H.C., & Thissen, D. (in press). IRTPRO: Flexible, multidimensional, multiple categorical IRT modeling. Chicago: Scientific Software Internatonal, Inc. Computer software.Google Scholar
Cai, L., Maydeu-Olivares, A., Coffman, D.L., Thissen, D. (2006). Limited-information goodness-of-fit testing of item response theory models for sparse 2p tables. British Journal of Mathematical and Statistical Psychology, 59, 173–194.CrossRefGoogle ScholarPubMed
Chen, W.H., Thissen, D. (1997). Local dependence indices for item pairs using item response theory. Journal of Educational and Behavioral Statistics, 22, 265–289.CrossRefGoogle Scholar
Cudeck, R., Harring, J.R., du Toit, S.H.C. (2009). Marginal maximum likelihood estimation of a latent variable model with interaction. Journal of Educational and Behavioral Statistics, 34, 131–144.CrossRefGoogle Scholar
de Boeck, P., Wilson, M. (2004). Explanatory item response models: A generalized linear and nonlinear approach, New York: Springer.CrossRefGoogle Scholar
de la Torre, J., Patz, R.J. (2005). Making the most of what we have: A practical application of multidimensional item response theory in test scoring. Journal of Educational and Behavioral Statistics, 30, 295–311.CrossRefGoogle Scholar
Edwards, M.C. (in press). A Markov chain Monte Carlo approach to confirmatory item factor analysis. Psychometrika. doi: 10.1007/S11336-010-9161-9.CrossRefGoogle Scholar
Ennett, S.T., Foshee, V.A., Bauman, K.E., Hussong, A.M., Cai, L., Luz, H.et al. (2008). The social ecology of adolescent alcohol misuse. Child Development, 79, 1777–1791.CrossRefGoogle ScholarPubMed
Gibbons, R.D., Bock, R.D., Hedeker, D., Weiss, D.J., Segawa, E., Bhaumik, D.K.et al. (2007). Full-information item bifactor analysis of graded response data. Applied Psychological Measurement, 31, 4–19.CrossRefGoogle Scholar
Gibbons, R.D., Grochocinski, V.J., Weiss, D.J., Bhaumik, D.K., Kupfer, D.J., Stover, A.et al. (2008). Using computerized adaptive testing to reduce the burden of mental health assessment. Psychiatric Services, 59, 361–368.CrossRefGoogle ScholarPubMed
Gibbons, R.D., Hedeker, D. (1992). Full-information item bifactor analysis. Psychometrika, 57, 423–436.CrossRefGoogle Scholar
Gibbons, R.D., Hedeker, D. (2007). Bifactor, Chicago: Center for Health Statistics, University of Illinois at Chicago. Computer softwareGoogle Scholar
Glas, C.A.W., Wainer, H., Bradlow, E.T. (2000). Maximum marginal likelihood and expected a posteriori estimation in testlet-based adaptive testing. In van der Linden, W.J., Glas, C.A.W. (Eds.), Computerized adaptive testing: Theory and practice (pp. 271–288). Boston: Kluwer Academic.CrossRefGoogle Scholar
Haberman, S.J. (1977). Log-linear models and frequency tables with small expected cell counts. The Annals of Statistics, 5, 1148–1169.CrossRefGoogle Scholar
Hill, C.D. (2006). Two models for longitudinal item response data. Unpublished doctoral dissertation, Department of Psychology, University of North Carolina at Chapel Hill.Google Scholar
Jöreskog, K.G., Sörbom, D. (2001). LISREL user’s guide, Chicago: Scientific Software Internatonal, Inc.Google Scholar
Lehman, A.F. (1988). A quality of life interview for the chronically mentally ill. Evaluation and Program Planning, 11, 51–62.CrossRefGoogle Scholar
Li, Y., Bolt, D.M., Fu, J. (2006). A comparison of alternative models for testlets. Applied Psychological Measurement, 30, 3–21.CrossRefGoogle Scholar
Lord, F.M., Novick, M.R. (1968). Statistical theories of mental test scores, Reading: Addison-Wesley.Google Scholar
Mardia, K.V., Kent, J.T., Bibby, J.M. (1979). Multivariate analysis, London: Academic Press.Google Scholar
Maydeu-Olivares, A., Cai, L. (2006). A cautionary note on using G 2(dif) to assess relative model fit in categorical data analysis. Multivariate Behavioral Research, 41, 55–64.CrossRefGoogle Scholar
Maydeu-Olivares, A., Coffman, D.L. (2006). Random intercept item factor analysis. Psychological Methods, 11, 344–362.CrossRefGoogle ScholarPubMed
Maydeu-Olivares, A., Joe, H. (2005). Limited and full information estimation and testing in 2n contingency tables: A unified framework. Journal of the American Statistical Association, 100, 1009–1020.CrossRefGoogle Scholar
Maydeu-Olivares, A., Joe, H. (2006). Limited information goodness-of-fit testing in multidimensional contingency tables. Psychometrika, 71, 713–732.CrossRefGoogle Scholar
Moustaki, I. (2007). Factor analysis and latent structure of categorical and metric data. In Cudeck, R., Maccallum, R.C. (Eds.), Factor analysis at 100: Historical developments and future directions, Mahwah: Laurence Erlbaum Associates.Google Scholar
Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16, 159–176.CrossRefGoogle Scholar
Muraki, E., Carlson, J.E. (1995). Full-information factor analysis for polytomous item responses. Applied Psychological Measurement, 19, 73–90.CrossRefGoogle Scholar
Muthén, & Muthén (2008). Mplus (Version 5.0). Los Angeles: Author. Computer software.Google Scholar
Orlando, M., Thissen, D. (2000). Likelihood-based Item-Fit indices for dichotomous item response theory models. Applied Psychological Measurement, 24, 50–64.CrossRefGoogle Scholar
Reckase, M.D. (2009). Multidimentional item response theory, New York: Springer.CrossRefGoogle Scholar
Reeve, B.B., Hays, R.D., Bjorner, J.B., Cook, K.F., Crane, P.K., Teresi, J.A.et al. (2007). Psychometric evaluation and calibration of health-related quality of life items banks: Plans for the patient-reported outcome measurement information system (PROMIS). Medical Care, 45, 22–31.CrossRefGoogle Scholar
Reise, S.P., Morizot, J., Hays, R.D. (2007). The role of the bifactor model in resolving dimensionality issues in health outcomes measures. Quality of Life Research, 16, 19–31.CrossRefGoogle ScholarPubMed
Rijmen, F. (2009a). Efficient full information maximum likelihood estimation for multidimensional IRT models (Tech. Rep. No. RR-09-03). Educational Testing Service.CrossRefGoogle Scholar
Rijmen, F. (2009b). A hierarchical factor IRT model for items that are clustered at multiple levels. Paper presented at the 2009 international meeting of the Psychometric Society. Cambridge, UK.Google Scholar
Rijmen, F. (2009c). Three Multidimensional models for testlet-based tests: formal relations and an empirical comparison (Tech. Rep. No. RR-09-37). Educational Testing Service.CrossRefGoogle Scholar
Rijmen, F. (in press-a). Formal relations and an empirical comparison between the bi-factor, the testlet, and a second-order multidimensional IRT model. Journal of Educational Measurement.Google Scholar
Rijmen, F. (in press-b). The use of graphs in latent variable modeling: Beyond visualization. In G.R. Hancock & G.B. Macready (Eds.), Advances in latent class analysis: A Festshrift in honor of C. Mitchell Dayton.Google Scholar
Rijmen, F., Vansteelandt, K., De Boeck, P. (2008). Latent class models for diary method data: Parameter estimation by local computations. Psychometrika, 73, 167–182.CrossRefGoogle ScholarPubMed
Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometric Monographs, 17.CrossRefGoogle Scholar
Schilling, S., Bock, R.D. (2005). High-dimensional maximum marginal likelihood item factor analysis by adaptive quadrature. Psychometrika, 70, 533–555.Google Scholar
Stout, W. (1990). A new item response theory modeling approach with application to unidimensional assessment and ability estimation. Psychometrika, 55, 293–325.CrossRefGoogle Scholar
te Marvelde, J., Glas, V.G.C., van Damme, J. (2006). Application of multidimensional item response theory models to longitudinal data. Educational and Psychological Measurement, 66, 5–34.CrossRefGoogle Scholar
Thissen, D. (1982). Marginal maximum likelihood estimation for the one-parameter logistic model. Psychometrika, 47, 175–186.CrossRefGoogle Scholar
Thissen, D. (2003). MULTILOG 7 user’s guide, Chicago: Scientific Software Internatonal, Inc.Google Scholar
Thissen, D., Cai, L., & Bock, R.D. (in press). The nominal categories item response model. In M. Nering & R. Ostini (Eds.), Handbook of polytomous item response theory models: developments and applications. New York: Taylor & Francis.Google Scholar
Thissen, D., Steinberg, L. (2010). Using item response theory to disentangle constructs at different levels of generality. In Embretson, S. (Eds.), Measuring psychological constructs: Advances in model-based approaches (pp. 123–144). Washington: American Psychological Association.CrossRefGoogle Scholar
Thissen, D., Steinberg, L., Mooney, J.A. (1989). Trace lines for testlets: A use of multiple-categorical-response models. Journal of Educational Measurement, 26, 247–260.CrossRefGoogle Scholar
Thissen, D., Wainer, H. (2001). Test scoring, Mahwah: Lawrence Erlbaum Associates.CrossRefGoogle Scholar
Tisak, J., Meredith, W. (1989). Exploratory longitudinal factor analysis in multiple populations. Psychometrika, 54, 261–281.CrossRefGoogle Scholar
Wainer, H., Bradlow, E.T., Wang, X. (2007). Testlet response theory and its applications, New York: Cambridge University Press.CrossRefGoogle Scholar
Wirth, R.J., Edwards, M.C. (2007). Item factor analysis: Current approaches and future directions. Psychological Methods, 12, 58–79.CrossRefGoogle ScholarPubMed
Woods, C.M., Thissen, D. (2006). Item response theory with estimation of the latent population distribution using spline-based densities. Psychometrika, 71, 281–301.CrossRefGoogle ScholarPubMed
Yeatts, K., Stucky, B.D., Thissen, D., Irwin, D., Varni, J., DeWitt, E.M.et al. (2010). Construction of the Pediatric Asthma Impact Scale (PAIS) for the patient-reported outcome measurement information system (PROMIS). Journal of Asthma, 47, 295–302.CrossRefGoogle ScholarPubMed
Yung, Y.F., McLeod, L.D., Thissen, D. (1999). On the relationship between the higher-order factor model and the hierarchical factor model. Psychometrika, 64, 113–128.CrossRefGoogle Scholar