Hostname: page-component-cd9895bd7-gbm5v Total loading time: 0 Render date: 2025-01-05T03:10:15.407Z Has data issue: false hasContentIssue false
  • English
  • Français

A Tensor-EM Method for Large-Scale Latent Class Analysis with Binary Responses

Published online by Cambridge University Press:  01 January 2025

Zhenghao Zeng ,
Yuqi Gu  and
Gongjun Xu Open the ORCID record for Gongjun Xu [Opens in a new window]
Zhenghao Zeng
Affiliation:
Carnegie Mellon University
Yuqi Gu
Affiliation:
Columbia University
Gongjun Xu*
Affiliation:
University of Michigan
*
Correspondence should be made to Gongjun Xu, Department of Statistics, University of Michigan, 456 West Hall,1085 South University, Ann Arbor, MI 48109, USA. Email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Latent class models are powerful statistical modeling tools widely used in psychological, behavioral, and social sciences. In the modern era of data science, researchers often have access to response data collected from large-scale surveys or assessments, featuring many items (large J) and many subjects (large N). This is in contrary to the traditional regime with fixed J and large N. To analyze such large-scale data, it is important to develop methods that are both computationally efficient and theoretically valid. In terms of computation, the conventional EM algorithm for latent class models tends to have a slow algorithmic convergence rate for large-scale data and may converge to some local optima instead of the maximum likelihood estimator (MLE). Motivated by this, we introduce the tensor decomposition perspective into latent class analysis with binary responses. Methodologically, we propose to use a moment-based tensor power method in the first step and then use the obtained estimates as initialization for the EM algorithm in the second step. Theoretically, we establish the clustering consistency of the MLE in assigning subjects into latent classes when N and J both go to infinity. Simulation studies suggest that the proposed tensor-EM pipeline enjoys both good accuracy and computational efficiency for large-scale data with binary responses. We also apply the proposed method to an educational assessment dataset as an illustration.

Keywords

large-scale latent class analysistensor decompositiontensor power methodEM algorithmclustering consistency
Type
Theory and Methods
Information
Psychometrika , Volume 88 , Issue 2 , June 2023 , pp. 580 - 612
Copyright
Copyright © 2022 The Author(s) under exclusive licence to The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Supplementary Information The online version contains supplementary material available at https://doi.org/10.1007/s11336-022-09887-1.

References

Allman, E. S.,Matias, C., &Rhodes, J. A.(2009).Identifiability of parameters in latent structure models with many observed variables.The Annals of Statistics,37(6A),3099–3132.CrossRefGoogle Scholar
Anandkumar, A., Foster, D. P., Hsu, D. J., Kakade, S. M., & Liu, Y.-K. (2012a). A spectral algorithm for latent dirichlet allocation. In Advances in neural information processing systems (pp. 917–925).Google Scholar
Anandkumar, A.,Ge, R.,Hsu, D.,Kakade, S. M., &Telgarsky, M.(2014).Tensor decompositions for learning latent variable models.The Journal of Machine Learning Research,15(1),2773–2832.Google Scholar
Anandkumar, A., Hsu, D., & Kakade, S. M. (2012b). A method of moments for mixture models and hidden Markov models. In Conference on learning theory (pp. 33–1).Google Scholar
Balakrishnan, S.,Wainwright, M. J., &Yu, B.(2017).Statistical guarantees for the EM algorithm: From population to sample-based analysis.The Annals of Statistics,45(1),77–120.CrossRefGoogle Scholar
Bandeen-Roche, K.,Miglioretti, D. L.,Zeger, S. L., &Rathouz, P. J.(1997).Latent variable regression for multiple discrete outcomes.Journal of the American Statistical Association,92(440),1375–1386.CrossRefGoogle Scholar
Bucholz, K.,Hesselbrock, V.,Heath, A.,Kramer, J., &Schuckit, M.(2000).A latent class analysis of antisocial personality disorder symptom data from a multi-centre family study of alcoholism.Addiction,95(4),553–567.CrossRefGoogle ScholarPubMed
Celeux, G., &Govaert, G.(1992).A classification EM algorithm for clustering and two stochastic versions.Computational Statistics & Data Analysis,14(3),315–332.CrossRefGoogle Scholar
Chaganty, A. T., & Liang, P. (2013). Spectral experts for estimating mixtures of linear regressions. In International conference on machine learning (pp. 1040–1048). PMLR.Google Scholar
Chen, Y.,Li, X.,Liu, J., &Ying, Z.(2017).Regularized latent class analysis with application in cognitive diagnosis.Psychometrika,82(3),660–692.CrossRefGoogle Scholar
Chen, Y.,Li, X., &Zhang, S.(2019).Joint maximum likelihood estimation for high-dimensional exploratory item factor analysis.Psychometrika,84(1),124–146.CrossRefGoogle ScholarPubMed
Chen, Y.,Li, X., &Zhang, S.(2019).Structured latent factor analysis for large-scale data: Identifiability, estimability, and their implications.Journal of the American Statistical Association,115,1756–1770.CrossRefGoogle Scholar
Collins, L. M., &Lanza, S. T.Latent class and latent transition analysis: With applications in the social, behavioral, and health sciences,(2009).John Wiley & Sons.CrossRefGoogle Scholar
De, L.,De Moor, B., &Vandewalle, J.On the best rank-1 and rank-(R1,R2,…,Rn\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$${R}_1, {R}_2,\ldots,{R}_n$$\end{document}) approximation of higher-order tensors.SIAM Journal on Matrix Analysis and Applications,(2000).21(4),1324–1342.Google Scholar
De Lathauwer, L., & De Moor, B. (1998). From matrix to tensor: Multilinear algebra and signal processing. In Institute of mathematics and its applications conference series (vol. 67, pp. 1–16). Citeseer.Google Scholar
Dean, N., &Raftery, A. E.(2010).Latent class analysis variable selection.Annals of the Institute of Statistical Mathematics,62(1),11.CrossRefGoogle ScholarPubMed
Dempster, A. P.,Laird, N. M., &Rubin, D. B.(1977).Maximum likelihood from incomplete data via the EM algorithm.Journal of the Royal Statistical Society: Series B (Methodological),39(1),1–22.CrossRefGoogle Scholar
Dunn, K. M.,Jordan, K., &Croft, P. R.(2006).Characterizing the course of low back pain: A latent class analysis.American Journal of Epidemiology,163(8),754–761.CrossRefGoogle ScholarPubMed
Fan, Y., &Tang, C. Y.(2013).Tuning parameter selection in high dimensional penalized likelihood.Journal of the Royal Statistical Society: Series B (Statistical Methodology),75(3),531–552.CrossRefGoogle Scholar
Gelman, A.,Hill, J.Data analysis using regression and multilevel/hierarchical models,(2006).Cambridge University Press.CrossRefGoogle Scholar
George, A. C., &Robitzsch, A.(2015).Cognitive diagnosis models in R: A didactic.The Quantitative Methods for Psychology,11(3),189–205.CrossRefGoogle Scholar
Goodman, L. A.(1974).Exploratory latent structure analysis using both identifiable and unidentifiable models.Biometrika,61(2),215–231.CrossRefGoogle Scholar
Gu, Y., & Xu, G. (2021). A joint MLE approach to large-scale structured latent attribute analysis. Journal of the American Statistical Association. https://doi.org/10.1080/01621459.2021.1955689.CrossRefGoogle Scholar
Hagenaars, J. A., &McCutcheon, A. L.(2002).Applied latent class analysis,Cambridge University Press.CrossRefGoogle Scholar
Harshman, R. A.(1970).Foundations of the PARAFAC procedure: Models and conditions for an “explanatory” multimodal factor analysis,University of California at Los Angeles.Google Scholar
Hitchcock, F. L.(1927).The expression of a tensor or a polyadic as a sum of products.Journal of Mathematics and Physics,6(1–4),164–189.CrossRefGoogle Scholar
Hsu, D., & Kakade, S. M. (2013). Learning mixtures of spherical Gaussians: moment methods and spectral decompositions. In Proceedings of the 4th conference on innovations in theoretical computer science (pp. 11–20). ACM.CrossRefGoogle Scholar
Keel, P. K.,Fichter, M.,Quadflieg, N.,Bulik, C. M.,Baxter, M. G.,Thornton, L.,Halmi, K. A.,Kaplan, A. S.,Strober, M., &Woodside, D. B.(2004).Application of a latent class analysis to empirically define eatingdisorder phenotypes.Archives of General Psychiatry,61(2),192–200.CrossRefGoogle Scholar
Kolda, T. G., &Bader, B. W.(2009).Tensor decompositions and applications.SIAM Review,51(3),455–500.CrossRefGoogle Scholar
Kongsted, A., &Nielsen, A. M.(2017).Latent class analysis in health research.Journal of Physiotherapy,63(1),55–58.CrossRefGoogle ScholarPubMed
Kruskal, J. B.(1976).More factors than subjects, tests and treatments: An indeterminacy theorem for canonical decomposition and individual differences scaling.Psychometrika,41(3),281–293.CrossRefGoogle Scholar
Lanza, S. T., &Rhoades, B. L.(2013).Latent class analysis: An alternative perspective on subgroup analysis in prevention and treatment.Prevention Science,14(2),157–168.CrossRefGoogle ScholarPubMed
Lazarsfeld, P. F.,Henry, N. W.(1968).Latent structure analysis,Houghton Mifflin Co..Google Scholar
Lubke, G. H., &Muthén, B.(2005).Investigating population heterogeneity with factor mixture models.Psychological Methods,10(1),21.CrossRefGoogle ScholarPubMed
Ma, C.,Ouyang, J.,Xu, G.Learning latent and hierarchical structures in cognitive diagnosis models.Psychometrika,(2022to appear.Google ScholarPubMed
McCullagh, P.(2018).Tensor methods in statistics: Monographs on statistics and applied probability,Chapman and Hall/CRC.CrossRefGoogle Scholar
McLachlan, G.,Peel, D.(2004).Finite mixture models,John Wiley & Sons.Google Scholar
Muthén, B., &Shedden, K.(1999).Finite mixture modeling with mixture outcomes using the EM algorithm.Biometrics,55(2),463–469.CrossRefGoogle ScholarPubMed
Neyman, J., &Scott, E. L.(1948).Consistent estimates based on partially consistent observations.Econometrica: Journal of the Econometric Society,16,1–32.CrossRefGoogle Scholar
Nishii, R.(1984).Asymptotic properties of criteria for selection of variables in multiple regression.The Annals of Statistics,12,758–765.CrossRefGoogle Scholar
Nylund, K. L.,Asparouhov, T., &Muthén, B. O.(2007).Deciding on the number of classes in latent class analysis and growth mixture modeling: A Monte Carlo simulation study.Structural Equation Modeling: A Multidisciplinary Journal,14(4),535–569.CrossRefGoogle Scholar
Ouyang, J.,Xu, G.Identifiability of latent class models with covariates.Psychometrika,(2022to appear.CrossRefGoogle ScholarPubMed
Rupp, A. A., &Templin, J. L.(2008).Unique characteristics of diagnostic classification models: A comprehensive review of the current state-of-the-art.Measurement,6(4),219–262.Google Scholar
Schwarz, G.(1978).Estimating the dimension of a model.The Annals of Statistics,6(2),461–464.CrossRefGoogle Scholar
Sedat, Ş,Arican, M.A diagnostic comparison of Turkish and Korean students’ mathematics performances on the TIMSS 2011 assessment.Journal of Measurement and Evaluation in Education and Psychology,(2015Google Scholar
Shi, J., &Malik, J.(2000).Normalized cuts and image segmentation.IEEE Transactions on Pattern Analysis and Machine Intelligence,22(8),888–905.Google Scholar
Smilde, A.,Bro, R.,Geladi, P.(2005).Multi-way analysis: Applications in the chemical sciences,John Wiley & Sons.Google Scholar
Tatsuoka, K. K.(1983).Rule space: An approach for dealing with misconceptions based on item response theory.Journal of Educational Measurement,20,345–354.CrossRefGoogle Scholar
Tucker, L. R. (1964). The extension of factor analysis to three-dimensional matrices. Contributions to Mathematical Psychology, 110119.Google Scholar
Tucker, L. R.(1966).Some mathematical notes on three-mode factor analysis.Psychometrika,31(3),279–311.CrossRefGoogle ScholarPubMed
Van Buuren, S., &Groothuis-Oudshoorn, K.(2011).mice: Multivariate imputation by chained equations in R.Journal of Statistical Software,45,1–67.Google Scholar
Vermunt, J. K. (2003). Applications of latent class analysis in social science research. In European conference on symbolic and quantitative approaches to reasoning and uncertainty (pp. 22–36). Springer.CrossRefGoogle Scholar
Vermunt, J. K.(2010).Latent class modeling with covariates: Two improved three-step approaches.Political analysis,18,450–469.CrossRefGoogle Scholar
von Davier, M.,Lee, Y-S(2019).Handbook of diagnostic classification models,Springer.CrossRefGoogle Scholar
Wang, M., &Hanges, P. J.(2011).Latent class procedures: Applications to organizational research.Organizational Research Methods,14(1),24–31.CrossRefGoogle Scholar
Xu, G.(2017).Identifiability of restricted latent class models with binary responses.The Annals of Statistics,45(2),675–707.CrossRefGoogle Scholar
Xu, G., &Shang, Z.(2018).Identifying latent structures in restricted latent class models.Journal of the American Statistical Association,113(523),1284–1295.CrossRefGoogle Scholar
Zhang, Y., Chen, X., Zhou, D., & Jordan, M. I. (2014). Spectral methods meet EM: A provably optimal algorithm for crowdsourcing. In Advances in neural information processing systems, 27.Google Scholar
Supplementary material: File

Appendix 1-3

Appendix 1-3
Download Appendix 1-3(File)
File 578.5 KB