Hostname: page-component-745bb68f8f-kw2vx Total loading time: 0 Render date: 2025-01-07T16:00:20.688Z Has data issue: false hasContentIssue false
  • English
  • Français

Using Penalized EM Algorithm to Infer Learning Trajectories in Latent Transition CDM

Published online by Cambridge University Press:  01 January 2025

Chun Wang Open the ORCID record for Chun Wang [Opens in a new window]
Chun Wang*
Affiliation:
University of Washington
*
Correspondence should be made to Chun Wang, Measurement and Statistics, College of Education, University of Washington, 312E Miller Hall, 2012 Skagit Ln, Seattle, WA 98105, USA. Email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Cognitive diagnostic models (CDMs) have arisen as advanced psychometric models in the past few decades for assessments that intend to measure students’ mastery of a set of attributes. Recently, quite a few studies attempted to extend CDMs to longitudinal versions, and they all tended to model transition probabilities from non-mastery to mastery or vice versa for each attribute separately, with an exception of a few studies (e.g., Chen et al. 2018; Madison & Bradshaw 2018). However, these pioneering works have not taken into consideration the attribute relationships and the ever-changing attributes in a learning period. In this paper, we consider a profile-level latent transition CDM (TCDM hereafter), which can not only identify transition probabilities across the same attributes over time, but also the transition pathways across different attributes. Two versions of the penalized expectation-maximization (PEM) algorithms are proposed to shrink the probabilities associated with impermissible transition pathways to 0 and, thereby, help explore attribute relationships in a longitudinal setting. Simulation results reveal that PEM with group penalty holds great promise for identifying learning trajectories.

Keywords

cognitive diagnostic modelslatent transition analysispenalized expectation-maximizationlearning trajectory
Type
Original Paper
Information
Psychometrika , Volume 86 , Issue 1 , March 2021 , pp. 167 - 189
Copyright
Copyright © 2021 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Bolt, D. M., &Lall, V. F.(2003).Estimation of compensatory and noncompensatory multidimensional item response models using Markov chain Monte Carlo.Applied Psychological Measurement,27(6),395–414.CrossRefGoogle Scholar
Briggs, D. C.,Alonzo, A. C.,Schwab, C, &Wilson, M.(2006).Diagnostic assessment with ordered multiple-choice items.Educational Assessment,11(1),33–63.CrossRefGoogle Scholar
Broman, K. W., &Speed, T. P.(2002).A model selection approach for the identification of quantitative trait loci in experimental crosses.Journal of the Royal Statistical Society: Series B (Statistical Methodology),64(4),641–656.CrossRefGoogle Scholar
Chen, J., &Chen, Z.(2008).Extended Bayesian information criteria for model selection with large model spaces.Biometrika,95(3),759–771.CrossRefGoogle Scholar
Chen, Y.,Culpepper, S. A.,Wang, S., &Douglas, J.(2018).A hidden Markov model for learning trajectories in cognitive diagnosis with application to spatial rotation skills.Applied Psychological Measurement,42(1),5–23.CrossRefGoogle ScholarPubMed
Chung, H.,Flaherty, B. P., &Schafer, J. L.(2006).Latent class logistic regression: Application to marijuana use and attitudes among high school seniors.Journal of the Royal Statistical Society: Series A (Statistics in Society),169(4),723–743.CrossRefGoogle Scholar
Chung, H.,Lanza, S. T., &Loken, E.(2008).Latent transition analysis: Inference and estimation.Statistics in Medicine,27(11),1834–1854.CrossRefGoogle ScholarPubMed
Cizek, G. J.,Bunch, M. B., &Koons, H.(2004).Setting performance standards: Contemporary methods.Educational Measurement: Issues and Practice,23(4),31–31.CrossRefGoogle Scholar
Choi, I., & Wilson, M. (2016). An investigation on the measurement model for complex learning progressions: Structured constructs model for the continuous latent trait (SCM-C). Paper presented at the annual meeting of National Council on Measurement in Education.Google Scholar
Collins, L. M., & Lanza, S. T. (2010). Latent class analysis with covariates (pp. 149–177). Latent class and latent transition analysis: With applications in the social, behavioral, and health sciences.Google Scholar
Collins, L. M., &Wugalter, S. E.(1992).Latent class models for stage-sequential dynamic latent variables.Multivariate Behavioral Research,27(1),131–157.CrossRefGoogle Scholar
Corcoran, T. B., Mosher, F. A., & Rogat, A. (2009). Learning progressions in science: An evidence-based approach to reform.CrossRefGoogle Scholar
Daro, P., Mosher, F. A., & Corcoran, T. B. (2011). Learning trajectories in mathematics: A foundation for standards, curriculum, assessment, and instruction.CrossRefGoogle Scholar
De La Torre, J.(2011).The generalized DINA model framework.Psychometrika,76(2),179–199.CrossRefGoogle Scholar
Deng, N., Roussos, L., & LaFond, L. (2017). Multidimensional modeling of learning progression-based vertical scales. Paper presented at the annual meeting of the National Council on Measurement in Educaiton, San Antonio TX.Google Scholar
Diakow, R., Irribarra, D., & Wilson, M. (2011). Analyzing the complex structure of a learning progression: Structured construct models. Paper presented at the annual meeting of the national council of measurement in education, New Orleans, LA.Google Scholar
Embretson, S.(1984).A general latent trait model for response processes.Psychometrika,49(2),175–186.CrossRefGoogle Scholar
Embretson, S. E.van der Linden, W. J., &Hambleton, R. K.(1997).Multicomponent response models.Handbook of modern item response theory,Berlin:Springer.305–321.CrossRefGoogle Scholar
Embretson, S. E., &Yang, X.(2013).A multicomponent latent trait model for diagnosis.Psychometrika,78(1),14–36.CrossRefGoogle ScholarPubMed
Fan, J., &Li, R.(2001).Variable selection via nonconcave penalized likelihood and its oracle properties.Journal of the American Statistical Association,96(456),1348–1360.CrossRefGoogle Scholar
Fan, J., &Peng, H.(2004).Nonconcave penalized likelihood with a diverging number of parameters.The Annals of Statistics,32(3),928–961.CrossRefGoogle Scholar
Fan, Y., &Tang, C. Y.(2013).Tuning parameter selection in high dimensional penalized likelihood.Journal of the Royal Statistical Society: Series B (Statistical Methodology),75(3),531–552.CrossRefGoogle Scholar
Gu, Y., &Xu, G.(2019).Learning attribute patterns in high-dimensional structured latent attribute models.Journal of Machine Learning,20 1–58.Google Scholar
Haertel, E. H.(1989).Using restricted latent class models to map the skill structure of achievement items.Journal of Educational Measurement,26(4),301–321.CrossRefGoogle Scholar
Hastie, T.,Tibshirani, R., &Friedman, J.(2009).The elements of statistical learning: Data mining, inference, and prediction,Berlin:Springer.CrossRefGoogle Scholar
Henson, R. A.,Templin, J. L., &Willse, J. T.(2009).Defining a family of cognitive diagnosis models using log-linear models with latent variables.Psychometrika,74(2),191–CrossRefGoogle Scholar
Junker, B. W., &Sijtsma, K.(2001).Cognitive assessment models with few assumptions, and connections with nonparametric item response theory.Applied Psychological Measurement,25(3),258–272.CrossRefGoogle Scholar
Kaya, Y., &Leite, W. L.(2017).Assessing change in latent skills across time with longitudinal cognitive diagnosis modeling: An evaluation of model performance.Educational and Psychological Measurement,77(3),369–388.CrossRefGoogle ScholarPubMed
Lanza, S. T.,Collins, L. M.,Schafer, J. L., &Flaherty, B. P.(2005).Using data augmentation to obtain standard errors and conduct hypothesis tests in latent class and latent transition analysis.Psychological Methods,10(1),84–CrossRefGoogle ScholarPubMed
Lappan, G., & Briars, D. (1995). How should mathematics be taught? In I. Carl (Ed.), Prospects for school mathematics. Reston, VA: National Council of Teachers of Mathematics.Google Scholar
Lehrer, R., & Wilson, M. (2011). Developing assessments of data modeling: Construct maps as boundary objects. Paper presented at the annual meeting of the American Educational Research Association, New Orleans.Google Scholar
Li, F.,Cohen, A.,Bottge, B., &Templin, J.(2016).A latent transition analysis model for assessing change in cognitive skills.Educational and Psychological Measurement,76(2),181–204.CrossRefGoogle ScholarPubMed
Lu, J., & Wang, C. (2019). A sequential exploratory approach to learning attribute hierarchies from data. Paper presented at the International meeting of psychometric society. Chile: Santiago.Google Scholar
Madison, M. J., &Bradshaw, L.(2018).Evaluating intervention effects in a diagnostic classification model framework.Journal of Educational Measurement,55(1),32–51.CrossRefGoogle Scholar
Martineau, J. A.(2006).Distorting value added: The use of longitudinal, vertically scaled student achievement data for growth-based, value-added accountability.Journal of Educational and Behavioral Statistics,31(1),35–62.CrossRefGoogle Scholar
Nylund-Gibson, K.,Grimm, R.,Quirk, M., &Furlong, M.(2014).A latent transition mixture model using the three-step specification.Structural Equation Modeling: A Multidisciplinary Journal,21(3),439–454.CrossRefGoogle Scholar
Pellegrino, J. W.,Chudowsky, N., &Glaser, R.(2001).Knowing what students know: The science and design of educational assessment,Washington, DC:National Academy Press.Google Scholar
Pham, D., Monroe, S., Bauer, M., Wylie, C., & Wells, C. (2017). Evaluating a learning progression theory: An application of MIRT. Paper presented at the annual meeting of the National Council on Measurement in Educaiton, San Antonio, TX.Google Scholar
Popham, W. J.(2008).Formative assessment: Seven stepping-stones to success.Principal Leadership,9(1),16–20.Google Scholar
Popham, W. J.(2011).Transformative assessment in action: An inside look at applying the process,Alexandria:ASCD.Google Scholar
Roussos, L. A., DiBello, L. V., Stout, W., Hartz, S. M., Henson, R. A., & Templin, J. (2007). The fusion model skills diagnosis system (pp. 275–318). Cognitive diagnostic assessment for education: Theory and applications.Google Scholar
Rupp, A.,Templin, J., &Henson, R.(2010).Diagnostic assessment: Theory, methods, and applications,New York:Guilford.Google Scholar
Rupp, A. A., &Templin, J. L.(2008).Unique characteristics of diagnostic classification models: A comprehensive review of the current state-of-the-art.Measurement,6(4),219–262.Google Scholar
Shao, J.(1997).An asymptotic theory for linear model selection.Statistica Sinica,7 221–242.Google Scholar
Shepard, L. A.(2009).Commentary: Evaluating the validity of formative and interim assessment.Educational Measurement: Issues and Practice,28(3),32–37.CrossRefGoogle Scholar
Siegmund, D.(2004).Model selection in irregular problems: Applications to mapping quantitative trait loci.Biometrika,91(4),785–800.CrossRefGoogle Scholar
Simon, M. A., &Tzur, R.(2004).Explicating the role of mathematical tasks in conceptual learning: An elaboration of the hypothetical learning trajectory.Mathematical Thinking and Learning,6(2),91–104.CrossRefGoogle Scholar
Sinharay, S.(2006).Model diagnostics for Bayesian networks.Journal of Educational and Behavioral Statistics,31(1),1–33.CrossRefGoogle Scholar
Sinharay, S., &Almond, R. G.(2007).Assessing fit of cognitive diagnostic models a case study.Educational and Psychological Measurement,67(2),239–257.CrossRefGoogle Scholar
Tatsuoka, C.(2002).Data analytic methods for latent partially ordered classification models.Journal of the Royal Statistical Society: Series C (Applied Statistics),51(3),337–350.Google Scholar
Wang, C., & Lu, J. (2020). Learning attribute hierarchies from data: Two exploratory approaches. Journal of Educational and Behavioral Statistics. https://doi.org/10.3102/1076998620931094CrossRefGoogle Scholar
Wang, C., &Nydick, S. W.(2015).Comparing two algorithms for calibrating the restricted non-compensatory multidimensional IRT model.Applied Psychological Measurement,39(2),119–134.CrossRefGoogle ScholarPubMed
Wang, C.,Zheng, C., &Chang, H. H.(2014).An enhanced approach to combine item response theory with cognitive diagnosis in adaptive testing.Journal of Educational Measurement,51(4),358–380.CrossRefGoogle Scholar
Wang, H.,Li, B., &Leng, C.(2009).Shrinkage tuning parameter selection with a diverging number of parameters.Journal of the Royal Statistical Society: Series B (Statistical Methodology),71(3),671–683.CrossRefGoogle Scholar
Wang, S.,Yang, Y.,Culpepper, S. A., &Douglas, J. A.(2018).Tracking skill acquisition with cognitive diagnosis models: A higher-order, hidden Markov model with covariates.Journal of Educational and Behavioral Statistics,43 57–87.CrossRefGoogle Scholar
Wilson, M.(2004).Constructing measures: An item response modeling approach,Abingdon:Routledge.CrossRefGoogle Scholar
Xu, G.,Wang, C., &Shang, Z.(2016).On initial item selection in cognitive diagnostic computerized adaptive testing.British Journal of Mathematical and Statistical Psychology,69(3),291–315.CrossRefGoogle ScholarPubMed