Hostname: page-component-745bb68f8f-lrblm Total loading time: 0 Render date: 2025-01-08T07:42:17.970Z Has data issue: false hasContentIssue false
  • English
  • Français

A Bayesian Analysis of Finite Mixtures in the LISREL Model

Published online by Cambridge University Press:  01 January 2025

Hong-Tu Zhu  and
Sik-Yum Lee
Hong-Tu Zhu
Affiliation:
Department of Statistics, The Chinese University of Hong Kong
Sik-Yum Lee*
Affiliation:
Department of Statistics, The Chinese University of Hong Kong
*
Requests for reprints should be sent to S.Y. Lee, Department of Statistics, The Chinese University of Hong Kong, Shatin, N.T. HONG KONG. E-Mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

In this paper, we propose a Bayesian framework for estimating finite mixtures of the LISREL model. The basic idea in our analysis is to augment the observed data of the manifest variables with the latent variables and the allocation variables. The Gibbs sampler is implemented to obtain the Bayesian solution. Other associated statistical inferences, such as the direct estimation of the latent variables, establishment of a goodness-of-fit assessment for a posited model, Bayesian classification, residual and outlier analyses, are discussed. The methodology is illustrated with a simulation study and a real example.

Keywords

Bayesian analysisfinite mixturesLISREL modelsGibbs samplerconditional distributionsgoodness-of-fit assessmentBayesian classificationresidual and outlier analyses
Type
Articles
Information
Psychometrika , Volume 66 , Issue 1 , March 2001 , pp. 133 - 152
Copyright
Copyright © 2001 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

This research was supported by a Hong Kong UGC Earmarked grant CUHK 4026/97H. The authors are indebted to the Editor, the Associate Editor, and three anonymous reviewers for constructive comments in improving the paper, and also to ICPSR and the relevant funding agency for allowing the use of the data. The assistance of Michael K.H. Leung and Esther L.S. Tam is gratefully acknowledged.

References

Albert, J.H., Chib, S. (1993). Bayesian analysis of binary and polychotomous response data. Journal of the American Statistical Association, 88, 669–679.CrossRefGoogle Scholar
Albert, J.H., Chib, S. (1995). Bayesian residual analysis for binary regression models. Biometrika, 82, 747–759.CrossRefGoogle Scholar
Arminger, G., Muthén, B.O. (1998). A Bayesian approach to nonlinear latent variable models using the Gibbs sampler and the Metropolis-Hastings algorithm. Psychometrika, 63, 271–300.CrossRefGoogle Scholar
Barnett, V., Lewis, T. (1984). Outliers in statistical data 2nd ed, Chichester: Wiley & Sons.Google Scholar
Bentler, P.M. (1983). Some contributions to efficient statistics for structural models: Specification and estimation of moment structures. Psychometrika, 48, 493–517.CrossRefGoogle Scholar
Bentler, P.M. (1992). EQS: Structural equation program manual. Los Angeles, CA: BMDP Statistical Software.Google Scholar
Besag, J., Green, P., Higdon, D., Mengersen, K. (1995). Bayesian computation and stochastic systems. Statistical Science, 10, 3–66.Google Scholar
Bollen, K.A., Long, J.S. (1993). Testing structural equation models. Newbury Park, CA: Sage.Google Scholar
Broemeling, L.D. (1985). Bayesian analysis of linear models. New York, NY: Marcel Dekker.Google Scholar
Browne, M.W. (1984). Asymptotically distribution-free methods in the analysis of covariance structures. British Journal of Mathematical and Statistical Psychology, 37, 62–83.CrossRefGoogle ScholarPubMed
Byrne, B.M. (1994). Structural equation modeling with EQS and EQS/Windows. Thousand Oaks, CA: Sage.Google Scholar
Chaloner, K. (1991). Bayesian residual analysis in the presence of censoring. Biometrika, 78, 637–644.CrossRefGoogle Scholar
Chaloner, K., Brant, R. (1988). A Bayesian approach to outlier detection and residual analysis. Biometrika, 75, 651–659.CrossRefGoogle Scholar
Cook, R.D., Weisberg, S. (1982). Residuals and influence in regression. London: Chapman and Hall.Google Scholar
Crawford, S.L., DeGroot, M.H., Kadane, J.B., Small, M.J. (1992). Modeling lake chemistry distributions: Approximate Bayesian methods for estimating a finite mixture model. Techometrics, 34, 441–453.CrossRefGoogle Scholar
Day, N.E. (1969). Estimating the components of a mixture of normal distributions. Biometrika, 56, 463–474.CrossRefGoogle Scholar
Dempster, A.P., Laird, N.M., Rubin, D.B. (1977). Maximum likelihood from incomplete data via the EM algorithm (with discussion). Journal of the Royal Statistical Society, Series B, 39, 1–38.CrossRefGoogle Scholar
Diebolt, J., Robert, C.P. (1994). Estimation of finite mixture distributions through Bayesian sampling. Journal of the Royal Statistical Society, Series B, 56, 163–175.CrossRefGoogle Scholar
Do, K., McLachlan, G.J. (1984). Estimation of mixing proportions: A case study. Applied Statistics, 33, 134–140.CrossRefGoogle Scholar
Dolan, C.V., van der Maas, Han L.J. (1998). Fitting multivariate normal finite mixtures subject to structural equation modeling. Psychometrika, 63, 227–254.CrossRefGoogle Scholar
Ganesalingam, S., McLachlan, G.J. (1981). Some efficiency results for the estimation of the mixing proportion in a mixture of two normal distributions. Biometrics, 37, 23–33.CrossRefGoogle Scholar
Gelfand, A.E., Smith, A.F.M. (1990). Sampling-based approaches to calculating marginal densities. Journal of the American Statistical Association, 85, 398–409.CrossRefGoogle Scholar
Gelman, A., Carlin, J.B., Stern, H.S., Rubin, D.B. (1995). Bayesian data analysis. London: Chapman and Hall.CrossRefGoogle Scholar
Gelman, A., Meng, X.L., Stern, H. (1996). Posterior predictive assessment of model fitness via realized discrepancies. Statistics Sinica, 6, 733–807.Google Scholar
Gelman, A., Rubin, D.B. (1992). Inference from iterative simulation using multiple sequences. Statistical Science, 7, 457–472.CrossRefGoogle Scholar
Geman, S., Geman, D. (1984). Stochastic relaxation, Gibbs distribution, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 721–741.CrossRefGoogle ScholarPubMed
Geyer, C.J. (1992). Practical Markov chain Monte Carlo. Statistical Science, 7, 473–511.Google Scholar
Gilks, W.R., Richardson, S., Spiegelhalter, D.J. (1996). Introducing Markov chain Monte Carlo. In Gilks, W. R., Richardson, S., Spiegelhalter, D. J. (Eds.), Markov Chain Monte Carlo in practice (pp. 1–19). London: Chapman and Hall.Google Scholar
Hathaway, R.J. (1985). A constrained formulation of maximum-likelihood estimation for normal mixture distributions. The Annals of Statistics, 13, 795–800.CrossRefGoogle Scholar
Hasselblad, V. (1966). Estimation of parameters for a mixture of normal distributions. Technometrics, 8, 431–444.CrossRefGoogle Scholar
Hawkins, D.M. (1980). Identification of outliers. London: Chapman and Hall.CrossRefGoogle Scholar
Hoyle, R. (1995). Structural equation modeling: Concepts, issues and applications. Thousand Oaks, CA: Sage.Google Scholar
Jedidi, K., Jagpal, H.S., DeSarbo, W.S. (1997). Finite-mixture structural equation models for response-based segmentation and unobserved heterogeneity. Marketing Science, 16, 39–59.CrossRefGoogle Scholar
Jedidi, K., Jagpal, H.S., DeSarbo, W.S. (1997). STEMM: A general finite mixture structural equation model. Journal of Classification, 14, 23–50.CrossRefGoogle Scholar
Jöreskog, K.G. (1978). Structural analysis of covariance and correlation matrices. Psychometrika, 43, 443–477.CrossRefGoogle Scholar
Jöreskog, K.G., Sörbom, D. (1996). LISREL 8: Structural equation modeling with the SIMPLIS command language. Hove and London: Scientific Software International.Google Scholar
Lee, S.Y. (1981). A Bayesian approach to confirmatory factor analysis. Psychometrika, 46, 153–160.CrossRefGoogle Scholar
Lee, S.Y., Poon, W.Y. (1992). Two-level analysis of covariance structures for unbalanced designs with small level-one samples. British Journal of Mathematical and Statistical Psychology, 45, 109–124.CrossRefGoogle ScholarPubMed
Lee, S.Y., Tsui, K.L. (1982). Covariance structure analysis in several populations. Psychometrika, 57, 409–426.Google Scholar
Lindsay, B.G. (1989). Moment matrices: Applications in mixtures. The Annals of Statistics, 17, 722–740.Google Scholar
Lindsay, B.G., Baska, P. (1993). Multivariate normal mixtures: A fast consistent method of moments. Journal of the American Statistical Association, 88, 468–476.CrossRefGoogle Scholar
Lindley, D.V., Smith, A.F.M. (1972). Bayes estimates for the linear model (with discussion). Journal of the Royal Statistical Society, Series B, 34, 1–42.CrossRefGoogle Scholar
Louis, T.A. (1982). Finding the observed information matrix when using EM algorithm. Journal of the Royal Statistical Society, Series B, 44, 226–233.CrossRefGoogle Scholar
Meng, X.L. (1994). Posterior predictivep-values. The Annals of Statistics, 22, 1142–1160.CrossRefGoogle Scholar
Mood, A.M., Graybill, F.A., Boes, D.C. (1974). Introduction to the theory of statistics 3rd ed., London: McGraw-Hill.Google Scholar
Muthén, B. (1989). Latent variable modeling in heterogeneous populations. Psychometrika, 54, 557–585.CrossRefGoogle Scholar
Raiffa, H., Schlaifer, L.R. (1961). Applied statistical secision theory 2nd ed., Boston, MA: Harvard, Graduate School of Business Administration.Google Scholar
Redner, R.A., Walker, H.F. (1984). Mixture densities, maximum likelihood and the EM algorithm. SIAM Review, 26, 195–239.CrossRefGoogle Scholar
Richardson, S., Green, P.J. (1997). On Bayesian analysis of mixtures with an unknown number of components (with discussion). Journal of the Royal Statistical Society, Series B, 59, 731–792.CrossRefGoogle Scholar
Robert, C.P. (1996). Mixtures of distributions: Inference and estimation. In Gilks, W.R., Richardson, S., Spiegelhalter, D.J. (Eds.), Markov Chain Monte Carlo in practice (pp. 163–187). London: Chapman and Hall.Google Scholar
Roeder, K., Wasserman, L. (1997). Practical Bayesian density estimation using mixtures of normals. Journal of the American Statistical Association, 92, 894–902.CrossRefGoogle Scholar
Rubin, D.B. (1984). Bayesianly justifiable and relevant frequency calculations for the applied statistician. The Annals of Statistics, 12, 1151–1172.CrossRefGoogle Scholar
Scheines, R., Hoijtink, H., Boomsma, A. (1999). Bayesian estimation and testing of structural equation models. Psychometrika, 64, 37–52.CrossRefGoogle Scholar
Shi, J.Q., Lee, S.Y. (1998). Bayesian sampling-based approach for factor analysis models with continuous and polytomous data. British Journal of Mathematical and Statistical Psychology, 51, 233–252.CrossRefGoogle Scholar
Smith, A.F.M., Makov, U.E. (1978). A quasi-Bayes sequential procedure for mixtures. Journal of the Royal Statistical Society, Series B, 40, 106–112.CrossRefGoogle Scholar
Titterington, D.M., Smith, A.F.M., Makov, U.E. (1985). Statistical analysis of finite mixture distributions. Chichester: Wiley & Sons.Google Scholar
van Buuren, S. (1997). Fitting ARMA time series by structural equation models. Psychometrika, 62, 215–236.CrossRefGoogle Scholar
Weiss, R.E. (1994). Residuals and outliers in Bayesian random effects models. Los Angeles, CA: UCLA, School of Public Health, Department of Biostatistics.Google Scholar
Weiss, R.E., Lazaro, C.G. (1992). Residual plots for repeated measures. Statistics in Medicine, 11, 115–124.CrossRefGoogle ScholarPubMed
World Values Survey, 1981–1984 and 1990–1993 (ICPSR version). (1994). Ann Arbor, MI: Institute for Social Research [producer], Inter-university Consortium for Political and Social Research [distributor].Google Scholar
Yuan, K.E., Bentler, P.M. (1997). Mean and covariance structures analysis: theoretical and practical improvements. Journal of the American Statistical Association, 92, 767–774.CrossRefGoogle Scholar
Yung, Y.F. (1997). Finite mixtures in confirmatory factor-analysis models. Psychometrika, 62, 297–330.CrossRefGoogle Scholar
Zeger, S.L., Karim, M.R. (1991). Generalized linear models with random effects: A gibbs sampling approach. Journal of the American Statistical Association, 86, 79–86.CrossRefGoogle Scholar
Zellner, A. (1971). An Introduction to Bayesian Inference in Econometrics. New York, NY: Wiley & Sons.Google Scholar
Zhu, H.T., Lee, S.Y. (1999). Statistical analysis of nonlinear factor analysis models. British Journal of Mathematical and Statistical Psychology, 52, 225–242.CrossRefGoogle Scholar