Hostname: page-component-5f745c7db-j9pcf Total loading time: 0 Render date: 2025-01-06T06:53:43.973Z Has data issue: true hasContentIssue false
  • English
  • Français

Weighted Least Squares Fitting Using Ordinary Least Squares Algorithms

Published online by Cambridge University Press:  01 January 2025

Henk A. L. Kiers
Henk A. L. Kiers*
Affiliation:
University of Groningen
*
Requests for reprints should be sent to Henk A. L. Kiers, Department of Psychology (SPA), Grote Kruisstraat 2/1, 9712 TS Groningen, THE NETHERLANDS.
Get access Rights & Permissions [Opens in a new window]

Abstract

A general approach for fitting a model to a data matrix by weighted least squares (WLS) is studied. This approach consists of iteratively performing (steps of) existing algorithms for ordinary least squares (OLS) fitting of the same model. The approach is based on minimizing a function that majorizes the WLS loss function. The generality of the approach implies that, for every model for which an OLS fitting algorithm is available, the present approach yields a WLS fitting algorithm. In the special case where the WLS weight matrix is binary, the approach reduces to missing data imputation.

Keywords

weighted least squaresalternating least squaresmissing dataalgorithmsmajorizationmatrix approximationmaximum likelihood estimation
Type
Original Paper
Information
Psychometrika , Volume 62 , Issue 2 , June 1997 , pp. 251 - 266
Copyright
Copyright © 1997 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

This research has been made possible by a fellowship from the Royal Netherlands Academy of Arts and Sciences to the author.

References

Bailey, R. A., Gower, J. C. (1990). Approximating a symmetric matrix. Psychometrika, 55, 665–675.CrossRefGoogle Scholar
Bijleveld, C., de Leeuw, J. (1991). Fitting longitudinal reduced rank regression models by alternating least squares. Psychometrika, 56, 443–447.CrossRefGoogle Scholar
Bollen, K. A. (1989). Structural equations with latent variables, New York: Wiley.CrossRefGoogle Scholar
Browne, M. W. (1984). Asymptotically distribution-free methods for the analysis of covariance structures. British Journal of Mathematical and Statistical Psychology, 37, 62–83.CrossRefGoogle ScholarPubMed
Carroll, J. D., De Soete, G., Pruzansky, S. (1989). Fitting of the latent class model via iteratively reweighted least squares candecomp with nonnegativity constraints. In Coppi, R., Bolasco, S. (Eds.), Multiway data analysis (pp. 463–472). Amsterdam: Elsevier Science Publishers.Google Scholar
Cliff, N. (1966). Orthogonal rotation to congruence. Psychometrika, 31, 33–42.CrossRefGoogle Scholar
Commandeur, J. J. F. (1991). Matching configurations, Leiden: DSWO Press.Google Scholar
de Leeuw, J., Heiser, W. (1980). Multidimensional scaling with restrictions on the configuration. In Krishnaiah, P. R. (Eds.), Multivariate analysis V (pp. 501–522). Amsterdam: North Holland.Google Scholar
Dempster, A. P., Laird, N. M., Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B, 39, 1–38.CrossRefGoogle Scholar
Gabriel, K. R., Zamir, S. (1979). Lower rank approximation of matrices by least squares with any choice of weights. Technometrics, 21, 489–498.CrossRefGoogle Scholar
Gifi, A. (1990). Nonlinear multivariate analysis, Chichester: Wiley.Google Scholar
Green, B. F. (1952). The orthogonal approximation of an oblique structure in factor analysis. Psychometrika, 17, 429–440.CrossRefGoogle Scholar
Harman, H. H., Jones, W. H. (1966). Factor analysis by minimizing residuals (Minres). Psychometrika, 31, 351–368.CrossRefGoogle ScholarPubMed
Harshman, R. A. (1978, August). Models for analysis of asymmetrical relationships among N objects or stimuli. Paper presented at the First Joint Meeting of the Psychometric Society and the Society for Mathematical Psychology, Hamilton, Ontario.Google Scholar
Harshman, R. A., Green, P. E., Wind, Y., Lundy, M. E. (1982). A model for the analysis of asymmetric data in marketing research. Marketing Science, 1, 205–242.CrossRefGoogle Scholar
Harshman, R. A., Lundy, M. E. (1984). The PARAFAC model for three-way factor analysis and multidimensional scaling. In Law, H. G., Snyder, C. W., Hattie, J. A., McDonald, R. P. (Eds.), Research methods for multimode data analysis (pp. 122–215). New York: Praeger.Google Scholar
Heiser, W. J. (1987). Correspondence Analysis with least absolute residuals. Computational Statistics and Data Analysis, 5, 337–356.CrossRefGoogle Scholar
Heiser, W. J. (1995). Convergent computation by iterative majorization: theory and applications in multidimensional data analysis. In Krzanowski, W. J. (Eds.), Recent advances in descriptive multivariate analysis (pp. 157–189). Oxford: Oxford University Press.CrossRefGoogle Scholar
Jöreskog, K. G., Sörbom, D. (1993). LISREL 8 User's guide, Chicago: Scientific Software International.Google Scholar
Kiers, H. A. L. (1989). An alternating least squares algorithm for fitting the two- and three-way DEDICOM model and the IDIOSCAL model. Psychometrika, 54, 515–521.CrossRefGoogle Scholar
Kiers, H. A. L. (1990). Majorization as a tool for optimizing a class of matrix functions. Psychometrika, 55, 417–428.CrossRefGoogle Scholar
Kiers, H. A. L. (1993). An alternating least squares algorithm for PARAFAC2 and DEDICOM3. Computational Statistics and Data Analysis, 16, 103–118.CrossRefGoogle Scholar
Kiers, H. A. L., ten Berge, J. M. F. (1992). Minimization of a class of matrix trace functions by means of refined majorization. Psychometrika, 57, 371–382.CrossRefGoogle Scholar
Kiers, H. A. L., ten Berge, J. M. F., Takane, Y., de Leeuw, J. (1990). A generalization of Takane's algorithm for DEDICOM. Psychometrika, 55, 151–158.CrossRefGoogle Scholar
Takane, Y. (1985). Diagonal Estimation in DEDICOM. Proceedings of the 1985 Annual Meeting of the Behaviormetric Society (pp. 100–101). Sapporo, Japan: Behaviormetric Society.Google Scholar
ten Berge, J. M. F., Kiers, H. A. L. (1989). Fitting the off-diagonal DEDICOM model in the least-squares sense by a generalization of the Harman & Jones MINRES procedure of factor analysis. Psychometrika, 54, 333–337.CrossRefGoogle Scholar
ten Berge, J. M. F., Kiers, H. A. L. (1993). An alternating least squares method for the weighted approximation of a symmetric matrix. Psychometrika, 58, 115–118.CrossRefGoogle Scholar
ten Berge, J. M. F., Kiers, H. A. L., Commandeur, J. J. F. (1993). Orthogonal Procrustes rotation for matrices with missing values. British Journal of Mathematical and Statistical Psychology, 46, 119–134.CrossRefGoogle Scholar
Verboon, P. (1994). A robust approach to nonlinear multivariate analysis, Leiden: DSWO Press.Google Scholar
Verboon, P., Heiser, W. J. (1992). Resistant orthogonal Procrustes analysis. Journal of Classification, 9, 237–256.CrossRefGoogle Scholar
Verboon, P., Heiser, W. J. (1994). Resistant lower rank approximation of matrices by iterative majorization. Computational Statistics and Data Analysis, 18, 457–467.CrossRefGoogle Scholar