Hostname: page-component-745bb68f8f-kw2vx Total loading time: 0 Render date: 2025-01-08T12:11:43.588Z Has data issue: false hasContentIssue false
  • English
  • Français

Neither Cronbach’s Alpha nor McDonald’s Omega: A Commentary on Sijtsma and Pfadt

Published online by Cambridge University Press:  01 January 2025

Eunseong Cho Open the ORCID record for Eunseong Cho [Opens in a new window]
Eunseong Cho*
Affiliation:
Kwangwoon University
*
Correspondence should be made to Eunseong Cho, Republic of Korea College of Business Administration, Kwangwoon University, 20 Kwangwoonro, Nowon-gu, Seoul01897, Republic of Korea. Email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Sijtsma and Pfadt (2021) published a thought-provoking article on coefficient alpha. I make the following arguments against their work. 1) Kuder and Richardson (1937) deserve more credit for coefficient alpha than Cronbach (1951). 2) We should distinguish between the definition of reliability and its meaning. 3) We should be wary of overfitting in the use of FA reliability. 4) Our primary concern is to obtain accurate reliability estimates rather than conservative estimates. 5) Several reliability estimators, such as λ2\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$\lambda _{2}$$\end{document}, μ2\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$\mu _{2}$$\end{document}, congeneric reliability and the Gilmer-Feldt coefficient are more accurate than coefficient alpha. 6) The name omega should not be used to refer to a specific reliability estimator.

Keywords

coefficient alphacronbach’s alphatau-equivalent reliabilitycongeneric reliabilityclassical test theorycoefficient omega
Type
Revisiting Cronbach’s Alpha
Information
Psychometrika , Volume 86 , Issue 4: Special Section: Revisiting Cronbach’s Alpha , December 2021 , pp. 877 - 886
Copyright
Copyright © 2021 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

The present research was supported by the Research Grant of Kwangwoon University in 2020

References

Allen, M. P. (1973).Construction of composite measures by the canonical-factor-regression method.Sociological Methodology, 5, 51– 78 CrossRefGoogle Scholar
Bentler, P. M. (1968).Alpha-maximized factor analysis (alphamax): Its relation to alpha and canonical factor analysis.Psychometrika, 33 (3), 335– 345 CrossRefGoogle ScholarPubMed
Cho, E. (2016).Making reliability reliable: A systematic approach to reliability coefficients.Organizational Research Methods, 19 (4), 651– 682 CrossRefGoogle Scholar
Cho, E., & Chun, S. (2018). Fixing a broken clock: A historical review of the originators of reliability coefficients including Cronbach’s alpha. Survey Research, 19(2), 23–54. https://doi.org/10.20997/sr.19.2.4 CrossRefGoogle Scholar
Cho, E., & Kim, S. (2015). Cronbach’s coefficient alpha: Well known but poorly understood. Organizational Research Methods, 18(2), 207–230. https://doi.org/10.1177/1094428114555994 CrossRefGoogle Scholar
Cronbach, L. J. (1943).On estimates of test reliability.Journal of Educational Psychology, 34 (8), 485– 494 CrossRefGoogle Scholar
Cronbach, L. J. (1947). Test “reliability”: Its meaning and determination. Psychometrika, 12(1), 1–16. https://doi.org/10.1007/BF02289289 CrossRefGoogle Scholar
Cronbach, L. J. (1951).Coefficient alpha and the internal structure of tests.Psychometrika, 16 (3), 297– 334 CrossRefGoogle Scholar
Cronbach, L. J. (1978).Citation classics.Current Contents, 13, 263 Google Scholar
Cronbach, L. J., & Shavelson, R. J. (2004).My current thoughts on coefficient alpha and successor procedures.Educational and Psychological Measurement, 64 (3), 391– 418 CrossRefGoogle Scholar
Dunn, T. J., Baguley, T., & Brunsden, V. (2014).From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation.British Journal of Psychology, 105 (3), 399– 412 CrossRefGoogle Scholar
Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999).Evaluating the use of exploratory factor analysis in psychological research.Psychological Methods, 4 (3), 272 CrossRefGoogle Scholar
Feldt, L. S., & Brennan, R. L. (1989). Reliability. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 105–146). American Council on Education and Macmillan.Google Scholar
Feldt, L. S., & Charter, R. A. (2003).Estimation of internal consistency reliability when test parts vary in effective length.Measurement and Evaluation in Counseling and Development, 36 (1), 23– 27 CrossRefGoogle Scholar
Ferguson, G. A. (1951).A note on the Kuder-Richardson formula.Educational and Psychological Measurement, 11 (4–1), 612– 615 CrossRefGoogle Scholar
Gilmer, J. S., & Feldt, L. S. (1983).Reliability estimation for a test with parts of unknown lengths.Psychometrika, 48 (1), 99– 111 CrossRefGoogle Scholar
Graham, J. M. (2006).Congeneric and (essentially) tau-equivalent estimates of score reliability what they are and how to use them.Educational and Psychological Measurement, 66 (6), 930– 944 CrossRefGoogle Scholar
Guttman, L. (1945).A basis for analyzing test-retest reliability.Psychometrika, 10 (4), 255– 282 CrossRefGoogle ScholarPubMed
Hägglund, G. (2001). Milestones in the history of factor analysis. In R. Cudeck, S. du Toit, & D. Sörbom (Eds.), Structural equation modeling: Present and future–A festschrift in honor of Karl Jöreskog (pp. 11–38). Scientific Software International.Google Scholar
Hancock, G., & Mueller, R. O. (2001). Rethinking construct reliability within latent variable systems. In R. Cudeck, S. du Toit, & D. Sörbom (Eds.), Structural equation modeling: Present and future–A festschrift in honor of Karl Jöreskog (pp. 195–216). Scientific Software International.Google Scholar
Harzing, A.-W. (2007). Publish or Perish. https://harzing.com/resources/publish-or-perish Google Scholar
Hayes, A. F. , & Coutts, J. J. (2020). Use omega rather than Cronbach’s alpha for estimating reliability. But.... Communication Methods and Measures, 14(1), 1–24. https://doi.org/10.1080/19312458.2020.1718629 CrossRefGoogle Scholar
Heise, D. R., & Bohrnstedt, G. W. (1970).Validity, invalidity, and reliability.Sociological Methodology, 2, 104– 129 CrossRefGoogle Scholar
Hoyt, C. (1941).Note on a simplified method of computing test reliability.Educational and Psychological Measurement, 1 (1), 93– 95 CrossRefGoogle Scholar
Hoyt, C. (1941).Test reliability estimated by analysis of variance.Psychometrika, 6 (3), 153– 160 CrossRefGoogle Scholar
Hunt, T. D. (2013). Lambda4: Collection of internal consistency reliability coefficients. https://cran.r-project.org/web/packages/Lambda4/index.html Google Scholar
Hunt, T. D., & Bentler, P. M. (2015).Quantile lower bounds to reliability based on locally optimal splits.Psychometrika, 80 (1), 182– 195 CrossRefGoogle ScholarPubMed
Jackson, P. H., & Agunwamba, C. C. (1977).Lower bounds for the reliability of the total score on a test composed of non-homogeneous items: I: Algebraic lower bounds.Psychometrika, 42 (4), 567– 578 CrossRefGoogle Scholar
Jackson, R. W. B., & Ferguson, G. A. (1941).Studies on the reliability of tests. University of Toronto Department of.Educational Research Bulletin, 12, 132 Google Scholar
Jöreskog, K. G. (1966).Testing a simple structure hypothesis in factor analysis.Psychometrika, 31, 165– 178 CrossRefGoogle ScholarPubMed
Jöreskog, K. G. (1967a). A computer program for restricted maximum likelihood factor analysis. Research Memorandum 66-20. Educational Testing Service.Google Scholar
Jöreskog, K. G. (1967).Some contributions to maximum likelihood factor analysis.Psychometrika, 32 (4), 443– 482 CrossRefGoogle Scholar
Jöreskog, K. G. (1969).A general approach to confirmatory maximum likelihood factor analysis.Psychometrika, 34 (2), 183– 202 CrossRefGoogle Scholar
Jöreskog, K. G. (1971).Statistical analysis of sets of congeneric tests.Psychometrika, 36 (2), 109– 133 CrossRefGoogle Scholar
Kaiser, H. F., & Caffrey, J. (1965).Alpha factor analysis.Psychometrika, 30 (1), 1– 14 CrossRefGoogle ScholarPubMed
Kamata, A., Turhan, A., & Darandari, E. (2003).Estimating reliability for multidimensional composite scale scores, Chicago: Annual Meeting of American Educational Research Association. 1– 27. Google Scholar
Kristof, W. (1971).On the theory of a set of tests which differ only in length.Psychometrika, 36 (3), 207– 225 CrossRefGoogle Scholar
Kuder, G. F., & Richardson, M. W. (1937).The theory of the estimation of test reliability.Psychometrika, 2 (3), 151– 160 CrossRefGoogle Scholar
Lawley, D. N. (1940).The estimation of factor loadings by the method of maximum likelihood.Proceedings of the Royal Society of Edinburgh, 60 (1), 64– 82 CrossRefGoogle Scholar
Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Addison-Wesley.Google Scholar
McDonald, R. P. (1970).The theoretical foundations of principal factor analysis, canonical factor analysis, and alpha factor analysis.British Journal of Mathematical and Statistical Psychology, 23, 1– 21 CrossRefGoogle Scholar
McDonald, R. P. (1985). Factor analysis and related methods. Lawrence Erlbaum.Google Scholar
McDonald, R. P. (1999). Test theory: A unified treatment. Lawrence ErlbaumGoogle Scholar
Miller, M. B. (1995).Coefficient alpha: A basic introduction from the perspectives of classical test theory and structural equation modeling.Structural Equation Modeling: A Multidisciplinary Journal, 2 (3), 255– 273 CrossRefGoogle Scholar
Novick, M. R. (1966).The axioms and principal results of classical test theory.Journal of Mathematical Psychology, 3 (1), 1– 18 CrossRefGoogle Scholar
Oosterwijk, P. R. (2016). Statistical properties and practical use of classical test-score reliability methods. the Netherlands: Tilburg University. PhD dissertation.Google Scholar
Oosterwijk, P. R. , van der Ark, L. A., & Sijtsma, K. (2017). Overestimation of reliability by Guttman’s λ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\lambda $$\end{document} 4, λ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\lambda $$\end{document} 5, and λ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\lambda $$\end{document} 6 and the Greatest Lower Bound. In L. A. van der Ark, M. Wiberg, S. A. Culpepper, J. A. Douglas, & W.-C. Wang (Eds.), Quantitative psychology research: The 81th Annual Meeting of the Psychometric Society 2016, Asheville NC, USA (Vol. 196, pp. 159–172). Springer. https://doi.org/10.1007/978-3-319-56294-0_15 CrossRefGoogle Scholar
Raykov, T. (2007). Reliability if deleted, not ‘alpha if deleted’: Evaluation of scale reliability following component deletion. British Journal of Mathematical and Statistical Psychology, 60(2), 201–216. https://doi.org/10.1348/000711006X115954 CrossRefGoogle Scholar
Raykov, T., & Marcoulides, G. A. (2019).Thanks coefficient alpha, we still need you!.Educational and Psychological Measurement, 79 (1), 200– 210 CrossRefGoogle ScholarPubMed
Reuterberg, S.-E, & Gustafsson, J.-E (1992).Confirmatory factor analysis and reliability: Testing measurement model assumptions.Educational and Psychological Measurement, 52 (4), 795– 811 CrossRefGoogle Scholar
Revelle, W. (2020). psych: Procedures for psychological, psychometric, and personality research. https://cran.r-project.org/web/packages/psych/index.html.Google Scholar
Revelle, W., & Condon, D. M. (2019). Reliability from α \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha $$\end{document} to ω \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\omega $$\end{document} : A tutorial. Psychological Assessment, 31 (12), 1395– 1411 CrossRefGoogle Scholar
Rodgers, J., & Nicewander, A. (1988).Thirteen ways to look at the correlation coefficient.American Statistician, 42, 59– 66 CrossRefGoogle Scholar
Rönkkö, M., & Cho, E. (2020). An updated guideline for assessing discriminant validity. Organizational Research Methods. https://doi.org/10.1177/1094428120968614 CrossRefGoogle Scholar
Rulon, P. J. (1939).A simplified procedure for determining the reliability of a test by split-halves.Harvard Educational Review, 9 (1), 99– 103. Google Scholar
Savalei, V., & Reise, S. P. (2019). Don’t forget the model in your model-based reliability coefficients: A reply to McNeish (2018). Collabra: Psychology, 5(1), 36. https://doi.org/10.1525/collabra.247 CrossRefGoogle Scholar
Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika, 74(1), 107–120. https://doi.org/10.1007/s11336-008-9101-0 CrossRefGoogle Scholar
Sijtsma, K., & Pfadt, J. (2021). Invited review part II: On the use, the misuse, and the very limited usefulness of Cronbach’s alpha: Discussing lower bounds and correlated errors. Psychometrika. CrossRefGoogle Scholar
Sörbom, D. (2001). Karl Jöreskog and LISREL: A personal story. In Cudeck, R. du Toit, S., & Sörbom, D. (Eds.), Structural equation modeling: Present and future–A festschrift in honor of Karl Jöreskog (pp. 3–10). Scientific Software International.Google Scholar
Spearman, C. (1904a). “General intelligence,” objectively determined and measured. American Journal of Psychology, 15(2), 201–292. https://doi.org/10.2307/1412107 CrossRefGoogle Scholar
Spearman, C. (1904).The proof and measurement of association between two things.American Journal of Psychology, 15 (1), 72– 101 CrossRefGoogle Scholar
Tang, W., & Cui, Y. (2012). A simulation study for comparing three lower bounds to reliability. Paper Presented on April 17, 2012 at the AERA Division D: Measurement and Research Methodology, Section 1: Educational Measurement, Psychometrics, and Assessment., 1–25.Google Scholar
Ten Berge, J. MF. Zegers, F. E. (1978).A series of lower bounds to the reliability of a test.Psychometrika, 43 (4), 575– 579 CrossRefGoogle Scholar
Votaw, D. F. (1948).Testing compound symmetry in a normal multivariate distribution.Annals of Mathematical Statistics, 19 (4), 447– 473 CrossRefGoogle Scholar
Watkins, M. W. (2017).The reliability of multidimensional neuropsychological measures: From alpha to omega.The Clinical Neuropsychologist, 31 (6–7), 1113– 1126 CrossRefGoogle ScholarPubMed
Yang, Y., & Green, S. B. (2010).A note on structural equation modeling estimates of reliability.Structural Equation Modeling, 17 (1), 66– 81 CrossRefGoogle Scholar