Hostname: page-component-6587cd75c8-6rxlm Total loading time: 0 Render date: 2025-04-23T17:47:03.277Z Has data issue: false hasContentIssue false
  • English
  • Français

On Similarity Coefficients for 2×2 Tables and Correction for Chance

Published online by Cambridge University Press:  01 January 2025

Matthijs J. Warrens
Matthijs J. Warrens*
Affiliation:
Leiden University
*
Requests for reprints should be sent to Matthijs J. Warrens, Psychometrics and Research Methodology Group, Leiden University Institute for Psychological Research, Leiden University, Wassenaarseweg 52, P.O. Box 9555, 2300 RB Leiden, The Netherlands. E-mail: [email protected]
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

This paper studies correction for chance in coefficients that are linear functions of the observed proportion of agreement. The paper unifies and extends various results on correction for chance in the literature. A specific class of coefficients is used to illustrate the results derived in this paper. Coefficients in this class, e.g. the simple matching coefficient and the Dice/Sørenson coefficient, become equivalent after correction for chance, irrespective of what expectation is used. The coefficients become either Cohen’s kappa, Scott’s pi, Mak’s rho, Goodman and Kruskal’s lambda, or Hamann’s eta, depending on what expectation is considered appropriate. Both a multicategorical generalization and a multivariate generalization are discussed.

Keywords

indices of associationresemblance measurescorrection for chanceCohen’s kappaScott’s piMak’s rhoGoodman and Kruskal’s lambdaHamann’s etasimple matching coefficientDice/Sørenson coefficient
Type
Theory and Methods
Information
Psychometrika , Volume 73 , Issue 3 , September 2008 , pp. 487 - 502
Creative Commons
Creative Common License - CCCreative Common License - BYCreative Common License - NC
This is an article distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
Copyright
Copyright © 2008 The Author(s)

Footnotes

The author thanks two anonymous reviewers for their helpful comments and valuable suggestions on earlier versions of this article.

References

Albatineh, A.N., Niewiadomska-Bugaj, M., & Mihalko, D. (2006). On similarity indices and correction for chance agreement. Journal of Classification, 23, 301–313.CrossRefGoogle Scholar
Baulieu, F.B. (1989). A classification of presence/absence based dissimilarity coefficients. Journal of Classification, 6, 233–246.CrossRefGoogle Scholar
Blackman, N.J.-M., & Koval, J.J. (1993). Estimating rater agreement in 2×2 tables: Correction for chance and intraclass correlation. Applied Psychological Measurement, 17, 211–223.CrossRefGoogle Scholar
Bloch, D.A., & Kraemer, H.C. (1989). 2×2 Kappa coefficients: Measures of agreement or association. Biometrics, 45, 269–287.CrossRefGoogle ScholarPubMed
Bray, J.R. (1956). A study of mutual occurrence of plant species. Ecology, 37, 21–28.CrossRefGoogle Scholar
Brennan, R.L., & Light, R.J. (1974). Measuring agreement when two observers classify people into categories not defined in advance. British Journal of Mathematical and Statistical Psychology, 27, 154–163.CrossRefGoogle Scholar
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37–46.CrossRefGoogle Scholar
Czekanowski, J. (1932). Coefficient of racial likeliness und Durchschnittliche Differenz. Anothropologidcher, 14, 227–249.Google Scholar
Dice, L.R. (1945). Measures of the amount of ecologic association between species. Ecology, 26, 297–302.CrossRefGoogle Scholar
Fleiss, J.L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin, 76, 378–382.CrossRefGoogle Scholar
Fleiss, J.L. (1975). Measuring agreement between two judges on the presence or absence of a trait. Biometrics, 31, 651–659.CrossRefGoogle ScholarPubMed
Gleason, H.A. (1920). Some applications of the quadrat method. Bulletin of the Torrey Botanical Club, 47, 21–33.CrossRefGoogle Scholar
Goodman, L.A., & Kruskal, W.H. (1954). Measures of association for cross classifications. Journal of the American Statistical Association, 49, 732–764.Google Scholar
Gower, J.C., & Legendre, P. (1986). Metric and Euclidean properties of dissimilarity coefficients. Journal of Classification, 3, 5–48.CrossRefGoogle Scholar
Hamann, U. (1961). Merkmalsbestand und Verwandtschaftsbeziehungen der Farinose. Ein Betrag zum System der Monokotyledonen. Willdenowia, 2, 639–768.Google Scholar
Heuvelmans, A.P.J.M., & Sanders, P.F. (1993). Beoordelaarsovereenstemming. In Eggen, T.J.H.M., & Sanders, P.F. (Eds.), Psychometrie in de praktijk (pp. 443–470). Arnhem: Cito Instituut voor Toetsontwikkeling.Google Scholar
Hubálek, Z. (1982). Coefficients of association and similarity based on binary (presence-absence) data: An evaluation. Biological Reviews, 57, 669–689.CrossRefGoogle Scholar
Hubert, L.J. (1977). Nominal scale response agreement as a generalized correlation. British Journal of Mathematical and Statistical Psychology, 30, 98–103.CrossRefGoogle Scholar
Hubert, L.J., & Arabie, P. (1985). Comparing partitions. Journal of Classification, 2, 193–218.CrossRefGoogle Scholar
Jaccard, P. (1912). The distribution of the flora in the Alpine zone. The New Phytologist, 11, 37–50.CrossRefGoogle Scholar
Krippendorff, K. (1987). Association, agreement, and equity. Quality and Quantity, 21, 109–123.CrossRefGoogle Scholar
Light, R.J. (1971). Measures of response agreement for qualitative data: Some generalizations and alternatives. Psychological Bulletin, 76, 365–377.CrossRefGoogle Scholar
Mak, T.K. (1988). Analyzing intraclass correlation for dichotomous variables. Applied Statistics, 37, 344–352.CrossRefGoogle Scholar
Morey, L.C., & Agresti, A. (1984). The measurement of classification agreement: An adjustment to the Rand statistic for chance agreement. Educational and Psychological Measurement, 44, 33–37.CrossRefGoogle Scholar
Nei, M., & Li, W.-H. (1979). Mathematical model for studying genetic variation in terms of restriction endonucleases. Proceedings of the National Academy of Sciences, 76, 5269–5273.CrossRefGoogle ScholarPubMed
Pearson, E.S. (1947). The choice of statistical tests illustrated on the interpretation of data classed in a 2×2 table. Biometrika, 34, 139–167.Google Scholar
Popping, R. (1983). Overeenstemmingsmaten voor nominale data. Ph.D. thesis, Groningen, Rijksuniversiteit Groningen.Google Scholar
Rand, W. (1971). Objective criteria for the evaluation of clustering methods. Journal of the American Statistical Association, 66, 846–850.CrossRefGoogle Scholar
Rogot, E., & Goldberg, I.D. (1966). A proposed index for measuring agreement in test-retest studies. Journal of Chronic Disease, 19, 991–1006.CrossRefGoogle ScholarPubMed
Scott, W.A. (1955). Reliability of content analysis: the case of nominal scale coding. Public Opinion Quarterly, 19, 321–325.CrossRefGoogle Scholar
Sokal, R.R., & Michener, C.D. (1958). A statistical method for evaluating systematic relationships. University of Kansas Science Bulletin, 38, 1409–1438.Google Scholar
Sokal, R.R., & Sneath, P.H. (1963). Principles of numerical taxonomy, San Francisco: Freeman.Google Scholar
Sørenson, T. (1948). A method of stabilizing groups of equivalent amplitude in plant sociology based on the similarity of species content and its application to analyses of the vegetation on Danish commons. Kongelige Danske Videnskabernes Selskab Biologiske Skrifter, 5, 1–34.Google Scholar
Steinley, D. (2004). Properties of the Hubert–Arabie adjusted Rand index. Psychological Methods, 9, 386–396.CrossRefGoogle ScholarPubMed
Warrens, M.J. (2008, in press). On the indeterminacy of resemblance measures for binary (presence/absence) data. Journal of Classification.CrossRefGoogle Scholar
Zegers, F.E. (1986). A General family of association coefficients. Ph.D. thesis, Groningen, Rijksuniversiteit Groningen.Google Scholar
Zegers, F.E., & Ten Berge, J.M.F. (1985). A family of association coefficients for metric scales. Psychometrika, 50, 17–24.CrossRefGoogle Scholar