Hostname: page-component-745bb68f8f-d8cs5 Total loading time: 0 Render date: 2025-01-08T12:21:17.005Z Has data issue: false hasContentIssue false
  • English
  • Français

Likelihood-Based Clustering of Meta-Analytic SROC Curves

Published online by Cambridge University Press:  01 January 2025

Heinz Holling ,
Walailuck Böhning  and
Dankmar Böhning
Heinz Holling
Affiliation:
University of Münster
Walailuck Böhning
Affiliation:
University of Münster
Dankmar Böhning*
Affiliation:
University of Southampton
*
Requests for reprints should be sent to Dankmar Böhning, School of Mathematics and Southampton Statistical Sciences Research Institute, University of Southampton, B39, R2021, Southampton, SO17 1BJ, UK. E-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Meta-analysis of diagnostic studies experience the common problem that different studies might not be comparable since they have been using a different cut-off value for the continuous or ordered categorical diagnostic test value defining different regions for which the diagnostic test is defined to be positive. Hence specificities and sensitivities arising from different studies might vary just because the underlying cut-off value had been different. To cope with the cut-off value problem interest is usually directed towards the receiver operating characteristic (ROC) curve which consists of pairs of sensitivities and false-positive rates (1-specificity). In the context of meta-analysis one pair represents one study and the associated diagram is called an SROC curve where the S stands for “summary”. In meta-analysis of diagnostic studies emphasis has traditionally been placed on modelling this SROC curve with the intention of providing a summary measure of the diagnostic accuracy by means of an estimate of the summary ROC curve. Here, we focus instead on finding sub-groups or components in the data representing different diagnostic accuracies. The paper will consider modelling SROC curves with the Lehmann family which is characterised by one parameter only. Each single study can be represented by a specific value of that parameter. Hence we focus on the distribution of these parameter estimates and suggest modelling a potential heterogeneous or cluster structure by a mixture of specifically parameterised normal densities. We point out that this mixture is completely nonparametric and the associated mixture likelihood is well-defined and globally bounded. We use the theory and algorithms of nonparametric mixture likelihood estimation to identify a potential cluster structure in the diagnostic accuracies of the collection of studies to be analysed. Several meta-analytic applications on diagnostic studies, including AUDIT and AUDIT-C for detection of unhealthy alcohol use, the mini-mental state examination for cognitive disorders, as well as diagnostic accuracy inspection data on metal fatigue of aircraft spare parts, are discussed to illustrate the methodology.

Keywords

C.A.MANdiagnostic testingmeta-analysissensitivityspecificitysummary receiver operating characteristic (SROC)summary statistics approach
Type
Article
Information
Psychometrika , Volume 77 , Issue 1 , January 2012 , pp. 106 - 126
Copyright
Copyright © 2011 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Aergeerts, B., Buntinx, F., Kester, A. (2004). The value of the CAGE in screening for alcohol abuse and alcohol dependence in general clinical populations: A diagnostic meta-analysis. Journal of Clinical Epidemiology, 57, 30–39.CrossRefGoogle Scholar
Aitkin, M. (1999). A general maximum likelihood analysis of variance components in generalized linear models. Biometrics, 55, 117–128.CrossRefGoogle ScholarPubMed
Aitkin, M. (1999). Meta-analysis by random effect modelling in generalized linear models. Statistics in Medicine, 18, 2343–2351.3.0.CO;2-3>CrossRefGoogle ScholarPubMed
Biggerstaff, B.J., Tweedie, R.L. (1997). Incorporating variability in estimates of heterogeneity in the random effects model in meta-analysis. Statistics in Medicine, 16, 753–768.3.0.CO;2-G>CrossRefGoogle ScholarPubMed
Böhning, D., Dietz, E., Schlattmann, P. (1998). Recent developments in computer-assisted analysis of mixtures (C.A.MAN). Biometrics, 54, 367–377.CrossRefGoogle Scholar
Böhning, D. (2000). Computer-assisted analysis of mixtures and applications. Meta-analysis, disease mapping and others, Boca Raton: Chapman & Hall/CRC.Google Scholar
Böhning, D., Malzahn, U., Dietz, E., Schlattmann, P., Viwatwongkasem, C., Biggeri, A. (2002). Some general points in estimating heterogeneity variance with the DerSimonian–Laird estimator. Biostatistics, 3, 445–457.CrossRefGoogle ScholarPubMed
Böhning, D. (2003). The EM algorithm with gradient function update for discrete mixtures with known (fixed) number of components. Statistics and Computing, 13, 257–265.CrossRefGoogle Scholar
Böhning, D., Viwatwongkasem, C. (2005). Revisiting proportion estimators. Statistical Methods in Medical Research, 14, 1–23.CrossRefGoogle ScholarPubMed
Böhning, D., Holling, H., Böhning, W. (2008). Revisiting Youden’s index as a useful measure of the misclassification error in meta-analysis of diagnostic studies. Statistical Methods in Medical Research, 17, 543–554.CrossRefGoogle ScholarPubMed
Brockwell, S.E., Gordon, I.R. (2001). A comparison of statistical methods for meta-analysis. Statistical Methods in Medical Research, 20, 825–840.CrossRefGoogle ScholarPubMed
Clayton, D.G., Kaldor, J. (1987). Empirical Bayes estimates of age-standardized relative risks for use in disese mapping. Biometrics, 43, 671–681.CrossRefGoogle Scholar
Cooper, H., Hedges, L. (1994). The handbook of research synthesis, New York: Russell Sage Foundation.Google Scholar
Dempster, A.P., Laird, N.M., Rubin, D.B. (1977). Maximum likelihood estimation from incomplete data via the EM algorithm (with discussion). Journal of the Royal Statistical Society. Series B, 39, 1–38.CrossRefGoogle Scholar
DerSimonian, R., Laird, N. (1986). Meta-analysis in clinical trials. Controlled Clinical Trials, 7, 177–188.CrossRefGoogle ScholarPubMed
Egger, M., Smith, G.D., Altman, D.G. (2001). Systematic reviews in health care: Meta-analysis in context, London: BMJ Publishing Group.CrossRefGoogle Scholar
Gatsonis, C., Paliwal, P. (2006). Meta-analysis of diagnostic and screening test accuracy evaluations: Methodologic primer. American Journal of Roentgenology, 187, 271–281.CrossRefGoogle ScholarPubMed
Harbord, R.M., Deeks, J.J., Egger, M., Whiting, P., Sterne, J.A.C. (2007). A unification of models for meta-analysis of diagnostic accuracy studies. Biostatistics, 8, 239–251.CrossRefGoogle ScholarPubMed
Hardy, R.J., Thompson, S.G. (1996). A likelihood approach to meta-analysis with random effects. Statistics in Medicine, 15, 619–629.3.0.CO;2-A>CrossRefGoogle ScholarPubMed
Hardy, R.J., Thompson, S.G. (1998). Detecting and describing heterogeneity in meta-analysis. Statistics in Medicine, 17, 841–856.3.0.CO;2-D>CrossRefGoogle ScholarPubMed
Hasselblad, V., Hedges, L.V. (1995). Meta-analysis of screening and diagnostic tests. Psychological Bulletin, 117, 167–178.CrossRefGoogle ScholarPubMed
Hedges, L.V., Olkin, I. (1985). Statistical methods for meta-analysis, New York: Academic Press.Google Scholar
Hunter, J.E., Schmidt, F.L., Jackson, B.G. (1982). Meta-analysis: Cumulative research findings across studies, Beverly Hills: Sage Publications.Google Scholar
Irwig, L., Tosteson, A.N., Gatsonis, C., Lau, J., Colditz, G., Chalmers, T.C., Mosteller, F. (1994). Guidelines for meta-analyses evaluating diagnostic tests. Annals of Internal Medicine, 120, 667–676.CrossRefGoogle ScholarPubMed
Irwig, L., Macaskill, P., Glasziou, P., Fahey, M. (1995). Meta-analytic methods for diagnostic test accuracy. Journal of Clinical Epidemiology, 48, 119–130.CrossRefGoogle ScholarPubMed
Kiefer, J., Wolfowitz, J. (1956). Consistency of the maximum likelihood estimator in the presence of infinitely many incidental parameters. Annals of Mathematical Statistics, 27, 886–906.CrossRefGoogle Scholar
Kriston, L., Hölzel, L., Weiser, A., Berner, M.M., Härter, M. (2008). Meta-analysis: Are 3 questions enough to detect unhealthy alcohol use?. Annals of Internal Medicine, 149, 879–888.CrossRefGoogle ScholarPubMed
Kuhnert, R., Böhning, D. (2007). A comparison of three different models for estimating relative risk in meta-analysis. Statistics in Medicine, 28, 2277–2296.CrossRefGoogle Scholar
Laird, N.M. (1978). Nonparametric maximum likelihood estimation of a mixing distribution. Journal of the American Statistical Association, 73, 805–811.CrossRefGoogle Scholar
Le, C.T. (2006). A solution for the most basic optimization problem associated with an ROC curve. Statistical Methods in Medical Research, 15, 571–584.CrossRefGoogle ScholarPubMed
Lindsay, B.G. (1983). The geometry of mixture likelihoods: A general theory. Annals of Statistics, 11, 86–94.CrossRefGoogle Scholar
Lindsay, B.G. (1995). Mixture models: Theory, geometry, and applications, Hayward: Institute of Statistical Mathematics.Google Scholar
Macaskill, P., Glasziou, P., Irwig, L. Meta-analysis of diagnostic tests. In Armitage, P., Colton, T. (Eds.), Encyclopedia of biostatistics, 2005.Google Scholar
Martin, C.S., Liepman, M.R., Young, C.M. (1990). The Michigan alcoholism screening test: False positives in a college student sample. Alcoholism, Clinical and Experimental Research, 14, 853–855.CrossRefGoogle Scholar
McCullagh, P., Nelder, J.A. (1989). Generalized linear models, London: Chapman & Hall.CrossRefGoogle Scholar
McLachlan, G., Peel, D. (2000). Finite mixture models, New York: Wiley.CrossRefGoogle Scholar
Midgette, A.S., Stukel, T.A., Littenberg, B. (1993). A meta-analytic method for summarizing diagnostic test performances: Receiver-operating-characteristic-summary point estimates. Medical Decision Making, 13, 253–257.CrossRefGoogle ScholarPubMed
Mitchell, A.J. (2009). A meta-analysis of the accuracy of the mini-mental state examination in the detection of dementia and mild cognitive impairment. Journal of Psychiatric Research, 43, 411–431.CrossRefGoogle ScholarPubMed
Moses, L.E., Littenberg, B., Shapiro, D. (1993). Combining independent studies of a diagnostic test into a summary ROC curve: Data-analytical approaches and some additional considerations. Statistics in Medicine, 12, 1293–1316.CrossRefGoogle Scholar
Ray, S., Lindsay, B.G. (2008). Model selection in high dimensions: A quadratic-risk-based approach. Journal of the Royal Statistical Society. Series B, 70, 95–118.CrossRefGoogle Scholar
Rabe-Hesketh, S., Pickles, A., Skrondal, A. (2003). Correcting for covariate measurement error in logistic regression using nonparametric maximum likelihood estimation. Statistical Modelling, 3, 215–232.CrossRefGoogle Scholar
Reinert, D.F., Allen, J.P. (2002). The alcohol use identification test (AUDIT): A review of recent research. Alcoholism, Clinical and Experimental Research, 26, 272–279.CrossRefGoogle ScholarPubMed
Reitsma, J.B., Glas, A.S., Rutjes, A.W.S., Scholten, R.J.P.M., Bossuyt, P.M., Zwinderman, A.H. (2005). Bivariate analysis of sensitivity and specificity produces informative measures in diagnostic reviews. Journal of Clinical Epidemiology, 58, 982–990.CrossRefGoogle ScholarPubMed
Ross, S. (1985). Introduction to probability models, Orlando: Academic Press.Google Scholar
Rutter, C.M., Gatsonis, C.A. (2001). A hierarchical regression approach to meta-analysis of diagnostic test accuracy evaluations. Statistics in Medicine, 20, 2865–2884.CrossRefGoogle ScholarPubMed
Schulze, R., Holling, H., Böhning, D. (2003). Meta-analysis. New developments and applications in medical and social sciences, Göttingen: Hogrefe& Huber.Google Scholar
Selzer, M.L. (1971). The Michigan alcoholism screening test: The quest for a new diagnostic instrument. American Journal of Psychiatry, 127, 1653–1658.CrossRefGoogle ScholarPubMed
Sidik, K., Jonkman, J.N. (2005). Simple heterogeneity variance estimation for meta-analysis. Journal of the Royal Statistical Society. Series C, 54, 367–384.CrossRefGoogle Scholar
Skrondal, A., Rabe-Hesketh, S. (2004). Generalized latent variable modeling: Multilevel, longitudinal, and structural equation models, Boca Raton: Chapman & Hall/CRC.CrossRefGoogle Scholar
Storgaard, H., Nielsen, S.D., Gluud, C. (1994). The validity of the Michigan alcoholism screening test (MAST). Alcohol and Alcoholism, 29, 493–502.Google ScholarPubMed
Sutton, A.J., Abrams, K.R., Jones, D.R., Sheldon, T.A., Song, F. (2000). Methods for meta-analysis in medical research, New York: Wiley.Google Scholar
Sweeting, M.J., Sutton, A.J., Lambert, P.C. (2004). What to add to nothing? Use and avoidance of continuity corrections in meta-analysis of sparse data. Statistics in Medicine, 23, 1351–1375.CrossRefGoogle ScholarPubMed
Swets, J.A. (2009). Signal detection theory and ROC analysis in psychology and diagnostics, New York: Psychology Press.Google Scholar
van Houwelingen, H.C., Zwinderman, K.H., Stijnen, T. (1993). A bivariate approach to meta-analysis. Statistics in Medicine, 12, 2273–2284.CrossRefGoogle ScholarPubMed
Whitehead, A. (2002). Meta-analysis of controlled clinical trials, New York: Wiley.CrossRefGoogle Scholar