Hostname: page-component-cd9895bd7-hc48f Total loading time: 0 Render date: 2024-12-26T06:58:54.943Z Has data issue: false hasContentIssue false
  • English
  • Français

Machine learning enhances prediction of illness course: a longitudinal study in eating disorders

Published online by Cambridge University Press:  28 February 2020

Ann F. Haynos ,
Shirley B. Wang ,
Sarah Lipson ,
Carol B. Peterson ,
James E. Mitchell ,
Katherine A. Halmi ,
W. Stewart Agras  and
Scott J. Crow
Ann F. Haynos*
Affiliation:
Department of Psychiatry and Behavioral Sciences, University of Minnesota, Minneapolis, MN, USA
Shirley B. Wang
Affiliation:
Department of Psychology, Harvard University, Cambridge, MA, USA
Sarah Lipson
Affiliation:
Department of Psychology, Harvard University, Cambridge, MA, USA
Carol B. Peterson
Affiliation:
Department of Psychiatry and Behavioral Sciences, University of Minnesota, Minneapolis, MN, USA The Emily Program, Minneapolis, MN, USA
James E. Mitchell
Affiliation:
Department of Psychiatry and Behavioral Science, University of North Dakota School of Medicine and Health Sciences, Fargo, ND, USA
Katherine A. Halmi
Affiliation:
New York Presbyterian Hospital-Westchester Division, Weill Medical College of Cornell University, White Plains, NY, USA
W. Stewart Agras
Affiliation:
Department of Psychiatry, Stanford University School of Medicine, Stanford, CA, USA
Scott J. Crow
Affiliation:
Department of Psychiatry and Behavioral Sciences, University of Minnesota, Minneapolis, MN, USA The Emily Program, Minneapolis, MN, USA
*
Author for correspondence: Ann F. Haynos, E-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Background

Psychiatric disorders, including eating disorders (EDs), have clinical outcomes that range widely in severity and chronicity. The ability to predict such outcomes is extremely limited. Machine-learning (ML) approaches that model complexity may optimize the prediction of multifaceted psychiatric behaviors. However, the investigations of many psychiatric concerns have not capitalized on ML to improve prognosis. This study conducted the first comparison of an ML approach (elastic net regularized logistic regression) to traditional regression to longitudinally predict ED outcomes.

Methods

Females with heterogeneous ED diagnoses completed demographic and psychiatric assessments at baseline (n = 415) and Year 1 (n = 320) and 2 (n = 277) follow-ups. Elastic net and traditional logistic regression models comprising the same baseline variables were compared in ability to longitudinally predict ED diagnosis, binge eating, compensatory behavior, and underweight BMI at Years 1 and 2.

Results

Elastic net models had higher accuracy for all outcomes at Years 1 and 2 [average Area Under the Receiving Operating Characteristics Curve (AUC) = 0.78] compared to logistic regression (average AUC = 0.67). Model performance did not deteriorate when the most important predictor was removed or an alternative ML algorithm (random forests) was applied. Baseline ED (e.g. diagnosis), psychiatric (e.g. hospitalization), and demographic (e.g. ethnicity) characteristics emerged as important predictors in exploratory predictor importance analyses.

Conclusions

ML algorithms can enhance the prediction of ED symptoms for 2 years and may identify important risk markers. The superior accuracy of ML for predicting complex outcomes suggests that these approaches may ultimately aid in advancing precision medicine for serious psychiatric disorders.

Keywords

Anorexia nervosabinge-eating disorderbulimia nervosacomputational psychiatryeating disordermachine learning
Type
Original Articles
Information
Psychological Medicine , Volume 51 , Issue 8 , June 2021 , pp. 1392 - 1402
Check for updates
Copyright
Copyright © The Author(s), 2020. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

*

The first two authors contributed equally to this work.

References

Agras, W. S., Crow, S., Mitchell, J. E., Halmi, K. A., & Bryson, S. (2009). A 4-year prospective study of eating disorder NOS compared with full eating disorder syndromes. The International Journal of Eating Disorders, 42, 565–570. doi: 10.1002/eat.20708CrossRefGoogle ScholarPubMed
American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders: DSM-IV-TR (4th ed.). text revision. Washington, DC: American Psychiatric Association.Google Scholar
Askland, K. D., Garnaat, S., Sibrava, N. J., Boisseau, C. L., Strong, D., Mancebo, M., … Eisen, J. (2015). Prediction of remission in obsessive compulsive disorder using a novel machine learning strategy. International Journal of Methods in Psychiatric Research, 24, 156–169. doi: 10.1002/mpr.1463CrossRefGoogle ScholarPubMed
Bardone-Cone, A. M., Hunt, R. A., & Watson, H. J. (2018). An overview of conceptualizations of eating disorder recovery, recent findings, and future directions. Current Psychiatry Reports, 20, 79. doi: 10.1007/s11920-018-0932-9CrossRefGoogle ScholarPubMed
Beck, A. T., Steer, R. A., & Brown, G. K. (1996). Manual for the Beck Depression Inventory-II. San Antonio, TX: Psychological Corporation.Google Scholar
Beck, A. T., Steer, R. A., & Carbin, M. G. (1988). Psychometric properties of the Beck Depression Inventory: Twenty-five years of evaluation. Clinical Psychology Review, 8, 77–100. doi: 10.1016/0272-7358(88)90050-5CrossRefGoogle Scholar
Berg, K. C., Peterson, C. B., Frazier, P., & Crow, S. J. (2012a). Psychometric evaluation of the eating disorder examination and eating disorder examination-questionnaire: A systematic review of the literature. The International Journal of Eating Disorders, 4, 428–438. doi: 10.1002/eat.20931CrossRefGoogle Scholar
Berg, K. C., Stiles-Shields, E. C., Swanson, S. A., Peterson, C. B., Lebow, J., & Le Grange, D. (2012b). Diagnostic concordance of the interview and questionnaire versions of the eating disorder examination. The International Journal of Eating Disorders, 45, 850–855. doi: 10.1002/eat.20948CrossRefGoogle Scholar
Berkman, N. D., Bulik, C. M., Brownley, K. A., Lohr, K. N., Sedway, J. A., Rooks, A., & Gartlehner, G. (2006). Management of eating disorders. Evidence Report/Technology Assessment, 135, 1–166.Google Scholar
Bilder, R. M., Howe, A. G., & Sabb, F. W. (2013). Multilevel models from biology to psychology: Mission impossible? Journal of Abnormal Psychology, 122, 917–927. doi: 10.1037/a0032263CrossRefGoogle ScholarPubMed
Burke, T. A., Ammerman, B. A., & Jacobucci, R. (2019). The use of machine learning in the study of suicidal and non-suicidal self-injurious thoughts and behaviors: A systematic review. Journal of Affective Disorders, 245, 869–884. doi: 10.1016/j.jad.2018.11.073CrossRefGoogle Scholar
Bzdok, D., & Meyer-Lindenberg, A. (2018). Machine learning for precision psychiatry: Opportunities and challenges. Biological Psychiatry. Cognitive Neuroscience and Neuroimaging, 3, 223–230. doi: 10.1016/j.bpsc.2017.11.007CrossRefGoogle ScholarPubMed
Canty, A., & Ripley, B. (2019). boot: Bootstrap R (S-Plus) Functions. Version 1.3-22.Google Scholar
Carter, J. C., Blackmore, E., Sutandar-Pinnock, K., & Woodside, D. B. (2004). Relapse in anorexia nervosa: A survival analysis. Psychological Medicine, 34, 671–679.CrossRefGoogle ScholarPubMed
Carter, J. C., Mercer-Lynn, K. B., Norwood, S. J., Bewell-Weiss, C. V., Crosby, R. D., Woodside, D. B., & Olmsted, M. P. (2012). A prospective study of predictors of relapse in anorexia nervosa: Implications for relapse prevention. Psychiatry Research, 200, 518–523. doi: 10.1016/j.psychres.2012.04.037CrossRefGoogle ScholarPubMed
Challen, R., Denny, J., Pitt, M., Gompels, L., Edwards, T., & Tsaneva-Atanasova, K. (2019). Artificial intelligence, bias and clinical safety. Artificial intelligence, bias and clinical safety. BMJ Quality & Safety, 28, 231–237. doi: 10.1136/bmjqs-2018-008370CrossRefGoogle ScholarPubMed
Cho, G., Yim, J., Choi, Y., Ko, J., & Lee, S.-H. (2019). Review of machine learning algorithms for diagnosing mental illness. Psychiatry Investigation, 16, 262–269. doi: 10.30773/pi.2018.12.21.2CrossRefGoogle ScholarPubMed
Cooper, Z., & Fairburn, C. (1987). The eating disorder examination: A semi-structured interview for the assessment of the specific psychopathology of eating disorders. The International Journal of Eating Disorders, 6, 1–8. doi: 10.1002/1098-108X(198701)6:1<1::AID-EAT2260060102>3.0.CO;2-93.0.CO;2-9>CrossRefGoogle Scholar
Crow, S. J., Agras, W. S., Halmi, K., Mitchell, J. E., & Kraemer, H. C. (2002). Full syndromal versus subthreshold anorexia nervosa, bulimia nervosa, and binge eating disorder: A multicenter study. The International Journal of Eating Disorders, 32, 309–318. doi: 10.1002/eat.10088CrossRefGoogle ScholarPubMed
Crow, S. J., Peterson, C. B., Swanson, S. A., Raymond, N. C., Specker, S., Eckert, E. D., & Mitchell, J. E. (2009). Increased mortality in bulimia nervosa and other eating disorders. The American Journal of Psychiatry, 166, 1342–1346. doi: 10.1176/appi.ajp.2009.09020247CrossRefGoogle ScholarPubMed
DuBois, R. H., Rodgers, R. F., Franko, D. L., Eddy, K. T., & Thomas, J. J. (2017). A network analysis investigation of the cognitive-behavioral theory of eating disorders. Behavior Research and Therapy, 97, 213–221. doi: 10.1016/j.brat.2017.08.004CrossRefGoogle ScholarPubMed
Eddy, K. T., Tabri, N., Thomas, J. J., Murray, H. B., Keshaviah, A., Hastings, E., … Franko, D. L. (2017). Recovery from anorexia nervosa and bulimia nervosa at 22-year follow-up. The Journal of Clinical Psychiatry, 78, 184–189. doi: 10.4088/JCP.15m10393CrossRefGoogle ScholarPubMed
First, M. B., Gibbon, M., Spitzer, R. L., Williams, J. B. W., & Benjamin, L. S. (1997). Structured Clinical Interview for DSM-IV Axis II Personality Disorders (SCID-II). Washington, DC: American Psychiatric Press, Inc.Google Scholar
First, M. B., Spitzer, R. L., Gibbon, M., & Williams, J. B. W. (1995). Structured Clinical Interview for DSM-IV Axis I Disorders: Patient Edition (SCID-I/P). Version 2.0. Biometrics Research Department, New York State Psychiatric Institute: New York.Google Scholar
Fox, K. R., Huang, X., Linthicum, K. P., Wang, S. B., Franklin, J. C., & Ribeiro, J. D. (2019). Model complexity improves the prediction of nonsuicidal self-injury. Journal of Consulting and Clinical Psychology, 87, 684–692. doi: 10.1037/ccp0000421CrossRefGoogle ScholarPubMed
Franklin, J. C., Ribeiro, J. D., Fox, K. R., Bentley, K. H., Kleiman, E. M., Huang, X., … Nock, M. K. (2017). Risk factors for suicidal thoughts and behaviors: A meta-analysis of 50 years of research. Psychological Bulletin, 143, 187–232. doi: 10.1037/bul0000084CrossRefGoogle ScholarPubMed
Franko, D. L., Tabri, N., Keshaviah, A., Murray, H. B., Herzog, D. B., & Thomas, J. J. (2018). Predictors of long-term recovery in anorexia nervosa and bulimia nervosa: Data from a 22-year longitudinal study. Journal of Psychiatry Research, 96, 183–188. doi: 10.1016/j.jpsychires.2017.10.008CrossRefGoogle ScholarPubMed
Gormally, J., Black, S., Daston, S., & Rardin, D. (1982). The assessment of binge eating severity among obese persons. Addictive Behaviors, 7, 47–55. doi: 10.1016/0306-4603(82)90024-7CrossRefGoogle ScholarPubMed
Greeno, C. G., Marcus, M. D., & Wing, R. R. (1995). Diagnosis of binge eating disorder: Discrepancies between a questionnaire and clinical interview. The International Journal of Eating Disorders, 17, 153–160. doi: 10.1002/1098-108x(199503)17:2<153::aid-eat2260170208>3.0.co;2-v3.0.CO;2-V>CrossRefGoogle ScholarPubMed
Guarda, A. S., Wonderlich, S., Kaye, W., & Attia, E. (2018). A path to defining excellence in intensive treatment for eating disorders. The International Journal of Eating Disorders, 51, 1051–1055. doi: 10.1002/eat.22899CrossRefGoogle ScholarPubMed
He, H., & Garcia, E. A. (2009). Learning from imbalanced data. IEEE Transactions on Knowledge and Data Engineering, 21, 1263–1284. doi: 10.1109/TKDE.2008.239Google Scholar
Jordan, M. I., & Mitchell, T. M. (2015). Machine learning: Trends, perspectives, and prospects. Science, 349, 255–260. doi: 10.1126/science.aaa8415CrossRefGoogle ScholarPubMed
Kazdin, A. E., Fitzsimmons-Craft, E. E., & Wilfley, D. E. (2017). Addressing critical gaps in the treatment of eating disorders. The International Journal of Eating Disorders, 50, 170–189. doi: 10.1002/eat.22670CrossRefGoogle ScholarPubMed
Kessler, R. C., van Loo, H. M., Wardenaar, K. J., Bossarte, R. M., Brenner, L. A., Cai, T., … Zaslavsky, A. M. (2016). Testing a machine-learning algorithm to predict the persistence and severity of major depressive disorder from baseline self-reports. Molecular Psychiatry, 21, 1366–1371. doi: 10.1038/mp.2015.198CrossRefGoogle ScholarPubMed
Koutsouleris, N., Kahn, R. S., Chekroud, A. M., Leucht, S., Falkai, P., & Wobrock, T. (2016). Multisite prediction of 4-week and 52-week treatment outcomes in patients with first-episode psychosis: A machine learning approach. The Lancet. Psychiatry, 3, 935–946. doi: 10.1016/S2215-0366(16)30171-7CrossRefGoogle ScholarPubMed
Kuhn, M. (2008). Building predictive models in R using the caret package. Journal of Statistical Software, 28, 1–26. doi: 10.18637/jss.v028.i05CrossRefGoogle Scholar
Kuhn, M., & Johnson, K. (2013). Applied predictive modeling. New York: Springer.CrossRefGoogle Scholar
Linthicum, K. P., Schafer, K. M., & Ribeiro, J. D. (2019). Machine learning in suicide science: Applications and ethics. Behavioral Science and the Law, 37, 214–222. doi: 10.1002/bsl.2392CrossRefGoogle ScholarPubMed
Lock, J., Agras, W. S., Le Grange, D., Couturier, J., Safer, D., & Bryson, S. W. (2013). Do end of treatment assessments predict outcome at follow-up in eating disorders? The International Journal of Eating Disorders, 46, 771–778. doi: 10.1002/eat.22175CrossRefGoogle ScholarPubMed
Marquand, A. F., Wolfers, T., Mennes, M., Buitelaar, J., & Beckmann, C. F. (2016). Beyond lumping and splitting: A review of computational approaches for stratifying psychiatric disorders. Biological Psychiatry. Cognitive Neuroscience and Neuroimaging, 1, 433–447. doi: 10.1016/j.bpsc.2016.04.002.CrossRefGoogle ScholarPubMed
Marucci, S., Ragione, L. D., De Iaco, G., Mococci, T., Vicini, M., Guastamacchia, E., & Triggiani, V. (2018). Anorexia nervosa and comorbid psychopathology. Endocrine, Metabolic, and Immune Disorders Drug Targets, 18, 316–324. doi: 10.2174/1871530318666180213111637CrossRefGoogle ScholarPubMed
McMahon, F. J. (2014). Prediction of treatment outcomes in psychiatry – where do we stand? Dialogues in Clinical Neuroscience, 16, 455–464.Google ScholarPubMed
Mitchell, E., & Crow, E. (2006). Medical complications of anorexia nervosa and bulimia nervosa. Current Opinions in Psychiatry, 19, 438–443. doi: 10.1097/01.yco.0000228768.79097.3eCrossRefGoogle ScholarPubMed
Ogutu, J. O., Schulz-Streeck, T., & Piepho, H.-P. (2012). Genomic selection using regularized linear regression models: Ridge regression, lasso, elastic net and their extensions. BMC Proceedings, 6(Suppl 2), S10. doi: 10.1186/1753-6561-6-S2-S10CrossRefGoogle ScholarPubMed
R Development Core Team (2013). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing.Google Scholar
Ribeiro, J. D., Huang, X., Fox, K. R., Walsh, C. G., & Linthicum, K. P. (2019). Predicting imminent suicidal thoughts and nonfatal attempts: The role of complexity. Journal of Consulting and Clinical Psychology, 87, 684–692. doi: 10.1177/2167702619838464Google Scholar
Rosenberg, M. (1965). Society and the adolescent self-image. Princeton, NJ: Princeton University Press.CrossRefGoogle Scholar
Smink, F. R., van Hoeken, D., & Hoek, H. W. (2012). Epidemiology of eating disorders: Incidence, prevalence and mortality rates. Current Psychiatry Reports, 14, 406–414. doi: 10.1007/s11920-012-0282-yCrossRefGoogle ScholarPubMed
Smith, G. C. S., Seaman, S. R., Wood, A. M., Royston, P., & White, I. R. (2014). Correcting for optimistic prediction in small data sets. American Journal of Epidemiology, 180, 318–324. doi: 10.1093/aje/kwu140CrossRefGoogle ScholarPubMed
Steinhausen, H.-C. (2002). The outcome of anorexia nervosa in the 20th century. The American Journal of Psychiatry, 159, 1284–1293. doi: 10.1176/appi.ajp.159.8.1284CrossRefGoogle ScholarPubMed
Steinhausen, H.-C., & Weber, S. (2009). The outcome of bulimia nervosa: Findings from one-quarter century of research. The American Journal of Psychiatry, 166, 1331–1341. doi: 10.1176/appi.ajp.2009.09040582CrossRefGoogle ScholarPubMed
Stice, E., Fisher, M., & Lowe, M. R. (2004). Are dietary restraint scales valid measures of acute dietary restriction? Unobtrusive observational data suggest not. Psychological Assessment, 16, 51–59. doi: 10.1037/1040-3590.16.1.51CrossRefGoogle ScholarPubMed
Stunkard, A. J., & Messick, S. (1985). The three-factor eating questionnaire to measure dietary restraint, disinhibition and hunger. Journal of Psychosomatic Research, 29, 71–83. doi: 10.1016/0022-3999(85)90010-8CrossRefGoogle ScholarPubMed
Suvisaari, J., Mantere, O., Keinänen, J., Mäntylä, T., Rikandi, E., Lindgren, M., … Raij, T. T. (2018). Is it possible to predict the future in first-episode psychosis? Frontiers in Psychiatry, 9, 580. doi: 10.3389/fpsyt.2018.00580CrossRefGoogle ScholarPubMed
Thompson-Brenner, H., Franko, D. L., Thompson, D. R., Grilo, C. M., Boisseau, C. L., Roehrig, J. P., … Wilson, G. T. (2013). Race/ethnicity, education, and treatment parameters as moderators and predictors of outcome in binge eating disorder. Journal of Consulting and Clinical Psychology, 81, 710–721. doi: 10.1037/a0032946CrossRefGoogle Scholar
Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Methodological), 58, 267–288. doi: 10.1111/j.2517-6161.1996.tb02080.xGoogle Scholar
Tikhonov, A. N. (1963). Solution of incorrectly formulated problems and the regularization method. Soviet Mathematics, 4, 1035–1038.Google Scholar
Traviss-Turner, G. D., West, R. M., & Hill, A. J. (2017). Guided self-help for eating disorders: A systematic review and metaregression. European Eating Disorder Review, 25, 148–164. doi: 10.1002/erv.2507CrossRefGoogle ScholarPubMed
Vall, E., & Wade, T. D. (2015). Predictors of treatment outcome in individuals with eating disorders: A systematic review and meta-analysis. The International Journal of Eating Disorders, 48, 946–971. doi: 10.1002/eat.22411CrossRefGoogle ScholarPubMed
van Buuren, S. (2018). Flexible imputation of missing data (2nd ed.). Boca Raton, FL: CRC/Chapman & Hall.CrossRefGoogle Scholar
Von Hippel, P. T. (2017). Regression with missing ys: An improved strategy for analyzing multiply imputed data. Sociological Methodology, 37, 83–117. doi: 10.1111/j.1467-9531.2007.00180.xCrossRefGoogle Scholar
Zou, H., & Hastie, T. (2005). Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society. Series B, 67, 301–320.CrossRefGoogle Scholar
Supplementary material: File

Haynos et al. supplementary material

Haynos et al. supplementary material

Download Haynos et al. supplementary material(File)
File 84.1 KB