Hostname: page-component-cd9895bd7-jn8rn Total loading time: 0 Render date: 2024-12-28T20:41:00.492Z Has data issue: false hasContentIssue false
  • English
  • Français

Checking the predictive accuracy of basic symptoms against ultra high-risk criteria and testing of a multivariable prediction model: Evidence from a prospective three-year observational study of persons at clinical high-risk for psychosis

Published online by Cambridge University Press:  23 March 2020

M.P. Hengartner ,
K. Heekeren ,
D. Dvorsky ,
S. Walitza ,
W. Rössler  and
A. Theodoridou
M.P. Hengartner*
Affiliation:
Department of Applied Psychology, Zurich University of Applied Sciences, Zurich, Switzerland
K. Heekeren
Affiliation:
Department of Psychiatry, Psychotherapy and Psychosomatics, University Hospital of Psychiatry, Zurich, Switzerland
D. Dvorsky
Affiliation:
Department of Psychiatry, Psychotherapy and Psychosomatics, University Hospital of Psychiatry, Zurich, Switzerland
S. Walitza
Affiliation:
Department of Child and Adolescent Psychiatry and Psychotherapy, University of Zürich, Zurich, Switzerland
W. Rössler
Affiliation:
Department of Psychiatry, Psychotherapy and Psychosomatics, University Hospital of Psychiatry, Zurich, Switzerland Department of Psychiatry and Psychotherapy, Charité–Universitätsmedizin Berlin, Berlin, Germany Institute of Psychiatry, Laboratory of Neuroscience (LIM 27), University of Sao Paulo, Sao Paulo, Brazil
A. Theodoridou
Affiliation:
Department of Psychiatry, Psychotherapy and Psychosomatics, University Hospital of Psychiatry, Zurich, Switzerland
*
*Corresponding author. Department of Applied Psychology, Zurich University of Applied Sciences (ZHAW), PO Box 707, 8037 Zurich, Switzerland. E-mail address:[email protected] (M.P. Hengartner).
Get access

Abstract

Background:

The aim of this study was to critically examine the prognostic validity of various clinical high-risk (CHR) criteria alone and in combination with additional clinical characteristics.

Methods:

A total of 188 CHR positive persons from the region of Zurich, Switzerland (mean age 20.5 years; 60.2% male), meeting ultra high-risk (UHR) and/or basic symptoms (BS) criteria, were followed over three years. The test battery included the Structured Interview for Prodromal Syndromes (SIPS), verbal IQ and many other screening tools. Conversion to psychosis was defined according to ICD-10 criteria for schizophrenia (F20) or brief psychotic disorder (F23).

Results:

Altogether n = 24 persons developed manifest psychosis within three years and according to Kaplan–Meier survival analysis, the projected conversion rate was 17.5%. The predictive accuracy of UHR was statistically significant but poor (area under the curve [AUC] = 0.65, P < .05), whereas BS did not predict psychosis beyond mere chance (AUC = 0.52, P = .730). Sensitivity and specificity were 0.83 and 0.47 for UHR, and 0.96 and 0.09 for BS. UHR plus BS achieved an AUC = 0.66, with sensitivity and specificity of 0.75 and 0.56. In comparison, baseline antipsychotic medication yielded a predictive accuracy of AUC = 0.62 (sensitivity = 0.42; specificity = 0.82). A multivariable prediction model comprising continuous measures of positive symptoms and verbal IQ achieved a substantially improved prognostic accuracy (AUC = 0.85; sensitivity = 0.86; specificity = 0.85; positive predictive value = 0.54; negative predictive value = 0.97).

Conclusions:

We showed that BS have no predictive accuracy beyond chance, while UHR criteria poorly predict conversion to psychosis. Combining BS with UHR criteria did not improve the predictive accuracy of UHR alone. In contrast, dimensional measures of both positive symptoms and verbal IQ showed excellent prognostic validity. A critical re-thinking of binary at-risk criteria is necessary in order to improve the prognosis of psychotic disorders.

Keywords

Clinical high riskPrognostic validityPsychosisSchizophreniaTransitionConversion
Type
Original article
Information
European Psychiatry , Volume 45 , September 2017 , pp. 27 - 35
Copyright
Copyright © European Psychiatric Association 2017

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Fusar-Poli, PCappucciati, MRutigliano, GSchultze-Lutter, FBonoldi, IBorgwardt, S, et al.At risk or not at risk? A meta-analysis of the prognostic accuracy of psychometric interviews for psychosis prediction. World Psychiatry 2015;14(3):322–332.CrossRefGoogle ScholarPubMed
Tandon, RNasrallah, HAKeshavan, MSSchizophrenia, “just the facts” 4. Clinical features and conceptualization. Schizophr Res 2009;110(1–3):1–23.CrossRefGoogle ScholarPubMed
Schultze-Lutter, FMichel, CSchmidt, SJSchimmelmann, BGMaric, NPSalokangas, RK, et al.EPA guidance on the early detection of clinical high risk states of psychoses. Eur Psychiatry 2015;30(3):405–416.CrossRefGoogle ScholarPubMed
Fusar-Poli, PYung, ARMcGorry, Pvan Os, JLessons learned from the psychosis high-risk state: towards a general staging model of prodromal intervention. Psychol Med 2014;44(1):17–24.CrossRefGoogle ScholarPubMed
Hengartner, MPHeekeren, KDvorsky, DWalitza, SRossler, WTheodoridou, ACourse of psychotic symptoms, depression and global functioning in persons at clinical high risk of psychosis: Results of a longitudinal observation study over three years focusing on both converters and non-converters. Schizophr Res 2017. http://dx.doi.org/10.1016/j.schres.2017.01.040.CrossRefGoogle Scholar
van Os, JGuloksuz, SA critique of the “ultra-high risk” and “transition” paradigm. World Psychiatry 2017;16(2):200–206.CrossRefGoogle ScholarPubMed
Fusar-Poli, PBonoldi, IYung, ARBorgwardt, SKempton, MJValmaggia, L, et al.Predicting psychosis: meta-analysis of transition outcomes in individuals at high clinical risk. Arch Gen Psychiatry 2012;69(3):220–229.CrossRefGoogle ScholarPubMed
Hengartner, MPLoch, AALawson, FLGuarniero, FBWang, YPRossler, W, et al.Public stigmatization of different mental disorders: a comprehensive attitude survey. Epidemiol Psychiatr Sci 2013;22(3):269–274.CrossRefGoogle ScholarPubMed
Rusch, NHeekeren, KTheodoridou, AMuller, MCorrigan, PWMayer, B, et al.Stigma as a stressor and transition to schizophrenia after one year among young people at risk of psychosis. Schizophr Res 2015;166(1–3):43–48.CrossRefGoogle ScholarPubMed
Fusar-Poli, PSchultze-Lutter, FCappucciati, MRutigliano, GBonoldi, IStahl, D, et al.The dark side of the moon: meta-analytical impact of recruitment strategies on risk enrichment in the clinical high risk state for psychosis. Schizophr Bull 2016;42(3):732–743.CrossRefGoogle ScholarPubMed
Fusar-Poli, PRutigliano, GStahl, DDavies, CBonoldi, IReilly, T, et al.Development and validation of a clinically based risk calculator for the transdiagnostic prediction of psychosis. JAMA Psychiatry 2017. http://dx.doi.org/10.1001/jamapsychiatry.2017.0284.CrossRefGoogle Scholar
Cannon, TDCadenhead, KCornblatt, BWoods, SWAddington, JWalker, E, et al.Prediction of psychosis in youth at high clinical risk: a multisite longitudinal study in North America. Arch Gen Psychiatry 2008;65(1):28–37.CrossRefGoogle ScholarPubMed
Michel, CRuhrmann, SSchimmelmann, BGKlosterkotter, JSchultze-Lutter, FA stratified model for psychosis prediction in clinical practice. Schizophr Bull 2014;40(6):1533–1542.CrossRefGoogle Scholar
Riecher-Rossler, APflueger, MOAston, JBorgwardt, SJBrewer, WJGschwandtner, U, et al.Efficacy of using cognitive status in predicting psychosis: a 7-year follow-up. Biol Psychiatry 2009;66(11):1023–1030.CrossRefGoogle ScholarPubMed
Ruhrmann, SSchultze-Lutter, FSalokangas, RKHeinimaa, MLinszen, DDingemans, P, et al.Prediction of psychosis in adolescents and young adults at high risk: results from the prospective European prediction of psychosis study. Arch Gen Psychiatry 2010;67(3):241–251.CrossRefGoogle Scholar
Fusar-Poli, PSchultze-Lutter, FPredicting the onset of psychosis in patients at clinical high risk: practical guide to probabilistic prognostic reasoning. Evid Based Ment Health 2016;19(1):10–15.CrossRefGoogle ScholarPubMed
Cannon, TDYu, CAddington, JBearden, CECadenhead, KSCornblatt, BA, et al.An individualized risk calculator for research in prodromal psychosis. Am J Psychiatry 2016;173(10):980–988.CrossRefGoogle ScholarPubMed
Carrion, RECornblatt, BABurton, CZTso, IFAuther, AMAdelsheim, S, et al.Personalized prediction of psychosis: external validation of the NAPLS-2 psychosis risk calculator with the EDIPPP project. Am J Psychiatry 2016;173(10):989–996.CrossRefGoogle ScholarPubMed
Klosterkotter, JSchultze-Lutter, FBechdolf, ARuhrmann, SPrediction and prevention of schizophrenia: what has been achieved and where to go next?. World Psychiatry 2011;10(3):165–174.CrossRefGoogle ScholarPubMed
Schultze-Lutter, FKlosterkotter, JRuhrmann, SImproving the clinical prediction of psychosis by combining ultra-high risk criteria and cognitive basic symptoms. Schizophr Res 2014;154(1–3):100–106.CrossRefGoogle ScholarPubMed
Theodoridou, AHeekeren, KDvorsky, DMetzler, SFranscini, MHaker, H, et al.Early recognition of high risk of bipolar disorder and psychosis: an overview of the ZInEP “Early Recognition” Study. Front Public Health 2014;2:166.CrossRefGoogle ScholarPubMed
Schultze-Lutter, FAddington, JRuhrmann, SKlosterkotter, JSchizophrenia Proneness Instrument. Adult version (SPI-A) Rome: Giovanni Fioriti Editore; 2007Google Scholar
Schultze-Lutter, FKoch, ESchizophrenia Proneness Instrument. Child and Youth Version (SPI-CY) Rome: Giovanni Fioriti Editore; 2010Google Scholar
Miller, TJMcGlashan, THRosen, JLSomjee, LMarkovich, PJStein, K, et al.Prospective diagnosis of the initial prodrome for schizophrenia based on the Structured Interview for Prodromal Syndromes: preliminary evidence of interrater reliability and predictive validity. Am J Psychiatry 2002;159(5):863–865.CrossRefGoogle ScholarPubMed
World Health Organization International Classification of Diseases ICD-10, 10th ed., Geneva: World Health Organization; 1992.Google Scholar
Yung, ARNelson, BThompson, AWood, SJThe psychosis threshold in Ultra High Risk (prodromal) research: is it valid?. Schizophr Res 2010;120(1–3):1–6.CrossRefGoogle ScholarPubMed
Kay, SRFiszbein, AOpler, LAThe positive and negative syndrome scale (PANSS) for schizophrenia. Schizophr Bull 1987;13(2):261–276.CrossRefGoogle Scholar
Hamilton, MA rating scale for depression. J Neurol Neurosurg Psychiatry 1960;23:56–62.CrossRefGoogle Scholar
Angst, JAdolfsson, RBenazzi, FGamma, AHantouche, EMeyer, TD, et al.The HCL-32: towards a self-assessment tool for hypomanic symptoms in outpatients. J Affect Disord 2005;88(2):217–233.CrossRefGoogle ScholarPubMed
Beck, ATEpstein, NBrown, GSteer, RAAn inventory for measuring clinical anxiety: psychometric properties. J Consult Clin Psychol 1988;56(6):893–897.CrossRefGoogle ScholarPubMed
Derogatis, LRSymptom Checklist 90, R-Version manual I: scoring, administration, and procedures for the SCL-90 Baltimore, MD: Johns Hopkins Press; 1977Google Scholar
Sheehan, DVLecrubier, YSheehan, KHAmorim, PJanavs, JWeiller, E, et al.The Mini-International Neuropsychiatric Interview (MINI): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry 1998;59(Suppl. 20):22–33.Google ScholarPubMed
Sheehan, DVSheehan, KHShytle, RDJanavs, JBannon, YRogers, JE, et al.Reliability and validity of the Mini International Neuropsychiatric Interview for children and adolescents (MINI-KID). J Clin Psychiatry 2010;71(3):313–326.CrossRefGoogle Scholar
Lehrl, SMehrfachwahl-Wortschatz-Intelligenztest; MWT-B Erlangen: Perimed; 1989Google Scholar
Dunn, LMDunn, LPeabody Picture Vocabulary Test: German Language Version for Adolescents and Adults Frankfurt am Main: Swets Test Services; 2003Google Scholar
Austin, PCBrunner, LJInflation of the type I error rate when a continuous confounding variable is categorized in logistic regression analyses. Stat Med 2004;23(7):1159–1178.CrossRefGoogle ScholarPubMed
MacCallum, RCZhang, SPreacher, KJRucker, DDOn the practice of dichotomization of quantitative variables. Psychol Methods 2002;7(1):19–40.CrossRefGoogle ScholarPubMed
Royston, PAltman, DGSauerbrei, WDichotomizing continuous predictors in multiple regression: a bad idea. Stat Med 2006;25(1):127–141.CrossRefGoogle ScholarPubMed
Babyak, MAWhat you see may not be what you get: a brief, nontechnical introduction to overfitting in regression-type models. Psychosom Med 2004;66(3):411–421.Google Scholar
Derksen, SKeselman, HJBackward, forward and stepwise automated subset-selection algorithms – frequency of obtaining authentic and noise variables. Brit J Math Stat Psy 1992;45:265–282.CrossRefGoogle Scholar
Steyerberg, EWEijkemans, MJCHarrell, FEHabbema, JDFPrognostic modeling with logistic regression analysis: in search of a sensible strategy in small data sets. Med Decis Making 2001;21(1):45–56.CrossRefGoogle ScholarPubMed
Hosmer, DW Jr.Lemeshow, SMay, SApplied survival analysis: regression modeling of time to event data, 2nd ed., New York: Wiley; 2008 [416 p.].CrossRefGoogle Scholar
Schisterman, EFPerkins, NJLiu, ABondell, HOptimal cut-point and its corresponding Youden Index to discriminate individuals using pooled blood samples. Epidemiology 2005;16(1):73–81.CrossRefGoogle ScholarPubMed
Pavlou, MAmbler, GSeaman, SRGuttmann, OElliott, PKing, M, et al.How to develop a more accurate risk prediction model when there are few events. BMJ 2015;351:h3868.CrossRefGoogle ScholarPubMed
Steyerberg, EWVickers, AJCook, NRGerds, TGonen, MObuchowski, N, et al.Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 2010;21(1):128–138.CrossRefGoogle ScholarPubMed
Pencina, MJD’Agostino, RB Sr.D’Agostino, RB Jr.Vasan, RSEvaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med 2008;27(2):157–172.CrossRefGoogle ScholarPubMed
Fusar-Poli, PCappucciati, MBorgwardt, SWoods, SWAddington, JNelson, B, et al.Heterogeneity of psychosis risk within individuals at clinical high risk: a meta-analytical stratification. JAMA Psychiatry 2016;73(2):113–120.CrossRefGoogle ScholarPubMed
Cornblatt, BACarrion, REDeconstructing the psychosis risk syndrome: moving the field of prevention forward. JAMA Psychiatry 2016;73(2):105–106.CrossRefGoogle ScholarPubMed
McGorry, PNelson, BWhy we need a transdiagnostic staging approach to emerging psychopathology, early diagnosis, and treatment. JAMA Psychiatry 2016;73(3):191–192.CrossRefGoogle Scholar
van Os, JMurray, RMCan we identify and treat “schizophrenia light” to prevent true psychotic illness?. BMJ 2013;346:f304.CrossRefGoogle ScholarPubMed
Ziermans, TBSchothorst, PFSprong, Mvan Engeland, HTransition and remission in adolescents at ultra-high risk for psychosis. Schizophr Res 2011;126(1–3):58–64.CrossRefGoogle ScholarPubMed
Koutsouleris, NRiecher-Rossler, AMeisenzahl, EMSmieskova, RStuderus, EKambeitz-Ilankovic, L, et al.Detecting the psychosis prodrome across high-risk populations using neuroanatomical biomarkers. Schizophr Bull 2015;41(2):471–482.CrossRefGoogle ScholarPubMed
Chan, MKKrebs, MOCox, DGuest, PCYolken, RHRahmoune, H, et al.Development of a blood-based molecular biomarker test for identification of schizophrenia before disease onset. Transl Psychiatry 2015;5:e601.CrossRefGoogle ScholarPubMed
Fusar-Poli, PDeste, GSmieskova, RBarlati, SYung, ARHowes, O, et al.Cognitive functioning in prodromal psychosis: a meta-analysis. Arch Gen Psychiatry 2012;69(6):562–571.CrossRefGoogle ScholarPubMed
Khandaker, GMBarnett, JHWhite, IRJones, PBA quantitative meta-analysis of population-based studies of premorbid intelligence and schizophrenia. Schizophr Res 2011;132(2–3):220–227.CrossRefGoogle Scholar
MacCabe, JHPopulation-based cohort studies on premorbid cognitive function in schizophrenia. Epidemiol Rev 2008;30:77–83.CrossRefGoogle Scholar
Gale, CRBatty, GDTynelius, PDeary, IJRasmussen, FIntelligence in early adulthood and subsequent hospitalization for mental disorders. Epidemiology 2010;21(1):70–77.CrossRefGoogle ScholarPubMed
Koenen, KCMoffitt, TERoberts, ALMartin, LTKubzansky, LHarrington, H, et al.Childhood IQ and adult mental disorders: a test of the cognitive reserve hypothesis. Am J Psychiatry 2009;166(1):50–57.CrossRefGoogle ScholarPubMed
Barnett, JHSalmond, CHJones, PBSahakian, BJCognitive reserve in neuropsychiatry. Psychol Med 2006;36(8):1053–1064.CrossRefGoogle ScholarPubMed
Nelson, BYuen, HPWood, SJLin, ASpiliotacopoulos, DBruxner, A, et al.Long-term follow-up of a group at ultra high risk (“prodromal”) for psychosis: the PACE 400 study. JAMA Psychiatry 2013;70(8):793–802.CrossRefGoogle ScholarPubMed
Velthorst, ENieman, DHBecker, HEvan de Fliert, RDingemans, PMKlaassen, R, et al.Baseline differences in clinical symptomatology between ultra high risk subjects with and without a transition to psychosis. Schizophr Res 2009;109(1–3):60–65.CrossRefGoogle ScholarPubMed
Bechdolf, ASchultze-Lutter, FKlosterkotter, JSelf-experienced vulnerability, prodromal symptoms and coping strategies preceding schizophrenic and depressive relapses. Eur Psychiatry 2002;17(7):384–393.CrossRefGoogle ScholarPubMed
Schultze-Lutter, FKlosterkötter, JPicker, HSteinmeyer, E.-M.Ruhrmann, SPredicting first-episode psychosis by basic symptom criteria. Clinical Neuropsychiatry 2007;4(1):11–22.Google Scholar
Varghese, DScott, JWelham, JBor, WNajman, JO’Callaghan, M, et al.Psychotic-like experiences in major depression and anxiety disorders: a population-based survey in young adults. Schizophr Bull 2009;37(2):389–393.CrossRefGoogle ScholarPubMed
Wigman, JTvan Nierop, MVollebergh, WALieb, RBeesdo-Baum, KWittchen, HU, et al.Evidence that psychotic symptoms are prevalent in disorders of anxiety and depression, impacting on illness onset, risk, and severity – implications for diagnosis and ultra-high risk research. Schizophr Bull 2012;38(2):247–257.CrossRefGoogle ScholarPubMed
Rossler, WHengartner, MPAjdacic-Gross, VHaker, HGamma, AAngst, JSub-clinical psychosis symptoms in young adults are risk factors for subsequent common mental disorders. Schizophr Res 2011;131(1–3):18–23.CrossRefGoogle ScholarPubMed
Werbeloff, NDrukker, MDohrenwend, BPLevav, IYoffe, Rvan Os, J, et al.Self-reported attenuated psychotic symptoms as forerunners of severe mental disorders later in life. Arch Gen Psychiatry 2012;69(5):467–475.CrossRefGoogle ScholarPubMed
McGorry, Pvan Os, JRedeeming diagnosis in psychiatry: timing versus specificity. Lancet 2013;381(9863):343–345.CrossRefGoogle ScholarPubMed
Fusar-Poli, PNelson, BValmaggia, LYung, ARMcGuire, PKComorbid depressive and anxiety disorders in 509 individuals with an at-risk mental state: impact on psychopathology and transition to psychosis. Schizophr Bull 2014;40(1):120–131.CrossRefGoogle ScholarPubMed
Woods, SWAddington, JCadenhead, KSCannon, TDCornblatt, BAHeinssen, R, et al.Validity of the prodromal risk syndrome for first psychosis: findings from the North American Prodrome Longitudinal Study. Schizophr Bull 2009;35(5):894–908.CrossRefGoogle ScholarPubMed
Hengartner, MPAjdacic-Gross, VWyss, CAngst, JRossler, WRelationship between personality and psychopathology in a longitudinal community study: a test of the predisposition model. Psychol Med 2016;46(8):1693–1705.CrossRefGoogle Scholar
Hengartner, MPKawohl, WHaker, HRossler, WAjdacic-Gross, VBig Five personality traits may inform public health policy and preventive medicine: evidence from a cross-sectional and a prospective longitudinal epidemiologic study in a Swiss community. J Psychosom Res 2016;84:44–51.CrossRefGoogle Scholar
Kotov, RGamez, WSchmidt, FWatson, DLinking “Big” personality traits to anxiety, depressive, and substance use disorders: a meta-analysis. Psychol Bull 2010;136(5):768–821.CrossRefGoogle ScholarPubMed
Kerns, JGSchizotypy facets, cognitive control, and emotion. J Abnorm Psychol 2006;115(3):418–427.CrossRefGoogle ScholarPubMed
Macare, CBates, TCHeath, ACMartin, NGEttinger, USubstantial genetic overlap between schizotypy and neuroticism: a twin study. Behav Genet 2012;42(5):732–742.CrossRefGoogle ScholarPubMed
Rossler, WAjdacic-Gross, VMuller, MRodgers, SHaker, HHengartner, MPAssessing sub-clinical psychosis phenotypes in the general population – a multidimensional approach. Schizophr Res 2015;161(2–3):194–201.CrossRefGoogle ScholarPubMed
Rossler, WHengartner, MPAjdacic-Gross, VHaker, HAngst, JDeconstructing sub-clinical psychosis into latent-state and trait variables over a 30-year time span. Schizophr Res 2013;150(1):197–204.CrossRefGoogle Scholar
Van Os, JJones, PBNeuroticism as a risk factor for schizophrenia. Psychol Med 2001;31(6):1129–1134.CrossRefGoogle Scholar
Kaymaz, NDrukker, MLieb, RWittchen, HUWerbeloff, NWeiser, M, et al.Do subthreshold psychotic experiences predict clinical outcomes in unselected non-help-seeking population-based samples? A systematic review and meta-analysis, enriched with new results. Psychol Med 2012;42(11):2239–2253.CrossRefGoogle ScholarPubMed
van Os, JReininghaus, UPsychosis as a transdiagnostic and extended phenotype in the general population. World Psychiatry 2016;15(2):118–124.CrossRefGoogle Scholar
Submit a response

Comments

No Comments have been published for this article.