Hostname: page-component-cd9895bd7-gvvz8 Total loading time: 0 Render date: 2024-12-27T02:55:36.731Z Has data issue: false hasContentIssue false

Short clinically-based prediction model to forecast transition to psychosis in individuals at clinical high risk state

Published online by Cambridge University Press:  11 March 2019

Magdalena Kotlicka-Antczak*
Affiliation:
aDepartment of Affective and Psychotic Disorders, Medical University of Lodz, ul. Czechosłowacka 8/1092-216Lodz, Poland
Michał S. Karbownik
Affiliation:
bDepartment of Pharmacology and Toxicology, Medical University of Lodz, ul. Żeligowskiego 7/990-752Lodz, Poland
Konrad Stawiski
Affiliation:
cDepartment of Biostatistics and Translational Medicine, Medical University of Lodz, ul. Mazowiecka 1592-215Lodz, Poland
Agnieszka Pawełczyk
Affiliation:
aDepartment of Affective and Psychotic Disorders, Medical University of Lodz, ul. Czechosłowacka 8/1092-216Lodz, Poland
Natalia Żurner
Affiliation:
dAdolescent Psychiatry Unit, Central Clinical Hospital, ul. Czechosłowacka 8/1092-216Lodz, Poland
Tomasz Pawełczyk
Affiliation:
aDepartment of Affective and Psychotic Disorders, Medical University of Lodz, ul. Czechosłowacka 8/1092-216Lodz, Poland
Dominik Strzelecki
Affiliation:
aDepartment of Affective and Psychotic Disorders, Medical University of Lodz, ul. Czechosłowacka 8/1092-216Lodz, Poland
Paolo Fusar-Poli
Affiliation:
eEarly Psychosis: Interventions and Clinical Detection Lab, Department of Psychosis Studies, Institute of Psychiatry, Psychology & Neuroscience, King's College London, 16 De Crespigny Park London, SE5 8AF, UK fDepartment of Brain and Behavioral Sciences, University of Pavia, Pavia, Italy
*
Corresponding author. E-mail addresses: [email protected] (M. Kotlicka-Antczak), [email protected] (M.S. Karbownik), [email protected] (K. Stawiski), [email protected] (A. Pawe³czyk), [email protected] (N. Ż urner), [email protected] (T. Pawe³czyk), [email protected] (D. Strzelecki), [email protected] (P. Fusar-Poli).

Abstract

Objective:

The predictive accuracy of the Clinical High Risk criteria for Psychosis (CHR-P) regarding the future development of the disorder remains suboptimal. It is therefore necessary to incorporate refined risk estimation tools which can be applied at the individual subject level. The aim of the study was to develop an easy-to use, short refined risk estimation tool to predict the development of psychosis in a new CHR-P cohort recruited in European country with less established early detection services.

Methods:

A cohort of 105 CHR-P individuals was assessed with the Comprehensive Assessment of At Risk Mental States12/2006, and then followed for a median period of 36 months (25th-75th percentile:10–59 months) for transition to psychosis. A multivariate Cox regression model predicting transition was generated with preselected clinical predictors and was internally validated with 1000 bootstrap resamples.

Results:

Speech disorganization and unusual thought content were selected as potential predictors of conversion on the basis of published literature. The prediction model was significant (p < 0.0001) and confirmed that both speech disorganization (HR = 1.69; 95%CI: 1.39–2.05) and unusual thought content (HR = 1.51; 95%CI: 1.27–1.80) were significantly associated with transition. The prognostic accuracy of the model was adequate (Harrell’s c- index = 0.79), even after optimism correction through internal validation procedures (Harrell’s c-index = 0.78).

Conclusions:

The clinical prediction model developed, and internally validated, herein to predict transition from a CHR-P to psychosis may be a promising tool for use in clinical settings. It has been incorporated into an online tool available at: https://link.konsta.com.pl/psychosis. Future external replication studies are needed.

Type
Research Article
Copyright
Copyright © European Psychiatric Association 2019

1. Introduction

The prediction of psychosis from a Clinical High Risk state (CHR-P afterwards) is an ongoing topic of research and a common theme in the mainstream of clinical psychiatry [Reference Fusar-Poli1]. Nowadays, a range of tools, such as the Comprehensive Assessment of At Risk Mental States (CAARMS) [Reference Yung, Yuen, McGorry, Phillips, Kelly and Dell’Olio2] or the Scale of Prodromal Risk-Syndromes (SOPS) [Reference Miller, McGlashan, Rosen, Cadenhead, Cannon and Ventura3], can be used for prospective recognition of clinical “at-risk” states and clinical interventions [Reference Fusar-Poli, Davies and Bonoldi4], and demonstrate comparable prognostic accuracy with other prognostic tests used in Medicine [Reference Fusar-Poli, Cappucciati, Rutigliano, Schultze-Lutter, Bonoldi and Borgwardt5, Reference Fusar-Poli, Rutigliano, Stahl, Davies, De Micheli and Ramella-Cravaro6]. However, a recent meta-analysis indicates that the CHR-P designation retains only modest specificity, despite being characterised by high sensitivity [Reference Oliver, Kotlicka-Antczak, Minichino, Spada, McGuire and Fusar-Poli7]. This tool is clinically useful only for samples who have undergone substantial risk enrichment during the recruitment phases (15% risk at three years) [Reference Fusar-Poli, Schultze-Lutter, Cappucciati, Rutigliano, Bonoldi and Stahl8, Reference Fusar-Poli, Rutigliano, Stahl, Schmidt, Ramella-Cravaro and Hitesh9]. Accordingly, while its negative predictive values are quite good, the positive predictive value after two years is only estimated to be between 20% and 30% [Reference Oliver, Kotlicka-Antczak, Minichino, Spada, McGuire and Fusar-Poli7]. This difficulty in accurately improving specificity by identifying individuals who progress to developing psychosis have required the inclusion of Attenuated Psychosis Syndrome in the “conditions for further study” section of the DSM-5 (DSM-5-APS); however, it has also been included in the main text as a subcategory of “Other Specified Schizophrenia Spectrum and Other Psychotic Disorders” [Reference American Psychiatric Association10]. The DSM-5-APS construct has shown comparable prognostic accuracy to other CHR-P tools [Reference Fusar-Poli, De Micheli, Cappucciati, Rutigliano, Davies and Ramella-Cravaro11].

The CHR-P state, known to be an important step in the development of psychotic disorders, is now recommended as a target of therapeutic preventive interventions [Reference Joa, Gisselgård, Brønnick, McGlashan and Johannessen12Reference Fusar-Poli15], although effective treatments for these patients have not yet been established [Reference Davies, Cipriani, Ioannidis, Radua, Stahl and Provenzani16]. Indeed, specialized clinical management for CHR-P individuals has been validated in clinical practice, and has officially become a key component of early intervention services worldwide [Reference NHS England17, 18]. Hence, interest is growing in improving the prognostic accuracy of tests for identifying CHR-P individuals with the highest risk of developing a psychotic disorder [Reference Fusar-Poli, Borgwardt, Bechdolf, Addington, Riecher-Rössler and Schultze-Lutter19]. Another important advancement would be to develop individualised approaches which would allow outcomes to be predicted at the individual subject level [Reference McGorry20].

One way by which the prognostic accuracy of the CHR-P construct may be improved would be by identifying specific clinical and biological predictors that account for the risk of developing psychosis [Reference Fusar-Poli, Borgwardt, Bechdolf, Addington, Riecher-Rössler and Schultze-Lutter19]. Of the established predictors, one of the most widely-replicated is the presence of attenuated psychotic symptoms [Reference Riecher-Rössler and Studerus21], which includes high levels of unusual thought content and/or suspiciousness [Reference Riecher-Rössler and Studerus21Reference Crump, Arndt, Grivel, Horga, Corcoran and Brucato32], as well as disorganized communication [Reference Cornblatt, Carrión, Auther, McLaughlin, Olsen and John28, Reference Addington, Liu, Perkins, Carrion, Keefe and Woods31, Reference Ruhrmann, Schultze-Lutter, Salokangas, Heinimaa, Linszen and Dingemans33Reference Francesconi, Minichino, Carrión, Delle Chiaie, Bevilacqua and Parisi36]. Another line of research points to negative symptoms and poor psychosocial functioning as psychosis predictors [Reference Yung, Phillips, Yuen and McGorry22Reference Addington, Liu, Buchy, Cadenhead, Cannon and Cornblatt26, Reference Cannon, Yu, Addington, Bearden, Cadenhead and Cornblatt29, Reference Carrión, Cornblatt, Burton, Tso, Auther and Adelsheim30, Reference Zhang, Li, Woodberry, Xu, Tang and Guo37Reference Demjaha, Valmaggia, Stahl, Byrne and McGuire40]; however, most of these studies have been conducted in Western European countries, USA or Australia, and only a few initial reports have been published from Asia and Africa [Reference Zhang, Li, Woodberry, Xu, Tang and Guo37, Reference Katsura, Ohmuro, Obara, Kikuchi, Ito and Miyakoshi41, Reference Mamah, Musau, Mutiso, Owoso, Abdallah and Cottler42].

The present study advances knowledge in this area by investigating a group of CHR-P individuals recruited in Eastern Europe (Poland). Early intervention services have only started to develop in this region of Europe, and training and stuffing resources are insufficient. Therefore we wanted to test a simplified version of CAARMS, which could facilitate the assessment of individuals at risk for psychosis in clinical settings with less-established early detection and intervention services network.

Hence, the principal aim of the study was to develop an easy-to use, clinically-based risk-estimation model that could improve the ability of the clinician to predict the onset of psychosis in CHR-P individuals from this population in a shorter period of time.

2. Materials and methods

2.1. Settings and subjects

This prospective cohort study was performed in the Programme Of Recognition and Therapy (PORT), which was created to provide a medical service to support CHR-P individuals. The programme is affiliated with the Medical University of Lodz, and operates as secondary mental health care in the Lodz voivodship. The community-focused informative campaign regarding PORT, comprising outreach with local authorities, educational meetings and individual workshops for groups of students, parents and teachers at high schools in the Lodz voivodship, was carried within years 2010-2013. Starting from 2009, training and workshops for local mental-health professionals are provided at least once a year on a regular basis during local psychiatric conferences and meetings of the local section of the Polish Psychiatric Association. Referrals for the programme have been received from 1) community mental health services and child and adolescent mental health services, 2) inpatient mental health services, 3) educational institutions, 4) relatives or caregivers, 5) social services and 5) self-referrals. All referred persons complete the self-made PORT pre-screening questionnaire, based on Structured Interview for Prodromal Symptoms [Reference Miller, McGlashan, Rosen, Cadenhead, Cannon and Ventura3] available on the PORT website: www.mojport.pl, before their first visit in PORT centre. For positively-screened help-seeking individuals, this is followed by assessment with CAARMS and a physical examination, which includes a neurological section. Additionally, to more accurately evaluate functioning, an interview is conducted with parents or legal guardians (for underage individuals), or with closest relatives (for adult participants who give their consent). The comorbid diagnoses are established based on Structured Clinical Interviews for DSM-IV-Axis I and Axis II Disorders [Reference First, Spitzer, Gibbon and Williams43, Reference First, Gibbon, Spitzer, Williams and Benjamin44] performed within several days following the initial visit. More details regarding PORT and the procedures have been given previously [Reference Kotlicka-Antczak, Pawełczyk, Rabe-Jabłońska and Pawełczyk45, Reference Kotlicka-Antczak, Pawełczyk, Podgórski, Żurner, Karbownik and Pawełczyk46].

CHR-P individuals included in the study were consecutively referred to the programme in the period between March 2010 and July 2015. The follow-up period ended in May 2017. In this period, a group of 152 individuals, inhabitants of the Lodz voivodship, underwent the post-screening procedure, of whom 108 met the CHR-P criteria. Three cases dropped out from the programme at the recruitment phase, because of stigmatization problems. Finally, 105 individuals (49 males, 56 females) aged 15–32 years, who met the CHR-P criteria according to the CAARMS12/2006 (At Risk Mental State, ARMS) [Reference Yung, Yuen, McGorry, Phillips, Kelly and Dell’Olio2] were included in the analysis.

The exclusion criteria comprised the presence of a neurological disorder, evidence of intellectual disability or a diagnosis of psychotic disorder made according to ICD 10 criteria [Reference World Health Organization47], which is approved as the main classification in Poland. More detailed information regarding the PORT population (data until the end of 2014) can be found in our previous publication [Reference Kotlicka-Antczak, Pawełczyk, Podgórski, Żurner, Karbownik and Pawełczyk46]. We have previously demonstrated that the recruitment strategies adopted by our CHR-P service are -in line with that adopted worldwide by other similar early detection clinics [Reference Kotlicka-Antczak, Pawełczyk, Podgórski, Żurner, Karbownik and Pawełczyk46]. Accordingly, the level of risk enrichment, which indexes the probability of developing psychosis in the sample undergoing a CHR-P assessment before the result of the assessment are known [Reference Fusar-Poli and Schultze-Lutter48], was previously calculated for PORT sample as 14% at 36 months [Reference Fusar-Poli, Schultze-Lutter, Cappucciati, Rutigliano, Bonoldi and Stahl8]. The risk is comparable to that observed worldwide at the meta-analytical level: 15% at average 38 months follow-up [Reference Fusar-Poli, Schultze-Lutter, Cappucciati, Rutigliano, Bonoldi and Stahl8]. Hence, the sample may be considered representative of CHR-P services from different countries.

2.2. Baseline assessments

Individuals with a CHR-P state were identified with the use of the Polish version of CAARMS [Reference Yung, Yuen, McGorry, Phillips, Kelly and Dell’Olio2, Reference Jaracz, Grzechowiak, Raczkowiak, Rataj and Rybakowski49] and hence classified as presenting the following characteristics: (1) positive symptoms which do not reach the level of psychosis either in their frequency or intensity (the attenuated psychotic symptoms or APS subgroup), (2) experience of a psychotic episode which resolved spontaneously within one week during the past year (the brief limited intermittent psychotic symptoms or BLIPS subgroup) or, (3) meeting the criteria of schizotypal personality disorder or having a family history of psychosis in a first-degree relative (the genetic risk and deterioration syndrome, GRD subgroup) [Reference Yung, Yuen, McGorry, Phillips, Kelly and Dell’Olio2]. A substantial deterioration in psychosocial functioning, defined as a drop of at least 30% on the Social and Occupational Functioning Assessment Scale (SOFAS) was additionally required in all cases [Reference Yung, Yuen, McGorry, Phillips, Kelly and Dell’Olio2]. At the time of recruitment, a semi-structured questionnaire was also used to obtain demographic data and a family history of mental disorders. The level of psychosocial functioning was evaluated with the SOFAS included in the CAARMS [Reference Yung, Yuen, McGorry, Phillips, Kelly and Dell’Olio2]. The evaluation of intelligence level (Intelligence Quotient-IQ), was carried out using Wechsler Intelligence Scales (for Children or for Adults), Polish versions [Reference Wechsler50, Reference Wechsler51] as a part of the comprehensive neurocognitive assessment provided for PORT participants [Reference Kotlicka-Antczak, Pawełczyk, Rabe-Jabłońska and Pawełczyk45]. The obtained score for intelligence level was used as the basis for exclusion of intellectual disability.

To ensure a high standard of analysis, the model was generated according to the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis Or Diagnosis (TRIPOD) checklist [Reference Moons, Altman, Reitsma, Ioannidis, Macaskill and Steyerberg52].

The study was carried out in accordance with the latest version of the Declaration of Helsinki. The study protocol was accepted by the Medical University of Lodz Ethics Committee. Written informed consent was provided by the participants, or both participants and their parents, in cases where the participant was underage.

2.3. Treatment

The study participants received needs-based interventions, targeting their individual problems. As described previously, all CHR-P individuals received psychological interventions, most commonly including psychological support focusing on social relationships, educational/vocational and family issues or structured psychotherapy (cognitive behavioural therapy) [Reference Kotlicka-Antczak, Pawełczyk, Rabe-Jabłońska and Pawełczyk45]. In order to treat comorbid disorders, antidepressants or mood stabilizers were used. Based on the International Clinical Practice Guidelines for Early Psychosis, three clinical situations were considered as indications for treatment with antipsychotics: (1) rapid deterioration of mental state; (2) the presence of a severe suicide risk, with treatment of any depression proving ineffective; or (3) aggression or hostility were increasing and posing a risk to others [53]. The details regarding pharmacological treatment received by the study participants are presented in supplementary eTable2.

2.4. Outcome

The outcome predicted in the study was transition from a CHR-P to psychotic disorder, which was determined on the basis of the CAARMS criteria for psychosis threshold [Reference Yung, Yuen, McGorry, Phillips, Kelly and Dell’Olio2]. This was defined as: 1) maximum score (six) on at least one of CAARMS Positive Symptoms Scale (Unusual Thought Content, Non-Bizarre Ideas, Disorganized Speech, for Perceptual Abnormalities minimum score of five was required); 2) the frequency of symptoms as at least four on a six-point scale (frequency three to six times per week, more than an hour per occasion or daily - less than an hour per occasion) 3) symptoms present for longer than one week [Reference Yung, Yuen, McGorry, Phillips, Kelly and Dell’Olio2].

In order to identify a potential transition, participants were assessed with the CAARMS every three months during the follow-up period, or more often, if required. Since the follow-up period was relatively long, as given in the Results section, it was not possible for each evaluation to be carried out by the same clinician; both researchers engaged in the procedure were blinded to the results of their evaluations. Their inter-rater reliability determined during the CAARMS training was high (kappa = 0.9).

To further validate the transition and to determine its clinical category, the identification of full-blown psychosis based on CAARMS was followed by applying ICD-10 criteria [Reference World Health Organization47].

2.5. Preselection of predictors of transition

Potential clinical predictors of the onset of psychosis from a CHR-P stage were preselected on the basis of availability in clinical settings and existing clinical knowledge [Reference Steyerberg and Vergouwe54Reference Fusar-Poli, Stahl and Steyerberg57]. This is regarded as the optimal approach to building robust clinical prediction models in psychiatry [Reference Studerus, Ramyead and Riecher-Rossler55, Reference Studerus, Ramyead and Riecher-Rossler55, Reference Wynants, Collins and Van Calster56, Reference Fusar-Poli, Stahl and Steyerberg57]. The model complexity was a priori limited by the number of events (transitions) to no more than two predictors. This was set to allow an event per variable (EPV) ratio of 10 and above, which is recommended to develop robust prognostic models [Reference Studerus, Ramyead and Riecher-Rossler55, Reference Studerus, Ramyead and Riecher-Rossler55, Reference Wynants, Collins and Van Calster56, Reference Fusar-Poli, Stahl and Steyerberg57]. The predictors were assumed to have a linear effect on the outcome, and to keep the model as simple as possible, no interactions were allowed [Reference Wynants, Collins and Van Calster56].

2.6. Data analysis

No data was missing from the dataset. The Kaplan-Meier method was utilized in survival analysis to estimate the cumulative risk function of transition to psychosis. The time-to-event model was developed using multivariate Cox proportional hazard regression with two prespecified predictors. The collinearity of these predictors was excluded with the use of Spearman’s rho. Because of the high imbalance in class representation (i.e. the numbers of converters and non-converters), model induction was performed after the application of a synthetic minority over-sampling technique [Reference Chawla, Bowyer, Hall and Kegelmeyer58]. The Shoenfeld residuals test was used to control for proportional hazard assumption. The degree of discrimination of the model, representing its ability to distinguish low-risk from high-risk individuals, was measured with the use of Harrell’s concordance index (c-index), a measure analogous to the area under the curve (AUC), but one that was tailored for censored data [Reference Uno, Cai, Pencina, D’Agostino and Wei59]. In addition, receiver-operating characteristics (ROC) curves against time [Reference Kamarudin, Cox and Kolamunnage-Dona60] were presented, with sensitivity, specificity, accuracy and AUCs for different time points [1, 2, 3, and 4 years, respectively].

The model was internally validated using bootstrap methods with 1000 samples [Reference Efron and Tibshirani61]. First, we applied the Cox proportional hazard regression model to each of these samples. In the next step, all resulting models were applied to the original dataset to estimate their performance. Average Somer's D calculated on all both bootstrap samples and original dataset and their difference was used to estimate model’s optimism. Optimism corrected c-index was then calculated based on its relationship to Somer's D (C = 0.5(D + 1)). In a further step we estimated the calibration (the agreement between the predicted risks and observed event rates) of the optimism-corrected model using calibration plots. The statistical analysis was conducted in R open source software, version 3.4.3 [Reference R Core Team62].

3. Results

The TRIPOD checklist for the study is presented in eTable 1.

3.1. Socio-demographic and clinical characteristics

The baseline demographic and clinical characteristics of the CHR-P participants are presented in Table 1. The CHR-P individuals most commonly presented APS, either as the only symptoms or in association with GRD syndrome (81 subjects in total; 77.1%). The median SOFAS score indicated severe impairment in psychosocial functioning. The detailed characteristics of the sample is additionally included in eTable2.

3.2. Transition to psychosis

The median follow-up period for the entire sample was 36 months (IQR: 10–59 months; mean time: 35.4 ± 25.0 months). The cumulative probability of developing psychosis of the CHR-P individuals and time to psychosis onset (failure function) is presented in Fig. 1. The median time from the entry to transition was five months (25th-75th percentile 3.5–10.5 months; mean time: 9.1 months; standard deviation: 8.9 months). All converters developed schizophrenia, diagnosed on the basis of ICD-10 criteria [Reference World Health Organization47].

Table 1 Baseline socio-demographic and clinical characteristics of CHR-P individuals (N = 105).

APS: attenuated psychotic symptoms.

BLIPS: brief limited intermittent psychotic symptoms.

CAARMS: Comprehensive Assessment of At Risk Mental States.

CHR-P: clinical high risk for psychosis.

GRD: genetic risk and deterioration syndrome.

IQ: intelligence quotient.

SOFAS: Social and Occupational Functioning Assessment Scale.

* Full CAARMS characteristics have been included in supplementary materials (eTable2).

Twenty four (22.9%) of the tested individuals had been using antipsychotic drugs; however, this was unlikely to affect the transition to psychosis (p > 0.05, multivariate Cox regression adjusted for Unusual Thought Content and Disorganized Speech CAARMS scores).

3.3. Selection of predictors

A thorough literature search identified two candidate predictors of transition, which had been most commonly used in previous studies: speech disorganization and unusual thought content. Of 24 analysed studies, 12 indicated speech disorganization to be a clinical feature predicting transition or a feature that could significantly differentiate converters from non-converters [Reference Addington, Liu, Buchy, Cadenhead, Cannon and Cornblatt26, Reference Cornblatt, Carrión, Auther, McLaughlin, Olsen and John28, Reference Addington, Liu, Perkins, Carrion, Keefe and Woods31, Reference Ruhrmann, Schultze-Lutter, Salokangas, Heinimaa, Linszen and Dingemans33Reference Francesconi, Minichino, Carrión, Delle Chiaie, Bevilacqua and Parisi36, Reference Demjaha, Valmaggia, Stahl, Byrne and McGuire40Reference Mamah, Musau, Mutiso, Owoso, Abdallah and Cottler42, Reference Tarbox, Addington, Cadenhead, Cannon, Cornblatt and Perkins63, Reference Armando, Pontillo, De Crescenzo, Mazzone, Monducci and Lo Cascio64] while 10 studies identified unusual thought content [Reference Cannon, Cadenhead, Cornblatt, Woods, Addington and Walker23Reference Addington, Liu, Buchy, Cadenhead, Cannon and Cornblatt26, Reference Cannon, Yu, Addington, Bearden, Cadenhead and Cornblatt29Reference Crump, Arndt, Grivel, Horga, Corcoran and Brucato32, Reference Mason, Startup, Halpin, Schall, Conrad and Carr38, Reference Nelson, Yuen, Wood, Lin, Spiliotacopoulos and Bruxner65]; three of the studies included samples larger than 400 CHR-P individuals [Reference Addington, Liu, Buchy, Cadenhead, Cannon and Cornblatt26, Reference Cannon, Yu, Addington, Bearden, Cadenhead and Cornblatt29, Reference Nelson, Yuen, Wood, Lin, Spiliotacopoulos and Bruxner65]. Both speech disorganization and unusual thought content have also been found to be independent predictors of transition [Reference Addington, Liu, Buchy, Cadenhead, Cannon and Cornblatt26, Reference Ruhrmann, Schultze-Lutter, Salokangas, Heinimaa, Linszen and Dingemans33]; and high unusual thought content score has been described as a relevant predictor of psychosis in a consensus paper summarizing 20 years of research in the CHR-P field [Reference Fusar-Poli, Borgwardt, Bechdolf, Addington, Riecher-Rössler and Schultze-Lutter19]. Additionally, a systematic review by Schmidt et al. [Reference Schmidt, Cappucciati, Radua, Rutigliano, Rocchetti and Dell’Osso66] found unusual thought content to be included in a model presenting the highest positive predictive value (PPV), and speech disorganization in that with the second highest PPV. Importantly, collinearity of the two parameters has been excluded in the tested population (Spearman’s rho = 0.10; p = 0.31). Hence, these two parameters clearly merit consideration for inclusion in the model used in the present study.

Fig. 1. The cumulative probability of developing psychosis of the CHR-P individuals and time to psychosis onset (failure function). The dashed lines represent 95% confidence intervals of the estimated probability. There were 84 individuals at risk followed up for at least 6 months (80.0% of the entire sample), 77- for 12 mo (73.7%), 69 - for 18 mo (65.7%) 66 - for 24 mo (62.9%), 57 - for 30 mo (54.3%), 54 - for 36 mo (51.4%), 42 - for 42 mo (40.0%), 41- for 48 mo (39.0%), 35 - for 54 mo (33.3%), 24 - for 60 mo (22.9%), 16 – for 66 mo (15.2%) and 12 - for 72 mo (11.4%).

3.4. Prediction of transition

The multivariate Cox proportional hazards model found both speech disorganization (HR = 1.69; 95% CI: 1.39–2.05) and unusual thought content (HR = 1.51; 95%CI: 1.27–1.80) to be statistically significant (p < 0.0001) predictors of conversion. As shown in supplementary material (eFig. 1 and eTable 3), the model met the proportional hazards assumption. The parameter estimates in the regression equation were 0.41 for unusual thought content and 0.52 for speech disorganization.

The developed model presented an adequate c-index of 0.79, and its predictive abilities at different time points were satisfactory (Fig. 2) [Reference Hosmer, Lemeshow and May67]. Based on the model, an algorithm was generated allowing the probability of transition from a CHR-P to psychosis to be estimated. The risk calculator is available online at: https://link.konsta.com.pl/psychosis. We recommend using Google Chrome to open it. We have estimated that recording the two key positive items in subjects undergoing a CHR-P assessment would require 10–15 minutes, while two hours are needed to administer the full CAARMS.

The results of the bootstrap internal validation procedure are shown in Fig. 3. The average c-index for the models developed on bootstrap samples was 0.78 and the mean c-index of their performance in the original dataset was 0.77. Optimism corrected c-index after 1000-sample bootstrap internal validation was 0.78. The calibration plots revealed a high degree of consistency between the observed probabilities and the model-predicted probabilities derived from 1000 bootstrap resamples. The internal validation of our model did not reveal substantial overfitting issues because a similar performance was observed after correction for optimism.

4. Discussion

The study develops and internally validates a pilot clinical prediction model to refine the prognosis of transition from a risk stage to established psychosis. The prognostic model has been developed in a new cohort from Poland, which is representing eastern European CHR-P patients. High levels of speech disorganization and unusual thoughts were chosen as key predictors, on the basis of a priori knowledge. The model showed satisfactory accuracy to predict psychosis and was internally validated, showing no major overfitting problems.

Fig. 2. Performance of the model predicting transition from CHR-P to psychosis at multiple time points. Time-dependent receiver operating characteristics (ROC) curves and the parameters are illustrated for 1-, 2-, 3- and 4-year outcomes. AUC – area under the curve. Accuracy is the proportion of all subjects for which conditions were accurately predicted (transition and non-transition). Sensitivity is the proportion of truly predicted transitions to all factual transitions. Specificity is the proportion of truly predicted non-transitions to all factual non-transitions. The cut-off points were set a priori as 0.5.

Our study followed state-of-the-art recommendation for developing clinical prediction models in psychiatry [Reference Steyerberg and Vergouwe54Reference Fusar-Poli, Stahl and Steyerberg57], although we have been unable to perform external validation (see below). Current methodological guidelines recommend selecting predictors on the basis of a priori knowledge [Reference Studerus, Ramyead and Riecher-Rossler55, Reference Fusar-Poli, Stahl and Steyerberg57]. As recently noted, preselection of predictors that can reflect the ability of clinicians to formulate a prognosis is essential to inform the development of robust prediction models in the field [Reference Nelson, Yung and McGorry68]. Accordingly, we rigorously selected predictors which have a rich literature and a proven position as predictors of psychotic transition from CHR-P. Disorganized communication was chosen, because it has been found to be a predictor of psychosis in CHR-P studies including children and adolescents [Reference Armando, Pontillo, De Crescenzo, Mazzone, Monducci and Lo Cascio64], adolescents [Reference Cornblatt, Carrión, Auther, McLaughlin, Olsen and John28, Reference Bearden, Wu, Caplan and Cannon34, Reference Mamah, Musau, Mutiso, Owoso, Abdallah and Cottler42]· or older samples [Reference Ruhrmann, Schultze-Lutter, Salokangas, Heinimaa, Linszen and Dingemans33, Reference DeVylder, Muchomba, Gill, Ben-David, Walder and Malaspina35]. In addition, severity of unusual thoughts was also chosen, because it has been recognized as a risk factor for transition of CHR-P into psychosis in both CAARMS [Reference Thompson, Nelson and Yung25, Reference Perkins, Jeffries, Cornblatt, Woods, Addington and Bearden27, Reference Velthorst, Nieman, Becker, van de Fliert, Dingemans and Klaassen39, Reference Nelson, Yuen, Wood, Lin, Spiliotacopoulos and Bruxner65] or SIPS-based studies [Reference Cannon, Cadenhead, Cornblatt, Woods, Addington and Walker23, Reference Cannon, Yu, Addington, Bearden, Cadenhead and Cornblatt29]. In particular, both speech disorganization and unusual thought content have been found to significantly differentiate converters to psychosis from non-converters or to act as predictors of transition in studies based on populations from different world regions [Reference Addington, Liu, Buchy, Cadenhead, Cannon and Cornblatt26, Reference Addington, Liu, Perkins, Carrion, Keefe and Woods31, Reference Katsura, Ohmuro, Obara, Kikuchi, Ito and Miyakoshi41]. This specific selection was also adopted for the study for pragmatic reasons: an easy-to-perform short assessment of two positive symptoms is a routine procedure in clinical practice.

The pre-specified mode was applied to the first Polish CHR-P cohort. There is converging evidence that the actual level of psychosis that is observed in CHR-P individuals is mostly determined by the way these individuals have been recruited, and therefore by their risk enrichment before they undergo CHR-P assessment [Reference Fusar-Poli, Schultze-Lutter, Cappucciati, Rutigliano, Bonoldi and Stahl8, Reference Fusar-Poli, Rutigliano, Stahl, Schmidt, Ramella-Cravaro and Hitesh9, Reference Fusar-Poli, Schultze-Lutter and Addington69Reference Fusar-Poli71]. As mentioned in the Methods section, our sample is representative of CHR-P services from different countries because the observed risk enrichment is very similar to the pooled meta-analytical mean [Reference Fusar-Poli, Schultze-Lutter, Cappucciati, Rutigliano, Bonoldi and Stahl8]. This suggests that our results are theoretically generalisable.

The model showed some promising clinical value, with its overall accuracy (Harrell’s c-index of 0.79) being found to be adequate [Reference Hosmer, Lemeshow and May67], confirming that these two clinical characteristics may be potentially used to estimate the risk of developing psychotic disorder in CHR-P individuals at the Pan-European level. Importantly, because our sample was rather small with rare events, it was validated internally to correct for potential overfitting problems. A small difference was observed in the optimism corrected c-index (0.78) and original c-index (0.79), suggesting that it demonstrates resilience to overfitting, further confirming its potential utility. It is however important to note that the broad generalisability of our findings need to be estimated in future external replication studies. External replication studies in samples undergoing CHR-P assessment are particularly needed given the high variance in risk enrichment which is associated with the recruitment phases for these individuals [Reference Fusar-Poli, Schultze-Lutter, Cappucciati, Rutigliano, Bonoldi and Stahl8].

Fig. 3. Calibration plots of the bootstrap validation of the model for specific time points showing the relationship between the original model and the average of bootstrap-based models.

The grey diagonal line shows the ideal prediction;The dashed line shows the predicted and observed values in the original model; The thin black line shows the change in calibration following the bootstrap-based correction for optimism;

The calibration plots show both the accuracy of prediction and the resilience of the modelling method to overfitting.

The median time to transition in the study was found to be five months, which is in line with meta-analytical median time to psychosis observed CHR-P populations [Reference Kempton, Bonoldi, Valmaggia, McGuire and Fusar-Poli72]. To further illustrate the speed of psychosis progression for the entire cohort, the failure function was reported in Fig. 1. As shown in the Results Section (3.2), treatment with antipsychotics had no influence on the outcome. This may seem surprising in the light of previous findings which have offered promising results for implementing antipsychotic treatment in CHR-P individuals [Reference van der Gaag, Smit, Bechdolf, French, Linszen and Yung73]. Also, according to the guidelines of European Psychiatric Association, such treatment is recommended if psychological interventions prove to be ineffective [Reference Schmidt, Schultze-Lutter, Schimmelmann, Maric, Salokangas and Riecher-Rössler74]. However, recent network meta-analyses show that no treatment for CHR-P individuals to date appears superior to other forms of therapy in preventing transition to psychosis or in reducing attenuated psychotic- or negative symptoms [Reference Davies, Cipriani, Ioannidis, Radua, Stahl and Provenzani16, Reference Devoe, Farris, Townes and Addington75, Reference Devoe, Peterson and Addington76].

The model presented in this study is based on two clinical predictors that are routinely collected as part of the CHR-P assessment. The overall prognostic accuracy of the optimism-corrected model (Harrell’s c-index = 0.78) was comparable to that of the full CAARMS assessment [Reference Oliver, Kotlicka-Antczak, Minichino, Spada, McGuire and Fusar-Poli7], with no apparent predictive gain. However, there are two potential benefits that may be associated with our clinical prediction model. First, our model allows the transition to psychosis to be predicted at the individual subject level, either when employed as a complement to full CAARMS assessment or as an independent tool. This is not currently possible using available prognostic tools, which can only indicate the risk of conversion at a group level. Second, since our model employs only two predictors, it may offer some practical advantages in that it is shorter than the administration the full CAARMS. In fact, we estimate that only 10–15 minutes are typically needed to assess the two predictors that this model is based on, compared to the two hours needed to administer the full CAARMS. Although brief versions of the CAARMS (still including four positive symptom items) already exist [77, Reference Yung, Phillips, Simmons, Ward, Thompson and French78], their prognostic accuracy has not been validated; in contrast, the current study employs state-of-the-art measures of prognostic accuracy. In addition, while the brief version of the CAARMS does not allow any individualised prediction of outcomes, our clinical prediction tool, including only two essential predictors, can potentially be used to formulate predictions at the individual subject level.

We are aware that the clinical presentation of the CHR-P state is much more complex and there is undoubtedly an unequivocal need to evaluate a broad range of symptoms, problems and levels of functioning in each individual with CHR-P. However, this is not always possible, especially in low resourced early detection services. Conversely, our revised prognostic tool requires substantially less time for its administration, compared to the standard prognostic tools in this area. This may be a significant consideration when applying this tool in European countries like Poland, which may have more limited early intervention services.

The proposed risk estimation model has, however, several limitations. Firstly, it is based on a small sample size and a small number of events, which limits the generalisability of the model [Reference Fusar-Poli, Stahl and Steyerberg57]. This is particularity concerning given the lack of proper external validation. Although we have performed an internal validation, our clinical prediction model should be considered cautiously and cannot be recommended for implementation in clinical routine before it is fully validated externally in independent samples. The online version of the tool is therefore provided to help independent researchers running this clinical prediction model in their samples and to measure its actual reproducibility.

The assumed recently recommended methodological approach limited the possibility to fully explore our dataset: the necessity to keep the EPV of minimum 10 a priori reduced the number of variables included in the analysis to two. This can, however, offer some pragmatic advantages for potential implementation in clinical routine: the presence of a high level of unusual thoughts or speech disorganization can be easily recorded as part of the clinical routine of CHR-P services and further facilitate the external validation of our model.

Another limitation is that our sample was not large enough to allow the differential impact of different CHR-P subgroups on the final level of risk to psychosis to be tested [Reference Fusar-Poli, Cappucciati, Rutigliano, Schultze-Lutter, Bonoldi and Borgwardt5, Reference Fusar-Poli, Rutigliano, Stahl, Davies, De Micheli and Ramella-Cravaro6, Reference Fusar-Poli, Cappucciati, Borgwardt, Woods, Addington and Nelson79Reference Fusar-Poli, Cappucciati, De Micheli, Rutigliano, Bonoldi and Tognin81]. Further studies analysing other clinical and biological features predicting transition from a CHR-P to psychosis may be needed to improve the prognostic accuracy of our model.

5. Conclusions

This study develops and internally validates a clinically-based risk estimation tool which is based on levels of speech disorganization and of unusual thoughts. The tool is simple and requires less clinical resources compared to the original CHR-P assessment. Future external replication studies in independent samples are required before generalisability of this model outside Polish samples can be demonstrated.

Declaration of interests

None.

Acknowledgement

The study was financed by the Polish Science National Centre as part of a larger project (grant No NN 402 1793 34).

We would like to thank all the psychiatrists referring participants to the study.

Appendix A. Supplementary data

Supplementary material related to this article can be found, in the online version, at doi:https://doi.org/10.1016/j.eurpsy.2019.02.007.

References

Fusar-Poli, PThe clinical high-risk State for psychosis (CHR-P), version II. Schizophr Bull 2017; 43:4447.CrossRefGoogle ScholarPubMed
Yung, ARYuen, HPMcGorry, PDPhillips, LJKelly, DDell’Olio, M et al. Mapping the onset of psychosis: the comprehensive assessment of at-risk mental states. Aust N Z J Psychiatry 2005; 39:964971.CrossRefGoogle ScholarPubMed
Miller, TJMcGlashan, THRosen, JLCadenhead, KCannon, TVentura, J et al. Prodromal assessment with the structured interview for prodromal syndromes and the scale of prodromal symptoms: predictive validity, interrater reliability, and trainnig to reliability. Schizophr Bull 2003; 29:703715.CrossRefGoogle ScholarPubMed
Fusar-Poli, PDavies, CBonoldi, IForecasting risk to prevent mental disorders. JAMA Psychiatry 2018 in press.Google Scholar
Fusar-Poli, PCappucciati, MRutigliano, GSchultze-Lutter, FBonoldi, IBorgwardt, S et al. At risk or not at risk? A meta-analysis of the prognostic accuracy of psychometric interviews for psychosis prediction. World Psychiatry 2015; 14:322332.CrossRefGoogle ScholarPubMed
Fusar-Poli, PRutigliano, GStahl, DDavies, CDe Micheli, ARamella-Cravaro, V et al. Long-term validity of the at risk mental State (ARMS) for predicting psychotic and non-psychotic mental disorders. Eur Psychiatry 2017; 42:4954.CrossRefGoogle ScholarPubMed
Oliver, DKotlicka-Antczak, MMinichino, ASpada, GMcGuire, PFusar-Poli, PMeta-analytical prognostic accuracy of the comprehensive assessment of at risk mental States (CAARMS): the need for refined prediction. Eur Psychiatry 2018; 49:6268.CrossRefGoogle ScholarPubMed
Fusar-Poli, PSchultze-Lutter, FCappucciati, MRutigliano, GBonoldi, IStahl, D et al. The dark side of the moon: meta-analytical impact of recruitment strategies on risk enrichment in the clinical high risk State for psychosis. Schizophr Bull 2016; 42:732743.CrossRefGoogle ScholarPubMed
Fusar-Poli, PRutigliano, GStahl, DSchmidt, ARamella-Cravaro, VHitesh, S et al. Deconstructing pretest risk enrichment to optimize prediction of psychosis in individuals at clinical High risk. JAMA Psychiatry 2016; 73:12601267.CrossRefGoogle ScholarPubMed
American Psychiatric Association, Diagnostic and statistical manual of mental disorders (DSM5) 5th ed. 2013, American Psychiatric Press Washington, DC.CrossRefGoogle Scholar
Fusar-Poli, PDe Micheli, ACappucciati, MRutigliano, GDavies, CRamella-Cravaro, V et al. Diagnostic and prognostic significance of DSM-5 attenuated psychosis syndrome in services for individuals at ultra high risk for psychosis. Schizophr Bull 2018; 44:264275.CrossRefGoogle ScholarPubMed
Joa, IGisselgård, JBrønnick, KMcGlashan, TJohannessen, JOPrimary prevention of psychosis through interventions in the symptomatic prodromal phase, a pragmatic Norwegian ultra high risk study. BMC Psychiatry 2015; 22:89.CrossRefGoogle Scholar
Seidman, LJNordentoft, MNew targets for prevention of schizophrenia: Is It time for interventions in the premorbid phase?. Schizophr Bull 2015; 41:795800.CrossRefGoogle ScholarPubMed
Sommer, IEBearden, CEvan Dellen, EBreetvelt, EJDuijff, SNMaijer, K et al. Early interventions in risk groups for schizophrenia: what are we waiting for?. NPJ Schizophr 2016; 16003.CrossRefGoogle ScholarPubMed
Fusar-Poli, PExtending the benefits of indicated prevention to improve outcomes of first-episode psychosis. JAMA Psychiatry 2017; 74:667668.CrossRefGoogle ScholarPubMed
Davies, CCipriani, AIoannidis, JPARadua, JStahl, DProvenzani, U et al. Lack of evidence to favor specific preventive interventions in psychosis: a network meta-analysis. World Psychiatry 2018; 17:196209.CrossRefGoogle ScholarPubMed
National Institute for Clinical Excellence NICE Psychosis and schizophrenia in children and young people: recognition and management 2013 http://www.nice.org.uk/Guidance/CG155 http://wwwniceorguk/Guidance/CG155.Google Scholar
Fusar-Poli, PBorgwardt, SBechdolf, AAddington, JRiecher-Rössler, ASchultze-Lutter, F et al. The psychosis high-risk state: a comprehensive state-of-the-art review. JAMA Psychiatry 2013; 70:107120.CrossRefGoogle ScholarPubMed
McGorry, PThe next stage for early intervention: transdiagnostic, personalized, universal. Early Interv Psychiatry 10(Suppl.1)2016; 3 Abstract.Google Scholar
Riecher-Rössler, AStuderus, EPrediction of conversion to psychosis in individuals with an at-risk mental state: a brief update on recent developments. Curr Opin Psychiatry 2017; 30:209219.CrossRefGoogle ScholarPubMed
Yung, ARPhillips, LJYuen, HPMcGorry, PDRisk factors for psychosis in an ultra high-risk group: psychopathology and clinical features. Schizophr Res 2004; 67:131142.CrossRefGoogle Scholar
Cannon, TDCadenhead, KCornblatt, BWoods, SWAddington, JWalker, E et al. Prediction of psychosis in youth at high clinical risk: a multisite longitudinal study in North America. Arch Gen Psychiatry 2008; 65:2837.CrossRefGoogle ScholarPubMed
Riecher-Rössler, APflueger, MOAston, JBorgwardt, SJBrewer, WJGschwandtner, U et al. Efficacy of using cognitive status in predicting psychosis: a 7-year follow-up. Biol Psychiatry 2009; 66:10231030.CrossRefGoogle ScholarPubMed
Thompson, ANelson, BYung, ARPredictive validity of clinical variables in the “at risk” for psychosis population: international comparison with results from the North American Prodrome Longitudinal Study. Schizophr Res 2011; 126:5157.CrossRefGoogle ScholarPubMed
Addington, JLiu, LBuchy, LCadenhead, KSCannon, TDCornblatt, BA et al. North American Prodrome Longitudinal Study (NAPLS 2). The prodromal symptoms. J Nerv Ment Dis 2015; 2039:328335.CrossRefGoogle Scholar
Perkins, DOJeffries, CDCornblatt, BAWoods, SWAddington, JBearden, CE et al. Severity of thought disorder predicts psychosis in persons at clinical high-risk. Schizophr Res 2015; 169:169177.CrossRefGoogle ScholarPubMed
Cornblatt, BACarrión, REAuther, AMcLaughlin, DOlsen, RHJohn, M et al. Psychosis prevention: a modified clinical High risk perspective from the recognition and prevention (RAP) program. Am J Psychiatry 2015; 172:986994.CrossRefGoogle ScholarPubMed
Cannon, TDYu, CAddington, JBearden, CECadenhead, KSCornblatt, BA et al. An individualized risk calculator for research in prodromal psychosis. Am J Psychiatry 2016; 173:980988.CrossRefGoogle ScholarPubMed
Carrión, RECornblatt, BABurton, CZTso, IFAuther, AMAdelsheim, S et al. Personalized prediction of psychosis: external validation of the NAPLS-2 psychosis risk calculator with the EDIPPP project. Am J Psychiatry 2016; 173:989996.CrossRefGoogle ScholarPubMed
Addington, JLiu, LPerkins, DOCarrion, REKeefe, RSWoods, SWThe role of cognition and social functioning as predictors in the transition to psychosis for youth with attenuated psychotic symptoms. Schizophr Bull 2017; 43:5763.CrossRefGoogle ScholarPubMed
Crump, FMArndt, LGrivel, MHorga, GCorcoran, CMBrucato, G et al. Attenuated first-rank symptoms and conversion to psychosis in a clinical high-risk cohort. Early Interv Psychiatry December)2017 doi:http://dx.doi.org/10.1111/eip.12529.Google Scholar
Ruhrmann, SSchultze-Lutter, FSalokangas, RKRHeinimaa, MLinszen, DDingemans, P et al. Prediction of psychosis in adolescents and young adults at high risk. Results from the prospective European prediction of psychosis study. Arch Gen Psychiatry 2010; 67:241251.CrossRefGoogle Scholar
Bearden, CEWu, KNCaplan, RCannon, TDThought disorder and communication deviance as predictors of outcome in youth at clinical high risk for psychosis. J Am Acad Child Adolesc Psychiatry 2011; 50:669680.CrossRefGoogle ScholarPubMed
DeVylder, JEMuchomba, FMGill, KEBen-David, SWalder, DJMalaspina, D et al. Symptom trajectories and psychosis onset in a clinical high-risk cohort: the relevance of subthreshold thought disorder. Schizophr Res 2014; 159:278283.CrossRefGoogle Scholar
Francesconi, MMinichino, ACarrión, REDelle Chiaie, RBevilacqua, AParisi, M et al. Psychosis prediction in secondary mental health services. A broad, comprehensive approach to the "at risk mental state" syndrome. Eur Psychiatry 2017; 40:96104.CrossRefGoogle ScholarPubMed
Zhang, THLi, HJWoodberry, KAXu, LHTang, YYGuo, Q et al. Two-year follow-up of a Chinese sample at clinical high risk for psychosis: timeline of symptoms, help-seeking and conversion. Epidemiol Psychiatr Sci 2017; 26:287298.CrossRefGoogle ScholarPubMed
Mason, OStartup, MHalpin, SSchall, UConrad, ACarr, VRisk factors for transition to first episode psychosis among individuals with’ at-risk mental states’. Schizophr Res 2004; 71:227237.CrossRefGoogle Scholar
Velthorst, ENieman, DHBecker, HEvan de Fliert, RDingemans, PMKlaassen, R et al. Baseline differences in clinical symptomatology between ultra high risk subjects with and without a transition to psychosis. Schizophr Res 2009; 109:6065.CrossRefGoogle ScholarPubMed
Demjaha, AValmaggia, LStahl, DByrne, MMcGuire, PDisorganization/cognitive and negative symptom dimensions in the at-risk mental state predict subsequent transition to psychosis. Schizophr Bull 2012; 38:351359.CrossRefGoogle ScholarPubMed
Katsura, MOhmuro, NObara, CKikuchi, TIto, FMiyakoshi, T et al. A naturalistic longitudinal study of at-risk mental state with a 2.4 year follow-up at a specialized clinic setting in Japan. Schizophr Res 2014; 158:3238.CrossRefGoogle Scholar
Mamah, DMusau, AMutiso, VNOwoso, AAbdallah, ABCottler, LB et al. Characterizing psychosis risk traits in Africa: a longitudinal study of Kenyan adolescents. Schizophr Res 2016; 176:340348.CrossRefGoogle ScholarPubMed
First, MBSpitzer, RLGibbon, MWilliams, JBWStructured clinical interview for DSM-IV axis I disorders, (SCID-I) 1997, American Psychiatric Press Washington, DC.Google Scholar
First, MBGibbon, MSpitzer, RLWilliams, JBWBenjamin, LSStructured clinical interview for DSM-IV axis II personality disorders, (SCID-II) 1997, American Psychiatric Press Washington, DC.Google Scholar
Kotlicka-Antczak, MPawełczyk, TRabe-Jabłońska, JPawełczyk, APORT (programme of recognition and therapy): the first Polish recognition and treatment programme for patients with an at-risk mental state. Early Interv Psychiatry 2015; 9:339342.CrossRefGoogle ScholarPubMed
Kotlicka-Antczak, MPawełczyk, TPodgórski, MŻurner, NKarbownik, MSPawełczyk, APolish individuals with an at-risk mental state: demographic and clinical characteristics. Early Interv Psychiatry 2018; 12:391399.CrossRefGoogle ScholarPubMed
World Health Organization, International classification of diseases 10th revision 2010, ICD10 Geneva.Google Scholar
Fusar-Poli, PSchultze-Lutter, FPredicting the onset of psychosis in patients at clinical high risk: practical guide to probabilistic prognostic reasoning. Evid Based Ment Health 2016; 19:1015.CrossRefGoogle ScholarPubMed
Jaracz, JGrzechowiak, MRaczkowiak, LRataj, KRybakowski, JPolish version of comprehensive assessment of at risk mental States (CAARMS) - the description of the method. Psychiatr Pol 2012; 46:95107.Google ScholarPubMed
Wechsler, DIntelligence scale for children, modified version. Manual, Polish version 2008, Psychological Testing Centre Warsaw.Google Scholar
Wechsler, DIntelligence scale for adults, revised. Manual, Polish version 2004, Psychological Testing Centre Warsaw.Google Scholar
Moons, KGAltman, DGReitsma, JBIoannidis, JPMacaskill, PSteyerberg, EWTransparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med 2015; 162:173.CrossRefGoogle ScholarPubMed
International Early Psychosis Association Writing Group International clinical practice guidelines for early psychosis. Br J Psychiatry 48(suppl)2005; 120124.CrossRefGoogle Scholar
Steyerberg, EWVergouwe, YTowards better clinical prediction models: seven steps for development and an ABCD for validation. Eur Heart J 2014; 35:19251931.CrossRefGoogle ScholarPubMed
Studerus, ERamyead, ARiecher-Rossler, APrediction of transition to psychosis in patients with a clinical high risk for psychosis: a systematic review of methodology and reporting. Psych Med 2017; 47:11631178.CrossRefGoogle ScholarPubMed
Wynants, LCollins, GSVan Calster, BKey steps and common pitfalls in developing and validating risk models. BJOG 2017; 124:423432.CrossRefGoogle ScholarPubMed
Fusar-Poli, Ziad H.Stahl, DSteyerberg, EThe science of prognosis in psychiatry. JAMA Psychiatry 2018 in press.CrossRefGoogle Scholar
Chawla, NVBowyer, KWHall, LOKegelmeyer, WPSMOTE: synthetic minority Over-sampling technique. JAIR 2002; 16: doi:http://dx.doi.org/10.1613/jair.953.CrossRefGoogle Scholar
Uno, HCai, TPencina, MJD’Agostino, RBWei, LJOn the C-statistics for evaluating overall adequacy of risk prediction procedures with censored survival data. Stat Med 2011; 30:11051117.Google ScholarPubMed
Kamarudin, ANCox, TKolamunnage-Dona, RTime-dependent ROC curve analysis in medical research: current methods and applications. BMC Med Res Methodol 2017; 17:53.CrossRefGoogle ScholarPubMed
Efron, BTibshirani, RAn introduction to the bootstrap 1993, Chapman & Hall New York, London.CrossRefGoogle Scholar
R Core Team, R: a language and environment for statistical computing URL 2017, R Foundation for Statistical Computing Vienna, Austriahttp://www.R-project.org/.Google Scholar
Tarbox, SIAddington, JCadenhead, KSCannon, TDCornblatt, BAPerkins, DO et al. Premorbid functional development and conversion to psychosis in clinical high-risk youths. Dev Psychopathol 2013; 25:11711186.CrossRefGoogle ScholarPubMed
Armando, MPontillo, MDe Crescenzo, FMazzone, LMonducci, ELo Cascio, N et al. Twelve-month psychosis-predictive value of the ultra-high risk criteria in children and adolescents. Schizophr Res 2015; 169:186192.CrossRefGoogle ScholarPubMed
Nelson, BYuen, HPWood, SJLin, ASpiliotacopoulos, DBruxner, A et al. Long-term follow-up of a group at ultra high risk ("prodromal") for psychosis: the PACE 400 study. JAMA Psychiatry 2013; 70:793802.CrossRefGoogle ScholarPubMed
Schmidt, ACappucciati, MRadua, JRutigliano, GRocchetti, MDell’Osso, L et al. Improving prognostic accuracy in subjects at clinical High risk for psychosis: systematic review of predictive models and meta-analytical sequtarboxential testing simulation. Schizophr Bull 2017; 43:375388.Google ScholarPubMed
Hosmer, DWLemeshow, SMay, SApplied survival analysis: regression modeling of time-to-event data 2nd ed. 2008, Wiley-Interscience Hoboken, N.J.CrossRefGoogle Scholar
Nelson, BYung, ARMcGorry, PDImportance of variable selection in multimodal prediction models in patients at clinical high risk for psychosis and recent-onset depression. JAMA Psychiatry 2019 doi:http://dx.doi.org/10.1001/jamapsychiatry.2018.4234.CrossRefGoogle ScholarPubMed
Fusar-Poli, PSchultze-Lutter, FAddington, JIntensive community outreach for those at ultra high risk of psychosis: dilution, not solution. Lancet Psychiatry 2016;3(18):.CrossRefGoogle Scholar
Fusar-Poli, PPalombini, EDavies, COliver, DBonoldi, IRamella-Cravaro, V. et al. Why transition risk to psychosis is not declining at the OASIS ultra high risk service. The hidden role of stable pretest risk enrichment. Schizophr Res 2018; 192:385390.CrossRefGoogle Scholar
Fusar-Poli, PWhy ultra high risk criteria for psychosis prediction do not work well outside clinical samples and what to do about it. World Psychiatry 2017; 16:212213.CrossRefGoogle ScholarPubMed
Kempton, MJBonoldi, IValmaggia, LMcGuire, PFusar-Poli, PSpeed of psychosis. Progression in people at ultra-high clinical risk: a complementary meta-analysis. JAMA Psychiatry 2015; 72:622623.CrossRefGoogle ScholarPubMed
van der Gaag, MSmit, FBechdolf, AFrench, PLinszen, DHYung, AR et al. Preventing a first episode of psychosis: meta-analysis of randomized controlled prevention trials of 12 month and longer-term follow-ups. Schizophr Res. 2013; 149:5662.CrossRefGoogle ScholarPubMed
Schmidt, SJSchultze-Lutter, FSchimmelmann, BGMaric, NPSalokangas, RKRiecher-Rössler, A et al. EPA guidance on the early intervention in clinical high risk states of psychoses. Eur Psychiatry 2015; 30:388404.CrossRefGoogle ScholarPubMed
Devoe, DJFarris, MSTownes, PAddington, JAttenuated psychotic symptom interventions in youth at risk of psychosis: a systematic review and meta-analysis. Early Interv Psychiatry 2019; 13:317.CrossRefGoogle ScholarPubMed
Devoe, DJPeterson, AAddington, JNegative symptom interventions in youth at risk of psychosis: a systematic review and network meta-analysis. Schizophr Bull. 2018; 44:807823.CrossRefGoogle ScholarPubMed
Yung, APhillips, LSimmons, MBWard, JThompson, KFrench, P et al. Comprehensive assessment of at risk mental States (CAARMS). Brief version for use in EDIT 2015 https://www.rcpsych.ac.uk/pdf/Brief%20CAARMS%20with%20SOFAS%202016.pdf.Google Scholar
Fusar-Poli, PCappucciati, MBorgwardt, SWoods, SWAddington, JNelson, B et al. Heterogeneity of psychosis risk within individuals at clinical High risk: a meta-analytical stratification. JAMA Psychiatry 2016; 73:113120.CrossRefGoogle ScholarPubMed
Fusar-Poli, PCappucciati, MBonoldi, IHui, LMRutigliano, GStahl, DR et al. Prognosis of brief psychotic episodes: a meta-analysis. JAMA Psychiatry 2016; 73:211220.CrossRefGoogle ScholarPubMed
Fusar-Poli, PCappucciati, MDe Micheli, ARutigliano, GBonoldi, ITognin, S et al. Diagnostic and prognostic significance of brief limited intermittent psychotic symptoms (BLIPS) in individuals at ultra high risk. Schizophr Bull 2017; 43:4856.CrossRefGoogle ScholarPubMed
Figure 0

Table 1 Baseline socio-demographic and clinical characteristics of CHR-P individuals (N = 105).

Figure 1

Fig. 1. The cumulative probability of developing psychosis of the CHR-P individuals and time to psychosis onset (failure function). The dashed lines represent 95% confidence intervals of the estimated probability. There were 84 individuals at risk followed up for at least 6 months (80.0% of the entire sample), 77- for 12 mo (73.7%), 69 - for 18 mo (65.7%) 66 - for 24 mo (62.9%), 57 - for 30 mo (54.3%), 54 - for 36 mo (51.4%), 42 - for 42 mo (40.0%), 41- for 48 mo (39.0%), 35 - for 54 mo (33.3%), 24 - for 60 mo (22.9%), 16 – for 66 mo (15.2%) and 12 - for 72 mo (11.4%).

Figure 2

Fig. 2. Performance of the model predicting transition from CHR-P to psychosis at multiple time points. Time-dependent receiver operating characteristics (ROC) curves and the parameters are illustrated for 1-, 2-, 3- and 4-year outcomes. AUC – area under the curve. Accuracy is the proportion of all subjects for which conditions were accurately predicted (transition and non-transition). Sensitivity is the proportion of truly predicted transitions to all factual transitions. Specificity is the proportion of truly predicted non-transitions to all factual non-transitions. The cut-off points were set a priori as 0.5.

Figure 3

Fig. 3. Calibration plots of the bootstrap validation of the model for specific time points showing the relationship between the original model and the average of bootstrap-based models.The grey diagonal line shows the ideal prediction;The dashed line shows the predicted and observed values in the original model; The thin black line shows the change in calibration following the bootstrap-based correction for optimism;The calibration plots show both the accuracy of prediction and the resilience of the modelling method to overfitting.

Supplementary material: File

Kotlicka-Antczak et al. supplementary material

Kotlicka-Antczak et al. supplementary material
Download Kotlicka-Antczak et al. supplementary material(File)
File 130.7 KB
Submit a response

Comments

No Comments have been published for this article.