Building capacity to provide innovative interventions for early psychosis in mental health professionals

Abstract

Despite international guidelines, cognitive behavioural therapy for early psychosis (CBTep) is still under-used in daily clinical practice, mainly due to the lack of specific skills among mental health professionals. The aim of the study was to evaluate the feasibility and efficacy of a CBTep training course and to investigate the impact of trainees’ variables on the level of skills acquisition. An intensive and graded CBTep training programme consisting of 112 hours of plenary lectures, 30 hours of group supervision and 3 months of practical training was offered to mental health professionals of 65 Italian community Mental Health Centers (CMHCs). CBT expert psychologists were used as the comparison group. Participants underwent pre-planned exams to test the level of skills acquisition and were requested to complete a satisfaction survey. The vast majority of participants (93%) completed the training with medium–high evaluation scores and reported to be highly satisfied with the course. CMHCs staff members achieved high scores in the examinations and no major differences between them and CBT expert psychologists were found in most of the final exam scores. Our results support the feasibility and the efficacy of the training to build specific CBTep capacity in a large cohort of professionals working in Italian Generalist Mental Health Services.

Key learning aims

1. (1) To understand the capacity building of a short training programme in CBT for early psychosis dedicated to community mental health professionals.

2. (2) To consider the optimal characteristics of a CBT training programme for early psychosis.

3. (3) To reflect on the feasibility of a CBT training programme for early psychosis in the context of Italian Community Mental Health Services.

Introduction

In the last two decades, many studies have proved the benefit and effectiveness of a multi-element psychosocial approach provided for severe psychiatric disorders like psychosis, especially in the early phases of the illness (Bird et al., 2010; Birchwood et al., 1998; Dixon et al., 2010; Dunn et al., 2012; Edwards et al., 2005; Kuipers et al., 1997; Lecomte et al., 2012; Morrison et al., 2004; Penn et al., 2005; Ruggeri et al., 2015; Sarin et al., 2011; Tarrier et al., 1998; Wykes et al., 2008). According to international guidelines [National Institute for Clinical Excellence (NICE), 2014], optimal intervention for early psychosis should include adequate anti-psychotic medication combined with individual structured psychological intervention. Specifically, cognitive behavioalur therapy for early psychosis (CBTep) is indicated as a first-line psychological treatment for patients at the onset of psychosis (NICE, 2003, 2009, 2014). The importance of professional competence in delivering CBTep has been consistently highlighted in the literature (Jolley et al., 2012, 2015; NICE, 2009, 2014). In particular, providing mental health staff with advanced and specific skills on assessment and therapeutic techniques should be guaranteed for the effectiveness of the intervention (Fowler et al., 1995; Rollinson et al., 2008; Ruggeri et al., 2008).

While the efficacy of CBT in reducing re-hospitalization and distress associated with both positive and negative symptoms has been recognized (Lecomte et al., 2012; NICE, 2003, 2009, 2014), few published studies have specifically investigated the feasibility and efficacy of a training programme on CBT for early psychosis provided to professionals who work daily with patients at the first period of the illness (Beidas and Kendall, 2010; Rakovshik and McManus, 2010). According to this perspective, Beidas and Kendall (2010) underline the importance of evaluating the impact of contextual variables of the training, such as the organizational support, the quality of the programme, and client and therapist variables. Some existing studies (Jolley et al., 2015; Waller et al., 2013, 2015) explored the efficacy and the feasibility of cognitive behavioural therapy for psychosis (CBTp) training for mental health staff. However, in these studies, staff were not specifically trained on early psychosis but more generally on psychotic patients, and the number of staff members involved was exiguous.

Therefore, in this study we wanted to address the shortcomings of the current evidence by evaluating the delivery of a CBTep training to staff working in different Italian Community Mental Health Centres (CMHCs).

The present work can be considered in line with the larger framework of the movement called ‘Capacity Building’, which is a worldwide movement with the aim to fill ‘the mental health treatment gap’ (the increasingly evident mismatch of mental health disease burden with the extant resources; Saxena et al., 2007) and to equip the local workforce to deliver high-quality, innovative, locally relevant, feasible and effective interventions (Becker and Kleinman, 2012; NICE, 2009), also in low- and middle-income countries (Fricchione et al., 2012; Thornicroft et al., 2012).

Specifically, the aims of the present study were: (i) to evaluate the feasibility and efficacy of a training programme developed in order to build specific expertise on CBT for early psychosis for staff working in CMHCs, and (ii) to investigate the possible association between the characteristics of the professionals and the competence acquisition.

Method

Study framework

The present study was conducted in the context of the ‘Psychosis early Intervention and Assessment of Needs and Outcome’ (PIANO) randomized clinical trial.

The PIANO study was part of a more general research project named GET UP (Genetics, Endophenotypes, and Treatment: Understanding early Psychosis), which was a large, multi-element research programme involving 117 CMHCs in Northern and Central Italy, covering a catchment population of approximately 10 million inhabitants.

The primary objectives of the PIANO were to compare the 9-month effectiveness of a multi-component psychosocial intervention versus ‘treatment as usual’ (TAU) for patients with first-episode psychosis (FEP) and their family members recruited from Italian CMHCs. CHMCs randomized in the experimental arm of the trial offered, in addition to the treatment as usual, CBTep for patients, family intervention and case management (Ruggeri et al., 2012, 2015). The PIANO study was a pragmatic cluster-randomized controlled trial, with the CMHCs as randomization clusters. In detail, 117 CMHCs were enrolled in the project; of these, 96 CMHCs entered in the randomization procedure because 32 small CMHCs were paired and therefore considered as 16 CMHCs. Forty-eight were randomized in the TAU arm and 48 in the experimental arm. Before the start of the trial, local mental health professionals in the experimental arm of the study received a specific training programme in CBTep, which represents the focus of the present study.

Participants

The CBTep training programme was delivered to psychiatrists and psychologists who worked in the 65 Italian CMHCs included in the experimental arm of the PIANO randomized clinical trial. At least two psychiatrists/psychologists per experimental CMHC took part in the training as they are the only professional figures allowed to provide any form of psychotherapy in Italy. Trainees were permanent salaried staff of the CMHC so they were not paid specifically for the delivery of the CBTep intervention.

The only selection requirement for participation in the training was that staff members should have worked for at least 5 years in clinical settings with people with psychosis. Expert psychologists with a specific 4-year CBT masters level qualification also joined the training course to support CMHC staff members in the use of CBTep within the PIANO trial. Expert psychologists also represented the comparison arm in order to compare the specific CBTep skills they developed during the training course with those acquired by CMHC staff members.

Training course

The training course was an innovative, focused and graded CBTep training developed to train local CMHC professionals to deliver CBTep in accordance with the NICE guidelines. It was developed by the staff members of the Italian Center for Early Psychosis ‘Programma 2000’, who had been working in this field for more than 10 years (Cocchi et al., 2008; Meneghelli et al., 2010). The training programme was included in a postgraduate course at Verona University and teachers involved were national and international experts in the field of the psychosis onset. The training was planned and organized on three main levels: teaching modules, practical training and supervision.

Specifically, the course started in June 2009 and finished in December of the same year and consisted of 112 hours of teaching (divided into four modules of 3 or 4 days), 30 hours of small group supervision and 3 months of practical training that included exercises, videos and role-playing.

The contents were conceived starting from the bases of the CBTep model developed by Kuipers and colleagues (1998), Garety and colleagues (2008) and Fowler and colleagues (1995), whose efficacy was demonstrated in previous randomized clinical trials (Fowler et al., 1995; Garety et al., 2008; Kuipers et al., 1998; Turkington et al., 2006). Table 1 reports the contents of the CBTep model used in this project, divided into four different modules.

Table 1. Training course content

A detailed CBTep intervention manual was developed according to the content covered in the course and by taking into account the critical aspects raised by the staff members. The intervention manual was provided to each participant to guide the use of CBT for early psychosis in clinical practice. Finally, after the course, supervision groups were carried out for the entire duration of the GET UP programme, in order to monitor the use of CBT and to maintain the skills acquired by the clinicians involved.

Assessments and measures

At the end of each module and end of the entire course, all participants (both CMHC staff members and CBT expert psychologists) were evaluated on acquired CBTep theoretical knowledge and practical skills.

The measures of training feasibility were: staff member drop-out, reasons for staff member drop-out, and the percentage of sessions attended.

The measures of training efficacy were: the written examination scores obtained after each course module, including theoretical and clinical CBT skills (maximum score: 40) and the supervisor’s judgement (maximum score: 10) – see the Supervisor assessment schedule in Appendix 1 of the Supplementary material. The final overall score to pass the course was 35/50.

Examination sessions included theoretical examination and case reports for all modules, except for the examination session of module 2 that included only questionnaires. Independent blinded expert trainers evaluated the participants’ results by assigning a score on a scale from 0 to 50.

Theoretical examinations consisted of multiple-choice questionnaires concerning the conceptual contents of the training. Case reports included case analysis, formulation and treatment strategies. In particular, experts assessed each participant’s level of knowledge of CBT theory and their skills to identify clinical problems, conceptualize the case following the CBT model (Fowler et al., 1995; Garety et al., 2008; Kuipers et al., 1998), recognize relevant goals and carry out a treatment programme.

Moreover, supervisors expert in CBTep at the end of the practical training and during the supervision period assessed each participant’s capacity building with regards to ‘Participation’, ‘Attitude’, ‘Therapeutic style’ and ‘Congruency with CBT contents’, by using the Supervisor Assessment Schedule (see Appendix 1 in Supplementary material). In order to avoid differences in the assessment style, supervisors were selected on purpose from the Programma 2000 team as they all shared a consistent theoretical and clinical approach and had long experience in working together on FEP cases within a comprehensive multi-modal protocol of early intervention in psychosis (Cocchi et al., 2008; Meneghelli et al., 2010).

To pass the final exam of the course, participants had to achieve a minimum total score of 35/50 at the end of the training. This score was made up by the combination of the final exam score and supervisory judgement. Lastly, participants’ course satisfaction was evaluated by using an ad hoc questionnaire for each training session on a score from 0 to 10.

At the end of the training, CBTep intervention manuals per international standards were given to the staff members as treatment references. During the 9 months of the trial, staff members were supported by CBT expert psychotherapists assigned to each CMHC and written reports of each session were produced (see the Schematic Report Schedule in Appendix 1). They were also supervised by external supervisors expert in CBTep who held 1-day meetings every 2 months, and were regularly available for consultation to ensure fidelity of the intervention. A random sample of sessions was audio-recorded to allow further fidelity measurement by independent raters.

Statistical analysis

The whole sample of participants was divided into two groups: group 1: CMHC staff members (psychiatrists and psychologists) and group 2: CBT expert psychologists, as a comparison group. Comparison was executed by t-test for independent groups (continuous variables) and Fisher’s exact test (categorical variables). All tests were bilateral at p = <0.01 (due to multiple testing). Data were analysed using Statistical Package for the Social Sciences (SPSS).

To test the role of demographic and background characteristics of CMHC staff members on competence acquisition, participants were dichotomized as follows: age <40 vs ≥40; years from graduation <10 vs ≥10; years from qualification test <10 vs ≥10; years of work experience <10 vs ≥10; theoretical orientation: CBT No vs Yes; and professional background: psychiatrists vs psychologists. Due to the descriptive nature of the study, no correction for multiple testing was applied and a conservative p-value was chosen (p < 0.01).

Results

Participants

One hundred and five CMHCs staff members were invited to join the course. Eight of them dropped out before the beginning of the course (four moved to other services and four for other reasons) and one after the beginning of the course (because of serious family health reasons), leading to a drop-out rate of 8.6%. Therefore, a total of 96 CMHCs staff members (group 1) and 24 CBT expert psychologists (group 2) attended and completed the course. The sample had a mean age of 41.47 years (SD = 8.16), and there was a preponderance of females (68.3%). Participants attended 93% of the course (minimum frequency required 70%, corresponding to 104 hours). The majority of the CMHC staff members were psychiatrists (75%), with no previous experience in CBTep and with a mean of 11.78 years (SD = 7.52) of work in CMHC. The CBT experts were all psychologists, on average younger [mean age: 33.21 years (SD = 3.43) vs 43.53 years (SD = 7.69), respectively; p < .001] and with less working experience in mental health clinical setting compared with CMHC professionals (mean years of work: 2.32 (SD = 2.67) vs 11.78 (SD = 7.52), respectively; p < .001). Among the 24 CBT expert psychologists, 21 had completed a specific 4-year Masters level in CBT and three had attended specific training courses with well-documented experience in CBT for psychosis. Table 2 shows in detail the demographic and background characteristics of the participants.

Table 2. Socio-demographic characteristics of participants

Theoretical capacity building

Overall, all trainees passed each examination session positively, and at the end of the training most of them (80.21% of the CMHC staff members and 95.65% of the CBT expert psychologists) reached medium–high total scores level (41–50/50) (see Table 3).

Table 3. Total score obtained by the participants at the end of training (%)

The data show that there was no significant difference between the two groups both in ongoing exam scores and in the final exam total score [group 1 = 34.94 (SD = 2.04) vs group 2 = 36.08 (SD=2.45), p = .020]; only in the ongoing exam of the training session 2 did we find a significant difference in favour of group 2 (p = 0.002) (see Table 4; Fig. 1).

Table 4. Examination scores (ongoing examination and final examination) of the whole sample and comparisons between CMHC staff members and previously trained CBT psychologists

Figure 1. Theoretical capacity building after training.¹

¹Original scores have been standardised according to a 1-100 scale

Clinical CBT capacity building

Considering supervisors’ judgements, we found a difference between the two groups in the final total score [total score: group 1 = 7.71 (SD = 1.23) vs group 2 = 9.30 (SD = 0.82), p < .001] and in two of the four evaluation areas: ‘Therapeutic style’ [group 1 = 1.39 (SD = 0.50) vs group 2 = 1.91 (SD = 0.29), p < .001] and ‘Congruency with CBT’ [group 1 = 1.83 (SD = 0.65) vs group 2 = 2.63 (SD = 0.58), p < .001] (see Table 5; Fig. 2). However, the score difference was small, suggesting no major difference between the capacity acquired by the participants of the two groups.

Table 5. Supervisor judgement of the whole sample and comparisons between CMHC staff members and of previously trained CBT psychologists

Figure 2. Clinical CBT capacity building (supervisor judgements, mean values) after the training.

After the training, the number of sessions delivered by the trained staff members during the 9 months has been recorded. In total, 272 patients were enrolled in CBTep. Out of the 96 staff members trained, 74 were actively working with the patients. A mean number of 65.4 sessions for each trained professional was done.

Impact of sociodemographic characteristics in CMHC staff members group

Descriptive analysis does not show a significant difference on the basis of age, years from graduation, years of work, and theoretical orientation, in training sessions 1 (fundamental principles of CBT) and 2 (theoretical basis of CBT for psychosis), both in the ongoing exam scores and in the final exam total scores. In training session 3 (specific CBT for first episode psychosis), we found a significant difference in favour of younger participants, with fewer years from graduation and less work experience; see Table 6). We detected some minor differences in training session 3 among professionals with different backgrounds. In detail, psychologists showed slightly higher scores than other professionals. Interestingly, previous knowledge in CBT did not strongly affect the outcome (Table 6).

Table 6. Association between examination scores (ongoing examination and final examination) and socio-demographic participant characteristics (CMHC staff sample)

Satisfaction with the course

The course was well received by trainee CMHC staff, with high scores on course satisfaction [minimum 0; maximum 10; mean 7.81, SD 1.14).

Discussion

The aim of this study was to evaluate the feasibility and efficacy of a brief and intensive CBTep training provided to mental health professionals of CMHCs.

The importance of providing this type of specific training to professionals has been pointed out by the NICE guidelines: ‘Trusts should provide access to training that equips healthcare professionals with the competencies required to deliver the psychological therapy interventions recommended in this guideline’ (NICE 2009, 2014).

As mentioned in the introduction, many studies evaluated the efficacy of CBT treatment for patients with psychosis or schizophrenia, but few described the results of a specific training for staff. For instance, Rakovshik and McManus (2010) conducted a review where they included 34 trials on CBT, emphasizing that only seven studies were specifically focused on training efficacy and underlining that an ‘extensive’ (more than 137 hours) and ‘graded’ training (in which the stages of therapist involvement progressed as competence developed) seems to increase training effectiveness, although having the drawback of being costly and often impractical. Furthermore, experiential and interactive approaches using clinical cases, co-therapy and supervision added to traditional type of training (workshop, reading, web-based instruction), are important to maintain competency over time.

Recently, Jolley and colleagues (2015) ran a pilot evaluation of therapist training in a CBT programme for psychosis targeted to a small number of professionals (n = 9) working in mental health services. They found that high-quality training in CBT had a beneficial effect on clinical outcomes of patients. In contrast to our study, they trained different-type mental health professionals (nurses, occupational therapists, clinical psychologists), and directed the therapy to patients in different phases of the psychosis illness.

The training course used in our study has been organized according to the main requirements that have been proven to be associated with better outcomes (Jolley et al., 2015), such as graded approach, extensive duration, interactive training and sustained supervision. Concerning the quality of training, the content and method used, we included the best evidence in this field according to the NICE guidelines (CBTep and interactive learning); similarly, organizational support was considered, with sustained supervision to facilitate the use of CBTep in routine clinical practice. The low drop-out rate and the successful completion of the course achieved by the majority of participants (120/129), along with the high frequency (104.46 hours; 93% vs 70% required) and the good performance (medium–high scores obtained by 80.21% of CMHC staff members), suggest the feasibility of the CBTep training programme.

Concerning the efficacy, the similar scores reported in the ‘ongoing’ and ‘final’ exam tests in the two groups (CMHC staff members vs CBT expert psychologists) support the efficacy of the course in terms of development of theoretical knowledge and competence. Furthermore, the sensitivity of the assessment method to test the capacity building of this brief and intensive course is proved by the difference detected between CMHC professionals and expert psychologists in the supervisor judgement, which probably reflects the effect of specific competences previously acquired by CBT expert psychologists, who already had a CBT Masters qualification or CBT-specific clinical experience. Interestingly, however, this difference is small and only concerned some evaluations such as ‘Congruency with CBT’ and ‘Therapeutic style’, which are probably more sensitive to previous specific training. Overall, these results are in line with those of Jolley et al. (2015), which proved that it is possible to train mental health staff in the field of CBTep to a high standard of competence, despite different professional backgrounds.

We found differences in the impact of other trainee characteristics (age, years from graduation, years of work, theoretical orientation and professional background) on competence acquisition only in the most specific part of the training (module 3), with a better performance of younger CMHC professionals. This observation, i.e. impact of age, years from graduation, years of work and theoretical orientation, is consistent with the reports of some previous studies (Brosan et al., 2006; Sholomskas et al., 2005) but in contrast with others (James et al., 2001; Manassis et al., 2009; McManus and Westbrook, 2010). This controversy in literature may be due to differences in methodology, settings and countries of the training, suggesting that the effects of such sociodemographic variables should be better addressed in future multi-sites studies.

Our findings, in conclusion, seem to confirm the possibility of delivering specific knowledge through a relatively short and intensive training programme to qualified mental health professionals, with no (or non-specific) previous knowledge in CBT.

Limitations and recommendations for future research

The CBTep training course presented here is the first attempt, conducted on a large scale, to build high-quality capacity in CBTep in CMHC staff working in a clinical practice setting, through a brief and intensive course. The results achieved by CMHC staff members, whose capacity building did not remarkably differ from CBT expert psychologists, seem to confirm not only the feasibility but also the efficacy of the course.

However, our results should be interpreted also in the light of some limitations. Firstly, course participants did not complete a baseline assessment, and therefore it is not possible to estimate the relative improvement in CBT specific skills from baseline, as only the number of working years in a CMHS was collected. However, if we consider that the baseline level of CBTep knowledge and skills of CBT expert psychologists were potentially higher than those of CHMC staff members, and that the final results observed in the two groups were not meaningfully different, the good performance achieved by CHMC staff members at the final examination is even more notable. Secondly, we did not perform a cost-efficacy analysis of the course, which should be a point to address in future research. Finally, as we did not use a standardized method to assess inter-rater reliability of supervisor judgements and nor did we use a blinding procedure in the assessment of the trainee by the supervisors, we cannot exclude a possible bias of their ratings.

This paper opens the way to future directions, especially in terms of implementation. After having passed the course examination, the staff members of CMHCs provided the CBTep in their local service during the 9 months of the PIANO study. The results and outcomes of the study have been reported in the paper from Ruggeri et al. (2015). Data proved the efficacy of CBTep psychotherapy in reductions in overall symptom severity, while no difference could be found for days of hospitalization (the second primary outcome). Improvements were also found for global functioning, subjective well-being and subjective burden of delusions. In particular, the patient subgroup of the experimental arm with more severe psychotic symptoms, showed a significant reduction in the subjective appraisal of delusions (emotional and cognitive components), which is a specific focus of the CBTep intervention. However, it must be said that as the trial design was based on a multi-element intervention (including – beside CBTep – also family intervention and case management), we could not disentangle the specific impact of the CBTep.

After the trial, many and different initiatives have been adopted by participating CMHCs in terms of clinical and training experiences arising from the GET-UP project. However, the implementation of the CBTep training for CMHC staff outside the trial context is still to be explored in a systematic way. As such, future investigations in this direction should centre on two main aspects. The first concerns the exploration of the possible implementation of the training as part of the routine clinical practice of services and the potential difficulties for its implementation. Barriers to the implementation of training that equips professionals with the competencies required to deliver high-specialist interventions may be fuelled by the lack of resources in terms of organizational factors (staff availability) and system factors (allocation of funding). This may be particularly true in the context of Italian generalist mental health services, which have fewer human resources for mental health care compared with other high-income countries, especially in terms of clinical psychologists working in the public mental health sector (Barbui et al., 2018). The second aspect concerns the impact of CBTep training course on long-term outcomes, including the maintenance of the acquired skills over time and the correct use of CBTep in clinical practice outside the trial.