Evaluation of teaching an integrated case formulation approach on the quality of case formulations: randomised controlled trial

Abstract

Aims and method

This paper introduces a new approach to producing case formulations, the integrated case formulation (ICF) approach, and a rating scale, the Case Formulation Scale (CFS). Twenty-four psychiatrists participated in a single-blind randomised controlled trial to evaluate the effectiveness of teaching using this new approach. The intervention group (n = 12) received teaching about using ICF, whereas the control group (n = 12) received teaching as usual. The scores on the CFS were compared before and after teaching.

Results

Prior to teaching, psychiatric trainees had poor scores on the CFS. The CFS score for the group taught the ICF approach improved five times more than the control group.

Clinical implications

This study demonstrates that case formulation skills can be taught. With further improvements, the ICF approach might be useful in the teaching of case formulation.

A case formulation summarises and integrates the important information regarding a patient and their problems. It provides a shared understanding of these problems between patient and professional, which can improve the therapeutic relationship and help identify useful interventions and potential difficulties. ¹ The training curriculum of the Royal College of Psychiatrists in the UK describes the ability to construct formulations as a competency to be achieved in core training. ²

Studies that have evaluated psychiatric case formulations have demonstrated poor standards in both clinical and academic settings. A US study rated case formulations in residents’ portfolios as less than competent on average. ³ Another US study showed that formulations contained mainly descriptive information, with little integration or inference of causes. ⁴ We are not aware of any published UK studies on this subject; however, in an unpublished British study, M.A. examined 150 new assessment letters, of which only 16% included any formulation, showing that case formulation is rarely attempted in routine psychiatric practice.

There is evidence that teaching case formulation leads to improvement. Mental health professionals who received 2 h of training produced better formulations in clinical practice, ⁵ and medical students taught mechanistic case diagramming reported being more comfortable in writing a formulation. ⁶ An increased emphasis on case formulation within a psychiatric training scheme led to improvement in portfolio case formulation entries. ³

There are many different approaches to case formulation, with most psychotherapies having formulations based on their models. ^7-9 The most common generic approach is a biopsychosocial one, although this term has been used in a number of different ways. As described by Engel, it is a systemic focusing on the current time frame. ^10,11 Currently, the term usually refers to identifying predisposing, precipitating and perpetuating factors within biological, psychological and social domains. It is often represented as a 3 × 3 grid, which has been criticised for encouraging the listing of factors, rather than integrating them. ¹²

We were able to identify only two tools for assessing psychiatric case formulations, the case formulation content coding manual ⁴ and the biopsychosocial formulation scoring rubric. ³ The case formulation content coding manual is a theoretically neutral research tool, ⁴ which seems to be very detailed, time consuming and does not produce an easily interpretable score. The biopsychosocial formulation scoring rubric rates formulations on a scale from one to six, each with a descriptor, ³ giving potentially broad, subjective evaluations of formulations, with little information on where improvement is needed.

Integrated case formulation approach

The integrated case formulation (ICF) approach was designed by M.A. and R.W. and is based on the standard psychiatric history. The aim is to provide a simple structure that can be used by professionals to integrate information and build a proposed understanding of a patient's problems, without having a detailed understanding of a specific theoretical model (see a fictitious example in Box 1).

This approach starts with demographic details that provide a general picture of the patient (Fig. 1). This is followed by a description of biological factors, which include genetic factors (family history of mental illness and maladaptive personality traits) and brain development (events during pregnancy, birth, developmental milestones and intellectual strengths and difficulties).

Fig 1 Diagrammatic representation of the integrated case formulation approach.

Next, there is a description of environmental factors (early life experiences and significant events). These include the reliability of care provided by others, significant losses, abusive, bullying or traumatic experiences as a child or adult, and beliefs and expectations held within the family. Positive experiences, which contribute to resilience, can also be identified.

In order to give a proposed explanation of someone's difficulties, all the above are then linked to the personality traits, vulnerabilities and strengths. A description of someone's personality may include descriptive words (such as shy), beliefs held by the individual (such as ‘I am a bad person’) and long-standing patterns of behaviour (for example tends to avoid difficult situations). There are also likely to be helpful resources within an individual's personality, which may prevent or help overcome mental health difficulties.

A link is then made between personality and functioning (relationships, occupation and interests). A full psychiatric history is then described, with a level of detail that would be useful for guiding treatment. This should cover the onset, severity and pattern of illness, the effectiveness of different treatments and support and the person's pattern of interaction with services. It should broadly cover the symptoms in the current episode of illness, grouping them rather than listing them all individually, and mention the duration of any current illness. In order to explain the development of illness, ICF links precipitating factors, personality and functioning to the psychiatric history. The impact of psychiatric illness on functioning is also described.

Box 1 Example case formulation

MrA is a 37-year-old, single, White British man with no children. He works as a website designer and lives alone in a house that he owns. His mother has generalised anxiety disorder, suggesting a possible vulnerability to anxiety in MrA. Pregnancy, birth and development were normal, with no evidenceofany perinatal complications. MrA's early life experiences were characterised by a critical and dominant father who used to verbally abuse him and he was bullied at school. These experiences seem to have contributed to long-standing low self-esteem, a belief that he is not good enough and a persistent fear of negative evaluation and rejection by others, resulting in a tendency to avoid others and an anxious personality trait. These personality traits have affected his functioning and could explain why he has only a few friends and has been unable to maintain a long-term relationship. However, one of his strengths is that he enjoys his work and has maintained his current job for the past12 years.

MrA has apsychiatric history ofan18-month depressive episode at the age of 25, precipitated by the end ofa short-termrelationship. He sought help fromhis generalpractitioner, who prescribed the antidepressant fluoxetine and didnot refer him to secondary care mentalhealth services. His current episode started about 7 months ago, precipitated by criticism from his new boss, which may have increased his underlying fears of rejection and not being good enough. Currently he describes affective, biological and cognitive symptoms of depression and recent suicidal thoughts, with no current suicide plan. He has not responded to adequate trials of fluoxetine or mirtazapine. His depression seems to be maintained by increased social isolation since being off work sick, and ruminating about the likely problems when he returns to work and his perception that he has failed in life. His mother is supportive and encourages him to maintain some activity, providing some resilience. His presentation is suggestive of a recurrent depressive disorder, with the current episode being moderate in severity. His depression has affected his ability to maintain work, resulting in him being off sick for the previous 4 months.

This is followed by a diagnosis or provisional diagnosis using ICD-10 ¹³ or DSM-IV, ¹⁴ to ensure that the formulation and diagnosis are compatible.

The ICF approach also identifies possible maintaining factors, including unhelpful behaviours (such as drinking excess alcohol), unhelpful thinking (such as rumination about upsetting events), the person's social situation and interactions with others (such as social instability), physical health and the person's help-seeking behaviour (such as not attending appointments). In addition, there may be positive factors in the current situation helping the person to recover.

The aims of the study were to (a) test the effectiveness of teaching the ICF approach on the ability of trainees to produce a case formulation and (b) to develop a rating scale to evaluate this.

Method

The study was approved and registered by the research and development department of Leicestershire Partnership NHS Trust, who stated that research ethics committee approval was not required. Participants were recruited from the East Midlands (South) specialty training programme in psychiatry by email invitation. Twenty-three psychiatric trainees and one consultant who had just finished training participated in the study.

Participants were stratified into core trainees and higher trainees/consultants. Participants within each stratum were allocated a number. An online computerised block randomisation programme (www.randomization.com) was used to randomly allocate the numbers within each stratum to two groups: the intervention and the control groups. The intervention group received a 45 min teaching session on ICF and the control group received a 45 min teaching session, ‘teaching as usual’. The control teaching session used slides and content from the current teaching provided on the Sheffield MRCPsych course, as formulation was not covered at all in the Leicester course. This teaching included discussion of assessment, the use of rating scales, diagnostic systems and classification and formulation using a biopsychosocial grid, including predisposing, precipitating and maintaining factors in the biological, social and psychological domains. The teaching was delivered by M.A. and R.W., who have both taught on the MRCPsych courses. Participants were given a written hypothetical psychiatric case history and asked to write a case formulation before and after the teaching session.

Tools and data collection

We collected data on our participants using the following methods and tools.

1. a Self-completed questionnaire on demographic and training information of participants.

2. b Self-completed questionnaire on knowledge, skills and attitudes regarding case formulation before and after teaching and on the teaching itself using a five-point Likert scale.

3. c The Case Formulation Scale (CFS; details available from the authors on request). This is an 18-item clinician-administered scale developed by the researchers (M.A. and R.W.). Twelve items are related to key factual information (content) in the case formulation and six related to the integration of this information. Items are scored either 0, 1 or 2, where 0 means not covered or extremely unclear or not plausible/relevant and 2 means fully covered and clear and plausible/relevant. Two independent raters (A.J. and M.C.) used this scale to rate all case formulations. The raters were higher trainees in their final year of training, who were not involved in the design of ICF or the study and received 3 h of teaching delivered by the researchers on administering the scale. They were masked to whether case formulations were written pre- or post-teaching or from the intervention or control group. Where there was a two-point difference between the scores of the two raters for an individual item, this was discussed and a consensus score given and all other score differences were averaged.

The existing available tools ^3,4 had many limitations, as described earlier, and therefore we did not feel it was appropriate to use these to validate the CFS against. The primary outcome for this study was the change in score on the CFS (post-teaching score minus pre-teaching score).

Analysis

All data were analysed using SPSS version 13.0 for Windows. Chi-squared and Fischer exact tests were used for categorical measures and independent sample t-test for continuous measures, which were found to be normally distributed using the Shapiro-Wilks test. For clarity and ease of reading, Likert scale results are presented in frequencies; however, the P-values were calculated using the Mann-Whitney U-test and Wilcoxon's signed ranks test on the raw ordinal data.

General linear regression analysis was used to adjust the results for the influence of significant potential confounding factors. The outcome variable was the change in the CFS score. Total hours of training in case formulation was not entered in the final analysis because of the missing data, which reduced the number of participants. However, a univariate regression analysis showed that there was no statistically significant relationship between the outcome and this variable.

Reliability of the CFS was calculated using Cronbach's alpha and interrater reliability was calculated using a intraclass correlation coefficient. The level of significance was set at P<0.05. The statistical analysis was carried out masked to group membership.

Results

Twenty-four individuals participated in the study. The intervention and control group were similar at baseline, with the only significant difference being that the intervention group were on average older (Tables 1 and 2).

Table 1 Characteristics of the participants

	Intervention group (n = 12)	Control group (n = 12)	P
Age, years: mean (s.d.)	32.8 (4.0)	29.4 (2.8)	0.03
Female, n (%)	9 (75)	7 (58)	0.4
Months in psychiatry, mean (s.d)	41.5 (39.0)	38 (28.3)	0.8
Grade, n (%)			0.5
ST1–3	9 (75)	9 (75)
ST4–6	2 (17)	3 (25)
Consultant	1 (8)	0 (0)
MRCPsych exams passed, n (%)			0.3
None	5 (42)	4 (33)
Part of MRCPsych	6 (50)	5 (42)
Completed MRCPsych	1 (8)	3 (25)
Case formulation training, n (%)			0.8
None	2 (17)	1 (8)
Lectures/tutorials	2 (17)	1 (8)
Informal teaching	5 (42)	5 (42)
Both	3 (25)	5 (42)
Hours of case formulation training, mean (s.d.)	1.2 (1.2)	1.5 (1.0)	0.5
Psychotherapy training, n (%)			0.8
None	1 (8)	0 (0)
Case discussion group	2 (17)	2 (17)
Supervised therapy cases	0 (0)	1 (8)
Lectures + case discussion	1 (8)	1 (8)
Lectures + therapy cases	0 (0)	1 (8)
Case discussion + therapy cases	2 (17)	1 (8)
All three	6 (50)	6 (50)
Total hours psychotherapy training: mean (s.d.)	60.6 (41.8)	57.3 (39.5)	0.9

ST, specialty trainee. Results in bold are significant.

Table 2 Knowledge, skills, attitude and practice regarding case formulation (CF) and teaching sessions

	Intervention group, n (%) (n = 12)			Control group, n (%) (n = 12)
	Strongly agree/ agree	Neither agree nor disagree	Strongly disagree/ disagree	Strongly agree/ agree	Neither agree nor disagree	Strongly disagree/ disagree	P
I write a CF after assessing new patients
Pre-teaching	4 (33)	4 (33)	4 (33)	6 (50)	4 (33)	2 (17)	0.4
I know what a CF is
Pre-teaching	8 (67)	2 (17)	2 (17)	11 (92)	1 (8)	0 (0)	0.2
Post-teaching	12 (100)	0 (0)	0 (0)	10 (83)	1 (8)	1 (8)	0.4
I am able to write a CF a
Pre-teaching	4 (33)	4 (33)	4 (33)	9 (75)	3 (25)	0 (0)	0.05
Post-teaching	11 (92)	1 (8)	0 (0)	9 (75)	3 (25)	0 (0)	0.7
CF is an important skill for psychiatrists
Pre-teaching	10 (83)	2 (17)	0 (0)	12 (100)	0 (0)	0 (0)	0.6
Post-teaching	12 (100)	0 (0)	0 (0)	12 (100)	0 (0)	0 (0)	1.0
CF leads to better care of patients
Pre-teaching	9 (75)	3 (25)	0 (0)	10 (83)	2 (17)	0 (0)	0.8
Post-teaching	12 (100)	0 (0)	0 (0)	11 (92)	1 (8)	0 (0)	0.6
	Not routinely	After initial assessment	After every contact	Not routinely	After initial assessment	After every contact
When to write a CF
Pre-teaching	2 (17)	7 (58)	3 (25)	1 (8)	9 (75)	2 (16)	0.7
Post-teaching	0 (0)	7 (58)	5 (42)	1 (8)	6 (50)	5 (42)	0.6
	Excellent good	Alright	Very poor/ poor	Excellent/ good	Alright	Very poor/ poor
How did you rate this teaching session?
Post-teaching	12 (100)	0 (0)	0 (0)	4 (33)	6 (50)	2 (17)	<0.001
How did you rate this formulation method?
Post-teaching	12 (100)	0 (0)	0 (0)	5 (42)	2 (17)	5 (42)	0.001
	Significantly/ a lot more	Somewhat	Not at all/ not much	Significantly/ a lot more	Somewhat	Not at all/ not much
Do you feel more confident in writing CFs?
Post-teaching	7 (58)	4 (33)	1 (8)	8 (67)	3 (25)	1 (8)	0.001
	Definitely/ probably	Possibly	Probably not	Definitely/ probably	Possibly	Probably not
Do you intend to write CFs more?
Post-teaching	11 (92)	1 (8)	0 (0)	8 (67)	3 (25)	1 (8)	0.1

Results in bold are significant.

a. Significant change in score pre- and post-teaching in the intervention group, P = 0.02.

There was no significant difference between the two groups’ case formulation scores prior to teaching (Table 3). The mean score on the CFS prior to teaching was 46%, with a mean score of 54% for content and a mean score of 28% for integration. On average the intervention groups’ formulations improved 8.5 points more than the control group (P<0.01) and this difference remained highly significant after adjusting for possible confounding variables (Table 3).

Table 3 Case formulation scores

	Intervention group, mean (95% CI) (n = 12)	Control group, mean (95% CI) (n = 12)	P
Pre-teaching score	16.4 (13.9 to 18.9)	16.4 (14.7 to 17.9)	1.0
Post-teaching score	25.7 (23.2 to 28.2)	17.2 (15.3 to 18.9)	<0.001
Change in score, post-teaching – pre-teaching	9.0 (6.8 to 11.2)	0.5 (–1.5 to 2.5)	<0.001
Adjusted change in score a	9.1 (4.7 to 13.5)	–1.2 (75.6 to 3.1)	0.02

a. Using general linear regression, adjusted for age, gender, grade, duration of training in psychiatry and duration of training in psychotherapy. Adjusted R ²=0.707.

The intervention group felt significantly more confident in writing case formulations following teaching (P = 0.001) and rated the teaching (P<0.001) and the method of case formulation (P = 0.001) significantly higher (Table 2). Teaching had no significant effect on participants’ beliefs regarding the importance of case formulation, its effect on clinical care or when it should be used (Table 2). Cronbach's alpha for the CFS was 0.83 and the interrater reliability was 0.81 (95% CI 0.78-0.83).

Discussion

This is the first single-blind randomised controlled trial that has investigated the effect of teaching a new approach to producing case formulations on trainees’ ability to write them. It proposes a new approach to formulation and developed a new tool for assessing them that could be used in clinical practice, training and future research. It demonstrates that teaching this approach improved trainees’ ability to write a formulation five times more than usual teaching, controlling for possible confounding factors. Those who had been taught the new approach felt significantly more confident in writing a formulation. This study shows that the ability of the participants to write formulations is poor, scoring less than 50% prior to teaching and particularly poor on integration elements, confirming previous research findings. ⁴ The grade of the trainee had no effect on their score.

Our approach emphases integration, providing a structure for psychiatrists to make links between elements within the formulation. It does not require an understanding of new concepts, although it is able to accommodate concepts from different theories and therapies. It is not specific to one diagnosis. It can be easily understood and taught in one session. It can be used as an effective way to summarise and communicate cases and to guide treatment. In common with many formulation approaches, it does not address prognosis or risk, which could be seen as a limitation.

We believe that the main reason for poor case formulation skills is the absence or poor quality of teaching of a good structured approach. Within current British psychiatric training, formulation tends to be taught using the biopsychosocial grid, if it is taught at all. This method identifies different aetiological factors but does not link these in a coherent way to provide a good understanding, which is essential in case formulation. In addition, most psychiatric textbooks give little information on this subject. Assessment of psychiatric trainees does not focus on formulation, either within workplace-based assessments or exams. This is something that might have been assessed in the long-case exam in the past. ¹⁵

There were only two tools for assessing case formulation and the CFS is a potentially useful addition to these. It has clearly defined items covering factual content and integration, with good psychometric properties. Raters can be easily trained to use this tool. If used within an exam, the item definitions could be operationalised for a specific case history to increase reliability. It has many advantages over the existing tools, however the lack of an established tool against which the CFS could be validated is a limitation.

A potential limitation of this study is the small number of participants; however, we did find a significant difference between groups. The control group intervention was based on the teaching of case formulation in Sheffield and Leicester. It is possible that teaching elsewhere focuses more on case formulation and is more effective in developing formulation skills. This study assessed participants’ formulation skills following teaching, but did not assess the effect of this teaching session on clinical practice. There is no consensus on the elements that should be included in a perfect case formulation, making it difficult to establish the content validity of the CFS and given the problems with existing tools, we were not able to establish concurrent validity. Another possible limitation of the CFS is that all factors have been given equal weight for simplicity, however some may be more important than others. The teaching as usual in the control group resulted in no significant change in learning as measured by the CFS. This is possibly because this teaching added very little to what they already received in their MRCPsych course or that teaching of case formulation in the usual way, as it is in the MRCPsych course, does not help trainees to apply their knowledge to formulations. We believe that any change would have been picked up by the CFS. It is possible that other teaching of the usual approach could lead to a better performance by the control group. Another limitation is that participants were not masked to study group and this may have affected their enthusiasm and score.

This study has proposed a new approach for producing a case formulation and a new assessment tool that could fill a gap in psychiatric training and potentially could have an impact on the quality of patient care. The CFS could be developed and used as a workplace-based assessment tool and also in future research.