Supported employment: randomised controlled trial

Abstract

Background

There is evidence from North American trials that supported employment using the individual placement and support (IPS) model is effective in helping individuals with severe mental illness gain competitive employment. There have been few trials in other parts of the world.

Aims

To investigate the effectiveness and cost-effectiveness of IPS in the UK.

Method

Individuals with severe mental illness in South London were randomised to IPS or local traditional vocational services (treatment as usual) (ISRCTN96677673).

Results

Two hundred and nineteen participants were randomised, and 90% assessed 1 year later. There were no significant differences between the treatment as usual and intervention groups in obtaining competitive employment (13% in the intervention group and 7% in controls; risk ratio 1.35, 95% CI 0.95–1.93, P = 0.15), nor in secondary outcomes.

Conclusions

There was no evidence that IPS was of significant benefit in achieving competitive employment for individuals in South London at 1-year follow-up, which may reflect suboptimal implementation. Implementation of IPS can be challenging in the UK context where IPS is not structurally integrated with mental health services, and economic disincentives may lead to lower levels of motivation in individuals with severe mental illness and psychiatric professionals.

Work represents an important goal for many people with severe mental illnesses, ¹ but in many economically developed countries rates of unemployment among people with severe mental illness often exceed 90%. ^2,3 However, several randomised controlled trials (RCTs), predominantly in North America, have found that supported employment using the individual placement and support (IPS) model can increase rates of competitive employment to 30–60%. ^4–6 It is less clear whether the IPS model is effective in populations with high rates of unemployment and relatively generous state welfare benefits for unemployed people with severe mental illness. Only one European study has been published to date investigating the IPS model in countries with diverse labour markets and different welfare systems ⁷ and in this study the IPS model was not effective in all countries included. We therefore aimed to assess the effectiveness and cost-effectiveness of the IPS model in the UK. Our primary hypothesis was that at 1-year follow-up a significantly greater percentage of individuals who received IPS would be in competitive employment compared with those receiving usual services. Secondary hypotheses were that participants allocated to IPS would be: working significantly more hours each week; have significantly longer job tenure; earn a significantly higher net income; have significantly higher job satisfaction; make significantly greater (net) contributions to the national economy; have a significantly higher quality of life in the domains of work, finance, leisure and social life; have more of their needs met; and have significantly improved self-esteem.

Method

Study design and sample characteristics

The SWAN (Supported Work and Needs) study is a pragmatic RCT of the IPS model. Participants were recruited from community mental health teams in two boroughs of South London (ISRCTN96677673). Borough A has an average index of deprivation score of 21.31 (a rank average score of 125 out of 354 for the UK), a 6.3% unemployment rate (compared with 5.3% nationally) and incapacity benefit (a benefit received by people unable to work because of ill health) is received by 5.6% of working-age residents (compared with 7.2% nationally). Borough B has an average index of deprivation score of 34.95 (a rank average score of 19 out of 354, i.e. a high level of deprivation), a 9.5% unemployment rate and incapacity benefit received by 7%.

Inclusion criteria were that participants should be receiving out-patient or community psychiatric care from local mental health services, have severe mental illness (duration of illness over 2 years, global asssessment of functioning (GAF) ⁸ score of 60 or less, and a diagnosis of a psychotic or chronic affective disorder), aged 18–65, able to read and speak English to a high enough standard to give informed written consent, to have been unemployed for at least 3 months and want to obtain competitive employment. Participants were excluded if they had been referred for IPS in the previous 6 months. Mental health professionals were asked to identify potential participants and ask them if they agreed to being approached by a researcher. After participants were given a complete description of the study by the researcher, written informed consent was obtained. Ethical approval was obtained from the Joint Institute of Psychiatry/South London and Maudsley Research Ethics Committee (reference no 319/03).

Participants were randomly allocated to the intervention group or the control group. Treatment allocation was stratified by gender and age (10-year bands). Randomisation with minimisation was used, performed by the Institute of Psychiatry Mental Health and Neuroscience Clinical Trials Unit, a unit independent of the study to maintain concealment. The intervention condition was an IPS programme integrated within community mental health teams provided by a well-established not-for-profit non-governmental supported employment agency. The intervention involved linking four experienced employment specialists (two for each borough) with community mental health teams. In addition to seeing clients, the employment specialists attended team meetings and multidisciplinary care planning meetings, and met with care coordinators whenever appropriate. They focused on rapid placement with continued follow-up support and sought to find employment opportunities that were consistent with participants' preferences, skills and abilities. However, the integration of the IPS programme was not structural or managerial, reflecting supported employment as it is provided in the vast majority of settings in the UK, which is the current provision referred to by the UK government in its implementation of IPS. ⁹

The control condition (treatment as usual, TAU) consisted of existing psychosocial rehabilitation and day care programmes available in the local area. There were 33 such services in the catchment areas of the RCT; these were visited or telephoned and a senior manager was asked about their operational characteristics. Each service employed, on average, 7 vocational staff, had a median of 58 places available to clients (range 6–3000) and 79% of services received referrals from clinical teams. A range of courses were offered, most commonly pre-employment preparation (e.g. interview skills, curriculum vitae coaching and application form practise), computers/information technology and confidence building/motivation.

The participating community mental health teams were given presentations on the study with opportunities for questions and answers and written information on the study, and employment consultants working with the community mental health teams would regularly explain the study when they attended team meetings and when addressing individual questions and concerns. Teams were also regularly sent newsletters with updates on study progress and examples of success in obtaining employment. Written information on the study was revised when the study team were aware of specific issues that needed to be addressed (e.g. concerns about equipoise or the referral process).

Procedure

Participants were assessed at baseline by the recruiting researcher and at 12 months after randomisation by a separate researcher who was masked to allocation status. Each participant was given £20.00 for their time at the baseline and 1-year follow-up interview. Participants were not reimbursed for any contacts they had with the employment consultants.

Measures

At baseline, participants were assessed using the Structured Clinical Assessment in Neuropsychiatry (SCAN) ¹⁰ to determine their diagnosis. Participants with a SCAN diagnosis of schizophrenia, schizoaffective disorder, delusional disorder or other psychotic disorders were categorised as having a psychotic disorder. Participants with a SCAN diagnosis of bipolar disorder, mania or depression were categorised as having a mood disorder.

At the baseline and 1-year follow-up interviews, data were collected on demographic information, service use and employment status over the previous 12 months. Measures of psychosocial functioning were:

1. (a) Manchester Short Assessment (MANSA) version 2, ¹¹ a brief modified version of the Lancashire Quality of Life Profile;

2. (b) Camberwell Assessment of Need (CAN) short version, ¹² a tool for assessing the unmet needs of people with severe mental illness;

3. (c) Rosenberg Self-Esteem Scale; ¹³

4. (d) Brief Psychiatric Rating Scale (BPRS); ¹⁴

5. (e) Global Assessment of Functioning (GAF); ⁸

6. (f) Client Service Receipt Inventory (CSRI), ¹⁵ a measure of health and social resource use during the previous 12 months.

At 1-year follow-up the primary outcome measure was competitive employment in the previous 12 months. Open competitive employment was defined as a job paying at least the minimum wage, located in a mainstream socially integrated setting not set aside for persons with disabilities, held independently (i.e. was not agency owned) and the participant was in continuous employment for at least 30 days (with part-time employment rated pro-rata). Information on this and the other employment outcomes was collected by employment consultants whenever they saw their clients or, for clients who were not seeing employment consultants regularly (i.e. those in the control arm and those not engaged with the IPS programme), by employment consultants at 4-monthly intervals through the study (either face-to-face or by telephone at the person's convenience). Where this was not possible (e.g. if the client was on holiday or unobtainable), data were obtained from clinical staff if possible. Data were then compared with data collected by the researcher and any discrepancies were investigated by asking clients for clarification, or, if they were not available, checking with other sources of information. Job satisfaction was assessed using the Indiana Job Satisfaction Scale. ¹⁶

Sample size

We calculated that a sample size of 75 in each group (150 in total) would be sufficient to detect a difference in employment from 10% in the TAU group compared with 30% in the experimental group, assuming a significance level P = 0.05 (double sided) at 80% power. Allowing for a 30% attrition rate, our target recruitment was 108 (216 in total) in each group. The base rate of 10% and the likely refusal and attrition rates were based on a study in Baltimore ¹⁷ and as discussed above are typical of many populations of people in the UK with schizophrenia.

Masking

Due to the nature of the study, it was not possible for participants or those administering the intervention to be masked to the participants' allocation status. However, the researcher conducting the follow-up assessment was not told the participants' allocation status and asked the participant at the beginning of follow-up interviews not to disclose the allocation if possible. A test of the success of masking was carried out by comparing the researcher's guessed allocation with the actual allocation.

Process data

The IPS fidelity scale ¹⁸ was completed by a lead occupational therapist independent from the study who interviewed service users in the intervention arm, the IPS workers and care coordinators in community mental health teams, and assessed the structure of the services. This scale rates good IPS implementation if a score of 66–75 is obtained, fair IPS implementation for scores 56–65, and a failure to implement IPS if a score of 55 or below is obtained (maximum rating is 75). During the course of the study, employment workers were required to record prospectively any events that occurred in relation to the client. These events included direct client contact and any non-client contacts (of more than 10 min) that were to do with the client, e.g. contact with the client's community psychiatric nurse. Employment workers were required to record the amount of time spent per contact, where and when the contact took place and the nature of the contact.

Statistical and health economic analysis

All data were analysed using SPSS for Windows (version 15.0). The primary and secondary hypotheses were tested on the whole group. For comparing groups, t-tests and χ²-tests were used to compare means and proportions respectively, unless the data were highly skewed, in which case non-parametric tests were used. Logistic and linear regression models were also fitted including potential confounding variables: (grouped) age, gender, ethnic group, educational level, symptomatology and diagnosis. All data were analysed in groups as randomised, whether or not receiving an intervention (i.e. intention-to-treat). Data were compared for those followed up with those not followed up, overall and by each treatment arm.

The amount of time spent by employment workers with clients was recorded and this was multiplied by their hourly cost. This was based on annual costs per employment worker of £40 000, a working week of 35 h and 42 weeks per year, and a ratio of direct to indirect client time of 0.67 (this ratio was not directly available from the employment services and therefore was based on the ratio of occupational therapist direct to indirect client time ¹⁹ and used to calculate the cost of the IPS intervention). The resulting cost was £45 per hour. The costs of other services were calculated by combining service data with appropriate unit cost information. ¹⁹ Total costs at follow-up were compared between the two groups using a regression model with the group indicator as the single independent variable. Confidence intervals around the cost difference were estimated using bootstrapping with 10 000 resamples due to the likely skewed cost distribution.

Results

Participants were recruited between November 2004 and September 2006. Of the 375 people referred to the study, 220 entered into the study (Fig. 1). One participant withdrew between the baseline assessments and randomisation leaving 219 participants. There were no substantial differences between the two randomised arms on any baseline variables (Table 1). Twenty-two participants were lost to follow-up at 1 year. There were no significant baseline differences at P<0.05 in sociodemographic or clinical variables between those who were and those who were not lost to follow-up (Table 2), although there was some weak evidence that the BPRS and GAF were different (at P<0.1), with those followed up being slightly less symptomatic and higher functioning.

Fig. 1. CONSORT flow diagram.

Table 1 Baseline sociodemographic and clinical variables of participants by randomisation group

	Control		Intervention
		IQR		IQR
Males, n (%)	72 (66)		75 (69)
Borough A, n (%)	47 (43)		38 (35)
Borough B, n (%)	63 (57)		71 (65)
Worked in the past 5 years, n (%)	61 (56)		56 (51)
Living alone, n (%)	58 (53)		60 (55)
Ethnicity, n (%)
White	45 (41)		37 (34)
Black	45 (41)		49 (45)
Other	19 (17)		22 (20)
Diagnosis, n (%)
Psychotic disorder	72 (69)		76 (76)
Mood disorder	32 (31)		24 (24)
Age, years: mean (s.d.)	38.3 (9.3)		38.4 (9.5)
Camberwell Assessment of Need unmet needs, median (range)	2 (11)	0-3	1 (8)	0-3
Rosenberg Self-Esteem Questionnaire, mean (s.d.)	2.29 (0.55)		2.24 (0.56)
Overall Manchester Short Assessment score, mean (s.d.)	3.86 (0.71)		3.98 (0.75)
Brief Psychiatric Rating Scale, median (range)	33 (41)	28-41	33 (32)	28-42
Global Assessment of Functioning score, mean (s.d.)	47.66 (6.26)		48.31 (7.00)

Table 2 Sociodemographic and clinical variables comparing those participants who were and were not lost to follow-up at 1 year^a

	Follow-up		Lost to follow-up		Statistical test	P
		IQR		IQR
Group, n (%)					χ2(1) = 0.001	0.98
Intervention	98 (50)		11 (50)
Control	99 (50)		11 (50)
Males, n (%)	134 (68)		13 (59)		χ2(1) = 0.715	0.40
Borough, n (%)					χ2(1) = 0.045	0.83
A	76 (39)		9 (41)
B	121 (61)		19 (59)
Worked in the past 5 years, n (%)	104 (53)		13 (62)		χ2(1) = 0.561	0.45
Living alone, n (%)	105 (53)		13 (59)		χ2(1) = 0.267	0.61
Ethnicity, n (%)						0.23b
White	75 (38)		7 (33)
Black	87 (44)		7 (33)
Other	34 (17)		7 (33)
Diagnosis, n (%)					χ2(1) = 0.271	0.78
Psychotic disorder	134 (72)		14 (78)
Mood disorder	52 (28)		4 (22)
Age, years: mean (s.d.)	38.60 (9.32)		35.91 (9.96)		t(217) = 1.274	0.20
Camberwell Assessment of Need unmet needs, median (range)	1.52 (0-11)	0-3	1 (0-8)	0-3	Z = -0.484c	0.63
Rosenberg Self-Esteem Questionnaire, mean (s.d.)	2.26 (0.56)		2.31 (0.52)		t(214) = 0.425	0.67
Overall Manchester Short Assessment score, mean (s.d.)	3.93 (0.72)		3.79 (0.80)		t(217) = 0.855	0.39
Brief Psychiatric Rating Scale, median (range)	38 (24-65)	40-43	33 (24-63)	28-42	Z = -1.765c	0.09
Global Assessment of Functioning score, mean (s.d.)	48.24 (6.49)		45.73 (7.58)		t(216) = -1.693	0.09

Process data

Individual placement and support fidelity was found to be high; the IPS programme in the two boroughs received a good IPS rating of 67 (Borough A) and 69 (Borough B); the IPS programme scored less well on the organisation dimension (specifically, integration of rehabilitation with mental health treatment) and, for Borough A, the services dimension (specifically, for the rapid search for a competitive job) compared with other dimensions. Of the 109 patients in the intervention arm, 73 (67%) engaged with staff (i.e. had at least one direct contact with an employment consultant); of these individuals, the mean number of contacts with or on behalf of clients was 14 (s.d. = 10).

The researchers who conducted the 1-year follow-up interviews were masked to allocation status, but guessed 119 correctly out of the 197 (60%) clients assessed compared with a hypothesised 50% (with random guesses); this is significant at P = 0.005 (single sample, double-sided test of a proportion).

One-year follow-up

One protocol deviation was noted in that one participant randomised to TAU received the intervention. Out of 219 individuals recruited, 20 (9%) reported having worked between baseline and the 1-year follow-up, in jobs that met the competitive employment criteria. Of those followed up, 7 out of 99 (7%) were from the control group and 13 out of 98 (13%) were from the intervention group (risk ratio 1.35, 95% CI 0.95–1.93, χ²(1) = 2.07, P = 0.15). Controlling for variables associated with loss to follow-up (BPRS and GAF) made no difference to the conclusions (P = 0.127). Sensitivity analyses were conducted assuming the 22 participants who were lost to follow-up had either all worked or all not worked, but this made little difference (P = 0.288 and P = 0.153 respectively). If the primary outcome had been alternatively defined (as in some other studies ⁷ – working at least 1 day) there would still have been no significant difference in obtaining competitive employment: 22/197 (11%) of participants worked in a job for at least 1 day, 8/99 (8%) from the control group and 14/98 (14%) from the intervention group (χ²(1) = 1.911, P = 0.167). When sensitivity analyses were conducted assuming the 22 participants who were lost to follow-up had either all worked or all not worked, this made little difference (P = 0.296 and P = 0.170 respectively).

The median number of hours worked per week was not significantly different in the control group (32.5 h, interquartile range (IQR) 4–45, range 3–63 for 7 jobs obtained) and the intervention group (8 h, IQR 4–16, range 3–50 for 15 jobs obtained) (Mann–Whitney Z = –1.45, P = 0.15). Similarly, the length of job tenure (median 13 weeks, IQR 8–15, range 4–26 for control group; 18 weeks, IQR 7–30, range 4–52 for intervention group; Mann–Whitney Z = –1.20, P = 0.23) and the median salary (£7.73, IQR 6.15–14.18, range 6–16 for control group; £6.00, IQR 5.35–7.20, range 4.85–30 for intervention group; Mann–Whitney Z = 1.48, P = 0.14) were not significantly different in the two groups. Out of the 20 participants who had worked in competitive employment in the 1-year follow-up, 2 had worked in two different competitive employment roles. The hypothesis that the intervention would increase net contributions to the national economy is closely linked to the hypotheses regarding net earnings and hours worked. Higher earnings would potentially lead to increased tax revenues. However, because there was no significant difference in earnings there is also no significant net gain to the economy.

Within the intervention arm, there was a significant difference in employment outcome by borough: within Borough A, 1 participant (3%) obtained competitive employment, whereas within Borough B, 12 participants (17%) obtained competitive employment (Fisher's exact test P = 0.03). At 1-year follow-up there were no adverse events for any clients.

Forty-five per cent (10 of 22) of the jobs gained at 1-year follow-up were elementary occupations (e.g. bar work, kitchen porter); 23% (5 of 22) of jobs were sales and customer service occupations; 14% (3 of 22) were administrative and secretarial occupations, 9% (2 of 22) were professional occupations. Other jobs obtained were skilled trade's occupation (n = 1), and process, plant and machine operative (n = 1).

Table 3 shows there were no significant differences in secondary outcomes when comparing outcomes by randomisation group. We also found no significant differences in outcomes for participants who had obtained work and those who had not.

Table 3 Clinical outcome for participants by randomisation status

	Intervention		Control		Statistical test	P
		IQR		IQR
Camberwell Assessment of Need unmet needs (n = 195), median (range)	1 (0.9)	0-3	1 (0-9)	0-3.5	Z = -0.409	0.68
Rosenberg Self Esteem questionnaire (n = 196), mean (s.d.)	2.26 (0.53)		2.25 (0.55)		t(194) = -0.124	0.90
Overall Manchester Short Assessment (MANSA) (n = 195), mean (s.d.)	4.03 (0.82)		3.95 (0.87)		t(194) = -0.705	0.48
Work	3.05 (1.50)		2.85 (1.39)		t(193) = -0.988	0.32
Finance	3.29 (1.64)		3.39 (1.58)		t(194) = 0.443	0.66
Leisure	3.44 (1.56)		3.47 (1.56)		t(194) = 0.137	0.89
Social life	4.39 (1.39)		4.30 (1.40)		t(189) = -0.424	0.67
Indiana Job Satisfaction Scale 16 (n = 16),a mean (s.d.)	2.08 (0.53)		1.78 (0.45)		t(14) = -1.10	0.29
Brief Psychiatric Rating Scale (n = 195), median (range)	36 (24-80)	29-44	37 (24-58)	30-44	Z = -0.484	0.63
Global Assessment of Functioning (n = 196), mean (s.d.)	53.34 (14.75)		52.46 (13.16)		t(194) = -0.440	0.66

Regression analyses were applied to control for ethnicity, age, gender, diagnosis, symptom level and level of functioning in order to test if there were any factors that were masking a difference between the two treatment arms in any of the outcomes. However, none of the possible masking or confounding factors were found to be significant, and there were no significant interactions between these variables and treatment group.

Service use and costs

During the 1-year follow-up, two-thirds of the intervention group received input from employment workers at mean cost of £296 (Table 4). There were not substantial differences in the number of people using other services. However, control group participants who were admitted spent substantially more days in hospital than in-patients in the intervention group. This resulted in a difference in in-patient costs of £1522. Total costs were £2176 significantly higher in the control group (bootstrapped 95% CI £445–4168).

Table 4 Service use and costs at 1-year follow-up

	Control						Intervention
	Using service		Contactsa		Costb		Using service		Contacts 11		Cost 2
	n	%	Mean	s.d.	Mean	s.d.	n	%	Mean	s.d.	Mean	s.d.
Received individual placement and support	1	1	12.0	-	5	47	73	67	9.3	7.8	296	416
Psychiatric in-patient	13	13	80.4	55.2	2241	7111	10	10	33.9	22.1	719	2533
Psychiatrist	45	46	1.7	2.0	190	390	33	34	2.0	2.4	117	229
Other doctor	13	13	2.8	2.7	31	114	20	20	1.8	2.0	30	98
Day care/education	35	35	18.8	20.0	320	858	35	36	10.9	12.9	149	424
Psychologist/therapy	4	4	5.8	5.9	21	139	11	11	5.5	5.6	67	320
Social care	7	7	4.6	4.0	17	90	8	8	5.9	4.9	25	111
General in-patient	5	5	27.2	26.7	323	1893	4	4	6.3	3.8	60	331
General practitioner	52	53	2.2	1.8	55	91	48	49	3.0	4.4	58	99
District nurse	1	1	3.0	-	< 1	1	5	5	4.2	1.8	2	11
Community mental health nurse	59	60	4.6	2.9	65	99	55	56	4.1	2.7	49	77
Occupational therapist	2	2	2	2.8	1	6	3	3	4.3	2.9	4	27
Total				5701	8677					3525	3854

Discussion

Main findings

In this largest RCT of supported employment in the UK to date, there was no evidence of significant benefit of the IPS model in helping people with severe mental illness obtain competitive employment. The study context – a socioeconomically deprived area with a somewhat rigid welfare benefits system, and an intervention provided by an external agency – may have made successful implementation of the IPS model more challenging. Although in this sample the IPS model achieved higher rates of competitive employment compared with standard services, this was not significant, and it is striking that in both arms the rates of competitive employment were very low (considerably lower than the rates reported for control groups in other studies). This suggests that even with a larger sample size that may have allowed us to detect a significant difference between the intervention arm and TAU, a very large number of people would need to receive the intervention in order to achieve relatively low rates of competitive employment.

Although we did not find evidence that IPS was effective in this study, we did not find the intervention arm to be significantly more expensive than the control arm; in fact, the control group spent more days in hospital and therefore was significantly more expensive (although this is as a result of a small number of outliers and has therefore skewed the results). This may be a chance finding or may reflect the supportive nature of the intervention, which may prevent in-patient admissions.

Successful effective implementation of IPS may be context-dependent, and it may be difficult to achieve even moderate rates of competitive employment in some settings. Our results contrast markedly with previous results reported in RCTs of the IPS model and there are a number of possible explanations for our findings. First, the IPS model may be less helpful in non-USA settings – a recent review of RCTs of supported employment found that two of the three studies with the lowest employment rates were non-US studies, ^7,20,21 although some non-US studies have found IPS to be effective (e.g. Latimer et al). ²⁰ These findings may be because of the different nature of the labour market; the lack of employer incentives in employing individuals with severe mental illness outside the USA; payment for healthcare in the USA, which may increase the pressure to find work in US settings; and the differences in welfare benefit systems found in different countries – Burns et al's EQOLISE study ⁷ found that the degree of economic disincentives for working across several countries predicted the success of IPS programmes. They also reported considerable recruitment difficulties, despite carrying out the study in a relatively more affluent area of London (Richmond) that is likely to have more entry-level jobs; this suggests that some of these disincentives existed at the London site in their study. However, whereas local unemployment rates do predict vocational outcomes for people with severe mental illness, ²² there is evidence from the USA that even in areas with high unemployment, people who receive IPS have superior outcomes to those in the control condition, ²² so local unemployment rates are unlikely to be the main reason for our results.

Second, the same review ²¹ found lower employment rates in a trial that did not require clients to have a goal of competitive employment. ¹⁶ Whereas our eligibility criteria included individuals only if they wanted to obtain competitive employment we did not test participants' motivation or restrict access to the trial to people with high levels of motivation. However, the lack of engagement with employment consultants that occurred in a quarter of the participants suggests that participants may have been less motivated compared with those in other trials. Recent studies have suggested that the most robust predictors of work outcomes include recent employment history, motivation and self-efficacy. ²³ Ethnographic research suggests that people who say they would like to work can be categorised as ‘active job-seekers and passive job-seekers’, and the latter make little progress in obtaining a job. ²⁴ We may therefore have demonstrated greater effectiveness of the intervention if we had selected only participants who were ‘active job-seekers’; 25% of participants obtained employment in the UK Sesami study, where participants were self-selected and had already engaged with supported employment programmes. ²⁵ Indeed, some of the American trials have held two participant consent meetings, to screen out those who are less highly motivated to participate, whereas we held one consent meeting only. These findings suggest that IPS programmes should routinely include tests of motivation before entry into the programme.

Third, the IPS programme was not structurally or managerially integrated with community mental health services and these may be important components that determine the effectiveness of IPS. Previous research has found that the degree to which integration of IPS with mental health services occurs predicts success. ²¹ It may therefore be relevant that the borough with the higher fidelity rating (i.e. where IPS was more successfully implemented, although still without structural or managerial integration) had better employment rates than the other borough. We have not been able to find many studies that report data on the number of contacts between IPS staff and clients to compare with our other process data, but the two studies (of 11 RCTs included in a recent review) ⁴ with such data report higher numbers of contacts, which also suggests IPS implementation may not have been optimal. Other aspects of IPS implementation, which are not captured by the IPS fidelity scale, such as the nature of the relationship between the IPS worker and the client (which was an independent predictor of outcome in the EQOLISE study) ²⁶ may also explain the poorer outcomes found in this study. A recently developed modified IPS fidelity scale (IPS+) also includes rating essential processes such as consumer choice, ²⁷ and the IPS fidelity rating carried out in this study may have been lower if the IPS+ had been used.

Fourth, the relatively high proportion of participants from ethnic groups other than White (62%) could also limit the success of IPS in the UK, as people from other ethnic groups are more likely to be unemployed in the UK. ²⁸ We also used an inclusion criteria of a maximum level of functioning of 60 (as measured by the GAF scale) in order to examine the effectiveness of IPS in people with severe mental illness. However, this means that our study population may have been more impaired than those recruited in other studies. In addition, qualitative research reported elsewhere ²⁹ found that the clinicians involved in this RCT had idiosyncratic interpretations of the eligibility criteria used for the trial (e.g. some staff would not refer individuals with positive symptoms to the study team even though this was not an exclusion criterion). This may have led to motivated potential participants not being referred and entering the trial, as staff wished to avoid the ‘stress’ of job-seeking in case this led to relapse. Finally, attitudes towards employing people with severe mental illness may differ in different contexts; there is evidence that experienced and anticipated discrimination do vary across countries, ³⁰ but there are no current data on international differences in discrimination for employment.

Strengths and limitations

Strengths of this study include the largest sample size for an RCT of IPS in the UK, the comparatively diverse group of participants and high levels of retention at follow-up. The limitations include the pragmatic nature of this trial in a ‘real world’ setting in a socially deprived inner-city catchment area where most people with severe mental illness do not obtain employment, and as is typical in the UK, the IPS programme was provided by an external supported employment agency. Such well-established but financially insecure non-governmental agencies may have less IPS-focused staff than new staff employed to set up IPS programmes (as in previous RCTs) where the enthusiasm and commitment of staff is likely to be particularly high. Despite the high IPS fidelity rating, the separation of the supported employment agency from mental health services may also have led to differences in the intervention received by participants in this study compared with that delivered in more fully integrated models of care.

Other studies have not usually reported on rates of engagement with IPS so it is difficult to compare data but the failure to engage all participants may reflect an individual's lack of motivation, severity of illness or staff attitudes. Mental health professionals involved had low expectations of their clients' ability to obtain competitive employment, as reported elsewhere. ^29,31,32 The UK site of the only European RCT of IPS also found recruitment difficult and slow, ⁷ suggesting these factors are not unique to our study. There is evidence from other UK studies that mental health professionals do not focus on vocational needs ³³ and may actually advise people not to work. ³⁴

There was evidence for researchers guessing allocation arm but as the primary outcome was derived from administrative methods in addition to participant and healthcare professional recall this is unlikely to affect the main findings. The methods to calculate patient costs relied on recall of service utilisation. There may be some concerns about recall accuracy but this is an accepted method (especially in the absence of databases designed to measure service use for insurance systems) and previous studies have shown the method to be acceptable. ^35,36

Implications

This study therefore suggests that implementing IPS in the context studied here can be challenging. Even in the studies carried out in North America, IPS is successful in obtaining employment for only around 50% of people with serious mental illness in the total population of people with disablility. Thus it is not clear that IPS alone can overcome an individual's reluctance to join the workforce, deficits in social skills and neurocognitive impairments that militate against enduring employment, ³⁷ and research into additional interventions such as motivational interviewing, cognitive interventions and benefits counselling ²¹ may help identify further ways of helping people with severe mental illness into work. We conclude from our study that IPS will not be the only intervention necessary to improve social inclusion for people with severe mental illness. Alternative or additional interventions such as motivational interviewing and/or cognitive remediation may be needed to help obtain work; for those who do not obtain work, other measures will be necessary to promote social inclusion.

Funding

This study was supported by the Wellcome Trust (GR071272MA); the supported employment programme was funded partly by the King's Fund and the South London and Maudsley Charitable Trust.