In-patient acute mental health services have been found to be unpopular with service users, Reference Quirk and Lelliott1,Reference Rose2 often failing to address individuals' needs or provide a safe and therapeutic environment. Reference Muijen3,Reference Barker4 Home treatment is not appropriate for all at times of acute illness because of levels of risk or adverse social circumstances, so residential acute services offering an alternative to standard in-patient care are of considerable interest. Several types of innovative residential acute service have been developed in Europe and North America over recent decades. Reference Warner5 However, their effectiveness remains uncertain. Two relevant Cochrane reviews have been published. Johnstone & Zolese Reference Johnstone and Zolese6 review length of stay on acute wards, but include as brief-stay wards services with no maximum length of stay or admission for up to 4 weeks, arguably too similar to current standard care to be seen as an alternative to it. Joy & Saylan Reference Joy and Saylan7 have reviewed mother and baby units, finding no studies for inclusion. A number of reviews without meta-analyses identify some relevant studies. Reference Mattes8–Reference Calton, Ferriter, Huband and Spandler14 All however are either old, reliant on search strategies too limited to be considered systematic, or narrowly focused on a specific service model. There is no systematic synthesis of the current evidence regarding all types of residential and in-patient alternatives to standard in-patient care.
A recent UK survey proposed criteria to define a service as a residential alternative to standard acute in-patient care. Reference Johnson, Gilburt, Lloyd-Evans, Boardman and Leese15 This review uses these criteria and reviews the evidence for each type of alternative. The aims of the review are to examine the effectiveness and cost-effectiveness of and satisfaction with alternative services, and to identify the major research gaps.
Method
Inclusion criteria
Studies meeting the following criteria were included in the review.
-
(a) Study type: randomised controlled trials (RCTs), two-group non-randomised cohort studies or one-group interrupted time series studies providing a specific quantitative comparison of the effectiveness and/or acceptability of residential alternatives and standard acute in-patient services.
-
(b) Participants: adults aged 16–65 years assessed by a mental health professional as needing acute in-patient admission.
-
(c) Interventions: residential acute mental health services that offer an alternative to standard acute psychiatric wards in one of the following five ways: based in the community (non-hospital services such as crisis houses); time limited (services offering admission with a time limit or planned maximum stay of 14 days or fewer); dedicated to a specific diagnostic group (e.g. first-episode psychosis or borderline personality disorder); dedicated to a specific socio-demographic group (e.g. wards for specific ethnic groups); implementing a specific therapeutic model involving changes to the working practices of more than one professional group.
-
(d) Outcomes: any outcome relating to clinical improvement or social functioning, service use, costs or cost-effectiveness, satisfaction with services.
Search strategy
A systematic search was undertaken of seven electronic databases – Medline, PsycINFO, Web of Science, CINAHL, EMBASE, the Cochrane Library (Database of Abstracts of Reviews of Effects and Central Controlled Trials Register) and the National Health Service (NHS) Economic Evaluation Database – covering the period from January 1966 to February 2008. Search terms for acute, residential mental health services were combined with search terms for different types of alternatives: terms were searched in the title and abstract except where stated. No language restrictions were applied. The Medline search was conducted on PubMed as follows:
-
(a) crisis intervention[MeSH].exp OR crisis OR acute OR emergency
AND
-
(b) residential treatment[MeSH].exp OR residential OR hospital* OR inpatient* OR ‘crisis house’ OR ‘community beds’ OR ‘crisis beds’
AND
-
(c) mental disorders[MeSH].exp OR mental OR psychiatr*
AND
-
(d) ‘disorder specific’ OR ‘disorder-specific’ OR specialist OR ‘socio-demographic’ OR sociodemographic OR ethnic* OR disabled OR disability OR ‘visual* impair*’ OR blindness OR deaf OR ‘mother and baby’ OR ‘parent and child’ OR ‘therapeutic model’ OR ‘model of care’ OR alternative OR innovative OR ‘brief admission’ OR ‘brief stay’ OR ‘short stay’ OR ‘time limited’ OR community OR ‘non-hospital’ OR ‘sponsor homes’ OR ‘accredited accommodation’ OR Soteria OR (‘crisis intervention’ AND Trieste) OR ‘Tidal Model’ OR ‘refocusing model’.
Search terms were modified as necessary to search other databases. Reference lists of all included articles and review articles were also hand searched. ‘Grey’ literature was searched through directories of conference proceedings and additional unpublished studies or papers in press were sought by contacting experts within the field.
Data abstraction
Two of the authors (B.L.-E. and D.J.) independently scanned titles from all identified studies and from abstracts where relevant and available, then retrieved and read the full text of all potentially relevant studies. Queries about inclusion were discussed and any disagreement resolved by a third reviewer (S.J.). Study details, including type of study, service and participant characteristics, duration of study period and study outcomes, were collected using a standard data extraction form.
Quality assessment
The methodological quality of each study included in the review was assessed using a standard form adapted from the quality assessment tool of Thomas. Reference Thomas16 Studies were rated as strong, moderate or weak regarding potential selection bias, allocation bias, accounting for confounders, masking, data collection methods and withdrawals, according to the criteria set out in the dictionary accompanying the quality assessment tool. Reference Thomas16 The analysis strategy and intervention integrity of studies were also noted.
Operational criteria were created from quality ratings to distinguish studies of high, moderate and low quality overall. Studies that were rated as high quality reported allocation concealment during randomisation, analysed data based on intention-to-treat and rated strong in all domains in the Thomas tool bar masking. All other RCTs were rated as moderate quality, as were non-randomised studies which demonstrated no significant difference or adjusted in analyses for difference between experimental and control groups for the confounder of severity of illness at admission and rated at least moderate for all Thomas criteria assessed except masking.
Reporting of study results
This review includes outcomes only from studies that were rated as moderate or high quality. Outcomes are reported from all these studies: mean figures for service use and cost data are presented if reported; effect size and P-values are reported for all outcomes where there is a significant difference between alternative and standard services.
Results
Twenty-seven studies were identified for inclusion in the review, of which nine were rated as moderate or high quality. Figure 1 summarises the study flow through the review.
Fig. 1 Selection of studies for inclusion in systematic review.
Types of service studied
The review identified studies of community-based Reference Timko, Chen, Sempel and Barnett17–Reference Goveia and Tutko31 and time-limited Reference Olfson, Plotke, Harding, Jones and Mayfield32–Reference Mendel38 services, and services with a specific therapeutic model. Reference Berger39–Reference Dodds and Bowles43 No studies of in-patient services for specific diagnostic or sociodemographic groups were found, although some studies of community-based services also included clinical or sociodemographic inclusion criteria for participants.
Community-based services
Fifteen studies of community-based services published from 1969–2006 were identified (Table 1; see online Table DS1 for a more detailed version of this table). Eleven of these were of US services.
Table 1 Characteristics of studies of community-based services a
| Study reference | Service description | Study design and duration | Outcomes assessed b | Quality rating and main limitations c |
|---|---|---|---|---|
| Timko et al Reference Timko, Chen, Sempel and Barnett17 (2006) | Veterans' community residential facilitites, California, USA | RCT 30-day follow-up (from discharge) | 1, 2, 4 | Moderate 1, 2 |
| Hawthorne et al Reference Hawthorne, Green, Gilmer, Garcia, Hough and Lee18 (2005) | 6 crisis hostels (11–14 bedded), San Diego, USA | RCT 2-month follow-up | 1, 2, 3, 4 | Moderate 1, 2, 7 (some satisfaction data collected by service staff) |
| Boardman et al Reference Boardman, Hodgson, Lewis and Allen19 (1999) | Community mental health centre beds, UK | Prospective non-randomised quasi-experiment 1-year follow-up | 1, 2, 3, 4 | Moderate 1, 2 |
| Fenton et al Reference Fenton, Mosher, Herrell and Blyler20 (1998) | Crisis hostel (8 beds), Maryland, USA | RCT 6-month follow-up | 1, 2, 3, 4 | Moderate 1 |
| Mosher et al Reference Mosher, Vallone and Menn21 (1995) (Soteria study 2) | Soteria crisis hostel, California, USA | RCT 6-week follow-up | 1 | Moderate 1, 2, 6 |
| Polak & Kirby Reference Polak and Kirby22 (1976) | Adult family placement, Colorado, USA | RCT 4-month follow-up | 1, 3 | Moderate 1, 2, 3 |
| Readhead et al Reference Readhead, Henderson, Hughes and Nickless23 (2002) | Adult family placements, UK | Interrupted time series study 1-year comparison period | 2, 4 | Low 2, 3 |
| Hawthorne et al Reference Hawthorne, Green, Lohr, Hough and Smith24 (1999) | 5 crisis hostels, San Diego, USA | Prospective non-randomised quasi-experiment 4-month follow-up | 1, 2, 3 | Low 1, 2, 4 |
| Ciompi et al Reference Hawthorne, Green, Lohr, Hough and Smith24 (1993) | Soteria crisis hostel, Switzerland | Prospective non-randomised quasi-experiment 2-year follow-up | 1, 2, 4 | Low 2, 3 |
| Ciompi et al Reference Ciompi, Dauwalder, Maier, Aebi, Trütsch and Kupper26 (1992) | Soteria crisis hostel, Switzerland | Non-randomised quasi-experiment (not stated if retrospective) 6-week follow-up | 1, 4 | Low 2, 3 |
| Rappaport et al Reference Rappaport, Goldman, Thornton, Stegner, Moltzen and Hall27 (1987) | 45 bed crisis hostel, California, USA | Retrospective non-randomised cohort study Assessment at discharge | 1 | Low 1, 2, 4 |
| Bittle et al Reference Bittle28 (1986) | 2 crisis hostels (10 bedded), Illinois, USA | Retrospective non-randomised cohort study 40-month follow-up | 2 | Low 2, 3 |
| Mosher & Menn Reference Mosher and Menn29 (1978) (Soteria study 1) | Soteria crisis hostel, California, USA | Prospective non-randomised (pseudo-randomised) quasi-experiment 2-year follow-up | 1, 2 | Low 1, 2, 4 |
| Brook Reference Brook30 (1973) | Crisis hostel, Denver, USA (time limited to 7 days) | Non-randomised cohort study (not specified if retrospective) 6-month follow-up | 1, 2 | Low 2, 3 |
| Goveia & Tutko Reference Goveia and Tutko31 (1969) | Crisis hostel, California, USA | Prospective non-randomised quasi-experiment (some but not all participants randomised) 12-month follow-up | 1, 2, 3 | Low 1, 2, 3, 4, 5 |
RCT, randomised controlled trial
a. A more detailed version of this table (Table DS1) is available online
b. Domains: 1, improvement; 2, service use; 3, satisfaction; 4, cost
c. Key to aspects of study quality: 1, analysis based on completer data not all intended to treat; 2, allocation concealment unclear (RCTs); not randomised (non-RCTs); 3, confounders (including severity of illness) not measured and if necessary adjusted for in analysis; 4, more than 40% of potential participants declined to participate or number not stated; 5, more than 40% participants lost at follow-up; 6, unspecified or previously unpublished outcome measure; 7, other
A range of non-hospital service models have been evaluated. Four studies Reference Mosher, Vallone and Menn21,Reference Ciompi, Kupper, Aebi, Dauwalder, Hubschmid and Trutsch25,Reference Ciompi, Dauwalder, Maier, Aebi, Trütsch and Kupper26,Reference Mosher and Menn29 concern Soteria houses. Developed by Loren Mosher in California in the 1970s, Reference Mosher44 Soteria services provided care for people with first- or second-episode psychosis in informal settings. Primary staff were not clinically trained and worked long shifts (36–48 h) designed to help them to attune to and engage with residents. Staff and residents shared responsibility for household tasks and there was minimal reliance on antipsychotic medication. The model has been replicated more recently in a number of European countries (Switzerland, Germany, Sweden, Hungary, Finland), and evaluated primarily by Luc Ciompi and colleagues in Bern, Switzerland. Reference Ciompi, Kupper, Aebi, Dauwalder, Hubschmid and Trutsch25,Reference Ciompi, Dauwalder, Maier, Aebi, Trütsch and Kupper26 Other crisis hostels described in studies in this review also provide care in small, homely settings, typically about 8- to 12-bedded. Services display variation in closeness of links with statutory services. Boardman et al Reference Boardman, Hodgson, Lewis and Allen19 describe a residential unit embedded within a community mental health resource centre, staffed by clinically qualified mental health professionals including supervision by psychiatrists; Timko et al Reference Timko, Chen, Sempel and Barnett17 by contrast describe independently run services contracted by statutory agencies. Two studies in this review Reference Timko, Chen, Sempel and Barnett17,Reference Hawthorne, Green, Gilmer, Garcia, Hough and Lee18 describe services funded by American Veterans Associations, providing care mainly to veterans. Apart from Soteria hostels, crisis houses in studies in this review are not described as guided by a clear, manualised model of care: types of care provided are not described in detail.
Placement with individual families is the other model of community-based alternative to hospital identified in this review. Polak & Kirby evaluated family sponsor homes developed in the 1970s in Denver Colorado, Reference Polak and Kirby22 where host families supported one or two acutely ill people, with training and assistance from local crisis services. This scheme specifically aimed to divert people from acute hospital admission. Support included involving individuals in a normal family environment with participation in meals and other domestic tasks and activities. The model has been used on a small scale in Powys, Wales in the last decade. Reference Readhead, Henderson, Hughes and Nickless23
Time-limited services
Seven studies evaluated time-limited hospital in-patient services ranging from 1 day to 8 days maximum planned admission (Table 2). Studies dated from 1966 to 1996 and evaluated services in North America and the UK. All the brief-stay wards in studies in this review were located within larger hospital complexes and accepted general acute admissions. Brief-stay services report an aim to increase the intensity with which care was provided compared with standard acute wards (e.g. providing assessment, medication review, help solving psychosocial problems, after-care planning within the period of a brief admission), but distinctly different interventions are not described.
Table 2 Characteristics of studies of time-limited services
| Study reference | Service description | Study design and duration | Participants and total n (alternative n/comparison n) | Outcomes assessed a | Quality rating and main limitations b |
|---|---|---|---|---|---|
| Olfson Reference Olfson, Plotke, Harding, Jones and Mayfield32 (1990) | 5-day time-limited crisis admission ward, New England, USA | Prospective non-randomised quasi-experiment 3-month follow-up | Adults assessed as requiring acute admission: diagnosis of schizophrenia, one or more previous admissions, stable housing, no current substance misuse or major medical problems n = 26 (8/18) | 1, 2 | Moderate 2 |
| Hirsch et al Reference Hirsch, Platt, Knights and Weyman33 (1979) | 8-day target hospital admission ward, UK | RCT 1-year follow-up | Adults aged 16+ requiring acute admission No diagnosis of brain injury or major physical health problem n=224 (115/109) | 1, 2 | Moderate 1, 2 |
| Herz et al Reference Herz, Endicott and Spitzer34 (1975) | 1-week target hospital admission ward, New York, USA | RCT 7-month follow-up | Adults aged 16+ requiring acute admission who live with a responsible adult, have a diagnosis of mental illness. Limitations on comorbidity n = 175 (51: brief hospital; 61: brief hospital and day care; 63: standard hospital) | 1, 2 | Moderate 2, 3 |
| Schneider & Ross Reference Schneider and Ross35 (1996) | 3-day crisis admission ward, Connecticut, USA | Retrospective non-randomised cohort 30-day follow-up | Adults assessed as requiring acute admission n = 1370 (590/780) | 2 | Low 2, 3 |
| Ianzito et al Reference Ianzito, Fine, Sprague and Pestana36 (1978) | 24-h admission ward, Massachusetts, USA | Retrospective non-randomised cohort study 2-week follow-up | Adults assessed as requiring acute admission n = 184 (83/101) (also 193 non-admitted patients evaluated) | 1, 3 | Low 2, 3, 6 |
| Voineskos et al Reference Voineskos, Morrison and Jain37 (1972) | 3-day crisis admission ward, Canada | Retrospective non-randomised cohort Duration of initial admission assessed | Adults assessed as requiring acute admission n = 868 (439/429) | 2 | Low 2, 3 |
| Mendel Reference Mendel38 (1966) | 7-day time-limited admission ward, California, USA | Prospective non-randomised quasi-experiment 18-month follow-up | Adults 18+ assessed as requiring acute admission: diagnosis of schizophrenia, admission from community, voluntary or on 72-h section n = 443 (114/329) | 1, 2 | Low 1, 2, 3, 5 |
RCT, randomised controlled trial
a. Domains: 1, improvement; 2, service use; 3, satisfaction; 4, cost
b. Key to aspects of study quality: 1, analysis based on completer data not all intended to treat; 2, allocation concealment unclear (RCTs); not randomised (non-RCTs); 3, confounders (including severity of illness) not measured and if necessary adjusted for in analysis; 4, more than 40% of potential participants declined to participate or number not stated; 5, more than 40% participants lost at follow-up; 6, unspecified or previously unpublished outcome measure; 7, other
Services with a distinctive therapeutic model
Five studies of services with a specific therapeutic model involved one of two UK nursing-led models of care developed in the last decade, the Tidal model or the Refocusing model (Table 3). The Tidal model, developed by Phil Barker in the 1990s Reference Barker45 seeks to avoid a perceived reductionist approach of relating to people purely as patients with symptoms that need to be treated, by valuing people's own narrative of illness and perception of problems. Frequent, collaborative contact between staff and patients is encouraged through regular assessment of problems and goals. This involves documenting individuals' expressed needs and problems verbatim. The Refocusing Model, developed on acute wards in Bradford, UK, in the late 1990s, Reference Dodds and Bowles43 increased nurses' authority to take risk management decisions and thus minimise or eradicate formal observations on wards, with time instead spent on more collaboratively agreed contact between staff and patients.
Table 3 Characteristics of studies of services with a distinct therapeutic model
| Study reference | Service description | Study design and duration | Participants and total n (alternative n /comparison n) | Outcomes assessed a | Quality rating and main limitations b |
|---|---|---|---|---|---|
| Berger et al Reference Berger39 (2006) | Tidal model ward, Canada | Interrupted time series study 6-month comparison period: outcomes during admission assessed | Adults on an acute admission ward who consent to participate n = 46 (not stated) | 3 | Low 1, 2, 3, 4 |
| Lafferty & Davidson Reference Lafferty and Davidson40 (2006) | Tidal model ward, Glasgow, UK | Interrupted time series study 1-year comparison period | Adults on an acute ward n not stated (service level data only collected) | 1/3 | Low 2, 3 |
| Gordon et al Reference Gordon, Morton and Brooks41 (2005) | Tidal model ward, Birmingham, UK | Retrospective non-randomised cohort study 1-year comparison period: outcomes during admission assessed | Adults on an acute ward n not stated (service level data only collected) | 1/3 | Low 2, 3 |
| Stevenson et al Reference Stevenson, Barker and Fletcher42 (2002) | Tidal model ward, Newcastle, UK | Interrupted time series study 6-month comparison period Duration of initial admission assessed | Adults on an acute ward n = 150 (81/69) | 2 | Low 2, 3 |
| Dodds & Bowles Reference Dodds and Bowles43 (2001) | Refocusing model ward, Bradford, UK | Interrupted time series study 6-month comparison period: outcomes during admission assessed | Adults on an acute ward n not stated (service level data only collected) | 1 | Low 2, 3 |
a. Domains: 1, improvement; 2, service use; 3, satisfaction; 4, cost
b. Key to aspects of study quality: 1, analysis based on completer data not all intended to treat; 2, allocation concealment unclear (randomised controlled trials (RCTs)); not randomised (non-RCTs)); 3, confounders (including severity of illness) not measured and if necessary adjusted for in analysis; 4, more than 40% of potential participants declined to participate or number not stated; 5, more than 40% participants lost at follow-up; 6, unspecified or previously unpublished outcome measure; 7, other
Study characteristics
Twenty-seven studies included in this review comprised 7 RCTs, 7 non-randomised prospective quasi-experimental studies and 13 before/after comparison studies. Most were of small or medium size, only four studies having more than 250 participants. Reference Hawthorne, Green, Lohr, Hough and Smith24,Reference Bittle28,Reference Voineskos, Morrison and Jain37,Reference Mendel38 Duration of studies ranged from the period of admission only up to 40-month follow-up.
No studies identified by this review were assessed as high quality. Nine studies were rated as moderate quality, including two well-designed quasi-experimental studies in addition to seven RCTs. Three of these studies were of brief-stay wards Reference Olfson, Plotke, Harding, Jones and Mayfield32–Reference Herz, Endicott and Spitzer34 and six of community-based services, residential crisis beds Reference Timko, Chen, Sempel and Barnett17–Reference Mosher, Vallone and Menn21 or family placement. Reference Polak and Kirby22 None rated strong on all criteria of the Thomas assessment tool. Reference Thomas16 Only one study Reference Fenton, Mosher, Herrell and Blyler20 clearly described allocation concealment procedures during randomisation. Where participants were lost to follow-up, no studies based analysis on intention-to-treat; all provided completer data only. Reported levels of loss of participants during the course of studies varied substantially for studies of moderate quality. Only two Reference Polak and Kirby22,Reference Herz, Endicott and Spitzer34 included in outcomes data all potential participants assessed as eligible; reported overall attrition rates on individual outcomes in other moderate-quality studies range from 3 to 52% of potential participants. Service use data, gathered from routinely collected records, was generally more comprehensive than assessment of participants' functioning or satisfaction.
Of the 18 remaining studies assessed as low quality, 15 did not adequately measure or adjust for confounders. Five had high risk of selection bias (more than 40% of those eligible declining to participate); two had unacceptably high withdrawal rates (more than 40%) for all outcomes and one used only unpublished outcome measures. Full details of the quality assessment of all studies can be found in online Table DS2.
Study outcomes
Outcomes assessed and results from studies of higher (moderate) quality found in this review are presented in Table 4 (a more detailed version, including details of the outcomes assessed, can be found as online Table DS3).
Table 4 Results from studies of moderate or high quality a
| Study reference | Results |
|---|---|
| Community-based services | |
| Timko et al Reference Timko, Chen, Sempel and Barnett17 (2006) | Favours alternative: total out-patient visits (104 v. 130: P <0.001), 1-year cost ($22 000 v. $33 000: P = 0.002) |
| Favours standard service: length of index admission (26 v. 55 days: P <0.001), 1-year total in-patient bed days (78 v. 86 days: P <0.01) | |
| No significant difference: ASI psychiatric subscale score at 1 year | |
| Hawthorne et al Reference Hawthorne, Green, Gilmer, Garcia, Hough and Lee18 (2005) | Favours alternative: discharge SF-36V (P = 0.02) and POC (P = 0.05) scores, cost of index admission (P = 0.001), homelessness (P = 0.001) at discharge |
| No significant difference: discharge PANSS and ASI scores, PANSS, SF-36V, ASI scores, homelessness and number of readmissions at 2-month follow-up | |
| Boardman et al Reference Boardman, Hodgson, Lewis and Allen19 (1999) | Favours alternative: GAF (P = 0.02), HRSD (P = 0.01), PSE (P = 0.001), VSSS overall satisfaction (P = 0.02) |
| No significant difference: HoNOS, SBS, CAN, length of index admission, number readmitted in 12-month follow-up, cost to all public services (although cost to NHS significantly higher at alternative services) | |
| Fenton et al Reference Fenton, Mosher, Herrell and Blyler20 (1998) | Favours alternative: cost of index admission significantly less ($3046 v. $5549: effect size 0.78, P <0.001) |
| Favours standard service: length of index admission (12 v. 19 days: P <0.002) | |
| No significant difference: PANSS scores, satisfaction, 6-month costs, cost-effectiveness | |
| Mosher et al Reference Mosher, Vallone and Menn21 (1995) | No significant difference |
| Polak & Kirby Reference Polak and Kirby22 (1976) | Favours alternative: satisfaction: TES score (patient report) at discharge (P <0.001) and 4-month follow up (P <0.01) |
| No significant difference: all measures of clinical improvement | |
| Time-limited services | |
| Olfson Reference Olfson, Plotke, Harding, Jones and Mayfield32 (1990) | No significant differences (only 3/8 participants discharged from brief-stay service within planned 5-day limit) |
| Hirsch et al Reference Hirsch, Platt, Knights and Weyman33 (1979) | No significant differences (median length of stay but not mean length significantly shorter at alternative) |
| Herz et al Reference Herz, Endicott and Spitzer34 (1975) | Favours alternative: length of index admission (9 days v. 50 days: no P stated); in-patient days over 2-year follow-up (17 v. 115: P <0.001) |
| No significant difference: PSS or GAS total scores at 3 months or 2 years, number of participants readmitted over 2-year follow-up |
ASI, Addiction Severity Index; SF-36V, Health Survey - Short Form (Veterans Version); POC, Perceptions of Care Questionnaire; PANSS, Positive and Negative Symptom Scale; GAF, Global Assessment of Functioning; HRSD, Hamilton Rating Scale for Depression; PSE, Present State Examination; VSSS, Verona Service Satisfaction Scale; HoNOS, Health of the Nation Outcome Scale; SBS, Social Behaviour Schedule; CAN, Camberwell Assessment of Need; NHS, National Health Service; TES, Treatment Effectiveness Scale; PSS, Psychiatric Status Schedule; GAS, Global Assessment Scale
a. A more detailed version of this table is available online as Table DS3
All six moderate-quality studies of community-based alternatives reported measures of symptoms and/or global improvement: four Reference Timko, Chen, Sempel and Barnett17,Reference Fenton, Mosher, Herrell and Blyler20–Reference Polak and Kirby22 found no significant differences from standard services, whereas two Reference Hawthorne, Green, Gilmer, Garcia, Hough and Lee18,Reference Boardman, Hodgson, Lewis and Allen19 found some results favouring alternatives and some showing no significant difference. Three Reference Hawthorne, Green, Gilmer, Garcia, Hough and Lee18,Reference Boardman, Hodgson, Lewis and Allen19,Reference Polak and Kirby22 out of four studies that assessed satisfaction reported moderately and significantly greater satisfaction with the alternative than the standard service. Three Reference Timko, Chen, Sempel and Barnett17,Reference Hawthorne, Green, Gilmer, Garcia, Hough and Lee18,Reference Fenton, Mosher, Herrell and Blyler20 out of four studies that assessed cost also reported results favouring the alternative service. The exception was the community mental health team community beds assessed by Boardman et al: Reference Boardman, Hodgson, Lewis and Allen19 a statutory service staffed by qualified mental health professionals, in contrast to the other community-based services described in this review, this service cost the NHS more per patient than standard acute wards, although with no significant overall difference to public services. Only with regard to service use were any results favourable to standard care: two of three studies reported length of index admission greater at community alternatives than standard wards; Reference Timko, Chen, Sempel and Barnett17,Reference Fenton, Mosher, Herrell and Blyler20 one of four found in-patient bed-days, including index admission, over the study follow-up period significantly fewer for standard services. Reference Timko, Chen, Sempel and Barnett17
Of the three moderate-quality studies of time-limited alternatives, only one found the brief-stay ward able to discharge individuals within the planned admission period and significantly more quickly than standard wards. Reference Herz, Endicott and Spitzer34 One study Reference Olfson, Plotke, Harding, Jones and Mayfield32 found a majority of people required transfer to a standard ward at the end of the brief-stay period; one study Reference Hirsch, Platt, Knights and Weyman33 found the 8-day planned admission period not rigorously adhered to and mean length of stay not significantly shorter than standard care. None found any significant differences in clinical outcomes or readmission rates. All five low-quality studies of services using a distinctive therapeutic model reported some outcomes favourable to alternative services regarding patient satisfaction or levels of untoward incidents on wards.
Discussion
Findings of the review
Current research is insufficient to provide convincing evidence about the effectiveness or acceptability of residential alternatives to standard acute in-patient mental health services. No studies of services for specific demographic groups or people with specific diagnoses were identified, despite literature providing descriptions of some such services as promising service models, e.g. mother and baby units 46 or psychosis, affective disorder and personality disorder wards. Reference Bonsack, Borgeat and Dubois47 Studies of services with a distinctive therapeutic model were all of low quality, failing to account for differences between groups in analysis. The before and after comparison provided by most studies of services with a distinctive therapeutic model and their lack of stated primary outcomes also exacerbate risks of reporting and publication biases. The feasibility of brief-stay acute wards is brought into question by the fact that in three moderate-quality studies, in only one Reference Herz, Endicott and Spitzer34 was the alternative service able to discharge a majority of individuals within the planned admission period. The applicability of this finding to contemporary mental health service settings may be limited however, as the moderate-quality studies of time-limited services identified in this review all pre-date the advent of modern community resources such as home-treatment teams. The most recent study of time-limited services included in the review, from 1996, Reference Schneider and Ross35 found that 69% of those admitted to a 3-day admission ward could be discharged into the community within this period, but the comparability of individuals with those admitted to general acute wards was unclear.
Despite the larger number of studies of community-based alternatives, the strength of evidence provided is nevertheless limited by the quality of included studies and the heterogeneity of services and participants studied. A crucial question is to what extent community-based alternatives can admit a population comparable with standard acute wards. All six moderate-quality studies of community-based services imposed different inclusion criteria for participants and all but one Reference Polak and Kirby22 included some criteria (such as veterans only, consenting to participate, no admissions in the previous 12 months) beyond those normally required for real-life acute admission. Two of the studies specifically excluded detained patients; Reference Hawthorne, Green, Gilmer, Garcia, Hough and Lee18,Reference Fenton, Mosher, Herrell and Blyler20 the remaining four did not report whether or how many participants were detained. The systematic exclusion from studies of some people who require acute admission and the not insignificant drop-out rates reported by some studies limit the strength and applicability of their results. Findings from studies of community-based services may only be applicable to a subgroup of people requiring acute admission, excluding some of those who are most severely unwell or least cooperative.
Notwithstanding its limitations, the evidence overall for community-based services retains the possibility that non-hospital residential crisis services are a useful service model: no clinical outcomes from studies in this review were worse than standard wards for community-based alternatives; the few differences reported tend to favour community-based models. That five studies successfully randomised participants assessed as requiring acute admission to either community alternatives or standard wards does suggest some similarity in populations served. As a whole, the studies provide preliminary evidence that for some people with acute mental health problems, community-based alternatives may be as effective and potentially less costly and more acceptable than standard in-patient wards. More generally, current research provides no contraindication to any of the types of alternatives included in this review.
Methodological issues and limitations
Residential alternatives are not clearly defined or described by a consistent terminology, providing a challenge for comprehensive retrieval of relevant studies. This review defined alternative services consistently with a current UK study of residential alternatives. Reference Johnson, Gilburt, Lloyd-Evans, Boardman and Leese15 Over-inclusive search terms were used in the initial search to minimise the risk of missing relevant studies, but this search found no studies that appeared to describe an innovative acute residential service and did not meet the review's inclusion criteria. This suggests the list of studies included in the review was relatively comprehensive.
In order to avoid ignoring available evidence in an under-researched area, non-randomised studies were included in the review. Assessment of study quality was conducted to inform consideration of the strength of evidence provided. The tool used in this review Reference Thomas16 was recommended by Deeks et al Reference Deeks, Dinnes, D'Amico, Sowden, Sakarovitch and Song48 in a review of quality assessment tools and recommended as suitable for randomised and non-randomised studies. It assesses the domains of quality identified as important in the most recent Cochrane handbook Reference Higgins and Green49 (although with less detailed assessment of randomisation procedures) and additionally allows higher and lower quality non-randomised trials to be distinguished. Study ratings for masking were not used to assess overall study quality: masking of participants and guarantee of consistent concealment from raters of participants' care status were not considered possible for health services research of this type. In order to rate high overall, in addition to rating strong on all other assessment tool criteria, studies were required to report adequate allocation procedures during randomisation and analyse data based on intention-to-treat, as these two dimensions of quality have been identified as most associated with estimates of treatment effects. Reference Schulz, Chalmers, Hayes and Altman50 Only from studies rated moderate quality or above, based on the aspects of quality rated with the Thomas tool, Reference Thomas16 were results presented in this paper. This provides some safeguard from creating a false impression of the weight and strength of evidence concerning alternative services by collating results from numerous poor-quality studies. However, the absence of any studies rated as highest quality in this review indicates the need for additional caution about the precision of results.
A meta-analysis of data from studies included in this review was initially planned. However, only a minority of moderate-quality studies, describing services of considerable heterogeneity, could provide any data usable in meta-analyses. In these circumstances, a meta-analysis was not considered appropriate. Data from moderate-quality studies which were potentially usable or not usable for meta-analyses are however presented in online Table DS4.
Clinical implications
Current research evidence provides clinicians and commissioners with only very limited guidance about effective models of acute in-patient mental healthcare. Several service models identified in this review – Soteria houses, adult family placements, time-limited wards – have been developed in more than one country or time period without ever becoming a well-established part of a national acute service system. This suggests some doubt about their sustainability and/or usefulness, although also a persisting perception of a need to seek alternatives to standard acute care. Residential services that can only cater for a proportion of people requiring acute admission may be perceived by service planners and commissioners as a luxury and be vulnerable to losing funding. The failure of alternative service models to endure may also reflect a reliance of innovative services on charismatic leaders and local champions, without whom they may not thrive. The community beds embedded in a community mental health resource centre evaluated by Boardman and colleagues Reference Boardman, Hodgson, Lewis and Allen19 allay some of these concerns. They were able to admit a reasonably high proportion (65%) of people assessed as requiring acute admission during the study period, can accept detained patients and are still running currently, a decade later. Drayton Park, a women's crisis house in North London, has also been established for more than a decade and evaluated in qualitative studies as providing a valuable role in local acute care. Reference Killaspy, Dalton, McNicholas and Johnson51,Reference Johnson, Bingham, Billings, Pilling, Morant and Bebbington52 This suggests that in a contemporary UK context, community crisis beds can constitute an important and sustainable part of local acute in-patient provision.
The dissatisfaction of many service users with standard psychiatric wards Reference Quirk and Lelliott1,Reference Rose2 suggests a need for alternatives. The potential for emergency residential accommodation outside the hospital setting to improve service user choice and thus the acceptability of services, while relieving bed pressure on acute wards, was identified 10 years ago. 53 Even if alternative service models can only divert a subgroup of people requiring acute admission, the increased scope this might bring for focusing appropriate facilities and expertise in in-patient services for a higher risk, predominantly detained client group is potentially useful. This review found no evidence against alternative models of care and, consistent with previous qualitative research, Reference Johnson, Bingham, Billings, Pilling, Morant and Bebbington52,Reference Faulkner, Petit-Zeman, Sherlock and Wallcraft54 provides an indication that crisis beds in non-hospital settings may increase satisfaction with acute residential services for users. Certainly, this review provides no discouragement to service managers and commissioners to consider innovation in the provision of acute in-patient care.
Future research
A recent service mapping study indicates that alternatives, as defined in this review, constitute almost 10% of current acute in-patient beds in England, with each type of alternative included in this review represented. Reference Johnson, Gilburt, Lloyd-Evans, Boardman and Leese15 Many of the service models are far from new, with some to which papers included in the review relate dating back to the 1960s. Yet this review found only limited evidence for any and none for some types of alternative. This means that, where alternatives have been established, clinical practice is running ahead of the research evidence base. It goes some way towards explaining why some service models first described as promising several decades ago, such as crisis placements in family homes and brief-stay admission wards, have yet to be widely adopted despite the wish among service planners, clinicians and service users to develop alternatives to standard acute wards. The conclusion that there is a need and an opportunity for more research is an inescapable one in this area. A further research question beyond the scope of this review was how users of residential alternatives compare with people receiving crisis home treatment: information about the extent to which alternatives accommodate people who cannot be adequately treated at home would also illuminate their role and potential utility in the acute care system.
Only 9 of 27 studies identified for inclusion in this review were assessed as moderate quality and none as high quality, limiting the certainty with which any conclusions about the services being evaluated can be drawn. This highlights the need for research to be of good quality if it is to be useful. Key quality indicators that were absent from studies of moderate quality included in this review were arranging adequate allocation concealment and conducting analysis based on intention-to-treat and stating primary outcomes in advance. Insufficiently thorough description of participants' characteristics (e.g. whether people were detained or not) and inadequate reporting of variance in outcomes data were also common shortcomings. Only two studies Reference Timko, Chen, Sempel and Barnett17,Reference Olfson, Plotke, Harding, Jones and Mayfield32 included in the review provided detailed quantitative comparison of the content of care provided at alternative and standard services, although five more, Reference Mosher, Vallone and Menn21,Reference Ciompi, Kupper, Aebi, Dauwalder, Hubschmid and Trutsch25–Reference Rappaport, Goldman, Thornton, Stegner, Moltzen and Hall27,Reference Mosher and Menn29 including all the studies of Soteria houses, compared participants' medication use. More information about care provided would help identify differences in alternative service models and the extent and nature of difference from standard services, open the black box of service provision Reference Mechanic55 and aid understanding of service outcomes. The studies included in this review evaluate young services or recently established service innovations. Evaluation of more established, enduring alternatives would also be desirable, in order to investigate whether outcomes, perhaps especially satisfaction, are sustainable and not merely a function of service novelty.
There are particular challenges to conducting RCTs in acute mental health settings, where the need for immediate intervention makes both the logistics of randomisation and the process of informing participants and obtaining consent problematic. In such circumstances, quasi-experimental studies may be more feasible and have strong real-world applicability by evaluating outcomes for cohorts that include all service users. Reference Gilbody and Whitty56 Two non-randomised, natural experiment studies were assessed as moderate quality and reported results broadly congruent with those from RCTs included in the review. This indicates that a well-designed non-randomised study, which accounts for important confounders, may have a useful place in acute mental health service research. The development of clear protocols for ethically acceptable recruitment in mental health crises would also be very helpful, addressing issues such as how to conduct urgent randomisation out of hours when researchers are not available and how to deal with the often transient loss of capacity experienced by many people at the time of a crisis.
Compared with many models of community care, developments in acute in-patient care remain unevaluated. The treatment of people at times of crisis is clearly a crucial part of mental healthcare: establishing effective models of providing residential acute care should be a priority for future mental health services research.
Acknowledgements
This report presents independent research commissioned by the National Institute for Health Research Service Delivery and Organisation Programme. The views expressed in this publication are those of the authors and not necessarily those of the NHS, the National Institute for Health Research or the Department of Health.
eLetters
No eLetters have been published for this article.