In-patient treatment is an important element in mental healthcare, but an efficient service requires treatment teams to minimise length of hospital stay while maintaining quality of care. Further, a report from the independent Mental Health Taskforce to the NHS in England highlighted the variability across in-patient length of stay (LOS) and a desire to minimise this variability. 1 , 2 Given concerns such as these, substantial research has focused on identification of variables associated with treatment length,Reference Warnke, Rössler and Herwig 3 – Reference Melchior, Hergert, Hofreuter-Gaetgens, Bergelt, Morfeld and Schulz 6 but the general consensus is that they are of minimal value in informing clinical decisions around discharge. A potential explanation is the focus on cross-sectional measurements (e.g. demographic and clinical status at admission), whereas it may be of greater benefit to examine utility of dynamic variables available across a hospital stay. Thus, the goal of this paper is to outline a novel use of symptom monitoring in mental health that could be used to inform decisions about length of hospital stay.
That is, patients’ symptoms fluctuate and will be a major determinant in the decision to discharge a patient. In psychiatric clinical trials, patients are often identified as achieving a clinically significant improvement by a 50% criterion rule, that is, obtaining a 50% or greater reduction in initial symptoms.Reference Nierenberg and DeCecco 7 – Reference Rush, Kraemer, Sackeim, Fava, Trivedi and Frank 9 The advantage of such a criterion is that it is simple to calculate, is easy to interpret (comprising two categories, either met or not met), and makes reference to initial symptom severity. If a 50% reduction were indicative of sufficient response to treatment in psychiatric in-patient settings, then an efficient hospital would discharge patients upon meeting this criterion. However, during in-patient psychiatric treatment, symptoms are rarely formally routinely examined during the course of treatment and are typically only measured at admission and discharge. Although assessment at admission and discharge provides information regarding the proportion of patients who have experienced a clinically significant treatment response by the end of their hospital stay it is not clear whether this treatment response had occurred substantially before discharge. On the contrary, formally monitoring symptoms could provide information regarding the rate and patterns of treatment response, and could in future inform decisions around discharge readiness.
The potential benefit of monitoring symptoms during treatment can be seen in the psychotherapy literature.Reference Lambert, Whipple, Hawkins, Vermeersch, Nielsen and Smart 10 – Reference Lambert, Hansen and Finch 12 Repeated assessment of symptoms has allowed modelling of expected patient response to therapy, and analyses have identified a negatively accelerating curve, characterised by rapid early gains that peak and then taper.Reference Lutz, Lowry, Kopta, Einstein and Howard 13 , Reference Kopta, Howard, Lowry and Beutler 14 An explanation is that the curve arises because of early termination of patients who responded quickly to treatment and reached a ‘good enough level’ of change to cease treatment.Reference Barkham, Rees, Stiles, Shapiro, Hardy and Reynolds 15 Thus, the ‘good enough level (GEL)’ model posits that clients respond to treatment at different rates and will cease treatment upon achieving a level of change that is ‘good enough’. If psychiatric in-patients also reach a ‘good enough level’ and then taper, then this would suggest that further time spent in hospital after reaching this level does not provide additional value in terms of symptom reduction. Using a criterion indicative of a ‘good enough level’ of response such as the 50% criterion may put an upper boundary on the number of people in hospital who no longer require hospitalisation, therefore assisting in allocation of resources within an in-patient setting.
Aims of the study
The objective of the present study was to examine the potential utility of monitoring symptoms of in-patients over the course of a psychiatric admission to inform discharge decisions. The first aim was to describe the proportions of in-patients demonstrating significant change (a) at discharge and (b) during their hospital stay. It was expected that, because a decision to discharge a patient will be made when the psychiatrist is of the opinion that in-patient care is no longer needed and because of support for the GEL model, most patients would meet this criterion of meaningful symptom reduction before discharge and then show little further improvement after this point. The second aim was to determine whether there were subgroups of patients who exhibited distinct patterns of symptom change after meeting the 50% criterion. The third aim was to examine the trajectories of patients who did not meet the 50% criterion. Correspondence between the 50% criterion and another indicator of clinically significant change, the reliable change index (RCI),Reference Jacobson and Truax 16 was also examined.
Method
Participants
Participants were recruited from a private psychiatric hospital in Australia. The total number of in-patients (with LOS over 1 day) in the hospital between 15 April 2013 and 23 June 2014 was N=1621. Eligible patients were English-speaking in-patients who had completed symptom assessments on at least three occasions (to permit modelling of trajectories over time). The resulting subsample comprised 1102 consecutive in-patients. Percentage of data collection was calculated (the number of daily measures completed compared with possible number completed given LOS). Patients could complete up to 60 daily ratings. On average, patients completed 61% (range 5–100) of their possible daily ratings. The sample was mostly female (70.9%) and ages ranged from 14 to 90 (M=40.3, s.d.=15.7). Mean LOS was 21 days (s.d.=13.8; which was longer than the mean LOS of 16 days for the unrestricted sample).
Patients were diagnosed by their treating psychiatrist according to ICD-10-AM criteria. 17 The majority of patients received primary diagnoses of affective disorders (56.5%) and neurotic disorders (20.6%). Other common primary diagnoses were substance use disorders (11.4%), personality disorders (4.4%), and schizophrenia (3.8%).
Data were collected as part of ongoing evaluation programme at the hospital, and written informed consent was obtained upon admission to the hospital. The research was approved by the Human Research Ethics Office at the University of Western Australia.
Measures
The Five Item Daily Symptom Index (DI-5; 18) was used to measure change in patients’ psychological distress throughout hospitalisation. The DI-5 is a recently developed self-report measure comprising five items designed to assess patient's psychological distress and predominantly assesses neurotic and affective symptoms, including depression, anxiety, worthlessness, not coping, and suicidal ideation. The DI-5 consists of five items rated by a six-point Likert-type scale measuring frequency (‘at no time’, ‘some of the time’, ‘less than half the time’, ‘more than half the time’, ‘most of the time’, and ‘all of the time’) scored from 0 (‘at no time’) to 5 (‘all of the time’). Thus, DI-5 scores can range from 0 to 25. Patients endorse the appropriate option for the previous 24 hours. Patients were asked to complete the DI-5 at admission and discharge, as well as daily during hospitalisation. De-identified data were made available to researchers. The DI-5 has demonstrated high internal consistency and test–retest reliability in clinical samples,Reference Dyer, Hooke and Page 18 and sound construct validity, exhibiting high correlations with other mental health measures in clinical samples.Reference Dyer, Hooke and Page 18 – Reference Restifo, Kashyap, Hooke and Page 20
Data analysis
First, descriptive analyses were conducted to identify (a) patients who had demonstrated greater than or equal to 50% reduction in DI-5 score from admission to discharge and (b) patients who had demonstrated greater than or equal to 50% reduction in DI-5 score during admission (i.e. prior to discharge).
Second, once these patient groups had been identified, the trajectories of patients who had met the 50% criterion during admission were examined. Latent class growth analyses (LCGAs) were run using Mplus,Reference Muthén and Muthén 21 to examine symptom trajectories from the point at which patients first met this criterion to discharge. Six time points were included in LCGA models: patient's DI-5 score when they first reached a greater than or equal to 50% reduction, a DI-5 score for each quartile of the remainder of the patient's admission (using the first valid DI-5 score for each quartile), and patient's DI-5 discharge score.
The third aim was to examine the trajectories of patients who did not demonstrate greater than or equal to 50% reduction in their DI-5 scores. LCGAs were run, comprising six time points: patient's DI-5 admission score, the first valid DI-5 score for each quartile of the patient's admission, and patient's DI-5 discharge score.
To deal with missing data, full information maximum likelihood (FIML) was used. To identify the best-fitting LCGA solution, several fit indices were considered. The Bayesian Information Criteria (BIC)Reference Schwarz 22 was examined, which measures goodness of fit and parsimony of the model. Lower BIC indicates better fit. The Vuong-Lo-Mendell-Rubin Likelihood Ratio Test (LMR-LRT)Reference Lo, Mendell and Rubin 23 and the Parametric Bootstrapped Likelihood Ratio Test (BLRT)Reference Nylund, Asparouhov and Muthén 24 were also examined. These tests check whether change in values for models with increasing number of classes is significant. Although entropy is not recommended as a criterion when selecting number of classes,Reference Feldman, Masyn and Conger 25 entropy values closer to 1 indicate better class separation. Thus, entropy was inspected but was not considered a primary selection criterion in model selection.
There is evidence for log-linear (i.e. negatively accelerated curve) response to treatment in psychotherapy studies.Reference Howard, Moras, Brill, Martinovich and Lutz 11 , Reference Lutz, Lowry, Kopta, Einstein and Howard 13 However, other researchers have argued that outcome trajectories are linear and slopes vary as a function of number of sessions attended, with steeper slopes for patients who attended fewer sessions.Reference Barkham, Rees, Stiles, Shapiro, Hardy and Reynolds 15 Few studies have examined patterns of treatment response in psychiatric in-patients. Thus, linear and log-linear solutions were tested, and goodness of fit was compared for one-class models, to determine the most appropriate modelling approach. In both the patients who did reach 50% and those who did not, the linear solutions (BIC=22327.431 and BIC=9016.528, respectively) provided marginally better fits than the log-linear solutions (BIC=22339.498 and BIC=9027.207, respectively). As a result, linear latent class growth models were tested. To check that solutions were not local, the analysis for best-fitting solutions was repeated using the seed values from the two highest-ranked log-likelihood values and comparing model parameter estimates.
Results
Patients who recorded a ≥50% symptom reduction
Of the 1102 patients who had completed the DI-5 at admission and discharge, 604 patients (66.2%) obtained discharge scores that were less than 50% of their admission DI-5 score. Of the 1102 in-patients, 867 (78.7%) met the 50% criterion during admission. On average, patients met this criterion 8 days (s.d.=7.59, range 1–58) into their stay, and a paired-samples t-test demonstrated that this date was significantly earlier than patients’ mean LOS (M= 20.05, s.d.=12.58), t(866)=−31.77, P < 0.001, which implies that many patients met this criterion before discharge. On average, this criterion was met by 44.7% (s.d.=26.60, 2–100) of their total LOS.
Of those who had halved their DI-5 scores during admission, 78.6% also obtained discharge scores that were less than 50% of their admission score. Of those who had not halved their DI-5 scores during admission, 81.0% had not halved their score by discharge. The RCI was also calculated, and the correspondence between the RCI (‘improved’ and ‘recovered’ categories) and 50% criterion (‘met’ category) in classifying patients as obtaining clinically significant change was examined: these analyses revealed that 82% of in-patients obtained the same classification by these methods. Thus, the degree of symptom change during admission was a reliable predictor of discharge functioning in the majority of patients.
Table 1 LCGA model fit indices for patients obtaining greater than or equal to 50% reduction in DI-5 symptoms
| Classes | Log-likelihood | BIC | Entropy | Posterior probabilities | LMR-LRT | BLRT |
|---|---|---|---|---|---|---|
| 2 | −10645.932 | 21366.279 | 0.84 | 0.93, 0.96 | 0.000 | 0.000 |
| 3 | −10515.053 | 21124.817 | 0.78 | 0.93, 0.84, 0.90 | 0.326 | 0.000 |
BIC, Bayesian Information Criteria; LMR-LRT, Vuong-Lo-Mendell-Rubin Likelihood Ratio Test; BLRT, Bootstrapped Likelihood Ratio Test.
Patient trajectories
The next step was to examine whether distinct subgroups existed among individuals who had obtained a greater than or equal to 50% reduction in their DI-5 symptom score at some point during their hospital stay (n=867). Table 2 presents the fit indices for the 2- and 3-class linear solutions for the DI-5. The 2-class linear solution was chosen as the optimal solution after considering fit indices and graphs: although BIC continued to reduce for the three-factor model, LMR-LRT was no longer significant beyond two classes, suggesting no significant improvement was obtained by adding a third class (Table 1).
Table 2 LCGA model fit indices for DI-5 for patients not obtaining 50% or greater change during hospital stay (n=277)
| Classes | Log-likelihood | BIC | Entropy | Posterior probabilities | LMR-LRT | BLRT |
|---|---|---|---|---|---|---|
| 2 | −4107.780 | 8277.425 | 0.92 | 0.97, 0.98 | 0.000 | 0.000 |
| 3 | −3929.474 | 7937.684 | 0.90 | 0.95, 0.95, 0.98 | 0.000 | 0.000 |
| 4 | −3870.655 | 7836.918 | 0.90 | 0.94, 0.94, 0.95, 0.92 | 0.035 | 0.000 |
| 5 | −3834.602 | 7781.648 | 0.85 | 0.92, 0.95, 0.94, 0.87, 0.89 | 0.112 | 0.000 |
BIC, Bayesian Information Criteria; LMR-LRT, Vuong-Lo-Mendell-Rubin Likelihood Ratio Test; BLRT, Bootstrapped Likelihood Ratio Test.
Figure 1 shows the trajectories of classes for the 2-class solution. In addition to the six time points included in the LCGAs (score at time first reached 50%, first score for each quartile of admission, and discharge score) and for the sake of completeness, mean admission score for each class (which was not included as a time point in the LCGA) is also plotted. Class 1 (26.8%) was characterised by relatively stable and low symptoms for the remainder of the admission. Class 2 (73.2%) was characterised by a slight overall increase but overall decreased symptoms in comparison to admission symptoms. Thus, once a patient met the criterion of a 50% reduction in symptoms, patients appeared to maintain those gains (and show little further change) for the remainder of the hospital stay.
Fig. 1 LCGA showing two trajectories of patients who obtained a greater than or equal to 50% reduction in symptoms during hospital stay. T0=score at which 50% criterion first met; Q1–Q4=first score from each quartile of the remainder of hospital stay.
Patients who did not demonstrate a ≥50% symptom reduction
Patient trajectories
LCGA was used to test for subgroups among patients who did not demonstrate a 50% or greater reduction in DI-5 score during admission. Table 2 presents the fit indices for the 2, 3, 4, and 5 linear solutions for the DI-5. The 4-class linear solution was chosen as the optimal solution after considering fit indices and graphs, as LMR-LRT ceased to be significant when five classes were specified (Fig. 2).
Fig. 2 LCGA showing four trajectories of patients who had not obtained a 50% or greater reduction in symptoms. Q1–Q4 = first score at each quartile of hospital stay.
Figure 2 shows the trajectories of the four groups from admission to discharge. The classes varied in their initial symptom severity, showing relatively flat trajectories across their admission and only slight reductions in symptom severity by discharge. Class 1 (45.3% of the sample) was characterised by high symptoms at admission with only a small mean reduction in severity over time. Class 2 (9.1%) was characterised by low DI-5 scores at admission and showed minimal change across time points. Class 3 (27.0%) was characterised by severe symptoms at admission and showed minimal change across time points. Class 4 (18.6%) was characterised by moderate symptoms, with minimal change in symptom severity across time points. One-way ANOVA revealed that the patient groups did not differ on mean LOS, F(3)=1.09, P=0.20.
Discussion
In psychiatric clinical trials, a 50% reduction in symptoms is often used as an indicator of clinically significant symptom improvement. However, in many settings, symptoms are measured solely at beginning and end of treatment. This study used symptom data recorded during hospital stay to identify the percentage of patients who reached a criterion indicating clinically significant response during hospital stay and examined patterns of change in symptoms after this criterion was met. The first aim was to describe the percentage of patients that met a criterion indicating meaningful symptom change – that is, a greater than or equal to 50% reduction in DI-5 admission score at (a) discharge and (b) during hospital stay. About two-thirds of patients obtained discharge DI-5 scores that were less than or equal to 50% of their admission score, and thus, this criterion appeared to describe the status of most patients when discharged. Given that a 50% or greater reduction in symptoms is not only meaningful, but is frequently used in clinical trials to signal clinically significant improvement,Reference Nierenberg and DeCecco 7 , Reference Rush, Kraemer, Sackeim, Fava, Trivedi and Frank 9 it is not surprising that the criterion describes staff behaviour because the decision to discharge a patient will be informed by a reduction in symptoms sufficient that hospitalisation is no longer warranted. Thus, although a 50% or greater reduction in symptoms is not a requirement for discharge and there is no evidence that staff consciously or formally used this criterion, the observation that two-thirds of patients had met the criterion at discharge supports the use of the criterion as a partial description of discharge readiness. The findings are consistent with the GEL model, in that the 50% criterion appeared to constitute a clinically meaningful level of change associated with discharge readiness: the majority of patients who met the criterion during their hospital stay also met the criterion at discharge and tended to exhibit stable or further reductions in symptoms. In addition, although most patients had met the criterion by discharge, approximately three-quarters of the sample met the 50% criterion at some other point during their hospital stay. In terms of symptom reduction, patients did not appear to demonstrate substantial change in symptoms during additional days in hospital after meeting the 50% criterion.
Despite the value of the dichotomous criterion, it was apparent that there was individual variability in treatment response that went beyond this simple classification. Therefore, the second aim was to determine whether distinct subgroups of in-patients could be identified amongst those who demonstrated a greater than or equal to 50% reduction in their DI-5 symptoms during their hospital stay. Two key patterns of change were identified: about three-quarters of the sample continued to exhibit stable, low symptoms across the remainder of the admission, whereas approximately a quarter of the sample showed a slightly less stable trajectory, with a slight increase in symptoms (relative to when they first reached the criterion) by discharge. However, symptom levels were still substantially lower than at admission. Thus, in terms of a prediction of the final end state at discharge, a 50% reduction in symptoms at some point after admission was a strong indicator for these patients.
The third aim was to examine symptom trajectories of patients who did not meet the 50% criterion. Individuals could be grouped based on symptoms from admission to discharge. Not all patients who did not obtain substantial reduction in symptoms followed the same trajectory. The key difference was in their initial severity: all four groups tended to show similar, flat trajectories, and no real change or deteriorating class was detected. A class comprising approximately half of the patients who had not met the 50% criterion reported low symptoms across the whole of their admission. Given that this subgroup reported low symptoms on admission, it is not necessarily surprising that they did not exhibit a greater than or equal to 50% reduction in symptoms. These findings suggest that the 50% criterion may be beneficial for identifying patients who are not responding to treatment, and who are less likely to demonstrate a ‘good enough level’ of change. A lack of change may indicate need for feedback or further treatment planning.
As an important caveat, it is imperative to note that this study is observational and that latent class growth modelling (LCGA) is an exploratory technique which suggests – but does not confirm – number of classes. It is not possible to conclude that patients who meet a 50% criterion are ready for discharge. However, this criterion may represent a flag for clinicians to indicate whether their patients are doing well. Clinically, the 50% criterion is easy to calculate and communicate to both staff and in-patients and could potentially be used as a flag for clinicians to consider discharge.Reference Hiller, Schindler and Lambert 26 Furthermore, the 50% criterion classified 82% of patients the same as another indicator of clinically significant change, the RCI, further supporting the validity of the criterion and would suggest that similar conclusions would be drawn using different indices of ‘recovery’.
Owing to the observational nature of the study, we cannot know what might have happened if patients had been discharged earlier or later than they were. It is possible that staff knew that despite a meaningful reduction in symptoms, discharge may not be indicated for many reasons. The patient may have additional problems (perhaps not measured by the brief index) that require treatment, ongoing monitoring and supervision may be indicated (e.g. monitoring blood levels of a medication to determine an optimal dose), or additional treatment is needed to consolidate gains. On its own, an algorithm (50% criterion or other criteria) based on a self-report measure cannot direct clinical practice. Rather, treatment decisions need to continue to be made by psychiatric staff using the full array of information available. However, monitoring data do have potential to provide additional clinical information to staff as they continually monitor patients and decide upon the optimal time for discharge.
Further research is required to corroborate the present findings and to provide further evidence for utility of self-report-based criteria in informing decision-making around discharge readiness. Future research may aim to examine whether patients who had improved were ready for discharge upon reaching the criterion, and whether those who do not reach the criterion benefit from reassessment and/or changes in treatment. Although the 50% criterion has been used previously in psychotherapy and pharmacotherapy research and performed similarly to the RCI, the decision to use a single criterion of 50% reduction remains somewhat arbitrary and may not represent the optimal criterion indicative of meaningful change. Further studies are required to determine whether other criteria have increased validity as indicators of meaningful change. In addition, this study used a single self-report instrument, and it is possible that the performance of the 50% criterion as an indicator of meaningful change may vary across instruments and/or patient groups (e.g. patients with different diagnoses). The sample was restricted to patients who had completed at least three daily symptom questionnaires, and findings may not generalise to patients with shorter LOS. The study used a single five-item measure of distress, which is beneficial due to ease of completion and scoring, but may not capture the most useful symptoms associated with discharge readiness. Although there is evidence for validity of the DI-5 in daily monitoring of distress in clinical samples,Reference Dyer, Hooke and Page 18 the information provided by a single five-item instrument is likely to be limited. Thus, future research may further investigate the utility of using a greater range of measures and symptoms. It is also noted that the sample in this study predominantly comprised patients with affective and neurotic disorders; hence, the questionnaire primarily assessed these symptoms. Although affective and neurotic symptoms, as well as suicidal ideation, are likely to be important symptoms to capture in various settings, other questionnaires and symptoms may also be relevant for monitoring in other in-patient units comprising different diagnostic case mixes (e.g. higher incidence of psychotic disorders).
In summary, it is evident that provision of information to staff about patient progress during treatment leads to reductions in rate of clinical deterioration.Reference Newnham, Hooke and Page 27 The current findings highlight the potential utility of monitoring symptoms. The important question raised by these findings is, are staff decisions improved by feedback of this information? Since provision of data about patients being not on track in terms of their treatment trajectory appears to help clinicians identify and respond to potential treatment failures,Reference Newnham, Hooke and Page 27 , Reference Dyer, Hooke and Page 28 the present data hold promise for assisting staff making decisions about the optimal duration of hospital stay.
Funding
A.C.P. and G.R.H. acknowledge the support of Moira Munro and the Australian Research Council grant LP150100503.
Open access
eLetters
No eLetters have been published for this article.