Use of a routine mental health measure in an adolescent secure unit

Summary

We examined the use of a staff-completed measure, the Health of the Nation Outcome Scales for Children and Adolescents (HoNOSCA), to record mental health problems in adolescents in local authority secure accommodation. It proved possible to train staff and implement completion of the HoNOSCA on 64 consecutive admissions. Interrater reliability was high. The HoNOSCA identified high levels of psychological problems on admission (mean 18.5, s.d.=5.5). Follow-up HoNOSCA ratings proved sensitive to change; however, correlation between HoNOSCA and adolescent-completed questionnaires was poor. We concluded that HoNOSCA can be helpful in documenting mental health problems among young people admitted to secure local authority units.

The number of young people placed in secure accommodation because of their severely disturbed or disruptive behaviour has steadily increased in the UK (Department of Health, 1999). Mental health problems contribute to disturbed and antisocial behaviour in these young people (Kroll et al, 2002; Gosden et al, 2003). However, their case management is primarily provided by professionals from social and education services, and the young people's mental health needs are thought to be inadequately recognised, understood and met (Kurtz et al, 1998). There is a case for the development of brief, specific mental health assessment tools that can be taught easily, for everyday use in these units.

The Health of the Nation Outcome Scales for Children and Adolescents (HoNOSCA; Gowers et al, 1999) is a reliable, useful instrument for multidisciplinary child and adolescent mental health service teams to use in routine monitoring of main clinical features and outcomes of service use (Gowers et al, 1999; Garralda et al, 2000). We examined the feasibility of using the HoNOSCA in a local authority secure unit to describe psychiatric morbidity, and documented interrater reliability, concurrent validity and sensitivity to change.

METHOD

The study was conducted in a secure residential complex managed by social services for young people aged 13–16 years admitted either through the welfare system (children at significant risk because of problems such as persistent absconding from children's homes) or through the criminal justice system. The head of the unit oversaw a multidisciplinary staff with backgrounds in education, health, social services and the criminal justice system. Two psychiatrists and a clinical psychologist provided mental health support. The main researcher (P.Y.) trained staff in the use of the instrument through small group sessions and individual discussion with staff members using clinical vignettes. He collected background demographic and mental health information from existing documentation using the Paddington Complexity Scale (Yates et al, 1999).

All young people admitted over a 24-month period had HoNOSCA assessments completed by their keyworkers upon arrival. This measure scores 13 commonly occurring clinical features on a five-point severity scale. Total clinical, individual item and specific group sub-scores are derived. To assess sensitivity to change, given that the mean duration of stay was 6 months, follow-up HoNOSCA ratings were completed 3 months and 6 months later; comparative analysis of pooled sample data at these three time points was carried out. For reliability purposes, in a subsample of adolescents HoNOSCA was rated by two members of staff on admission.

To assess rating congruence between the HoNOSCA and adolescent-assessed symptoms, young people were requested to complete two questionnaires: the Moods and Feelings Questionnaire (MFQ; Wood et al, 1995) to assess depression and the Strengths and Difficulties Questionnaire (SDQ; Goodman et al, 1998) to screen for behavioural and emotional symptoms.

RESULTS

Demographic data

Most of the young people (56 out of 64; 88%) were admitted through the welfare system and the rest were criminal admissions (on remand or sentenced). Their median age was 14 years (range 10–17). There was a predominance of males (43; 67%), White ethnicity (59; 92%) and single-parent families (58; 91%); the household bread-winner was unemployed in half the cases. Approximately a fifth (12) were admitted from another institution. Virtually all (60) had evidence from existing documentation of psychiatric disorder, with conduct disorder being the most common primary diagnosis (46; 72%). Most disorders were rated as severe or extreme in severity, and two-thirds of cases had evidence of comorbidity. Over half the children had previously recognised learning difficulties. A fifth were in special schooling and a tenth were not attending school. Approximately half had previous contact with child and adolescent mental health services (CAMHS) and with the youth justice system. Most had also had contact with social and educational services: 60 (94%) and 50 (78%) respectively. Young people remaining at the unit for 3 months (n=39) and 6 months (n=29) were comparable with the initial sample on socio-demographic and clinical features.

HoNOSCA findings

Training of staff was well received and staff completed HoNOSCAs on 64 consecutive admissions. Two completed the admission HoNOSCA for the same 18 young people, except for two items (hallucinations/delusions and family life/relationships); all showed good interrater reliability (Pearson correlation α=0.6–0.9, all P<0.02).

On admission the mean total HoNOSCA score was 18.6 (s.d.=5.5) and nearly 60% of young people had a high score (4) on at least one item (see data supplement to the online version of this paper). Highest item scores were for disruptive, antisocial/aggressive behaviour; overactivity/attention deficit; and peer and family life relationships. There were statistically significant reductions in total HoNOSCA scores at both follow-up assessments, behaviour and social function group sub-scores changing first and most markedly. The symptoms group sub-score took longer to change and this was only significant at 6 months. The proportion of high scorers (any score of 4 or more) decreased from 58% at admission to 17% at 6 months (χ² test, P=0.0006).

Correlation between staff ratings and adolescent-completed questionnaires

Sixty-one adolescents completed the SDQ on admission. With a mean score of 20.5 (s.d.=6.5), abnormal total scores were present in 35 (57%) of this group, mainly accounted for by high behavioural and hyperactivity scores. Mean MFQ scores (n=62) were 14.6 (s.d.=5.3); one respondent (1.6%) was at risk of depressive disorder according to the cut-off score specified by Wood et al (1995). Reductions in SDQ total, emotional, conduct and hyperactivity scores were statistically significant at 3 months and 6 months, most markedly for behavioural and hyperactivity scores, with no significant change in peer problems or pro-social scores. Change in MFQ scores at 3 months and at 6 months approached statistical significance.

The HoNOSCA scores on admission showed poor and non-statistically significant correlation (Spearman) with SDQ total score (r=0.006) and group sub-scores (emotional, r=0.070; conduct, r=0.049; hyperactivity, r=–0.128; peer relationship, r=–0.094; pro-social, r=–0.114). Correlation with MFQ scores on admission was also poor (Spearman r=0.093). Individual item and group sub-scale HoNOSCA scores showed equally poor correlations with MFQ and SDQ scores; similarly poor correlations were seen with 3-month and 6-month HoNOSCA scores and SDQ and MFQ change scores.

DISCUSSION

Our study demonstrates that implementation of the use of HoNOSCA in a social services secure unit is feasible with appropriate training, and that HoNOSCA can be used reliably and is sensitive to change in this setting. However, correlation with adolescent-completed psychological measures was poor.

Mental health problems

The HoNOSCA proved useful in identifying high levels of mental health problems in young people. Total scores were higher than those reported for specialist CAMHS attenders (Yates et al, 1999) but were comparable with those found in adolescent psychiatric units (Gowers et al, 2002). Hyperactivity scores were high on both the HoNOSCA and the self-completed questionnaires: in some cases this would reflect hyperkinetic disorder which is potentially responsive to medical treatment. The use of HoNOSCA may thus help in the identification of mental health problems in a population with high levels of need.

Reliability and sensitivity to change

We achieved acceptable reliability in staff ratings of HoNOSCA and significant changes at follow-up indicative of improvement in the severity of mental health problems, in line with previous reports in populations of boys admitted to secure care for serious or persistent offending (Kroll et al, 2002).

Congruence with adolescent symptom ratings

The poor correlation between the professionally rated HoNOSCA and adolescent-rated measures of mood symptoms and behaviour is in line with findings by Gowers et al (2002). We considered whether the adolescents might have been not adept at completing questionnaires, as many had learning difficulties. However, consistency of adolescent scoring was suggested by acceptable correlation between mood symptoms on the MFQ and on the emotional sub-scale of the SDQ (Spearman's correlation, r=0.4, P<0.0001, n=61). Although the findings are in line with the long-established modest agreement between adult (normally parental) and adolescent mental health symptom reporting, further work to examine the reasons for and implications for differences in views by staff and adolescents is called for.