Are community-based interventions effective at improving mental health outcomes in refugee children and adolescents in high-income countries?

Summary

Multiple epidemiological studies have shown an increased prevalence of adverse mental health outcomes in refugee populations and have highlighted children and adolescents to be particularly at risk. This commentary considers a Cochrane Review examining the efficacy of community-based interventions at improving the mental health of refugee children and adolescents in high-income countries. The review concludes that community-based interventions are ineffective at improving mental health in such populations. Notably, the data are limited by significant risk of bias and a small sample size. This article aims to critically appraise this systematic review, extrapolate implications for current practice and identify avenues for further research.

In the 1951 Refugee Convention, a refugee is defined as ‘someone who is unable or unwilling to return to their country of origin owing to a well-founded fear of being persecuted for reasons of race, religion, nationality, membership of a particular social group, or political opinion’ (UN General Assembly 1951: p. 3). By the end of 2021, the total population of forcibly displaced individuals worldwide reached 89.3 million, of whom 27.1 million were recognised as refugees (United Nations 2023). More than 50% of the refugee population worldwide are children and adolescents (United Nations 2023), who have been acknowledged to be some of the most severely affected by forced migration (Garin 2016). Although exact figures vary between populations and between studies, multiple epidemiological studies have demonstrated a greater prevalence of mental health problems and elevated risk of adverse mental health outcomes in refugee populations in comparison with host populations (Fazel 2005; Porter 2005; Silove 2017). Several factors have been associated with poor mental health outcomes among refugee populations, including exposure to traumatic events prior to migration, as well as stressors in the post-migration environment such as prolonged periods of detention, uncertain residency status, restricted access to public services, lack of educational and employment opportunities, financial strain, racial discrimination and the challenges of assimilating in a foreign country, for example, language differences and cultural barriers (Ellis 2008; Fazel 2012; Hynie 2017). This calls attention to refugee mental health as a vital area of research to identify evidence-based interventions proven to promote refugee mental health and to ensure that forcibly displaced individuals are adequately supported to enable them to thrive in their new communities.

The Cochrane Review

This month's Cochrane Corner review, by Soltan et al (2022), aimed to assess the efficacy and acceptability of community-based interventions in preventing and treating mental health conditions in refugee children and adolescents in high-income countries. The population considered in the study were individuals aged 18 years or younger, of any sex/gender or ethnicity, with or without a previous diagnosis of a mental health condition.

This systematic review considered mental health using a non-binary conceptualisation, that is, rather than defining it as simply the presence or absence of a diagnosed mental health condition, mental health was considered as a continuum. Fittingly, the study considered individuals with a formal diagnosis of a mental disorder, individuals with subclinical symptoms of a mental disorder and individuals with no symptoms of a mental disorder. In accordance with this, the interventions considered in this study included (a) interventions aimed at treating a mental health condition in those with a formal diagnosis, (b) interventions aimed at mental health promotion in those with subclinical symptoms of a mental disorder and (c) interventions aimed at prevention of mental health problems in those with no symptoms of a mental disorder. All interventions must have been delivered via community institutions, that is, structures with the inherent imperative of engagement with members of the local population, such as schools, churches or voluntary agencies. All interventions conducted in a clinical setting were excluded.

The intervention effect was determined by comparing with a control group who received (a) no treatment, (b) allocation to a waiting list, (c) psychological placebo, that is, an intervention that was regarded as active treatment by participants but did not qualify as such by the research group, or (d) standard care. The review authors did not give details on what standard care entailed and this requires clarification. The primary outcome measures to assess the efficacy of the interventions in question were (a) self-reported symptom severity for post-traumatic stress disorder (PTSD), anxiety disorders, depression and psychological distress, and (b) adverse events, for example counts of self-harm and suicide. Adverse events were used as a proxy for intervention acceptability, that is, the extent to which participants were satisfied by the intervention and willing to engage with it. This is not a standard measure of acceptability, which is often determined from ‘treatment discontinuations due to any cause’, whereas the ‘type and number of adverse events’ usually measure the safety of an intervention or its tolerability (in terms of ‘treatment discontinuations due to adverse events’). Several other secondary outcomes were considered alongside this but we do not discuss them here since this would go beyond the scope of this commentary.

Method

To collate data, the review authors performed an extensive search of academic sources, including 10 databases and 2 trial registries, from which 38 studies were thought suitable for inclusion. Considerable efforts were made to comprehensively retrieve relevant data, including checking reference lists of the included studies and inquiring about unpublished or ongoing trials to identify any further studies that were suitable for inclusion. This provides a broader evidence base for the meta-analysis and reduces publication bias (Box 1). To further reduce bias in this respect, the authors should have performed a thorough search of grey literature in addition to academic sources.

BOX 1 What is publication bias?

In statistical models, we use a sample from a population to derive estimations about the whole population. There is always the possibility of a difference between the derived estimation and the true value, owing to random chance. ‘Bias’ describes any feature of the statistical model that may lead to variability between the derived estimation and true value that is not a product of random chance.

Publication bias refers to the selective publication of data on the basis of their findings. For instance, trials that demonstrate a significant treatment effect may be more likely to be published, skewing the overall impression within that field.

Initially, studies of any design were considered, and at a second stage this was narrowed down to randomised controlled trials (RCTs) only. Evidence on the efficacy of interventions was derived from RCTs alone, thereby giving the most reliable evidence about treatment effects. Where cluster RCTs (Box 2) were considered, the data were adjusted for intra-cluster correlation. Where cross-over RCTs (Box 2) were considered, only data from the first cycle were included, that is, before each group received a second intervention, to avoid carry-over effects.

BOX 2 Cluster versus cross-over RCTs

A randomised controlled trial (RCT) is the gold-standard experimental design to prove the efficacy of an intervention in comparison with a control group. Participants are randomly allocated (often with single or double masking of allocation) to either the intervention group or control group and followed up to measure treatment effect. This helps to minimise selection bias.

A cluster RCT describes an experimental design in which pre-existing groups or clusters of participants (such as school classes or church groups) are randomly allocated to either the intervention group or control group. This may occur when it is not possible to randomly allocate participants on an individual basis. Cluster randomisation can introduce an element of bias within the trial design.

In a cluster RCT the intracluster correlation coefficient (ICC) describes the extent to which the outcomes of the participants within a cluster resemble each other. For example, if a group of 100 school children are divided into four classes or clusters with an ICC of 0, the resemblance between students within each cluster is no greater than that within the total population of school children. This means that the trial design would be no different than if each school child had been independently allocated to either the intervention or control group as opposed to within their clusters. With an ICC of 1, the outcome for any school child completely resembles that of any other school child within their respective cluster. This means that each cluster acts as a single data point as opposed to 25 individual data points.

A cross-over RCT describes an experimental design in which all participants within the trial receive all interventions, but the order in which they are received is randomised. The carry-over effect describes the effect of a previous treatment on the efficacy of a current treatment.

Baseline and end-point data, or change from baseline data, were collected for each RCT. If trials used the same continuous measure for comparison of treatment effect, data were collated and the mean difference (m.d.) was calculated (Box 3). If trials used different measures to assess the same outcome, data were collated and the standardised mean difference (s.m.d.) was calculated (Box 3). A standardised mean difference <0 would indicate that the intervention was more successful at reducing symptom severity in comparison with the control intervention. The lower the score the greater the treatment effect. Standardised mean differences could also be translated to reflect the units of a commonly used scale for the outcome measure in question, for example the Children's Depression Inventory as a measure of symptom severity. To accommodate for trials of varying length, studies were subcategorised into the following: (a) short term: up to 6 months; (b) medium-term: 6–12 months; and (c) long-term: more than 12 months.

BOX 3 Mean difference and standardised mean difference

The change from baseline data between the intervention and control groups within an RCT can be compared to produce a mean difference (m.d.). When different RCTs consider outcomes with the same units, the mean differences can be pooled in a meta-analysis.

When different RCTs consider outcomes with different units, the mean differences cannot be pooled in their raw form. Instead, each m.d. value can be divided by its standard deviation (s.d.) to calculate a standardised mean difference (s.m.d.). Therefore, the data from multiple RCTs can be presented as the product of their standard deviations, and the resulting uniformity in units means that the data can be pooled in a meta-analysis. An s.m.d. of 0 means that there is no treatment effect, whereas an s.m.d. of 0.2–0.5 is considered a small treatment effect, an s.m.d. of 0.5–0.8 is considered a moderate treatment effect and an s.m.d. > 0.8 is considered a large treatment effect.

The risk of bias in the included RCTs was assessed using Cochrane's RoB 2 tool, which considered bias in the following five domains: (a) the randomisation process, (b) deviations from intended interventions, (c) missing outcome data, (d) outcome measurement and (e) selection of the reported result (Sterne 2019). A study was considered at low risk of bias overall if it was classified as low risk with respect to all five of these domains. A study was considered at high risk of bias overall if it was classified as at high risk of bias with respect to one domain or at moderate risk for multiple domains.

The review authors planned to assess heterogeneity by using the χ² test (with a P value of 0.1) and the I ² statistic, which calculate the degree to which variability between the control group and intervention group is attributable to chance. Additionally, they intended to create forest plots to visually present the data. The certainty of the available evidence was categorised as ‘high’, ‘moderate’, ‘low’ and ‘very low’ using the GRADE approach (Box 4) (Higgins 2023).

BOX 4 The GRADE approach

The Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach is a means of evaluating the certainty of evidence, namely, our confidence that the true effect of an intervention lies close to the estimated effect in a study. The GRADE approach assesses certainty of evidence using five domains (Higgins 2023; Siemieniuk 2023):

• • Risk of bias: this refers to an evaluation of inherent limitations of study design, which can be assessed using instruments such as RoB 2 (Sterne 2019).

• • Inconsistency of effect: this refers to an evaluation of heterogeneity between studies, which can be assessed using the χ² (chi-squared) test and I ² statistic.

• • Imprecision: this refers to an evaluation of the 95% confidence interval, that is, the range of values that is likely to contain the true effect of an intervention with 95% confidence. Certainty may be reduced: (a) if the clinical conclusion would be different if the true effect was at the upper versus the lower range of the confidence interval, (b) if the 95% confidence interval includes a relative risk of 1.0, (c) if the total sample size is small and (d) if the total number of events is small (Guyatt 2011).

• • Indirectness: this refers to an evaluation of the applicability of data to the clinical question in focus. Evidence may be considered to be indirect if the patient group, intervention or outcome measure differ from that of specific interest to the clinical question. Indirectness may also be bred if intervention effect is compared without direct comparison, for example if data from Intervention 1 versus Intervention 3 and from Intervention 2 versus Intervention 3 is used to extrapolate conclusions regarding Intervention 1 versus Intervention 2 (Rasch 2012).

• • Publication bias: see Box 1.

Results

Two cluster RCTs (Ooi 2016; Walg 2020) and one cross-over RCT (Baker 2006) were identified from the 38 studies found from the original literature search, providing data for a total population of 83 children and adolescents. Each study compared a community-based intervention with a waiting list. The primary outcomes considered across the three RCTs were (a) symptom severity of PTSD, (b) symptom severity of depression and (c) severity of psychological distress. No studies reported data on the primary outcomes of (a) symptom severity of anxiety and (b) adverse events such as self-harm or suicide counts. The meta-analysis showed (a) no evidence of a difference in severity of symptoms of depression between intervention and control groups at 3 months post intervention, with an associated moderate risk of bias, (b) no evidence of a difference in severity of symptoms of PTSD between intervention and control groups at 3 months post intervention, with an associated moderate risk of bias and (c) no evidence of a difference in severity of psychological distress between intervention and control groups directly following the final intervention session, with an associated high risk of bias. Therefore, the meta-analysis showed no evidence to support the efficacy of community-based interventions in preventing mental health problems and promoting mental health in refugee children and adolescents in high-income countries, although the certainty of the evidence remained low to very low.

Discussion

The main limitation of this systematic review is the small sample size, with only three RCTs and data for a total population of 83 participants available, in conjunction with the absence of data for two of the primary outcomes highlighted by the review authors, namely symptom severity of anxiety and occurrence of adverse events. Additionally, the data that were available were thought to be of poor quality, with the certainty of evidence classified as low to very low. The authors made considerable efforts to collate data as fully as possible, identifying several ongoing studies that can be considered when this systematic review is updated. Notably, where studies considered a population of a different age bracket, study authors were contacted to extract data concerning individual participants who were 18 years or younger. Studies in which 75% of the participants were aged 18 years or younger were thought suitable for inclusion, although it is unclear why a figure of 75% was chosen. Despite all efforts, the scarcity of available evidence highlights the need for further research in this important field.

In addition to this, this meta-analysis considered participants both with and without a previous diagnosis of a mental health condition. For example, one RCT considered participants with subclinical PTSD symptoms and excluded participants with a formal diagnosis of PTSD (Ooi 2016). The remaining two RCTs did not specifically include or exclude participants on the basis of the severity of their mental health symptoms at baseline (Baker 2006; Walg 2020). The interventions considered across the three RCTs included those aimed at either prevention or active treatment of mental disorders. Symptoms of a mental disorder in the absence of a formal diagnosis may be a reactive, short-term product of traumatic pre-migration events and stressors in the post-migration environment. Contrastingly, a diagnosed mental disorder is, at least partly, considered to be self-standing and more likely to persist once the initial stressor has been removed (Silove 2017). Individuals with a diagnosed mental disorder may be more likely to require more structured therapy and/or pharmacological treatment based in a clinical setting (Silove 2017). Therefore, combining data for participants with no symptoms of a mental disorder, subclinical symptoms and a diagnosis of a pre-existing mental health condition may underestimate the efficacy of community-based interventions. This meta-analysis would benefit from a subgroup analysis to assess whether the efficacy of the interventions varies depending on the severity of symptoms at baseline. However, this highlights the current scarcity of data, since two of the three RCTs did not comment on baseline symptom severity (Baker 2006; Walg 2020). Additionally, all three RCTs measured treatment effect either immediately following the intervention or at 3-month follow-up. Considering only short-term studies may not capture the chronicity of a diagnosed mental disorder and therefore may not accurately represent long-term outcomes.

Furthermore, this review considered populations from multiple different cultural backgrounds, introducing an element of transcultural measurement error. Different populations have been shown to express psychological distress in different ways (Chang 2008; Zhou 2020). Particularly, certain populations may report a lower symptom severity for a comparable level of psychological distress, owing to social stigma pertaining to mental illness in their native country (Bharadwaj 2017; Radez 2020; DeSa 2022). Therefore, an accurate measure of psychological distress would require scales of symptom severity that have been adjusted to represent local expressions of psychological distress.

Implications for practice

This meta-analysis shows no evidence to support the efficacy of community-based interventions at preventing mental health problems and promoting mental health in refugee children and adolescents in high-income countries. However, notably, the certainty of the evidence is low and the study is limited by a small sample size. In view of this, a change to current practice cannot be recommended on the basis of these data. Nonetheless, this study reveals the absence of data in this field and highlights a key area for further research to identify and implement evidence-based interventions to promote the mental health of refugee children and adolescents.