Meta-Analysis on the Health Effects Resulting from Evacuation or Relocation

Abstract

Objective:

Evacuation and relocation are key actions used to protect the public in response to natural or technological disasters, but there are inherent risks to both. Unfortunately, these risks have not been fully quantified, which limits the ability of emergency managers and the public to effectively balance the risks and benefits of evacuation or relocation. This work provides quantitative data on the risks of health effects from displacement following evacuation or relocation.

Methods:

Researchers performed a literature review and meta-analysis of published studies and quantified risks of 14 different health effects, including both physical and socio-behavioral outcomes, from studies of 9 different disaster types.

Results:

The findings show statistically significant increases in 9 of the 14 health effects in displaced populations, indicating an increased likelihood of experiencing detrimental health effects compared with nondisplaced populations. A pooled analysis of all negative health effects found an odds ratio of 1.49 (95% confidence interval: 1.24-1.79), which shows a significant relationship between displacement and negative health outcomes.

Conclusions:

These findings demonstrate that evacuated or relocated populations have an increased risk of experiencing negative health effects associated with displacement. The broad number of disaster types included mean that findings are applicable to any emergency evacuation or relocation.

Evacuation (temporarily displacing populations) and relocation (displacing populations for typically 1 or more years) are risk-mitigating strategies used by emergency managers to move populations away from hazards. ¹ These protective actions can be taken on small or large scales (eg, within individual buildings or entire cities) as required by the prevailing circumstances. Although evacuations and relocations are frequently used to protect the population from harm, these protective actions are not without their own risk of consequences. When deciding upon action to protect the public, emergency managers must balance the risks of the hazards against those posed by evacuation or relocation. Unfortunately, the health effects associated with evacuation or relocation on displaced populations are not well understood, complicating the overall process of risk-informing protective action decision-making.

In many cases, the risk posed by the hazard itself far exceeds the risk of evacuation or relocation. This is particularly true for a hazard that is immediately dangerous to life and health. For example, when a community is directly and imminently threatened by a wildfire, it would be a grave mistake not to move the population out of harm’s way. However, the balance between the hazard and the response to the hazard is not always clear. Risk adverse decision-makers may be tempted to evacuate even when the likelihood of an emergency event is low, or to evacuate a larger area than necessary out of an abundance of caution; however, recent studies have shown that this type of thinking may not be appropriate. ^2–4 Furthermore, studies of the 2011 Fukushima Daiichi triple-disaster involving an earthquake, tsunami, and nuclear power plant accident in Japan, have highlighted that there were significant harms to both the physical and mental health of people displaced from their homes ^5–7 ; in contrast, no significant radiological harm has been reported.

This study provides insights into the health effects associated with displacement. Published literature describing both physical and socio-behavioral health effects in displaced populations were analyzed using meta-analytic methods. This analysis was designed to include a broad range of health effects and event types and is discussed here with a focus on its significance and implications for emergency managers and decision-makers.

Methods

Data for this effort were collected from a literature review. The literature collection targeted studies examining health effects experienced by displaced populations following a variety of disaster-related evacuations and relocations. Studies for review were identified by searching Scopus and PubMed with a series of keyword strings and by a review of citations from relevant studies to identify related publications. The review of citations of relevant studies created an iterative cycle where new keyword strings were generated or refined by the results of the previous searches. While the literature search was not a systematic review of literature, this method allowed for the identification of high quality, widely cited, keystone, and influential studies. The Scopus and PubMed search used broad terms in search strings alongside more specific terms to capture studies pertaining to specific populations, health outcomes, or emergency events. In addition, search strings of non-specific singular terms, such as “hurricane” or “evacuation” were avoided, because these searches provided enormous amounts of nonrelevant literature. Specific search strings used are listed below:

• “disaster”+“evacuation”+“risk”

• “diabetes”+“disaster”+“evacuation”

• “disaster”+“evacuation”+“depression”

• “hurricane”+“risk”+“evacuation”

• “wildfire”+“risk”+“evacuation”

• “flood”+“risk”+“evacuation”

• “terrorist”+“risk”+“evacuation”

• “bomb”+“risk”+“evacuation”

• “volcano”+“risk”+“evacuation”

• “war”+“risk”+“evacuation”

• “chemical”+“risk”+“evacuation”

• evacuation associated accidents

• hurricane Rita evacuation

• “earthquake”+“risk”+“evacuation”

• ((Evacuee) NOT Fukushima) NOT Japan

• disaster evacuation relocation

• “Black Saturday” fire

• 2004 hurricane displ*

• hurricane AND relocation

• earthquake AND relocation NOT Japan

• 2007 England floods

• Wildfires AND (evacu* OR displaced OR relocate)

• “disaster”+“relocation”+“risk”

• “disaster”+“relocation”

• Botchway 2019 Citation Review

• Pfefferbaum 2017 Citation Review

• TITLE-ABS-KEY (“vehicle” AND “evacuation” AND “fatal*”) AND (LIMIT-TO (DOCTYPE, “ar”) OR LIMIT-TO (DOCTYPE, “re”))

• hospital AND morbidity AND evacuation AND disaster

• TITLE-ABS-KEY (evacuation AND mortality AND disaster)

Studies of various disaster types and locations, health effects, and study design were included in the initial collection process. The collected studies covered a time period from 1977 to March 2020. ^8,9 The literature search identified a total of 1210 unique, potentially relevant studies. Although the search strings were carefully chosen, not all studies returned by the search were useful for further analysis. The research team initially evaluated every study for relevancy based on the title and abstract using several inclusion criteria:

• Study was available in English.

• Study reported a statistical analysis of health effects in evacuated or relocated populations.

• Study reported that populations were displaced from their place of residence or temporary dwelling (eg, homes, hospitals, long-term care facility, but excluding building evacuations during fires).

• No unpublished studies were included in the data, nor were any authors contacted for more information.

Nonrelevant studies were discarded after the abstract review. The research team further evaluated the entire text of the relevant articles to confirm the study met the inclusion criteria above and by estimating the quality of the study’s methodology using the Newcastle-Ottawa Scale (NOS) system. ¹⁰

Of the 1210 studies originally identified, 235 were determined to be relevant based on title and abstract. Twenty-six studies were excluded as they were found to examine the same group and effect as other studies, that is, both papers studied the same population evacuated from the same disaster and produced duplicative data. Based on the examination of the specific statistical data presented in the studies, 127 studies were identified that contained useful qualitative insights, but were excluded from the quantitative analysis for 1 or more of the exclusion criteria listed below:

• Study focused on emergency incidents related to terrorism. Terrorism is designed to create disproportionate psychological and emotional reactions in both the affected and general population, creating additional confounding factors and responses that were outside the scope of this study.

• Studies where populations are not evacuated to truly safe areas. This exclusion criteria applied most often to armed conflicts in which the battlefront moved following evacuation.

• As the point of this study was to estimate the risk from evacuation or relocation on a generalized population, studies too specific to be generalized were excluded (eg, World Trade Center attack survivors who also experienced a specific hurricane, or specific groups like elderly aboriginal Taiwanese people ^11,12 ).

• Study made no clear distinction between nondisplaced and displaced populations when reporting health effects, making it impossible to determine differences in effect size.

• Study data were not usable in the meta-analysis with reported statistical data without considerable manipulation or access to original study data (eg, the effect size was reported as a least squares regression coefficient but did not involve meta-analysis of regression coefficients).

After screening, a total of 82 studies were included in the meta-analysis. Figure 1 provides an overview of the literature review and down selection process.

Figure 1. Literature review down selection process.

As described below, the 82 studies included in this analysis covered 14 total health effects, including 9 specific health effects and 5 broad health effects. Specific health effects included specific diseases or outcomes, such as posttraumatic stress disorder (PTSD) and mortality. Broad health effects included health effects studied in studies that captured many different potential symptoms or outcomes such as psychological distress or lack of health-care accessibility. Broad health effects included:

• General health effects, including health effects where insufficient data were available for their own specific category. Health effects included in this category were: worse general health, ¹³ metabolic syndrome, ^14,15 chronic kidney disease, ⁵ injury, ^9,16 hypo-high density lipoprotein cholesterolemia, ^17,18 dyslipidemia, ¹⁹ worse or much worse health status after disaster, ²⁰ and reporting medically unexplained physical symptoms ²¹ (18 studies).

• Health-care accessibility, including disruption of access to primary care, pharmacy care, and emergency care as well as any other lack of availability of health-care following an emergency event (5 studies).

• Psychological distress, ⁸ which is a health effect that captures mental anguish, including symptoms of depression or anxiety that do not necessarily rise to the level of an independent diagnosis (23 studies).

• Substance abuse, including both alcohol, medications, and other drugs (11 studies).

• Other miscellaneous including the experience of abuse during evacuation, children with reported memory problems, loss of support or social networks, and other symptoms that were referred to general practitioners following emergency department visits. These studies represented high quality studies of interesting health effects that could not be grouped with other broad or specific health effects and do not have a sufficient number of studies for separate analysis. To include these studies in the analysis, these health effects were grouped into a miscellaneous category and analyzed together (8 studies).

Specific health effects included:

• Anxiety (10 studies)

• Depression (17 studies)

• Post-traumatic stress disorder (PTSD; 17 studies)

• Diabetes (10 studies)

• Heart disease (12 studies)

• Mortality (8 studies)

• Weight problems, including any studies reporting sudden weight loss or gain, or incidence of underweight or overweight populations following evacuation/relocation (6 studies).

• Respiratory illness such as infectious respiratory disease, pneumonia, and acute bronchitis. This health effect did not include COVID-19 (as all papers used in the study were published before 2019) or lung cancer (5 studies).

• Sleep problems, such as difficulty falling asleep or inability to get enough sleep (4 studies).

In addition to the various health effects covered, studies included in the literature review covered 9 different event types, including both natural and technological disasters. While the plurality of the studies focused on nuclear power plants, these studies primarily represent data from either the Fukushima Daiichi nuclear power plant accident in 2011 (which occurred simultaneously with a major earthquake and tsunami) or the Chernobyl Nuclear Power Plant accident in 1986. Emergency event types included were:

• Hurricanes/cyclones (19 studies)

• Wildfires (6 studies)

• Nondisaster relocation events where populations were relocated from their home; for example, populations displaced as part of dam construction (2 studies)

• Nuclear power plant accidents (32 studies)

• Earthquakes (9 studies)

• Floods (6 studies)

• Explosions (3 studies)

• Earthquakes accompanied by tsunamis (3 studies)

• War (3 studies).

Meta-analysis

Meta-analytic methods were used to estimate a combined effect size of the association between displacement and each health outcome. In general, an odds ratio effect size is used to determine whether an exposure (eg, displacement) is associated with, or a risk factor for, a specific outcome (eg, PTSD). While odds ratios are well suited to this type of analysis, they cannot provide the relative risk of that outcome (eg, outcome X is 4 times more likely in the displaced population), nor can they be used to estimate the number of people affected by that outcome following an emergency event. However, pooling the odds ratios from several studies provides an estimate of the odds that a specific outcome (eg, an individual suffers from psychological distress) will occur if someone is displaced following an emergency incident, compared with the odds of the outcome occurring if someone is not displaced.

To support a meta-analysis, ideally, a study would include both a nondisplaced population (control group) and a displaced population (experimental group). In practice, many studies reviewed did not include data on both population groups, and instead only reported the results of an effect (eg, PTSD) in a displaced population. Without a control group, determining the magnitude of an effect due to displacement is difficult, although the displaced population can be qualitatively compared with other nondisplaced groups or quantitatively analyzed alongside other displaced populations in the meta-analysis. Because a study of a hurricane in Florida involves a much different type of displacement event and different population demographics than a study of a nuclear power plant accident in Japan, accounting for between-study variance was critical in developing a robust estimate of the effect. These differences were accounted for using a random effects (RE) model in the meta-analysis to estimate an overall, population-level effect size for each of the 14 health outcomes. The RE model was chosen because it explicitly assumes error due to sampling (within-study variance) and due to differences between study design or study populations (between-study variance). ²²

Combined odds ratio effect sizes were calculated using computed and extracted odds ratios from individual study studies, which were transformed into log-odds to better approximate the sampling variance. An empirical Bayes method was used to estimate between-study variance with individual effect sizes weighted according to their inverse variance (a measure of within-study variance). The RE model calculations were conducted and final estimates were visualized with forest plots using the metafor package (version 3.0.2) in R statistical software.

Results

Table 1 shows a summary of the odds ratio meta-analysis for each health effect along with the respective 95% confidence intervals around each estimated effect size. Each of the estimated odds ratios examines the likelihood that a displaced population experiences a health effect compared with populations that did not evacuate or relocate. The odds ratio effect size was greater than 1.0 for all 14 of the health effects examined and statistically significant (at the 0.05 level) in 9 of the 14 meta-analyses. An odds ratio greater than 1.0 indicates a greater likelihood that displaced individuals experience these negative health outcomes compared with nondisplaced individuals.

Table 1. Summary of odds ratio meta-analyses for each of the fourteen health effects

Significance codes: p-value < 0.001 = ***; < 0.01 = **; < 0.05 = *, <0.1 = ~

Limitations

There are 2 major limitations to this study, 1 related to the underlying data and another related to interpretation. These data are a sample of studies exploring health effects in populations all over the world experiencing a range of different disasters. Cultural and socio-economic differences as well as differences in underlying health and access to health-care between populations are bound to create differences in outcomes, which could ameliorate or exacerbate the effect observed. Using an RE model helps mitigate this inter-study variation by incorporating an additional variance term that accounts for differences between studies, but does not completely remove it as a limitation, especially for health effects investigated in just a few studies. The second major limitation is related to the interpretation of odds ratios. While odds ratios are a valuable tool for helping identify whether a population experiences negative health effects at a higher or lower rate than another, odds ratios cannot be used to determine the number of people who might experience that outcome in either group. This inability to apply the results directly to a population limits the utility of these findings, particularly for emergency planners and managers concerned with the allocation of resources to mitigate health effects caused by displacement. Additional analysis is needed to estimate the number of people in a population experiencing each health effect following displacement.

Discussion

The overall study results, shown in Table 1, suggest that displaced populations experience these negative health effects at higher rates than nondisplaced populations, for both broad and specific health outcomes. Perhaps most notably, mortality is significantly higher in the displaced populations. While the studies underpinning the mortality meta-analysis often focused on elderly or hospitalized populations rather than the general population, this higher odds ratio for mortality is still of concern for emergency managers as it relates to protective action decisions. The high estimated odds ratio for mortality is reflected in the individual studies used in the meta-analysis, as shown in Figure 2. In these studies, displaced populations experienced higher mortality particularly in the first 60 to 90 d following displacement. It is worth noting, however, that these studies focus on evacuation of hospitals, elderly care or nursing facilities, and admittees at hospitals in receiving communities. As a result, the findings specific to mortality may not be generalizable to the entire population. Instead, the high odds ratio for mortality emphasizes the vulnerability of these populations. Additionally, the prevalence of mortality was quite low in these studies (estimated effect size of 4% [95% confidence interval {CI}: 1-9%] in displaced populations).

Figure 2. Meta-analysis of the odds ratio data for mortality.

By contrast, the odds ratio data for PTSD are around the same magnitude as the estimated odds ratio for mortality but is associated with much higher prevalence. PTSD had an estimated prevalence of 32% in displaced populations (as shown in Figure 3), compared with a prevalence of only 15% (95% CI: 8-25%) in nondisplaced populations. This finding highlights the fact that odds ratios are useful in understanding relative risks between populations, although they cannot be used to directly estimate the proportion of a population experiencing a specific health effect.

Figure 3. Meta-analysis of the prevalence of PTSD among displaced populations.

In addition to examining individual health effects, a meta-analysis was performed across all fourteen health effects included in this analysis. As duplicate study effects were excluded, the final analysis included no fewer than 3 studies per health outcome. This part of the meta-analysis found a significant association between evacuation or relocation and an increase in any negative health effect, as shown in Figure 4. The odds ratio for this overall analysis was 1.49 (95% CI: 1.24-1.79) which suggests that displaced populations experience a statistically significant increase in negative health effects overall.

Figure 4. Forest plot of the odds ratio meta-analysis for all health-effects combined. Far left column lists the author and year of publication. Center shows the centroid estimate and confidence interval. Far right columns list central estimate and confidence intervals in square brackets.

Conclusions

The meta-analysis in this research has aggregated evidence of a clear association between a range of negative health outcomes and evacuation or relocation in response to an emergency event. The estimated magnitude of these health effects in the displaced population is large, with nearly 25% of displaced populations suffering from “other health effects,” which includes increases in domestic abuse, memory problems in children, and disruption of social support networks, among others. Additionally, for displaced population, the study found statistically significant increases in 9 health effects, including depression, psychological distress, PTSD, sleep problems, and mortality, among others. While many of the health effects identified in this analysis are potentially short-lived or not disabling, these health effects still represent a considerable health burden on displaced populations.

These findings indicate that emergency managers considering evacuation, relocation, or similar protective actions must carefully balance the risks from the disaster against the potential health effects identified in this analysis. However, this comparison is complex. For example, most of the identified health effects in this analysis are not lethal, while the disaster may potentially be life-threatening. Comparing lethal and nonlethal outcomes is difficult as emergency managers must consider how transient physical or psychosocial health effects compare with permanent physical harms. While some of the health effects included in this analysis (eg, heart disease, lack of medical care) are associated with shorter lifespans, many of the health effects are acute, manifesting within days of displacement and potentially resolving on their own with time with or without additional treatment. For example, studies of PTSD showed decreasing proportions of the population experiencing PTSD symptoms as the time between the disaster and the study increased. This finding suggests that some people who experience PTSD symptoms initially after the disaster will have their symptoms resolve with time. In a similar way, the potential health effects due to the disaster or hazard itself can vary; such effects may be acute (eg, smoke inhalation or trauma) or long-term (eg, radiation-induced cancer, which develops over years or decades). Using average reduction in lifespan, quality-adjusted life-years (QALYs), or similar measures could allow for some comparison between disaster-related injuries/illness and health effects from displacement, but any such method would have its own set of limitations. Given the limitations of a direct comparison between physical and socio-behavioral injury to acute and chronic health effects, quantitative methods may ultimately be ill-suited to balancing these risks. Instead, emergency managers could qualitatively balance these risks based on their experience and whatever additional data are available to them.

Perhaps the most important conclusion of this research, however, is the fact that there are significant deleterious effects associated with the displacement of populations during disasters. When emergency managers are considering potential protective actions, evacuation and relocation should not be used purely out of an abundance of caution. Populations displaced unnecessarily may suffer serious health effects for years after the incident following initial evacuation or relocation. Evacuation decisions need to be carefully optimized to protect populations from the hazard while ensuring as few people as possible suffer harms from displacement itself.