Rising sea levels and extreme weather events are already impacting people all over the world and are predicted to continue. 1 The Intergovernmental Panel on Climate Change warns that low-lying coastal areas are increasingly vulnerable to the effects of sea level rise, storm surge, flooding, and extreme storm events. 1 Such potential for disasters in the future necessitates the need for emergency preparedness at all levels, to include the individual/household level. Despite the lack of evidence-based emergency preparedness recommendations, Reference Heagele2 researchers endeavor to determine whether emergency preparedness education regarding what community members need to obtain to be prepared for emergencies can improve preparedness levels Reference McNeill, Alfred and Mastel-Smith3 or whether such emergency preparedness really matters. Reference Clay, Goetschius and Papas4 Hurricane Katrina decimated New Orleans in August of 2005; the impacts are still felt today. Reference Gibbens5 Superstorm Sandy devastated New York City (NYC) in October 2012, Reference Benimoff, Fritz and Kress6 Hurricane Maria demolished Puerto Rico in 2017, Reference Pasch, Penny and Berg7 and Hurricane Florence destroyed areas of eastern North Carolina in 2018. 8 These events point toward an underlying need to create an evidence-based instrument to measure household emergency preparedness (HEP), guide education efforts to improve HEP, and ultimately determine the impact of HEP among community-dwelling residents.
A Delphi study was recently completed for the purpose of generating a consensus on the concept definition of HEP from experts and community stakeholders representing multiple disciplines and countries to develop an all-hazards, valid and reliable Household Emergency Preparedness Instrument (HEPI). Reference Heagele, McNeill and Adams9 Delphi participants came to a consensus that adequate HEP is defined as completion of several preparedness actions in addition to assembling a home disaster supply kit that would enable households to endure the common conditions that disasters present, regardless of location. Reference Heagele, McNeill and Adams9 These common conditions include power outages, deficiencies with drinking water, and being forced to shelter in place for a few days. Reference Heagele, McNeill and Adams9
The creators of the HEPI defined the construct of HEP and developed the initial instrument questions while establishing evidence to support face, content, and criterion validity of the instrument. Reference Heagele, McNeill and Adams9 The HEPI - Delphi yielded a 51-question instrument built through qualitative content analysis and group consensus achieved with 3 rounds of data collection. Scaling of the original Delphi instrument was based on similarity of items as presented by the expert panel before pilot-testing in a general population. The next step of instrument development is to administer the instrument to a representative sample, perform psychometric testing on the instrument to establish construct validity, and make revisions if necessary. The purpose of this cross-sectional study was to pilot test the HEPI to perform psychometric testing and generate reliability and construct validity data. A second purpose was to measure and compare the HEP levels of a convenience sample of faculty, staff, and students from an urban university that is comprised of a socially vulnerable population in a physically vulnerable geographical location.
New York City is the most populous city in the United States (US) and is characterized by diverse people, neighborhoods, landscapes, and aging infrastructure. The city is vulnerable to coastal storms (erosion and flooding), rising sea levels, high winds, high heat, winter weather, earthquakes, fires, chemical/biological/radiological/nuclear incidents, emerging infectious diseases (epidemics and pandemics), and the approximately 22 million people are heavily dependent on energy, public transportation, water, and wastewater treatment. 10
Emergency management agencies in NYC began to aggressively implement emergency preparedness educational interventions after the 9/11 attacks in 2001. Reference Martins, Louis-Charles and Nigg11 Martins et al. analyzed data about the emergency preparedness levels of 2,001 NYC residents collected in the months before the 2012 Superstorm Sandy. Reference Martins, Louis-Charles and Nigg11 They found that NYC residents engaged in emergency preparedness activities at a greater rate than community members in most geographical locations in the United States. The NYC residents were considered well prepared for disasters, particularly with gathering supplies and developing a family communication plan. Martins et al. attribute these high preparedness levels to high levels of social capital, trust in the government, and community emergency preparedness educational interventions. Reference Martins, Louis-Charles and Nigg11 Some of these community interventions include: (a) training health-care providers (HCPs) about how to educate their medically vulnerable patients about emergency preparedness resources 12 ; (b) the Ready New York and Know Your Zone public education campaigns 13 ; (c) the Notify NYC emergency alert system for risk communications 13 ; (d) the NYC Community Emergency Response Teams and Community Organizations Active in Disasters teams of community volunteers who assist their neighbors before and during disasters 13,14 ; (e) training community- and faith-based organizations about how to prepare their employees, services, and facilities for emergencies 13 and; (f) free HEP classes with distribution of disaster supply kits for participants. 15
Assessments of college and university campus preparedness levels and plans are abundant. Reference Abedin and Shaw16–25 However, little is known about the emergency preparedness levels among college students. College students are seldom considered a vulnerable population even though they are living in large institutions and are often geographically separated from families that provide important instrumental and social support. Reference Claborn26 Less than half of students receive emergency preparedness information or training during student orientation. Reference Cheung, Basiaga and Olympia17 Although little research has been done to determine the preparedness levels among college students, what studies have been done demonstrate that college students are unprepared for disasters. Reference Claborn26–Reference Tkachuck, Schulenberg and Lair29
The City University of New York (CUNY) is the third-largest university system in the United States, in terms of enrollment, with more than 274,000-degree-credit students, continuing, and professional education students enrolled at 25 campuses located in all 5 NYC boroughs. The university has 1 of the most diverse student bodies in the United States, with students hailing from 208 countries, but mostly from NYC. 30 As of fall of 2019, the race/ethnicity of all CUNY students was 0.3% American Indian/Alaska Native, 21.2% Asian/Pacific Islander, 25.2% Black, 30.2% Hispanic, and 23.1% White. 31 Undergraduate enrollment in fall 2019 consisted of 97.2% under the age of 45 and 2.7% were 45 or older. 32 As of fall 2016, 34.5% of CUNY’s approximately 7600 full-time faculty were from underrepresented groups. Reference Torres33
Recruiting CUNY faculty, staff, and students as a convenience sample for this pilot study provided a large, diverse, and possibly globally representative group of participants. Culture bias is manifested when an instrument is not conceptually equivalent for cultural subgroups, resulting in different psychometric properties for these groups. Reference Waltz, Strikland and Lenz34 The assumption of universal applicability of standardized instruments normed on particular cultural or racial/ethnic majority population needs to be tested. Reference Ramirez, Ford and Stewart35 The diverse sample obtained from CUNY faculty, staff, and students was ideal to evaluate culture bias of the HEPI.
Methods
This pilot study received institutional review board (IRB) approval from Hunter College, CUNY (protocol #2019-1037). The participants’ consent was implied by completing the questionnaire. The participants were not compensated for their participation.
No specific race, ethnicity, gender, socioeconomic status, employment status, marital status, citizen status, or education level was targeted for recruitment. Inclusion criteria was noninstitutionalized, English-reading, adults who identified as either a CUNY faculty, staff, or student. Exclusion criteria was persons less than 18 y of age and non-CUNY faculty, staff, or students.
Given that this was a pilot study, assuming a simple design of 1 population self-classified into 2 groups based on responses to the survey (prepared or not prepared), the minimum sample size was calculated as N > 50 + 8m (where m is the number of groups), which is the common method for determining sample sizes in this scenario. Reference Green36 In this case, m = 2 (prepared or not prepared) so the minimum sample size required per group was 66 participants. Based on potential missing data of approximately 10%, the minimum sample size sought for each group was 75 participants, or 150 participants total. Nine of the 25 CUNY campuses granted permission for the researchers to recruit participants from their campuses. Deans and department heads chose whether or not to forward the IRB-approved recruitment and consent script by means of e-mail and served as gatekeepers for recruitment. As a result, varying numbers of CUNY faculty, staff, and students on participating campuses received the link to the survey.
Data were collected using Qualtrics online survey software version February 2020 (Qualtrics, Provo, UT) to distribute the HEPI. Reference Heagele, McNeill and Adams9 The HEPI is an all-hazards, comprehensive, 51-question instrument used to ascertain if a respondent is prepared for disasters. There is support for face, content, and criterion validity of this instrument. Reference Heagele, McNeill and Adams9 The HEPI questions are objective and ask about what the respondent presently owns or does in a dichotomous format. As this was a pilot test of the new instrument, virtual focus group-style questions were asked at the conclusion of the HEPI. Participants were asked to rate the perceived clarity of the HEPI instructions and questions, the difficulty of the HEPI, and the reasonableness of the length of the HEPI. In addition, 1 open-ended question was included to inquire about anything the participant would like the researchers to know about the experience of completing the HEPI. Open-ended interview questions provide the participants the opportunity to describe their experience in greater detail. Reference Streubert and Carpenter37 Focus groups representative of the intended respondents are useful for (a) developing insights into lay-person perceptions of HEP, (b) informing authors about the language and terminology that particular groups of people use, and (c) revealing unexpected additional issues related to HEP that were not identified through the conventional means of the literature review or the Delphi study. Reference DeVellis38
Data Analysis
The open-ended question responses were analyzed with qualitative content analysis by means of NVivo software for Mac version 11.4.1 (QSR International Pty Ltd, Doncaster, Victoria, AU). Participants’ responses were coded, counted, and placed into categories by 2 researchers separately first, then discrepancies were discussed and agreed upon, and then the data was summarized. These data were primarily used to learn about the participants’ experience with completing the HEPI.
The level of statistical significance for Type I error was set at P < 0.05. Evidence-based estimation of effect size is not currently available. On the basis of expert judgment, Type II error was set at beta = 0.20. All data analyses were completed with the Statistical Package for the Social Sciences for Windows version 26.0 software (International Business Machines Corporation, Armonk, NY).
Nine nondemographic items dispersed randomly throughout the scales were left blank. The most commonly occurring subscale response was used to replace those missing items before summing the subscale scores for Preparedness Actions and Planning (PAP) and Disaster Supplies and Resources (DSR). These subscales represent basic preparedness applicable to all individuals/households. Assumptions appropriate for statistical testing were evaluated and corrections were applied as required for the various statistical analyses used to test hypotheses.
It is appropriate to measure reliability of the HEPI by looking at the internal consistency of the questions. Internal consistency indicates how well the questions fit together conceptually, which can be assessed by looking at the inter-item correlations. Reference DeVellis38,Reference Devon, Block and Moyle-Wright39 This looks at the relationship among questions that are all indicators of the latent variable of HEP. Questions that have a strong relationship to HEP will have a strong relationship to one another. Reference DeVellis38 The HEPI would be considered internally consistent to the extent that its questions are highly intercorrelated, as evidenced by coefficient alphas greater than .65. Reference DeVellis38
The HEPI is intended to measure actual preparedness for individuals/households; resulting in questions restricted to a dichotomous (yes or no) response format. For example: Have you prepared and discussed a family emergency plan? Scales using this type of response format do not perform well when subjected to traditional factor or component analyses. An alternative approach that is based on polychoric correlations was used to examine the factor structure of the new instrument. Reference Lorenzo-Seva and Ferrando40
The Access and Functional Needs (AFN) subscale is highly individualized and only applicable to a specific subset of the population (participants with a disability, aged greater than 65, on at least 1 prescription medication, or who are pregnant) and was not included in the factor or reliability analysis. The same is true for the Special Actions (SA) subscale that is applicable to only those who have specific lifestyle or family characteristics (Table 1).
Table 1. HEPI scaling after factor analysis
Note: Items included in Factor Analysis are shaded in gray. Original Scaling: Preparedness Actions (PA), Communication Plans (CP), Evacuation Plans (EP), Disaster Supplies (DS), and Access & Functional Needs (AFN). Scaling after Factor Analysis: Preparedness Actions and Planning (PAP) and Disaster Supplies & Resources (DSR). Scales remaining but not applicable to all: Special Actions (SA) and Access & Functional Needs (AFN).
a One-point weight added for items included in an emergency supply kit.
Results
The survey was accessed by 348 participants with 295 consenting, meeting the inclusion criteria, and beginning the survey. Data were reviewed for completeness and retained for the 284 participants who completed at least 95% of the survey (Table 2), 26.7% (n = 71) of which identified as having a country of origin outside of the United States. Approximately 50% of the surveys were completed by students (n = 140) compared with faculty/staff (n = 138); likewise, almost 50% reported they were married or partnered (n = 145) compared with not married or partnered (n = 133). Caucasian participants (n = 148) were most frequently represented, followed by Asian (n = 52), Hispanic/Latino (n = 39), Black/African American (n = 23), and Other (n = 14) participants. Faculty/staff were better prepared than students, people 50 y and older were more prepared than those 29 and younger, and members of the Hispanic/Latino and White/Caucasian group were significantly better prepared (P ≤ 0.001) than those in the other race/ethnicity groups. See Table 2 for additional information on the demographic characteristics and general preparedness of the CUNY sample.
Table 2. Demographic characteristics and GP of CUNY sample
Note: c = comparison group. HCP discussion of HEP = health-care provider discussion of household emergency preparedness).
* P < 0.05.
** P < 0.01.
*** P ≤ 0.001.
Regarding factor analysis, the adequacy of the polychoric correlation matrix was assessed favorably resulting in a highly significant Bartlett’s statistic, a very good KMO (.92) indicating the sample size was sufficient for factor-analysis, and a narrow bootstrap 95% CI of .902 to .904. This indicates the quantity and quality of the data were appropriate for this method of factor analysis. The first factor analysis, based on a 4-factor solution (representing the 4 scales organized from the data generated in the Delphi study of experts), did not yield a reasonable solution. The factor analysis output indicated a 2-factor solution might be optimal and when run, 31 of the 34 general preparedness questions loaded on 1 of 2 dimensions and explained 35% of the variance represented in this sample. The study participants did not discriminate types of preparedness items as precisely as expected. The sample represented in this study is likely more consistent with the general population than the Delphi experts and changes to scaling were made to reflect the 2 dimensions represented in the factor analysis. The first dimension presented as the PAP scale (Cronbach’s alpha = .86) and the second dimension presented as the DSR scale (Cronbach’s alpha = .80). Together, the 2 scales represent the General Preparedness (Cronbach’s alpha = .89) of individuals or families. Yes-responses to the questions were summed across the subscales and for the General Preparedness (GP) scores. Nine of the DSR items were assigned an extra point if the item was included in an emergency supply kit. Refer to Table 1 for more information on the HEPI before and after factor analysis and how survey questions were re-clustered to better fit the 2 dimensions of GP identified in the factor analysis process.
Higher scores on the HEPI indicate higher levels of preparedness. Excluding the AFN and SA subscales that are not applicable to all respondents, the minimum score a participant could receive on the GP scale is 0 and the maximum score is 40 (1 point for each yes-response and 0 points for each no-response). The mean GP score for this sample was 15.28 and the range was 1 to 37.
Of the 127 participants reporting AFN, 122 responded to the question of HCP discussion of HEP. Only 10 of those individuals and 5 individuals who did not report access and/or functional needs (a total of 5.4% of the total sample) responded positively to the question. Those who had a discussion with their HCP were some of the best prepared study participants with a GP M = 19.33 (8.89) compared with M = 15.06 (6.83) for those whose HCPs did not discuss HEP with them (t(282) = 2.32; P = 0.021). See Table 2 for a complete description of sample characteristics and group differences in GP.
To examine the overall impact of the significant demographic variables on GP, the following variables were dichotomized: married or partnered to not married or partnered, faculty/staff to students, Hispanic/Latino and Caucasian/White to any other race/ethnicity, graduate degree to no graduate degree, annual income $75,000 or greater to less than $75,000, HCP discussed HEP to HCP did not discuss HEP, owned home to did not own home, and previous home damage due to disaster to no previous home damage due to disaster. Using standard multiple regression, GP was regressed on the 9 variables, together, they explained 20% of the variance found in the level of preparedness for this diverse group (R Reference Heagele2 = .205; adjusted R Reference Heagele2 = .178; F(9, 268) = 7.67; P < 0.001). However, only 1 of the 9 dichotomized variables, previous home damage due to disaster event, provided a significant independent impact on GP (unstandardized B = 3.17; t = 3.70; P < 0.000) having the greatest unique impact on GP represented by a part correlation of r = .20.
The core theme from the qualitative data was that participation in the HEPI helped participants learn what is needed to be prepared, made them realize that they were unprepared, and inspired them to develop a HEP plan. Several participants recommended that HEP educational resources be included at the completion of the HEPI. Five participants wrote they felt anxious after completing the HEPI. The authors will include in the instructions for researchers to disclose to their IRB and potential participants that completion of the HEPI may cause anxiety. The second most common theme from the qualitative data was that the HEPI was user-friendly, with descriptors of “easy,” “quick,” “understandable,” and “clear” commonly mentioned. This theme was supported by the quantitative data, with 98% (n = 278) of the sample rating the clarity of the instructions for the survey as extremely clear, 98% (n = 277) rating the clarity of the questions as extremely clear, 77% (n = 218) rating the length of the survey as extremely reasonable, and 75% (n = 212) rating the level of difficulty of the survey as extremely easy on a 5-point rating scale.
Discussion
The HEPI Delphi study was conducted to begin the process of developing the “gold standard” HEP instrument. Reference Heagele, McNeill and Adams9 Disaster supply kits are often used as a potential measure of preparedness; however, because disaster supply kits are not empirically confirmed, using supply kits as an indicator of HEP calls into question the validity of the instruments that are used. The most appropriate form of validity for HEP instruments, predictive validity, has yet to be explored. Reference Heagele2 “Predictive validity indicates the extent to which an individual’s future level of performance on a criterion can be predicted from knowledge of performance on a prior measure” (p. 176). Reference Waltz, Strikland and Lenz34 To assess predictive validity, researchers are encouraged to use the HEPI in longitudinal studies evaluating survival and resilience outcomes associated with suitable HEP.
The finding that students were less prepared than faculty and staff is consistent with findings of other studies that indicate students were generally unprepared for disasters. Reference Simms, Kusenbach and Tobin27–Reference Tkachuck, Schulenberg and Lair29 A small study of 80 university students in Ontario, Canada living off-campus and outside a family home revealed that despite the majority (69.2%) of participants believing they were personally responsible for their wellbeing during an emergency, 75.2% reported having no emergency preparedness kit or a designated stockpile of emergency preparedness items. Reference Tanner and Doberstein28 In another study of 503 students at a Florida university, only 28% of participants reported having completed some form of hurricane preparation. Reference Simms, Kusenbach and Tobin27 Follow-up related to the level of preparation revealed that this included possessing items typical to households or dormitory rooms such as canned food, bottled water, batteries, and a raincoat; thus, it is not clear if any of these items were obtained specifically for HEP.
The finding that previous home damage provided significant independent impact on HEP is consistent with a previous study of 756 graduate and undergraduate students at a southeastern university in which disaster experience was a strong predictor for students’ actual and perceived disaster preparedness. Reference Tkachuck, Schulenberg and Lair29 It differs from a study of 503 undergraduate students at a Florida university, which found that, although 80% of participants indicated they had some experience with hurricanes, only 28% reported having completed some level of hurricane preparation and only 29% had an evacuation plan. Reference Simms, Kusenbach and Tobin27
The current findings that faculty/staff were better prepared than students, those over 50 are better prepared than those 29 and younger, and that Hispanic/Latino and White/Caucasian participants were better prepared than those in the other race/ethnicity groups are consistent with findings of a study of preparedness using the 2013 American Housing Survey. Reference Malmin41 In Malmin’s study, the most resilient households based on cumulative preparedness scores included those with the presence of married couples, older adults, and those with higher incomes; the least resilient households included Asian households and those that included children or disabled individuals. In a study on HEP behaviors in Virginia, a greater proportion of Asian-American households reported being “unprepared” compared with Caucasian or Black/African American households. Reference Ferguson, Kiernan and Spannhake42
The current study found those with higher income levels and graduate degrees had higher levels of GP. Ferguson et al. found that households in which high school was the highest level of education attained reported feeling “unprepared” at twice the proportion of households with a baccalaureate degree. Reference Ferguson, Kiernan and Spannhake42 Bell et al. completed a study with 2256 community-dwelling older adults, in which an income less than $30,000 per year was associated with lower odds of having a multi-day supply of food and water; and having a high school education or less was associated with lower odds of having a stocked emergency kit or having discussed evacuation plans with family or friends. Reference Bell, Singer and Solway43 In contrast, a study exploring the effectiveness of household emergency plans during tornadoes found no significant differences between older and younger adults in level of education on plan effectiveness, although the plans of older adults were less likely to be as detailed. Reference Cong, Liang and Luo44
The finding that the best-prepared study participants were those who had discussed preparedness with their HCP is consistent with the findings of studies of those with health needs, Reference McNeill, Killian and Moon45,Reference Killian, Moon and McNeill46 providing further evidence that HCP education is an important component of preparedness education to those most vulnerable. The authors recommend the inclusion of preparedness measures in HCP education as a part of regular patient/client care.
New York has been impacted by numerous disasters throughout history. 47 However, even with robust community-level HEP interventions targeting NYC residents, 12–15 the sample assessed in this study was not considered particularly well prepared, exhibiting a mean GP score of 15.28 of 40 possible points. The participants who previously had damage to their homes from a disaster were significantly more prepared for emergencies. This is consistent with researchers in Japan who found that those who experienced damage from the Great East Japan Earthquake in 2011 were more prepared for emergencies. Reference Onuma, Shin and Managi48 This disaster experience, previously linked to improved preparedness levels Reference McNeill, Killian and Moon45 may impact perceived risk, which has been linked to higher HEP levels. Reference Han, Wang and Cui49
Limitations
This cross-sectional study used a convenience sample and retains the expected limitations of such a study design. Although the CUNY population is diverse and housed on multiple campuses on all 5 boroughs in NYC, the sample size was small and drawn from the same general geographic location. Data collection took place during the COVID-19 pandemic, which may have affected response rate. Some participants indicated that they experienced anxiety when completing the survey; concerns related to personal or family well-being during the pandemic may have exacerbated emotional response to the subject matter and altered participants’ responses to the survey.
Conclusions
Although the results from previous studies indicate a wide variation in preparedness levels among different demographics, making comparisons is problematic as HEP was measured inconsistently across such demographics. Having an empirically developed instrument facilitates HEP measurement that can be compared across demographics, improving interventions and targeted education to improve HEP levels. Furthermore, future studies should endeavor to assess associations between HEP, survival, postdisaster needs, and outcomes, particularly among vulnerable community members with chronic conditions and/or AFNs. The next steps for the HEPI evaluation should include a predictive validity study, a known-groups validity examination, and a test-retest reliability assessment.
The HEPI is easy to administer in-person or online, it accurately captures the various indicators of HEP and explains an acceptable amount of the variance. The internal consistency reliability was strong in this assessment, particularly for a pilot test. Criterion, construct, content, and face validity support the adequacy of the new instrument to capture essentials of HEP. The HEPI takes approximately 15 min to complete and does not ask about sensitive information, making respondent and researcher burden low.
Researchers, emergency planners, and educators may use the HEPI for noncommercial purposes without cost under the following conditions: (a) the HEPI developers are properly credited in publications and presentations; (b) if the HEPI is translated to a language other than English or modified in any way, these changes are disclosed in publications; and (c) psychometric data analysis for the instrument is provided to the HEPI developers. This information will inform future modifications of the HEPI. For a copy of the survey and instructions on how to use and score the HEPI, contact the corresponding author of this article.
Acknowledgments
We acknowledge the work of an anonymous reviewer for their assistance in using the methods proposed by Lorenzo-Seva & Ferrando Reference Lorenzo-Seva and Ferrando40 for factor analysis of dichotomously scaled items.
Funding
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
Conflict(s) of interest
The authors have no financial or personal relationships with other people or organizations that could inappropriately influence or bias their work. We have no commercial associations that could pose a conflict of interest or financial bias.
