Mass casualty incidents (MCIs) have become more frequent in recent decades and often involve pediatric populations. MCIs are defined as events which overwhelm the available resources of a health care system. 1 What overwhelms a given health care system is dependent on the resources available at each entity, and the type of situation. Unfortunately, current literature suggests that overall preparedness for such events remains sub-optimal, leading to calls for continued efforts and attention to the methods of training for such events.Reference Biswas, Bahouth and Solomonov 2 As an example, Rassin et al. surveyed emergency department staff at a single institution that has frequent MCI drills and real-world MCI experience, yet their results suggested knowledge deficits regarding pediatric mass casualty.Reference Rassin, Avraham and Nasi-Bashari 3 In the US, emergency medicine residency programs have developed core disaster medicine topics utilizing a modified Delphi approach.Reference Sarin, Biddinger and Brown 4 Amongst PEM fellows, the majority of whom are trained in pediatric residency programs,Reference Murray, Woolridge and Colletti 5 there is varying, and oftentimes limited, experience regarding training in disaster medicine and MCI. Disaster preparedness education requirements exist for emergency medicine trainees, leading to an imbalance in disaster preparedness among PEM fellows. 6 , 7 Within the emergency department at our institution, the departmental mass casualty plans underwent a large update during the 2021-22 academic year. These changes were seen as an opportunity to address the education of MCIs amongst PEM fellows.
Available Knowledge
Historically, pediatrics populations did not receive prominent consideration in MCI preparedness efforts amongst hospital systems.Reference Lozon and Baldin 8 In fact, a survey of disaster preparedness by Shirm et alReference Shirm, Liggin and Dick 9 revealed that although 72.9% of agencies surveyed reported having mass casualty plans, only 13.3% had plans that involved children. Data from a scoping literature review, however, indicates that over the past 20 years there has been a growing body of pediatric relevant disaster medicine literature.Reference Jeong, Clark and Chung 10 Accounting for the unique physical, psychological, physiologic and developmental needs of children is important as these factors may place children at higher risk. One pivotal component of preparedness for mass casualty scenario is the concept of accurate and timely triage. Multiple tools and schema exist that have been employed to assist with accurate triage such as the jumpSTART, SALT, or SMART tools.Reference Cicero, Overly and Brown 11 , Reference Nadeau and Cicero 12 In 2008 recommendations to use SALT triage for a national standard for mass casualty triage were put forth. 13 Unfortunately, there is no single triage modality that outperforms others.Reference Stéfani, de Melo and Zardeto 14 , Reference Jones, White and Tofil 15 , Reference Cheng, Staats and Kaji 16 Triage training tools have been gamified to allow for simulated practice with educational tools such as the Emergency Medicine Services for Children’s 60 seconds to survival and, more recently, implementing virtual reality technology simulations have also been piloted.Reference Cicero, Whitfill and Munjal 17 , Reference Lowe, Peng and Winstead-Derlega 18 , Reference Pandit, Healy and Todman 19
MCI preparedness and response requires more than physicians who are competent in accurate and rapid triage, it also requires thoughtful system-based planning and knowledge of resources. While initial triage occurs in the field, hospital triage is also necessary to quickly allocate limited resources. A shift of ethos is also required, with the emphasis on the greatest good for the greatest number in accordance with crisis standards of care.Reference Leider, DeBruin and Reynolds 20 Departmental disaster planning for MCIs is vital to aid in the overall response. QI initiatives have not been frequently utilized to improve disaster preparedness efforts but have been suggested as both feasible and effective in both a hospital and clinic setting.Reference Codispoti, Rodriguez and Kaziny 21 Examples of QI efforts focusing on disaster and mass casualty preparedness have been reported amongst a group of trauma surgery staff and emergency nurses with statistically significant improvements reported in perceived knowledge in an adult hospital.Reference Hollister, Zhu and Edwards 22 , Reference Amberson, Wells and Gossman 23 Bank et al. performed an experiential effort amongst PEM, Pediatric critical care and pediatric surgery providers and evaluated the impact of such training on long-term knowledge retention at 6 months with significant self-perceived improvements in triage (including of pediatric patients), knowledge of hospital plans, and in their ability to respond to a MCI.Reference Bank and Khalil 24
Recent increases in disasters and reports of disasters in the media including mass shooting incidents affecting children such as the Sandy Hook Elementary School shooting in Newtown, CT and the Robb Elementary School Shooting in Uvalde, TX, as well as the recent SARS-CoV2 pandemic have led to increased awareness and interest in disaster medicine and preparedness. As aforementioned, there is a discrepancy of experience amongst physicians and trainees based on their prior medical training.Reference Grock, Aluisio and Abram 25 Given the predominance of PEM trainees with a pediatrics background, we recognized a need to provide a broad overview of disaster preparedness and systems and focused efforts on specific topic areas. For triage methodology, the regional trauma system adopted the use of SALT triage as the method of choice for MCI/disaster triage – interventions were planned with this in mind.
A focused QI initiative amongst the institution’s PEM fellows was undertaken based on literature of QI in MCI preparedness, desire for focused education in disaster medicine, and the need to implement and evaluate our departmental MCI plans.
Specific Aims
To increase the comfort with, and knowledge of, MCIs by PEM fellows at Nationwide Children’s Hospital from a baseline (2.9/3.7 respectively) to a goal of 8 on a 10-point Likert scale, and to sustain this for a period of 6 months.
Methods
This study, approved by the Nationwide Children’s Hospital institutional review board (Study00003573), was completed with 14 PEM fellows at a large, urban, tertiary children’s hospital with 62 emergency room beds and over 90,000 visits annually. Voluntary consent was obtained from participants.
Interventions
A multidisciplinary working group comprised of PEM attendings, a PEM fellow, ED nurses, QI analysts was assembled. The project was timed to align with the start of academic year 2023-2024 to obtain baseline data on all participating fellows. The baseline survey (appendix I) included information regarding any prior experience with MCIs, prior training program, hospital specific questions, such as where the MCI plans are located, and longitudinal questions for the initiative.
Using QI methodologies, a key driver diagram (KDD, Figure 1) was created to outline areas of focus for the initiative. The main key drivers focused on education, supplies, and application of concepts; with interventions produced to support and improve each key driver. Following the creation of the KDD, an effort impact matrix was developed to help focus initial efforts on high impact interventions (appendix II).
Figure 1. Key Driver Diagram developed for the QI Initiative.
For the first intervention, a short overview of the project was presented, followed by a focused lecture on the importance of timely and accurate triage, and how to utilize 2 of the most common triage systems – jumpSTART (reviewed briefly as not all hospital systems utilize the same triage method) and SALT. Following the lecture, an active exercise in triaging utilizing a patient injury deck allowed trainees to put their new knowledge into practice.
The second intervention consisted of a disaster-based scavenger hunt through the emergency department. A mock scenario was created to prompt utilization of the department’s MCI plan, including determination of activation level based on the expected number of inbound patients and current ED capacity. As part of the exercise, fellows were also prompted to locate specific items pertinent to disaster response, including the departmental disaster plan, equipment for decontamination, and PPE.
As part of the educational interventions, a series of dedicated lectures were given over a 2-month period. This lecture series included various topics related to hospital specific preparedness and incident command structure, and the disaster cycle and reunification plans specific to this institution. Another intervention included the creation and distribution of reference cards with the SALT triage algorithm and activation guidelines for use on badge reels. These were distributed to all the PEM fellows as well as to any interested staff, and larger triage cards with the SALT triage algorithm were placed in specific locations within the emergency department including the workstations and critical care rooms.
The active intervention period culminated with a multidisciplinary, simulated tabletop exercise that allowed for hands on experience of MCI response using our departmental plans and with a layout of our emergency department. The fictitious simulation involved a mass shooter incident during a holiday parade. The exercise was attended by ED faculty, ED nursing staff, and our institution’s office of emergency management. The variety of attendees present for the exercise helped provide oversight and feedback to the fellows after completing the simulated MCI of an active shooter incident during a parade.
After each intervention’s completion, trainees were surveyed with intervention-specific questions and longitudinal questions evaluating their comfort and knowledge of MCI response. During the initiative, the working group held regularly scheduled meetings to review progress and plan the upcoming PDSA cycles.
The total time commitment from all interventions during this initiative’s active phase totaled 6 hours.
Measures
The primary measures for assessing the impact of the interventions were the self-reported comfort level with, and knowledge of, MCI response, based on a Likert scale of 1-10. By including these questions in each post-intervention survey, responses were tracked for the entire initiative. Overall attendance for individual cycles was tracked, and only fellows who participated in each intervention were surveyed after each given cycle. As a balancing measure, fellows were surveyed on whether they felt that MCI education had detracted from their overall board-specific learning, and if learning about MCI preparedness had negative psychological effects.
Analysis
Quantitative data were analyzed in control charts using statistical process control (SPC) methodology.Reference Provost and Murray 26 , 27 Results from the baseline, post-intervention, and sustain surveys were analyzed using SPC methodology, entered onto X-bar S charts with the corresponding month the survey was distributed.Reference Provost and Murray 26 , 27 Surveys also included qualitative data with subjective feedback on recommendations from fellows regarding frequency of drills and modifications to the interventions to improve learning, which will be considered for future iterations of the training.
Results
Outcome Measures
This project sought to improve 2 outcome measures: PEM fellow comfort with the MCI process and PEM fellow self-reported knowledge. For both measures, special cause variation coincided with the first intervention, Triage Exercise, using the SPC rule of 1 point outside of the control limits. By the end of the last intervention in December 2023, level of comfort with the MCI process had improved from a baseline of 2.93 of 10 to a new baseline of 6.50 of 10 (Figure 2). Similarly, self-reported knowledge increased from a baseline of 3.71 of 10 to 6.55 of 10 (Figure 3).
Figure 2. Run chart data of fellow reported comfort with MCI process on 0-10 Likert Scale.
Figure 3. Run chart data of fellow reported knowledge of MCI process on 0-10 Likert Scale.
To assess for retention, fellows were surveyed at the 1-month, 3-month, and 6-month marks following the last intervention in December 2023. Level of comfort remained unchanged at the 1-month survey period with an average of 6.50, however this improved to 6.86 in the 6-month sustain survey. Self-reported knowledge initially decreased from the intervention period average of 6.55 to 6.29 with the 1-month survey, however increased to an average of 7.0 at the 6-month survey.
Overall, level of comfort increased to an average of 6.56 of 10, and self-reported knowledge increased to 6.58 of 10. While both measures had improved scores at the 6-month period from the intervention period average, neither had special cause variation, indicating a stable system.Reference Provost and Murray 26
Process Measures
Number of attendees and survey responses were measured to track attendance and engagement over the intervention and sustain periods. From a cohort of 14 PEM fellows, there was an average of 11.2 fellows present for each intervention. Overall, 98.2% (56/57) of surveys were completed after each intervention. The 1-month and 6-month sustain surveys had 100% (14/14) completion rates; 13/14 were completed during the 3-month sustain period due to 1 fellow being on maternity leave.
Balancing Measures
Balancing measures were identified as negative impacts on other learning opportunities and increased psychological response regarding focus on disaster preparedness and mass causalities. Surveys were sent to the PEM fellows at the 3-month and 6-month sustain marks, asking “Has learning about disaster medicine and mass casualty preparedness negatively impacted other learning opportunities?” and “Has becoming more aware of disaster preparedness and mass casualty response led to increased psychologic response (worry, anxiety, confidence etc.)?” Overall, respondents rated the negative impact of the mass casualty preparedness as 2.1 of 10 on a 10-point Likert scale. When provided a scale of Significant Negative Impact, Negative Impact, Neutral, Positive Impact, and Significant Positive Impact regarding psychologic response to the trainings, 14.8% of responses reported a Significant Positive Impact, 37.0% reported a Positive Impact, 44.4% indicated Neutral, and 3.7% reported Negative Impact.
Discussion
Summary
While it may be impossible to predict when or how a mass casualty situation may occur for at any given institution, training and preparing for such eventualities has been demonstrated to improve response through identification of high acuity patients. Although such events often involve adults, children are also often directly affected by these tragedies. Although disaster training for emergency medicine resident physicians has undergone standardization over recent years,Reference Rassin, Avraham and Nasi-Bashari 3 training for PEM physicians has not yet undergone such measures. QI initiatives have been utilized infrequently to improve comfort with this situation, but in general has shown success.Reference Hollister, Zhu and Edwards 22 –Reference Bank and Khalil 24
This study aimed to improve the comfort with, and knowledge of, MCIs among PEM fellows. An increase was seen in both comfort with MCI and knowledge of MCI processes from baseline averages of 2.9 out of 10 and 3.7 out of 10 to 6.56 out of 10 and 6.58 out of 10 respectively. There was notable special cause variation following the initial intervention of disaster triage, representing the largest increase in overall knowledge and comfort amongst the fellow cohort. After the initial intervention phase, data from the 1- and 3- month surveys indicated a mild decrease in both primary outcome measures. Data from the 6-month post intervention cycle, however, showed that the improvements were retained, with scores remaining within the previous range.
Interpretation
With relatively few studies utilizing QI within the realm of disaster medicine, and the infrequent nature of MCI events, these results speak to the feasibility of utilizing a QI approach to mass casualty incident preparedness. Additionally, the iterative nature of QI allows for individualized/systems-based adjustments that may best serve a given locale or institution.
During the intervention period, a marked increase was seen in the comfort and knowledge of PEM fellows regarding MCI despite a relatively low time commitment. The effects of this QI initiative were sustained for 6 months following the intervention period (Figures 1, 2). To maintain preparedness, ongoing monitoring to assess for the need of periodic review or skills refresher is prudent.
The significant increase in perceived comfort and knowledge of MCI response seen after the initial triage intervention imply that even just 1 or 2 interventions focused on disaster triage and application could significantly improve perceptions of preparedness. Further studies would be needed to assess whether similar long-term effects could be seen with fewer interventions.
It is important to note that the original goal of improvement to 8-out-of-10 on a 10-pt Likert scale was not attained. This may reflect an overly ambitious starting goal, the need for further efforts, or difficulty amongst the provider cohort to feel adequately prepared for events they have not experienced first-hand.
Limitations
There were several limitations to this QI initiative. This project was focused on PEM fellows and did not include other staff members. Due to clinical schedules, it was also not feasible for every fellow to be present for every intervention, although all interventions had a majority of fellows present. Therefore, the overall improvement in comfort and knowledge may have been higher if all fellows had participated in every intervention. Additionally, this study employed self-reported assessment surveys. During the intervention period, our institution began a major construction project to expand the hospital, which necessitated a loss of space in the ED. This impacted normal flow and daily operations of the ED as well as required overall plan amendments.
This QI initiative was implemented at a single center with 1 cohort of PEM fellows and thus may not be directly applicable to other hospital and trainee settings.
Conclusions
Utilizing the framework of QI for MCI education and to monitor preparedness for MCIs is a feasible methodology. There was a marked increase in comfort with the MCI process and overall MCI knowledge amongst PEM trainees due to the QI initiative. While the study’s focus was on PEM fellows, this initiative could be re-aimed to include all ED providers, as well as nursing staff. Such additional efforts would hopefully lead to improved overall system preparedness, given the shift-based work that comes with emergency medicine. These results will hopefully encourage further studies utilizing QI within the realm of disaster medicine.
Supplementary material
The supplementary material for this article can be found at http://doi.org/10.1017/dmp.2025.74.
Author contribution
All authors reviewed and contributed to the manuscript. CW & JM led the project from inception through completion. CC, HJ & RS provided subject matter expertise and guidance. JG provided statistical support and analysis.
Funding statement
None to disclose.
Open access
