Development of On-Site Medical System for Mass-Gathering Events During TOKYO 2020: Vulnerability Analysis Using Healthcare Failure Mode and Effect Analysis

Abstract

At mass-gathering events of the Olympic and Paralympic Games, a well-organized, on-site medical system is essential. This study evaluated the vulnerabilities of the prehospital medical system of the TOKYO 2020 Olympic and Paralympic Games (TOKYO2020) to propose corrections that can be generalized to other mass gatherings. The healthcare failure mode and effect analysis (HFMEA) was adopted to analyze vulnerabilities of the on-site medical system proposed by the organizing committee of TOKYO2020. Processes from detecting a patient on the scene to completing transport to a hospital were analyzed. Ten processes with 47 sub-processes and 122 possible failure modes were identified. HFMEA revealed 9 failure modes as vulnerabilities: misidentification of patient, delayed immediate care at the scene, misjudgment of disposition from the on-site medical suite, and inappropriate care during transportation to hospital. Proposed corrections included surveillance to decrease blind spots, first aid brochures for spectators, and uniform protocol for health care providers at the scene. The on-site medical system amended by HFMEA seemed to work appropriately in TOKYO2020.

As international sporting events, the Olympic and Paralympic Games draw thousands of athletes and hundreds of thousands of spectators from around the world. ^1–3 This temporary population surge in a local area challenges health care delivery systems and requires plans for a mass-gathering medical system. ^3,4 Previous studies of public health preparedness and procedures for summer sporting events, including Olympic and Paralympic Games, found that medical response plans should involve health-related agencies, volunteers, and sponsors, as well as risk assessment with system correction that should be performed during the planning process. ^3,5

Mass gatherings introduce various burdens on a local health care delivery medical system, for instance, transmission of infectious diseases and mass casualty incidents, along with overwhelming demands on local medical services by spectators with exacerbated comorbidities, thermal disorders, injuries, or alcohol-related symptoms. ^6,7 While a health organization needs to implement systems for infectious diseases, ⁸ development of a well-organized on-site medical response system is another principle of mass-gathering medicine. ^9,10 An interview-based study suggested that emergency care delivery at an out-of-hospital location by a designated on-site medical team would provide timely access to health care systems and eventually reduce unnecessary hospital visits. ¹⁰ Moreover, such an on-site medical response system should be tailored to event components, participant characteristics, geography, and availability of local resources. ^9,11

The medical service unit of the Tokyo Organizing Committee of the Olympic and Paralympic Games (TOCOG) developed a scheme of on-site mass-gathering medical system and assigned a designated physician as a venue medical officer (VMO) to each event arena. TOCOG then decided to collaborate with the VMO to adjust the on-site medical response system, considering the expected number of spectators and the available health resources (eg, hospital beds). Accordingly, this study aims to elucidate potential vulnerabilities of the scheme that TOCOG provided, and to propose corrections that can be generalized and used for other mass-gathering medical systems. This study targeted the on-site mass-gathering medical system at the main arena of the Tokyo 2020 Olympic and Paralympic Games (TOKYO2020), where opening and closing ceremonies were held. The healthcare failure mode and effect analysis (HFMEA), ¹² a systematic and prospective method of process mapping to identify how a complex task might fail and which corrective interventions are needed, ^12–15 was adopted for the vulnerability analysis.

Methods

Study Design and Setting

A prospective vulnerability assessment of the on-site mass-gathering medical system for the main arena of the Olympic and Paralympic Games in Tokyo in 2021 was performed. The main arena was targeted in this study because it has the largest capacity in Japan and results could be adapted to other mass-gathering events. The initially proposed on-site medical response system was examined with HFMEA by a risk assessment team. Team members were assembled by a board-certified emergency physician, rather than the physician assigned as VMO, because vulnerability analysis should be performed independently from the VMO or TOCOG. The HFMEA was conducted from April 2019 to July 2020. Institutional Review Board approval for conducting human research was waived because no human subjects were involved in this study.

On-Site Medical System

The main arena of the Tokyo 2020, Japan National Stadium, located at Tokyo City’s center, has a capacity of approximately 60 000 seats with 2 underground floors and 5 floors above ground. An on-site medical response system for spectators had been proposed by the medical service unit of TOCOG, following the scheme based on the capacity of venue: 2 physicians and 4 registered nurses at each venue, with an additional physician and 2 registered nurses for every 10 000 spectators, 1 pair of volunteers as first responders for every 1000 spectators, 1 on-site clinic for every 10 000 spectators, and at least 1 designated ambulance at each venue. Accordingly, 8 physicians, 16 registered nurses, 120 first responders, 6 on-site clinics, a main on-site medical suite, and 3 designated ambulances were initially proposed for the main arena by TOCOG. The rationale for this scheme was not disclosed by TOCOG.

An emergency care delivery system was then planned by referring to mass-gathering medical systems previously adopted by several organizations (Figure 1) ^10,16–19 : (1) Paired first responders (first aiders) are distributed across the venue and initially respond to patients on the scene; (2) first aiders triage patients into 3 categories (red, emergent activation of both a mobile medical unit and an ambulance; yellow, activation of a mobile medical unit; and green, completion of care at on-site medical suit); (3) a registered nurse patrolling each floor is activated by the first aiders and reassesses patients and re-triages as needed; (4) a mobile medical unit composed of a physician and 2 registered nurses is dispatched as needed, provides initial treatment on the scene, and transports patients to the main on-site medical suite; and (5) initial treatment continues at the main on-site medical suite until an ambulance loads the patient.

Figure 1. Process map of on-site medical response system. (1)(2) Paired first aiders are distributed across the venue and initially respond to patients or bystanders; (3)(4)(5) responded first aiders triage patients into 3 categories (red, emergent activation of both a mobile medical unit and an ambulance; yellow, activation of a mobile medical unit; and green, completion of care at on-site medical suit); (6) a registered nurse patrolling each floor is activated by the first aiders and reassesses patients and re-triages as needed; (7)(8) a mobile medical unit composed of a physician and 2 registered nurses is dispatched as needed, provides initial treatment on the scene, and transports patients to the main on-site medical suite; and (9)(10)(11) initial treatment continues at the main on-site medical suite until an ambulance loads the patient. FA, first aider; MMU, mobile medical unit; RN, registered nurse.

Healthcare Failure Mode and Effect Analysis

The proposed on-site medical response system was examined using HFMEA to elucidate the system’s vulnerability. The National Center of Patient Safety and the Commission on Accreditation in the United States developed HFMEA by modifying the failure mode and effect analysis that has been used as a risk prevention tool instead of a retrospective revision based on outcomes in the engineering industry. ^12,20 HFMEA was selected to examine the medical system because this tool can prospectively identify vulnerabilities before any outcomes of the system are obtained (analyses were conducted before the beginning of TOKYO2020 on July 23, 2021). HFMEA includes 5 steps to analyze potential failure modes within a medical system, and risks are classified and quantified by occurrence, severity, and detection controls already in place.

Step one: Definition of the HFMEA topic

Potential vulnerability topics in the on-site mass-gathering medical response system that should be examined by HFMEA were defined as the core process—from detecting a patient on the scene to completing transport to a hospital. The on-site medical response system’s outcome was defined as delivery of immediate care to patients with life-threatening conditions, timely patient transport without exacerbating disease, and appropriate selection of patients who do not need a hospital visit.

Step two: Definition of the risk assessment team

The multidisciplinary risk assessment team was established with 9 members, consisting of board-certified emergency physicians skilled to provide prehospital emergency care (also for obstetric, pediatric, and geriatric patients) at prehospital, board-certified general surgeons, trauma surgeons, members of the Disaster Medical Assistance Team who had experience in working with mass-casualty events, educational personnel, and administrative personnel. These specialties were selected because the prehospital on-site medical response system was the target topic. They had a training period to learn the basic concepts of HFMEA before the analyses were initiated.

Step three: Graphical description of the process/processes

Processes between patient identification and hospital transfer were outlined by the risk assessment team and then sub-processes were described. During this step, each team member answered a questionnaire, specifically designed to identify and describe the processes and sub-processes, based on literature and scenario-based simulations. Subsequently, all the team members were asked for agreement or disagreement with each process identified with the first questionnaire through a 3-round Delphi survey, until consensus was reached. All processes that fulfilled the predefined threshold for the inter-percentile range in agreement were then integrated, and appropriate modifications were adopted for similar sub-processes. The final diagram of processes and sub-processes was drawn by consensus.

Step four: Hazard analysis

First, team members independently listed all possible/potential failure modes for each sub-process. Then suggested potential failure modes were integrated via team discussion in which the failure mode was defined as a different way a particular sub-process could fail to accomplish its intended purpose. ¹²

Second, the severity and probability of potential failure modes were assessed by each member independently on a 4-point scale: 1 = remote, unlikely to occur; 2 = uncommon, possibly occurring once during the entire event; 3 = occasional, likely to occur once a day; and 4 = frequent, likely to occur several times a day (Supplementary Table S1). To determine the probability, each member independently performed a literature search on the proposed failure modes and scenario-based simulations. Each member’s probability scores were averaged, with consideration of a weight for each member’s total scores. The severity of failure was determined by team consensus with a 2-round Delphi method on a 4-point scale: 1 = minor, no injury and no increased level of care; 2 = moderate, increased length of stay or increased level of care; 3 = major, permanent lessening of bodily functioning; and 4 = catastrophic, death or major permanent loss of function (Supplementary Table S1). A hazard score was then calculated by multiplying the severity and probability scores. ^12,13

Third, potential failure modes with a hazard score equal to or greater than 8 were identified as unacceptable risks (this threshold was predefined in the formal HFMEA methods) and transferred to a 3-step decision tree, following the formal HFMEA decision tree (Supplementary Figure S1). ¹² Any potential failure mode identified as a single-point weakness in the system, even if the hazard score was less than 8, was sent to the decision tree for further evaluation.

Fourth, the risk assessment team used the 3-step decision tree to assess criticality in the process, existence of an effective control measure, and detectability. The failure mode that was determined to be critical, uncontrollable, and undetectable was validated as the critical vulnerability in the entire process of the on-site mass-gathering medical system. The assessment in the decision tree was conducted with Delphi rounds.

Finally, effect analysis was performed to identify effective corrections for validated failure modes. Corrections were categorized into either an intervention within the system or at a health care provider (HCP).

Step five: Action plans and outcome measures for test events

Each member developed and integrated a description of action as corrective intervention. At this step, the VMO examined the feasibility of each action to ensure execution of corrections. Then, outcome measures for test events before TOKYO2020 was held were proposed to ensure that the action plans effectively amended the mass-gathering on-site medical system.

Results

Number of proposed processes, sub-process, possible failure modes, failure modes with a high hazard score, failure mode assessed in HFMEA decision tree, and validated failure modes are shown in Table 1. Ten processes with 47 sub-processes were identified between recognizing a patient and completing transfer to a hospital (step 3 of HFMEA). Then, during hazard analysis (step 4 of HFMEA), 122 possible failure modes were proposed, and 58 were assessed with the HFMEA decision tree.

Table 1. Processes and failure modes in the on-site medical system identified with HFMEA

HFMEA, healthcare failure mode and effect analysis; HS, hazard score.

^a Hazard score was calculated by multiplying the scores of severity and probability obtained by team members.

The HFMEA identified 9 failure modes as critical vulnerabilities in the system (Table 2): 3 failure modes related to misidentification of patient by first aiders, 2 related to delayed immediate care by the mobile medical unit at the scene, 1 related to misjudgment of disposition from the main on-site medical suite, and 3 related to inappropriate care during transportation to hospital. Failure modes regarding inadequate on-site care were considered to happen when HCPs did not have enough experience of prehospital medicine. Six failure modes with hazard scores less than 8, which were sent to the HFMEA decision tree because single-point weakness was a concern, were eventually validated as critical vulnerabilities. All possible failure modes and results of triage in the decision tree are shown in Supplementary Table S2.

Table 2. Summary of validated failure modes and effective corrections

FM, failure mode; HCP, health care provider; and Y, yes.

^a FM number is the reference code assigned to each failure mode, in which the initial number indicates the process, middle letter indicates sub-process, and last number indicates failure mode (see Supplementary Table S2).

^b Failure modes with hazard scores less than 8 were assessed with the decision tree if it was considered a single-point weakness in the system

Proposed corrective measures included 5 interventions within the system and 2 for HCP: Surveillance to decrease blind spots is installed; first aid brochures are provided to spectators; a protocol is developed for patient assessment on site, at medical suite, and during transportation; and knowledge education and skill training regarding prehospital medicine are provided (see Table 2).

Step-by-step action plans were developed with backup plans for each effective correction, considering the VMO’s viewpoints for the feasibility (step 5 of HFMEA, Table 3). Outcome measures for TOKYO2020 test events were set as clinical consequences of patients identified at the venue and the duration of time spent on the core process. These outcomes were scheduled to be measured to examine whether proposed action plans would correct the on-site medical response system and to analyze whether the corrected system would function without new vulnerabilities.

Table 3. Action plans for effective corrections

TOCOG, Tokyo Organizing Committee of the Olympic and Paralympic Games.

^a Action plans were developed considering the viewpoints from the venue medical officer (VMO).

Discussion

In this vulnerability analysis of the on-site medical response system, 9 of 122 possible failure modes were identified as critical. Notably, these validated failure modes were related to delayed identification of the patient (eg, “collapsed at blind spot” and “misrecognizing patient”) or inappropriate assessment/provision of immediate care by the medical team, rather than a shortage of HCPs. This finding introduced action plans such as installation of a surveillance system at the venue and additional training and education of HCPs, not an incremental number of HCPs, which tends to be proposed without a clear rationale for preparing the on-site medical response system at mass-gathering events.

Previous studies have proposed several methods to calculate needs for medical resources at mass-gathering events. ^19,21–23 A retrospective study of over 200 mass-gathering events with more than 25 000 attendees revealed that the patient presentation rate was 1/1000 and the transportation to hospital rate was 0.03/1000. The same study developed regression models to estimate patient load using event variables, such as weather, mobility of the crowd, and availability of alcohol. ²² Another study evaluating patient presentations and ambulance transfers found that using historical data was more accurate than adopting an existing prediction model when the event is unique and periodic. ²¹ Although these models have been recommended for preparing mass-gathering medical systems in subsequent studies, ^24,25 literature is sparse on analyses that predict more than just the number of patient presentations.

While the number of medical resources (eg, medical supplies, HCPs, and transportation methods) should definitely be calculated in advance based on predicted patient load, a prospective evaluation of whether the on-site medical system would operate appropriately is essential for preparedness. The current study conducted a prospective risk analysis and showed that one of the vulnerabilities depended on possible inappropriate immediate care at the scene, not on a shortage of resources. Similar concerns were reported in a retrospective study, in which patients who had received inappropriate on-site care stayed significantly longer at the hospital after transportation. ²⁶ While additional education of HCPs to understand the unique aspect of on-site medicine has been conducted previously, the current study similarly proposed knowledge education and skill training as corrective measures for vulnerabilities. Along with other corrective measures to support HCPs with limited experience, such as first aid brochures and a protocol for the prehospital assessment, continuous education would be needed to provide the appropriate on-site mass-gathering medical system.

Although the number of critically ill patients has been reported as limited (0.6–4.2%), ^26,27 there are a few patients who need to be recognized rapidly and receive immediate care. ²⁸ Indeed, this study reported the lack of a rapid recognition system as a critical vulnerability, which is similar to concerns in another study, such as limited access due to uncontrolled crowds and/or large size of venue. ²⁹ Notably, a study of college festivals in the United States found that mobile, roving responders intimately familiar with the venue were capable of responding rapidly to nonambulatory patients and of identifying individuals in need. ³⁰ Development of such a first-response system should be prioritized to prepare for mass-gathering events.

Limitations

This study’s results must be interpreted within the context of the study design. Because we targeted the on-site medical response system’s functioning, specifically at the main arena of the TOKYO2020, generalizability of the critical vulnerabilities should be carefully evaluated, particularly in a different type of events such as music festivals in which spectators would have different behaviors. However, as the initially proposed scheme was simply based on the capacity of the venue, similar vulnerabilities would be found in most large, outdoor, bounded athletic events.

Another study limitation is that HFMEA only introduced action plans but did not confirm the effectiveness following those corrections. Although there are always possibilities that proposed plans would not ameliorate vulnerabilities, HFMEA has been well-defined as a useful tool for prospective vulnerability evaluation when outcome data is not available. While TOKYO2020 was actually held with less than the expected number of spectators, the amended on-site medical response system seemed to work appropriately. Further assessment with actual data will be performed in a future study.

Furthermore, this study analyzed vulnerabilities in the core process of delivering emergency care in normal or ordinary times. Therefore, an on-site medical system to respond to natural disasters should be discussed separately from the current analysis. Repeating vulnerability analysis with different core processes using HFMEA would be effective for developing a different on-site medical system.

Finally, all the steps in HFMEA were done by the same members and they had been gathered by the board-certified emergency physician, in which objectivity is a concern because the same experts were used to validate the a priori process and participated concurrently after each of the sequential HFMEA steps. It is important to note that the HFMEA members should be assembled by a physician rather than the VMO because the vulnerability analysis should be performed independently. Missed relevant information during each step in HFMEA due to a limited number of team members is also a limitation. Further validation of vulnerabilities found in this study would be needed.

Conclusions

TOKYO2020 was finally held and no obvious problems in the on-site medical system were noted, suggesting that HFMEA analyses seemed to work appropriately. This study conducted a vulnerability analysis of the on-site medical system for mass-gathering events of the TOKYO2020 and recognized a delayed identification of patients and inappropriate immediate care at the scene, rather than a shortage of HCPs, as critical vulnerabilities. These risks were corrected by several action plans before the TOKYO2020 was held and could be useful for planning other on-site medical systems at mass-gathering events. While selected interventions may not be universally adaptable to other mass-gathering or disaster events, proposed corrective measures were made for the system and providers.