The coronavirus disease 2019 (COVID-19) pandemic has presented complex ethical dilemmas about how to operationalize the allocation of scarce healthcare resources. Informed by experience with past infectious pandemics, the National Academy of Medicine (NAM) (previously the Institute of Medicine) and multiple other national and regional leaders have developed guidelines for the allocation of scarce life-sustaining intensive care resources, such as ventilators or intensive care unit (ICU) beds, during a disaster. Reference Bell, Hyland and DePellegrin1–Reference Ingram, Hertelendy and Molloy8 When available health-care resources are no longer sufficient to support usual standards of care, regional authorities may declare a state of crisis capacity. Under crisis standards of care, ICU resources are to be allocated in a way that is intended to optimize overall benefit to a population while also respecting equity and fairness. Reference White, Katz and Luce9 To operationalize this approach, many state and national scarce resource allocation algorithms center on a specialized scarce resource triage team, including members with expertise in critical care and/or emergency medicine and bioethics, which is deployed to help prioritize patients to receive scarce life-sustaining resources. A commonly agreed-upon goal for triage team deliberation is to categorize patients based on their likelihood of surviving to hospital discharge if they were to recieve all needed health-care resources. Reference White, Katz and Luce9 To facilitate this process, triage teams should be given that patient information that is needed to make prognostic decisions while avoiding information that might introduce implicit biases. Reference Daniels10–Reference Tolchin, Hull and Kraschel13
However, it remains unclear what specific demographic and clinical information should be provided to these triage teams. Because the development of such a set of information items involves integrating and weighing complex clinical, operational, and ethical factors, a formal process of consensus-building among experts is a well-suited empirical approach to fill this informational gap. We conducted a Delphi study to develop an expert consensus on the appropriate types and presentation of patient information items that would be needed to support the functioning of a triage team during crisis capacity settings in the COVID-19 pandemic.
Methods
Guiding Principles
In 2019, the Washington (WA) state Department of Health approved the Crisis Standards of Care Guidance Framework, which was based on ethical guidance included in NAM reports. 14–16 The NAM’s ethical framework also informed the approach to this study, including guiding principles of fairness, duty to care, duty to steward resources, transparency, consistency, proportionality, and accountability. 3,14 This framework was reviewed with Delphi participants at each step of the study and participants were asked to refer to these principles when making decisions and comments.
Generation of Candidate Patient Information Items
The Northwest Healthcare Response Network serves as the regional disaster preparedness collaborative for the Western region of WA state. 17 This group also administers a regional Disaster Clinical Advisory Committee and assists in administering the WA state Department of Health Disaster Medical Advisory Committee. Members of these 2 committees include public health officials and clinicians of different disciplines from 18 specialties representing a majority the major health-care systems in the state. Early in the initial wave of the COVID-19 pandemic in WA state, members of these committees were invited to attend workgroup meetings to create a patient information form that could be used to collect clinical data needed to inform triage team deliberation, as well as to devise an approach to reporting triage team decisions that would support transparency and standardization. Over a series of 10 meetings, bioethics specialists reviewed principles relevant to health-care resource allocation, guest speakers from regional COVID-19 epicenters presented their real-world experience, and the group discussed concerns from historically disenfranchised populations. The group then iteratively developed a candidate list of patient information items thought to be potentially relevant to the triage process and frameworks for reporting triage decisions.
Delphi Participant Recruitment and Delphi Rounds
Members of the WA state Disaster Clinical Advisory Committee and Disaster Medical Advisory Committee attended a virtual meeting where the Delphi process was introduced, Reference Niederberger and Spranger18 and they were invited to participate. All Delphi participants were asked to complete an online survey to collect their demographic information and clinical backgrounds. Participants then sequentially viewed candidate information items by means of an online survey. Participants rated their agreement with whether a candidate patient information item should be supplied to a scarce resource triage team (response options included disagree, neutral, or agree). Each group of related patient information items included a free text box for participants to explain their rationale for ratings, propose alternative wording for an item, and/or suggest other candidate items. Participants were instructed to leave a question blank when the item required expertise outside their scope of practice.
Following each round of the Delphi process, the percentage of participants who agreed with each patient information item was calculated. A priori, agreement of <70% was defined as unlikely to achieve consensus, and items achieving less than this threshold were removed. Items with an agreement of ≥80% were accepted into the final form. Items with an indeterminant agreement (70-79%) were included in a subsequent round of the Delphi.
One study team member (M.M.G.) acted as a Delphi moderator and reviewed free text entries and collated these into a deidentified summary report. The full study team developed new candidate items and/or modified phrasing based upon participant comments. In each subsequent round, participants were able to review the percentage agreement and collated comments for each item. Delphi rounds continued until all items either reached ≥80% agreement and were included on the final patient information form or were removed.
Qualitative Analysis
As part of the Delphi process, participants wrote detailed comments describing their opinions and reasoning regarding the relevance of patient information items and proposed new items or alternative phrasing. One study team member with experience in qualitative methodology (C.R.B.) conducted a directed content analysis of these written comments. Reference Hsieh and Shannon19 Informed by the conceptual framework of ethical considerations guiding triage, 3 this team member coded written comments line-by-line, then iteratively reviewed concepts to identify relationships and categories and ultimately developed a set of higher-level concepts describing the factors and considerations that Delphi participants reported in their comments. All study team members together reviewed preliminary results of the content analysis and together developed the final set of higher-level concepts.
Data Collection Timing
Patient information used for triage in crisis capacity settings must be collected efficiently. To better understand the feasibility of collecting data included in the final patient information form, 1 study team member with clinical experience in emergency nursing (L.B.W.) completed the information form for a sample of patients within our local institution (Virginia Mason Medical Center). Detailed data entry timings for 10 patient records were collected using production system software tools. Timing for each step was analyzed using descriptive statistics.
The Benaroya Research Institute at Virginia Mason Institutional Review Board (IRB) determined the study to be exempt from IRB review (ID: STUDY00005901). All participants provided verbal consent to participate. Study data were collected and managed using REDCap electronic data capture tools hosted at the Institute of Translational Health Sciences. Reference Harris, Taylor and Thielke20 Statistical analysis was supported by STATA software (2018, STATACorp LLP, College Station, TX).
Results
Members of health-care disaster advisory committees in WA state identified 33 candidate patient information items potentially relevant to the triage process. These candidate information items were categorized as follows: patient and provider identifiers, patient care preferences, patient information, comorbidities, frailty and malnutrition, current physiology and disease process, length of stay, and response to treatment.
Three Delphi rounds took place between August and December, 2020. Among 188 members of the WA state disaster advisory committees invited to participate, 50 participated in Delphi round 1 (26% initial response rate); 46 of those who participated in round 1 participated in round 2 (92% retention), and 35 of those who participated in round 1 participated in round 3 (70% retention). The majority of participants were White (77%) and non-Hispanic (93%) (Table 1). Participants’ focus of clinical experience included acute care (43%), intensive care (35%), emergency medicine (30%), outpatient care (23%), nonclinical settings (7%), or other settings (8%). Thirty-two percent of participants were bioethicists or had clinical ethics experience.
Table 1. Characteristics of multidisciplinary experts participating in the Delphi study
a Six participants did not complete the demographic survey.
b Non-exclusive.
During 3 rounds of the Delphi, 20 of 33 items from the original list of patient information items met the 80% agreement threshold for inclusion in the final form and 13 items were dropped. Participants suggested 4 additional items, all 4 of which reached ≥80% agreement in subsequent Delphi rounds. A total of 24 items were included in the final form, including patient preferences for care, age, select comorbidities (chronic lung, heart, liver, oncologic, and kidney diseases), presence of limited life expectancy, duration of and reason for hospitalization, specific information on the severity of the current illness, and the clinical trajectory. The wording and agreement for each item are reported in Tables 2 and 3. Reworded questions are presented in separate rows. Participants received a range of options for the degree of severity necessary for a comorbidity to be included on the patient information form and there was agreement that specified conditions should be reported if they were “severe” and/or “terminal/end-stage.”
Of 3 options for framing the goal of the triage team deliberation, only the goal of reporting likelihood of survival to discharge reached consensus (Table 4). Four options for how triage teams could report their decisions were offered. While the 80% agreement threshold was not reached for any of these reporting frameworks, the option of having 5 prognostic categories achieved the highest level of agreement (65%) (Table 4). This reporting framework included 4 color categories defined by likelihood of survival to hospital discharge and a fifth “striped” category defined by a limited set of severe conditions that had been previously established as having especially poor outcomes (ie, high body surface area burns, severe trauma, and persistent vegetative state).
Qualitative Analysis
Qualitative analysis of experts’ written comments during the Delphi resulted in 3 inter-related themes (Supplementary Material Table 1). First, experts delineated multiple categories of features by which to judge the appropriateness of candidate information items. They commented on the relative value of each item in supporting prognostication and the likely consistency and accuracy with which the item conveyed the relevant information. They considered and suggested opportunities to reduce the impact of implicit biases, which could be introduced either through data entry by the clinical team or during triage team deliberations. Although participants were familiar with the ethical principles guiding the WA state triage process, they nonetheless grappled with ethical gray areas (eg, whether a pregnant person constituted 2 lives).
Second, experts used several approaches to navigate conflicting considerations in deciding whether and how an information item was appropriate for inclusion in the triage team process. They considered the relative weight of conflicting values, such as the prognostic value of an item versus the possibility of introducing bias into the process. They suggested different ways to phrase items or alternative data items that more specifically and objectively conveyed only information relevant to the narrow task of the triage team. Ultimately, many acknowledged the inevitable imperfection of any set of information items and suggested that community input and ongoing monitoring were needed to iteratively improve the final set of information items.
Finally, experts emphasized the relevance of processes surrounding triage team decision-making and pragmatic considerations of how the broader process would be operationalized. Experts weighed the value of information items versus the time it would take to complete data entry and for the triage team to review and interpret these data. They considered how features of the process might be adapted to avoid exacerbating clinicians’ moral distress and negative public perceptions of the process.
Data Collection Timing
For a random sample of 10 patients in our local electronic health record, the median time to identify and enter all administrative and clinical patient information items included on the final patient information form was 227 seconds (s) (inter quartile range [IQR] = 205, 298). Administrative data entry (including the creation of a triage identifier linked to the medical record number) took a median of 35 s (IQR = 31, 39). Patient information entry (including all patient information items that would be provided to the triage team) required a median of 187 s (IQR = 166, 280) per patient. The most time-consuming components of data collection included identifying and recording relevant comorbidities, classifying the severity of acute respiratory distress syndrome, and identifying the admission diagnosis (Supplementary Material Table 2).
Table 2. Items describing patients’ pre-hospitalization status evaluated by Delphi participants and percentage agreement in each round
Note: Instances of ≥80% agreement are bolded. These items were accepted for the final patient information form.
Abbreviations: ICU, intensive care unit; POLST, portable orders for life sustaining treatment.
a Implicit biases include those based on stereotypes, assessments of the quality of life by persons other than the patient/surrogate, judgments about a person’s “relative worth,” etc.
Table 3. Items describing patients’ current hospital status evaluated by Delphi participants and percentage agreement in each round
Instances of ≥80% agreement are bolded. These items were accepted for the final patient information form.
Abbreviations: ARDS, acute respiratory distress syndrome; ICU, intensive care unit; COVID-19, coronavirus disease 2019.
Table 4. Candidate strategies to frame the goal output of the triage team decision-making process and a reporting framework evaluated by Delphi participants and percentage agreement
Instances of ≥ 80% agreement are bolded. These items were accepted for the final patient information form.
Limitations
This study has several limitations. Experts were recruited at a time of intense stress on the health-care system during the COVID-19 pandemic, which necessitated an asynchronous Delphi format and likely contributed to the modest retention of participants between Delphi rounds. These factors may have limited the diversity of perspectives and opportunity for more nuanced discussion and deliberation around information items. Due to the limited volume and richness of the data available for qualitative analysis, we did not expect to reach thematic saturation, and the findings may not represent a comprehensive set of themes describing the deliberative process. Furthermore, demographics of participants reflect the disproportionately White population of health-care professionals, and additional input from community stakeholders is needed to identify opportunities to further minimize any potential biases. Finally, this study was intentionally embedded in the contextual details of the COVID-19 pandemic, but may require updates as understanding of disease processes evolve. The specific set of patient information items would also need to be tailored to any future health-care emergency, but this Delphi process may serve as a model for this work.
Discussion
In this Delphi study, experts on health-care triage and emergency planning reached consensus on a set of clinically and ethically relevant and appropriate patient information items needed to support triage team decision-making in crisis capacity settings during the COVID-19 pandemic. In developing this list, experts worked to optimize and balance multiple goals by selecting information items that would support accurate prognostication and consistency while avoiding factors associated with implicit biases. A final set of patient information items included patient preferences for care, a list of select severe or end-stage chronic comorbidities (chronic lung, heart, liver, oncologic, and kidney diseases), the reason for hospitalization, duration of ICU care, specific information on the severity of current illness, clinical trajectory, and age. Furthermore, our results support the feasibility of collecting these patient information items from an institutional electronic health record.
Delphi participants grappled with multiple—and often conflicting—priorities, including the need to provide sufficient information to support accurate prognostication, careful avoidance of potentially biasing information, as well as pragmatic considerations around time efficiency in developing a final list of patient information items needed to support triage. Balancing these priorities involved complex ethical, clinical, and operational considerations, and experts expressed a range of diverse perspectives about the most appropriate approach.
Despite multiple existing guidelines and approaches to disaster triage advocating the use of physiology-based scoring systems, Reference Raith, Udy and Bailey21,Reference Grissom, Brown and Kuttler22 deliberation among Delphi participants aligned with emerging concerns about the limitations and pitfalls of using these composite scores for emergency triage reflected in the broader literature. First, these scores may not provide sufficiently granular prognostic predictions to differentiate between seriously ill patients to be of value in disaster triage. Reference Raith, Udy and Bailey21,Reference Grissom, Brown and Kuttler22 Furthermore, these scores have not been validated to predict mortality for patients with COVID-19 or other future viral pandemics. Reference Ferreira, Bota and Bross23–Reference Raschke, Agarwal and Rangan25 Finally, these scores may incorporate both recognized and implicit biases. Reference Cleveland Manchanda, Couillard and Sivashanker11,Reference Sederstrom and Wiggleton-Little26,Reference Schmidt, Roberts and Eneanya27 This last point is especially pertinent because health-care emergencies often exacerbate existing health disparities, Reference Solomon, Wynia and Gostin28,Reference Webb Hooper, Nápoles and Pérez-Stable29 and while the triage process is unlikely to effectively correct for the many inequities in the health-care system, it should at a minimum seek to avoid worsening inequities. Reference Cleveland Manchanda, Couillard and Sivashanker11,Reference Tolchin, Hull and Kraschel13,Reference Solomon, Wynia and Gostin28,Reference Cleveland Manchanda, Sanky and Appel30,Reference Bruce and Tallman31
The question of whether to include age as a factor in triage team decision-making generated substantial debate among Delphi participants, which is also reflected in the broader literature. Reference Farrell, Ferrante and Brown32 Age is a strong predictor of mortality for critically ill patients, including for patients with COVID-19, Reference Raschke, Agarwal and Rangan25 and because older adults, on average, have a lower expected longevity, consideration of age in triage would result in a disproportionate number of older adults being denied treatment. Reference Montero-Odasso, Hogan and Lam33 However, there is substantial variability in survival among individuals of similar age, Reference de Rooij, Abu-Hanna and Levi34,Reference Brunner-Ziegler, Heinze and Ryffel35 and prevalent negative implicit biases directed at older adults—as well as the very young—risk shaping triage decisions to an extent beyond the prognostic relevance of this factor. Reference FitzGerald and Hurst36,Reference Chapman, Kaatz and Carnes37 Our results reflect agreement about the prognostic value of age, but also reinforce the importance of ongoing real-time monitoring for the impact of implicit biases throughout the triage process.
Consistent with prior work identifying factors associated with ICU mortality, Reference Zilberberg and Epstein38–Reference Chelluri, Im and Belle40 our findings reinforce the relevance of severe or end-stage chronic lung, heart, liver, oncologic, and kidney diseases in predicting prognosis. Delphi experts also considered conditions that may have a somewhat weaker association with prognosis, but are also known to be strongly shaped by health inequities and social determinants of health (eg, diabetes and hypertension). Reference Harrell, Hall and Taliaferro41,Reference Walker, Strom Williams and Egede42 These factors may also lack commonly agreed-upon definitions. The group ultimately determined that these factors did not—on balance—warrant inclusion on a final patient information form. Clinical frailty and malnutrition, which may have prognostic value for critically ill patients, were also excluded because of concerns about a lack of a consistent application and risk of bias against people living with disabilities. Reference Solomon, Wynia and Gostin28,Reference Chelluri, Im and Belle40 The inclusion of free text boxes on an information form was suggested as an opportunity for clinical teams to provide other valuable clinical context but this option was similarly believed to involve a high risk of introducing irrelevant and potentially biasing information into the triage team process. Ultimately, relatively few patient information items were included in a final patient information form, perhaps reflecting a default to exclude potentially biasing or ambiguous information. The development of specific definitions of each information item may serve to further support consistency and limit introduction of implicit biases in data collection and interpretation.
Past consensus processes identified several discrete clinical conditions as having a uniquely poor prognosis, including high body surface area burns, Reference Saffle, Gibran and Jordan43 severe trauma, and persistent vegetative state. These conditions have historically served as exclusionary criteria in some triage algorithms, including for WA state. 14 Our results suggest that this very specific and limited set of conditions may appropriately constitute a separate “striped” triage category. People affected by these conditions would be prioritized lower than other groups in crisis capacity settings, but rather than being categorically excluded from consideration for scarce resources (as in existing algorithms), they would receive therapy as possible, guided by the patients’ preferences. This approach supports proportionality, that is, withholding resources only to the extent necessitated by scarcity.
These findings lay the foundation for additional studies to precisely define, operationalize, and determine the effectiveness of this set of patient information items in supporting triage team functioning. Further evaluation of this process through triage team simulations as well as additional input from public stakeholders, especially those representing underserved populations, would strengthen the ethical foundation of this set of patient information items to be used for allocating scarce life-sustaining resources in a health-care crisis. Reference Cleveland Manchanda, Sanky and Appel30
Conclusions
Through an iterative Delphi process, experts in health-care emergency preparedness developed consensus on a set of patient information items and a reporting system that would support the work of an institutional triage team in a crisis capacity setting during the COVID-19 pandemic. While patient information items were selected to support prognostication and to minimize the impact of biases, future work is needed to validate the approach, and ongoing monitoring for opportunities to minimize biases and support equity will be critical in future study and/or implementation.
Supplementary Material
To view supplementary material for this article, please visit https://doi.org/10.1017/dmp.2021.351.
Acknowledgments
The authors acknowledge Josh Edrich, MPH, for his contributions to study coordination and the many expert participants who made this study possible.
Author Contributions
Drs. Gray and Butler contributed equally to this work and should be considered co-primary authors. M.M.G. had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. M.M.G., C.R.B., L.B.W., D.S.D., M.R.T., and V.L.S. contributed substantially to the study design, analysis, interpretation, and writing of the manuscript.
Conflict(s) of interest
No conflicts of interest exist for the study authors.
Open access
