Sensitivity and Specificity of Spanish Prehospital Advanced Triage Method (META)

Abstract

Introduction:

Mass-casualty incident (MCI) triage systems aim to provide the best possible health care to the greatest number of affected people with the available resources in the context of a mass-casualty event. The Spanish Prehospital Advanced Triage Method (Modelo Extrahospitalario de Triaje Avanzado; META) was designed to improved patient sorting in MCIs.

Objective:

The aim of this study was to estimate and compare sensitivity and specificity of META and the Manchester Triage System (MTS) in MCIs by retrospectively applying both triage algorithms to real MCI patients from the Emergency and Disaster Research Unit (Unidad de Investigación en Emergencia y Desastres; UIED) MCI database.

Method:

This was a comparative study of two triage methods using sensitivity and specificity with the Revised Trauma Score (RTS) as the gold standard. A total of 134 MCI patients from the UIED database were included. An MCI in Asturias is defined as an incident that involves four or more victims that require ambulance mobilization.

Results:

Patients mean age was 39.85 years (95% CI, 35.9-43.8) with an age range from one to 88 years old. In total, 54.4% of the patients were female. The most common types of MCI involved were fires (51.0%), followed by road traffic accident (43.3%) and street fight (3.7%). For MTS, the overall sensitivity was 30.6% (95% CI, 22.9-39.1) and specificity was 66.0% (95% CI, 60.0-71.7). For META triage algorithm, the overall sensitivity was 79.9% (95% CI, 72.1-86.3) and specificity was 89.9% (95% CI, 85.7-93.3).

Conclusion:

The META triage algorithm is a reliable triage system; thus, it can be recommended to be used in an MCI.

Introduction

Mass-casualty incident (MCI) triage systems aim to provide the best possible health care to the greatest number of affected people with the available resources in the context of a mass-casualty event where the number of patients exceeds the capacity of the first responders at the scene, such as during disasters. ¹ The word “triage” originates from the French word “trier,” which refers to the categorization, classification, and prioritization of patients and management of casualties resulting from battlefield injuries in the 1800s. ² Triage is defined as: “the sorting of and allocation of treatment to patients and especially battle and disaster victims according to a system of priorities designed to maximize the number of survivors.” ³ There are many types of triage systems world-wide suited for different populations and countries. ⁴

In Spain, the Manchester Triage System (MTS) is used in more than 70 hospitals, including the Central University Hospital of Asturias (Asturias, Spain), ^5,6 so it would be used in MCIs upon arrival of patients to the emergency department. In 2011, the Unit for Research in Emergency and Disaster (Unidad de Investigación en Emergencia y Desastres; UIED) in University of Oviedo (Oviedo, Spain) developed a new Spanish Prehospital Advanced Triage Method (Modelo Extrahospitalario de Triaje Avanzado; META) ⁷ designed to be implemented as prehospital triage in MCIs. ⁸ The META triage system has two main steps: primary triage or “treatment triage,” referring to establishing on-scene treatment priority; and evacuation triage, for patients to be evacuated once evaluated and treated on scene. Treatment triage has four steps: establish treatment priority according to ABCD approach, establish need of urgent surgical evaluation, adequate “on scene” treatment, and establish evacuation priority (Figure 1 and Figure 2).

Figure 1. Treatment Triage in META Model.

Abbreviations: META, Modelo Extrahospitalario de Triaje Avanzado (Out-of-Hospital Advanced Triage Model); ATLS, Advanced Trauma Life Support.

Figure 2. Evacuation Triage in META Model.

Abbreviations: META, Modelo Extrahospitalario de Triaje Avanzado (Out-of-Hospital Advanced Triage Model); HPC, high priority criteria; GCSm, Glasgow Coma Scoremotor; SBP, systolic blood pressure.

Also, it establishes the so called “surgical priority” for those patients that benefit from urgent evacuation to an operating theater. The META triage system improves evacuation in MCIs ⁹ and is beneficial to rapidly identify severe trauma patients. ¹⁰ However, there is limited research available in comparing the reliability among different triage systems in MCIs.

The selection of a suitable triage system is vital for MCI and disaster preparedness. ¹¹ Appropriate triage protocols will increase the ability to correctly differentiate both high and low urgency patients, and the opposite will lead to misusing valuable resources on over-triaged patients and jeopardizing under-triaged ones. ¹² Nevertheless, most Emergency Medical Systems developed their own MCI triage and response protocols based on internationally accepted ideas and recommendations, but not based on best evidence practice. ¹³ In addition, the nature of MCIs is not suitable for randomized studies to compare different triage systems. ¹⁴

In this study, the aim was to estimate and compare the sensitivity and specificity of the META and MTS triage algorithms for MCIs in Spain by retrospectively applying both triage algorithms to real MCI patients from the UIED Asturias MCI database and to analyze the sensitivity and specificity by using the Revised Trauma Score (RTS) as the gold standard of reference – a well-established predictor of mortality in trauma populations ¹⁵ that commonly occurs in MCIs.

Methodology

This was a comparative study of two triage methods using sensitivity and specificity with RTS as the gold standard. The researchers retrospectively applied MTS and META triage algorithms to MCI patient data using Asturias MCI and UIED database ¹⁶ from January 2017 through August 2019. An MCI in Asturias is defined, as for the database, as an incident that involves four or more victims that require ambulance mobilization. ¹⁷ No explicit exclusion criteria were used in this research. Data were uploaded by one single person in each Emergency Medical Service, and a data dictionary was accessible for all of them after a training program to improve accuracy and reliability of the database. Researchers selected all patients involved in an MCI that were transported to Central University Hospital of Asturias (HUCA; Asturias, Spain); HUCA is the main reference hospital with Level 1 trauma center and 1,039 beds for Principality of Asturias in Spain, one of the northern Spanish regions covering more than one million in population. ¹⁸

All data necessary for triage color-coding patient were studied. All patients were triaged according to MTS, META, and RTS. For analysis purpose, orange and yellow categories of the MTS were merged into only the yellow category. Researchers then analyzed the triage color coding of all patients to determine their sensitivity, specificity, under-triage, and over-triage rates by using the Cribari Matrix with 95% CI. They then calculated Cohen Kappa’s interrater reliability coefficient to compare between MTS versus RTS and META versus RTS, according to three triage color – red, yellow, and green. The RTS was used as the gold standard for these calculations. StatPlus software version 8.0 (AnalystSoft Inc.; Walnut, California USA) was used for data analysis. The study was approved by the Research Ethical Committee of the Principality of Asturias, Spain (ref.: 2020/079).

Results

In total, 134 patients were included in this study; the mean age was 39.85 years (95% CI, 35.9-43.8) and age range from one to 88 years old. Overall, 54.4% of the patients were female. The most common types of MCI involved were fires (51.0%), followed by road traffic accident (43.3%) and street fight (3.7%). The number of patients triaged in different color codes of red (immediate), yellow (delayed), and green (minimal) are represented in Table 1.

Table 1. MTS, META, and RTS Triage Results According to Patient Clinical Severity and Priority

Abbreviations: META, Modelo Extrahospitalario de Triaje Avanzado (Out-of-Hospital Advanced Triage Model); MTS, Manchester Triage System; RTS, Revised Trauma Score.

Cohen Kappa’s interrater reliability coefficient results are tabulated in Table 2. For MTS versus RTS, the Cohen’s Kappa coefficient for red was 0.72, indicating substantial agreement (97.8% agreement). For yellow, Cohen’s Kappa coefficient was −0.02, indicating no agreement (31.3% agreement). Green’s Cohen’s Kappa coefficient was 0.04, indicating slight agreement (33.6% agreement). Cohen’s Kappa coefficient for META versus RTS in red was 0.69, indicating substantial agreement (96.3% agreement). For yellow, Cohen’s Kappa coefficient was 0.03, indicating slight agreement (82.1% agreement). Green’s Cohen’s Kappa coefficient was 0.34, indicating fair agreement (81.3% agreement). From the overall 134 cases, MTS resulted in 89 over-triage, three under-triage, and 42 correct triage levels. To identify red cases, MTS’ sensitivity was 66.7% (95% CI, 22.3-95.7) and specificity was 99.2% (95% CI, 95.7-100.0). For yellow/orange case, the sensitivity was 50.0% (95% CI, 6.8-93.2) and specificity was 30.8% (95% CI, 23.0-39.5). Green case’s sensitivity was 29.0% (95% CI, 20.5-37.0) and specificity was 90.0% (95% CI, 55.5-99.8). The overall sensitivity was 30.6% (95% CI, 22.9-39.1) and specificity was 66.0% (95% CI, 60.0-71.7). Overall under-triage rate for MTS was 2.2% (95% CI, 0.5-6.4) and over-triage rate was 66.4% (95% CI, 57.8-74.3). The positive predictive values ranged from 2.1% to 97.3% and negative predictive value from 9.3% to 98.4% (Table 3).

Table 2. Cohen Kappa’s Interrater Reliability Coefficient Comparison Result

Abbreviations: META, Modelo Extrahospitalario de Triaje Avanzado (Out-of-Hospital Advanced Triage Model); MTS, Manchester Triage System; RTS, Revised Trauma Score.

Table 3. Manchester Triage System (MTS) Results

The META triage resulted in 26 over-triage, one under-triage, and 107 correct triage levels. To identify red cases, META’s triage sensitivity was 100.0% (95% CI, 54.0-100.0) and specificity was 96.0% (95% CI, 91.1-98.7). For yellow/orange cases, the sensitivity was 25.0% (95% CI, 0.6-80.6) and specificity was 83.8% (95% CI, 76.3-89.7). Green case’s sensitivity was 80.6% (95% CI, 72.6-87.2) and specificity was 90.0% (95% CI, 55.5-99.8). The overall sensitivity was 79.9% (95% CI, 72.1-86.3) and specificity was 89.9% (95% CI, 85.7-93.3). Overall under-triage rate for META triage system was 0.7% (95% CI, 0.0-4.1) and over-triage was 19.4% (95% CI, 13.1-27.1). The positive predictive values ranged from 4.5% to 99.0% and negative predictive value from 27.2% to 100.0% (Table 4).

Table 4. META Results

Abbreviation: META, Modelo Extrahospitalario de Triaje Avanzado (Out-of-Hospital Advanced Triage Model).

Discussion

Mass-casualty incidents are low frequency phenomenon that need a more proper epidemiological approach that improves knowledge and research. By using a population-based MCI register for the first time, accuracy of the META triage system was calculated to detect severe patients in the prehospital setting for MCIs. Both MTS and META triage algorithms have been analyzed by using RTS as the gold standard. Although RTS is widely used and is a well-established predictor of mortality in trauma, the calculation of the RTS is too complicated to be used in the field, ¹⁹ such as during MCIs where the number of patients temporary exceeds the capability of the first responders at the scene. Moreover, the use of RTS might not have high reliability when the primary triage of the MCI was done by first responder who might be non-medical background personnel. ²⁰

Both MTS and META triage have a notable difference in overall sensitivity, specificity, under-triage rate, and over-triage rate. The META triage algorithm has more superior overall sensitivity (META 79.9% and MTS 30.6%) and specificity (META 89.9% and MTS 66.0%) and lower overall under-triage rate (META 0.7% and MTS 2.2%) and over-triage rate (META 19.4% and MTS 66.4%) as compared to MTS.

Two noteworthy findings from MTS are that it has a higher specificity to identify red cases (MTS 99.2% and META 96.0%) and a higher sensitivity to identify yellow cases (MTS 50.0% and META 25.0%). Nevertheless, the sensitivity in identifying red cases in the META triage algorithm is higher (META 100.0% and MTS 66.7%). In addition, to identify yellow cases, META triage also has a higher specificity (META 83.8% and MTS 30.8%).

Another interesting finding from MTS is that it has a lower over-triage rate for red cases (MTS 20.0% and META 45.0%). It is likely that the high sensitivity but relatively low specificity of the META triage algorithm contributed to this finding. Nevertheless, the overall over-triage rate and under-triage rate of META is lower than that of MTS.

Mass-casualty incident triage emphasizes sorting, prioritizing, and resource allocation to achieve best outcomes in survival, quality of life, and resource consumption in the temporary state of insufficiency. ²¹ Sensitivity and specificity of mass-casualty triage is important in making sure that the right patient was correctly sorted, correctly prioritized to be transported, and receives life-saving intervention without wasting resources. ²² It was found that META has a good performance and could be a useful tool to sort patients in cases of MCI, especially to detect severely injured patients.

Strengths

The main strength of this study is that data were used from an MCI database that included patients from real MCIs with a different profile from other trauma patients that are usually used for this type of studies.

Due to the scarcity of the MCI cases, MCI patients from all age groups were included into this study. Future research could study the sensitivity and specificity of different MCI triage in different age groups. Lastly, MCI patients who died on scene (triage color code black) were certified dead on the spot and were not transferred to the HUCA hospital, and thus, were not included in the database and this study.

On the other hand, this study is a pioneer in using population-based, real MCI cases to determine the sensitivity and specificity of triage algorithms. More studies from a public health perspective should be developed to improve MCI knowledge and triage systems.

Limitations

The limitations of this study are that this was a retrospective study and information was collected from the UIED database. Retrospectively triaging patients is not comparable to first-hand visualization of the patient from the scene. In addition, the data on outcome of the patients were not available. It is possible in the future to correlate the MCI triage with the clinical outcome of the patient.

Another limitation of this study is the sample size and demographic. The large majority of patient in this study involved fire, road traffic accident, and street fight. It might not accurately represent patients needing triage after a mass-casualty event such as chemical leak or explosion. The nature of disaster and MCI makes research on the response difficult, but not impossible. Future research could collect data from all MCI data available and classify them according to the type of disaster and to study the sensitivity and specificity on different triage systems in different disaster mass-casualty events. It may also be a limitation that there is no agreement about the gold standard to evaluate accuracy of triage methods in MCIs. This study used RTS as it is a reliable and well-known trauma score based on patient physiological data. As with other systems used as gold standards, their own internal validity may also have some bias on the results.

Conclusion

The META triage algorithm is a reliable triage system with very good sensitivity and specificity, not inferior to MTS, thus can be recommended to be used in MCIs. This finding has the usual limitation of the study of low frequency epidemiological phenomena. It is feasible to analyze triage algorithms using real MCI patients, but, as in this case, it requires a population-based registry.