A Modified Sequential Organ Failure Assessment Score for Critical Care Triage

Abstract

Objective: The Sequential Organ Failure Assessment (SOFA) score has been recommended for triage during a mass influx of critically ill patients, but it requires laboratory measurement of 4 parameters, which may be impractical with constrained resources. We hypothesized that a modified SOFA (MSOFA) score that requires only 1 laboratory measurement would predict patient outcome as effectively as the SOFA score.

Methods: After a retrospective derivation in a prospective observational study in a 24-bed medical, surgical, and trauma intensive care unit, we determined serial SOFA and MSOFA scores on all patients admitted during the 2008 calendar year and compared the ability to predict mortality and the need for mechanical ventilation.

Results: A total of 1770 patients (56% male patients) with a 30-day mortality of 10.5% were included in the study. Day 1 SOFA and MSOFA scores performed equally well at predicting mortality with an area under the receiver operating curve (AUC) of 0.83 (95% confidence interval 0.81-.85) and 0.84 (95% confidence interval 0.82-.85), respectively (P = .33 for comparison). Day 3 SOFA and MSOFA predicted mortality for the 828 patients remaining in the intensive care unit with an AUC of 0.78 and 0.79, respectively. Day 5 scores performed less well at predicting mortality. Day 1 SOFA and MSOFA predicted the need for mechanical ventilation on day 3, with an AUC of 0.83 and 0.82, respectively. Mortality for the highest category of SOFA and MSOFA score (>11 points) was 53% and 58%, respectively.

Conclusions: The MSOFA predicts mortality as well as the SOFA and is easier to implement in resource-constrained settings, but using either score as a triage tool would exclude many patients who would otherwise survive.

(Disaster Med Public Health Preparedness. 2010;4:277-284)

In an influenza pandemic, natural disaster, or human-made disaster, a mass influx of critically ill patients could overwhelm existing intensive care unit (ICU) resources. In this setting it would be helpful to predict which patients in the ICU would not survive despite full care and which patients require ongoing critical care resources, such as mechanical ventilators, so that equipment and personnel needs are anticipated. It may also be necessary to triage patients already in the ICU to reserve resources for those most likely to benefit from them. Useful triage tools would predict which patients have the greatest chance of survival, which patients are likely to die, and which patients will require mechanical ventilation. These are important issues not only for the initial allocation of resources but also for reallocation of resources over time to support the ongoing needs of the survivors of critical illness. The goals of this study are to evaluate the Sequential Organ Failure Assessment (SOFA) score and a simpler Modified SOFA (MSOFA) for their ability to predict mortality and the need for mechanical ventilation in critically ill patients after admission to the ICU.

The SOFA score is a validated measure of organ failure over time and a predictor of mortality in critically ill patients123 that has been incorporated into triage protocols for critical care in the event of an influenza pandemic or a mass influx of patients during a disaster.4567 The SOFA score combines a clinical assessment of 2 organ systems, cardiovascular system and central nervous system, with laboratory measurements for evaluation of 4 other organ systems: respiratory, hematologic, liver, and renal (Table 1). The requirement for arterial and venous blood specimens from each patient to calculate a SOFA score may prove impractical with a large number of patients and constrained resources. Given this limitation, we created the simpler MSOFA score8 (Table 2) to minimize reliance on laboratory resources. The MSOFA score eliminates the platelet count, replaces partial pressure of arterial oxygen (PaO ₂) with arterial oxygen saturation measured by a pulse oximeter (SpO₂), and replaces serum bilirubin with clinical assessment of scleral icterus or jaundice (Table 3). The only laboratory value required for the MSOFA is creatinine, which can be measured using a bedside point-of-care testing device.

TABLE 1 Sequential Organ Failure Assessment Score12

TABLE 2 Modified Sequential Organ Failure Assessment Score

TABLE 3 Comparison of the Sequential Organ Failure Assessment and Modified Sequential Organ Failure Assessment Scores

We hypothesized that the MSOFA score would predict mortality and need for mechanical ventilation as well as the SOFA score and thus could be used in triage protocols in resource-constrained critical care environments during a disaster or pandemic. We initially compared the MSOFA to the SOFA in a preliminary study using retrospective data from patients admitted to a 12-bed medical, surgical, and trauma ICU at LDS Hospital in Salt Lake City, Utah, during the 2006 calendar year. On the basis of this analysis, we prospectively compared the MSOFA with the SOFA in patients admitted to a 24-bed medical, surgical, and trauma ICU at Intermountain Medical Center in Murray, Utah, during the 2008 calendar year.

METHODS

In developing the MSOFA score, we identified bedside clinical parameters and electronic medical record data that could function as surrogates for the evaluation of respiratory and liver system organ function in the SOFA score. The cardiovascular, central nervous system, and renal scores for the MSOFA score remained the same as the SOFA score (Tables 1 to 3). We eliminated the hematologic system from the MSOFA score so that 5 organ systems are scored on a 0-to-4 scale according to specified criteria indicating severity, with a maximum score of 19 (Table 2). For the SOFA score, 6 organ systems are scored with a maximum score of 24 (Table 1).

To calculate the respiratory component of the MSOFA score, the SpO₂ is divided by the fraction of inspired oxygen (FiO₂) to calculate an SpO₂/FiO₂ ratio. For patients receiving nasal flow oxygen, we estimated FiO₂ by multiplying the liter flow per minute by 0.03 and adding that to 0.21. The SpO₂/FiO₂ ratio has been validated separately as a surrogate for the PaO ₂/FiO₂ ratio that is used by the SOFA score.910 The liver portion of the MSOFA score requires the observation of scleral icterus or jaundice documented by the bedside nurse.

MSOFA and SOFA scores were determined from parameters at the end of 24-hour time periods relative to ICU admission. For day 1 the time period was 6 hours before ICU admission to 18 hours after, for day 3 the time period was 48 to 72 hours after ICU admission, and for day 5 the time period was 96 to 120 hours after ICU admission. Scores were calculated only for patients who were alive and in the ICU for the entire duration of the day 1, day 3, or day 5 scoring period. The SOFA and MSOFA score on day 1 included only those patients who had survived the entire initial 18 hours in the ICU. The SOFA and MSOFA score on days 3 and 5 included only those patients who had survived at least 72 hours and 120 hours, respectively, and remained in the ICU at the end of the scoring period. We used only laboratory measurements that were obtained for the care of the patient during the specified time periods for the calculation of the MSOFA or SOFA scores. We selected the worst values for each parameter in the specified time period, except for the Glasgow Coma Scale (GCS), as was done in the primary studies on the SOFA score.1 For the GCS, we selected the maximum score in the 24-hour period. In the primary study on the SOFA, the presumed GCS score was used for sedated patients.1 Our standard protocol is to chart the actual GCS score determined regardless of sedation or paralysis; therefore, to evaluate the contribution of the GCS to the SOFA score, we also analyzed our data using the minimum GCS score in 24 hours for comparison. For a missing value, we used the closest available value up to 4 hours before the 24-hour scoring interval. Where no value was available, the predictor was assumed to be normal and assigned a score of 0.

LDS Hospital and the Intermountain Medical Center are both Intermountain Healthcare Hospitals and have mature electronic clinical information systems that include electronic charting of scleral icterus and jaundice, which allowed us to retrospectively determine the MSOFA and SOFA scores on ICU admission for every patient admitted to the LDS Hospital Shock Trauma Respiratory ICU in 2006. For the prospective study of the MSOFA and SOFA scores in the Shock Trauma ICU at Intermountain Medical Center, we instructed the clinical nursing staff of the study and specifically trained them to document jaundice consistently within the electronic medical record. All of the other components of the MSOFA and SOFA scores that required clinical evaluation were part of standard nursing bedside charting. The institutional review board of LDS Hospital and the Intermountain Medical Center, Intermountain Healthcare, approved this observational study with waiver of informed consent.

The endpoints used for validation of the SOFA and MSOFA scores were 30-day all-cause mortality and need for mechanical ventilation on days 3 and 5. Mechanical ventilation was prospectively recorded for all of the patients and the need for ventilation was defined as any receipt of mechanical ventilation on the relevant study day.

Statistical Methods

We compared MSOFA score mortality prediction to the SOFA score by areas under the receiver operating characteristic curve (AUC) using the technique of DeLong and colleagues11 and calculated an exact binomial confidence interval for the AUC. In the 2006 retrospective study, the MSOFA score performed well, but the evaluation revealed the need to standardize charting of jaundice by the nursing staff when they identified a patient with jaundice. This was done in 2007 and the prospective evaluation began in 2008.

To investigate the relevance of the evolution of organ dysfunction to mortality prediction for patients with scores on days 1, 3, and 5 during their ICU stay, we evaluated several summarization and trending transformations in a logistic regression model for both MSOFA and SOFA. For the day 3 model, we evaluated quantitative and qualitative change, summed scores, maximum score, and a linear prediction of the day 5 score. For the day 5 model, we included everything from the day 3 model (incorporating the day 1 and day 3 scores only, the day 3 and day 5 scores only, and all 3 days' scores). In addition, we fitted a nonlinear prediction model for day 5 to the available scores. The utility of adding additional terms to the model was evaluated within a forward stepwise bootstrap procedure. The “out of the bag” sample was used to avoid a biased estimate of AUC due to overfitting.1213 The most recent MSOFA was always selected first. Additional predictors were maintained in the model only if they significantly increased the AUC. A subsequent 1000-bootstrap sample estimate was used to confirm these findings.

In a secondary analysis we evaluated the day 1 scores as predictors of the receipt of mechanical ventilation on days 3 and 5, comparing AUC using the technique of DeLong et al11 and calculating exact binomial confidence intervals for the AUC. Analyses were performed using the R statistical package, version 2.9.1 (R Foundation for Statistical Computing, Vienna, Austria) and MedCalc statistical software, version 11.3.8 (MedCalc Software, Mariakerke, Belgium).

RESULTS

The retrospective derivation cohort in 2006 included 718 patients (58% male patients) with a 30-day inpatient mortality of 17.3%. In the 2006 cohort, day 1 SOFA and MSOFA scores predicted 30-day mortality with an AUC of 0.78 (95% confidence interval [CI] 0.74-0.82) and 0.77 (95% CI 0.73-0.82), respectively (P > .2 for comparison).

In 2008, 1803 patients (56% male patients) were admitted to the Shock Trauma ICU with a mortality of 12.1%. Of those 1803 patients, 33 patients died during the day 1 scoring period and were excluded from the analysis. The prospective study cohort in 2008 included 1770 patients (56% male patients) with a 30-day mortality of 10.5%. Table 4 compares the 2006 preliminary study retrospective cohort and the 2008 prospective cohort. Figure 1 shows the flow of patients admitted to the Shock Trauma ICU in 2008 and the number of patients included in the 2008 prospective days 1, 3, and 5 analyses. Table 5 shows the percentage of missing values for parameters collected for organ system scoring in calculating the SOFA and MSOFA scores during each scoring period. Table 6 shows the characteristics of survivors vs nonsurvivors in the 2008 prospective study including differences in SOFA and MSOFA scores.

FIGURE 1 Flowchart: Number of patients included in each scoring period.

TABLE 4 Comparison of 2006 and 2008 Groups

TABLE 5 Missing Values for Parameters Used in Organ System Scores for SOFA and MSOFA

TABLE 6 Comparison of Survivors and Nonsurvivors in the 2008 Prospective Study

In the 1770 patients alive at the end of the day 1 scoring period, the SOFA score calculated using the maximum GCS gave a greater AUC (0.83, 95% CI 0.81-0.85) as compared to the SOFA score calculated using the minimum GCS score (AUC 0.81, 95% CI 0.79-0.83; P < .001). The day 1 MSOFA score calculated using the maximum GCS score gave a greater AUC (0.84, 95% CI 0.82-0.85) as compared to the MSOFA score calculated using the minimum GCS score (AUC 0.82, 95% CI 0.80-0.84; P < .001). Because the SOFA and MSOFA scores calculated using the maximum GCS score on day 1 performed better than the SOFA and MSOFA scores calculated using the minimum GCS score, we used the SOFA and MSOFA scores calculated using the maximum GCS score in all further analyses.

There was no difference between the day 1 SOFA and MSOFA scores (P = .33) for predicting mortality (Figure 2). For day 3, SOFA and MSOFA scores performed similarly well at predicting mortality for the 828 patients remaining in the ICU, with an AUC of 0.78 (95% CI 0.75-0.81) and 0.79 (95% CI 0.76-0.82), respectively (P = .45). For day 5, SOFA and MSOFA scores performed less well at predicting mortality on the 369 patients remaining in the ICU, with an AUC of 0.72 (95% CI 0.67-0.76) and 0.74 (95% CI 0.70-0.79), respectively (P = .21). None of the novel predictors added significantly to the AUC for the day 1, 3, or 5 predictions.

FIGURE 2 A, Mortality by day 1 Sequential Organ Failure Assessment and Modified Sequential Organ Failure Assessment scores for prospectively collected data of patients admitted to the Shock Trauma ICU at Intermountain Medical Center during the 2008 calendar year. B, Analysis of area under the receiver operating curve for day 1 Sequential Organ Failure Assessment and Modified Sequential Organ Failure Assessment prediction of mortality in the 2008 cohort.

The day 1 SOFA and MSOFA scores predicted the need for mechanical ventilation on day 3 with an AUC of 0.83 (95% CI 0.81-0.84) and 0.82 (95% CI 0.80-0.84), respectively (P = .43). Of the 828 patients remaining in the ICU on day 3, 202 (24%) were receiving mechanical ventilation. The day 1 SOFA and MSOFA scores predicted the need for mechanical ventilation on day 5 with an AUC of 0.76 (95% CI 0.74-0.78) for both (P = .81). Of the 369 patients remaining in the ICU on day 5, 136 (37%) were receiving mechanical ventilation.

Previously published pandemic influenza triage protocols use threshold values of SOFA scores >11 for exclusion from ICU admission, 8 to 11 for intermediate priority, and <8 for high priority for ICU admission.456 Mortality for threshold levels of MSOFA and SOFA scores are shown in Figure 2A. Day 1 SOFA scores in the 2008 study had 1450 patients with a score of 0 to 7 and a mortality of 5% (70 patients), 216 patients with a score of 8 to 11 and a mortality of 28% (60 patients), and 104 patients with a score of >11 and a mortality of 53% (55 patients). Day 1 MSOFA scores in the 2008 study had 1440 patients with a score of 0 to 7 and a mortality of 4% (64 patients), 259 patients with a score of 8 to 11 and a mortality of 31% (80 patients), and 71 patients with a score of >11 and a mortality of 58% (41 patients).

We performed a subgroup analysis for prediction of mortality from day 1 SOFA or MSOFA score for medical patients (Table 4, n = 991, 13% mortality), postoperative surgical patients (n = 244, 4% mortality), and trauma patients (n = 495, 8% mortality). Among the medical patients, day 1 SOFA and MSOFA scores predicted mortality with an AUC of 0.82 (95% CI 0.79-0.84) and 0.82 (95% CI 0.80-0.85), respectively (P = .59). In the postoperative surgical patients, day 1 SOFA and MSOFA scores predicted mortality with an AUC of 0.70 (95% CI 0.64-0.76) and 0.84 (95% CI 0.79-0.88), respectively (P = .024). In the trauma patients, day 1 SOFA and MSOFA scores predicted mortality with an AUC of 0.87 (95% CI 0.84-0.90) and 0.84 (95% CI 0.80-0.87), respectively (P = .03).

DISCUSSION

Future H1N1 influenza pandemics,1415 outbreaks of avian influenza,16171819 natural disasters,7 or human-made disasters may overwhelm critical care capacities. The number of available mechanical ventilators and ICU beds may not be sufficient for the number of people who are affected.202122 Understanding which patients are most likely to benefit from receiving critical care resources may allow for triage of patients in a manner that provides the highest likelihood of survival for the most people. Complex mortality prediction models are labor and resource intensive and are difficult to apply in a mass patient influx.2324 Algorithms specific to mass casualty trauma triage25 are not generalizable for a mass influx of influenza patients or for the general population of nontrauma critically ill patients who will compete for critical care resources during a pandemic. A simple triage scoring system focusing on age, respiratory failure, and shock index for use during epidemics has been validated in a cohort of emergency department patients admitted to the hospital with nonsurgical infection and performed moderately well at predicting mortality (AUC 0.73) or need for mechanical ventilation (AUC 0.68),21 but restriction of the cohort to infected patients limits generalizability to triage of acutely injured patients.

The SOFA score has been recommended as a tool in critical care triage456726 because of ease of calculation, if the required laboratory tests are available, and validation takes place in a wide variety of critically ill patients.123 Reliance on multiple laboratory values, however, may make application impractical during mass critical care triage. We showed that an equivalent mortality prediction can be performed with an MSOFA score using variables easily collected at the bedside. A much smaller study of 144 surgical ICU patients in Indonesia suggested the utility of the MSOFA with an equivalent mortality prediction to the SOFA score.27 In our prospective study, the MSOFA score predicted mortality with a high AUC of 0.84 that was equivalent to that of the SOFA score. The MSOFA score also performed well in predicting mortality after 3 days of ICU stay, with an AUC of 0.79, again equivalent to the SOFA score. Prediction of mortality at day 5 of ICU stay in 369 critically ill patients was less robust for both the MSOFA and SOFA scores, which is a limitation for application of the SOFA and MSOFA to predict outcomes during the ICU stay. Both the SOFA and MSOFA scores have been proposed as tools to retriage patients during their ICU course, and the degradation of predictive utility over time may have implications for published protocols.4578 Beyond mortality, the initial SOFA and MSOFA scores both predicted ongoing requirements for mechanical ventilation with no clear difference between them.

In the subgroup analysis separating medical, surgical, and trauma patients, the SOFA and MSOFA performed equally well at predicting mortality in medical patients (AUC 0.82 for both). The MSOFA (AUC 0.84) performed much better at predicting mortality than the SOFA (AUC 0.70) in the postoperative surgical patients (P = .024), but the mortality was low (4%) and the number of patients (244) was small, so the significance is unclear. In the larger group of trauma patients (n = 495) with an 8% mortality, both the SOFA and MSOFA performed well with AUC of 0.87 and 0.84, respectively (P = .03). Medical and trauma patients are those most relevant to mass casualty and triage.

A convenient result of our study is that the MSOFA and SOFA scores directly correlate in the range of the thresholds used in pandemic influenza triage plans in which a score of >11 excludes a patient from critical care, whereas a score of 8 to 11 is an intermediate priority and <8 is a high priority for critical care.45 The MSOFA score predicts mortality in these categories equally as well as the SOFA score at the relevant thresholds of 8 and 11 (Figure 2). The SOFA and MSOFA, however, have different scoring ranges (0-24 for SOFA and 0-19 for MSOFA) and it will require further study to confirm that they can be used interchangeably. The threshold MSOFA and SOFA score of >11 used to exclude patients from critical care includes only about 6% of total patients in our cohort, and up to 40% of the excluded patients would survive in a setting where ample critical care resources were available. This emphasizes that these scoring systems can only be 1 part of an overall system of triage. We specifically did not evaluate the effect of significant comorbidities on triage, which is included in published triage protocols that use SOFA or MSOFA.5826

A recent study emphasized that the practical implementation of a triage system that includes a SOFA score requires training for those who will act as triage officers and protocol modification to minimize the exclusion from critical care of patients who may in fact benefit.28 Findings in that same study suggested that application of a SOFA score–based critical care triage protocol could direct scarce resources to patients who are most likely to benefit from them while increasing resource availability.28 Two other recent studies, however, have raised concern about using a SOFA score–based critical care triage. One small study evaluating application of the SOFA score retrospectively for critical care triage to 8 patients with 2009 H1N1 influenza found that 5 survivors would have been candidates for withdrawal of life support at 48 hours according to a SOFA score triage.29 A larger study applying SOFA score triage criteria to a mix of 255 patients admitted to an ICU found that 116 patients with 39% survival would have been denied admission.30

The results of these studies and our study suggest that further evaluation and public discussion is necessary to refine a plan for the allocation of scarce resources and life-sustaining treatments during a public health emergency with a mass influx of critically ill or injured patients.313233

The results from our study and others282930 suggest that a large number of critically ill patients who have SOFA or MSOFA scores >11 may survive in a mass critical care patient influx, such as an influenza pandemic, and require ongoing care and resource reallocation. For example, patients critically ill with pandemic influenza and SOFA scores >11 may be delegated in a triage system to receive a freestanding continuous positive airway pressure system, rather than positive pressure mechanical ventilation, with an FiO₂ of 1.0 to conserve mechanical ventilators for those who have the greatest chance of survival. Some of those patients delegated to continuous positive airway pressure will survive and may have SOFA or MSOFA scores that improve and assign them a higher priority than other critically ill patients for allocation of resources. Scenarios such as this include not only resource allocation but also resource reallocation, and will require further study and careful planning.

A limitation of our study is the focus on triage of patients after, rather than before, admission to the ICU. Determination of which critically ill patients should be admitted to an ICU is an important component of mass critical care triage. The SOFA score has been applied to initial critical care triage in some published triage protocols,4567 even though the SOFA score has only been validated as a severity of illness score for patients in the ICU.12 The MSOFA score performed similarly to the SOFA score in our study, suggesting that the MSOFA may also be useful for initial triage for admission to the ICU from the emergency department. The day 1 data in our study reflect the acute presentation of critically ill patients because the majority of admissions to our ICU come through the emergency department (75%), and the time spent in the emergency department for patients admitted to the Shock Trauma ICU during 2008 was brief: 3.7 ± 2.0 hours. Further study is warranted to determine the utility of the SOFA or MSOFA scores for triage of critically ill patients in the emergency department for admission to the ICU.

CONCLUSIONS

The MSOFA score predicts mortality in critically ill patients as effectively as the SOFA score and is easier to implement in resource-constrained or pandemic settings. The mortality in the highest scoring group with an SOFA or MSOFA of >11 was only 53% to 58%, which means that implementation of either the MSOFA or SOFA score as a triage tool during a mass influx of critically ill patients would exclude a significant number of patients who would survive with usual critical care resources. Our findings are consistent with recommendations that a triage tool, including the SOFA or MSOFA score, should be applied only when critical care resources are overwhelmed and all of the efforts to obtain additional resources have been instituted726 and should be combined with criteria that exclude patients based on comorbid conditions that increase mortality.58 These are serious issues for consideration and warrant further study and careful ethical and social discussion.