INTRODUCTION
Early warning scores (EWSs) identify patients at risk of critical deterioration, as defined by the need for intensive care unit (ICU) admission, cardiopulmonary resuscitation, or death. There have been a variety of EWSs proposed, and they share common features. EWSs combine commonly measured physiological parameters, including heart rate, respiratory rate, temperature, blood pressure, oxygen saturation, and level of consciousness. The magnitude of derangement of each parameter from normal is used to generate a score indicating the patient’s risk for deterioration. EWSs can predict risk of death and ICU admission among ward patients.Reference Smith, Prytherch and Meredith 1 Furthermore, combining an EWS with a rapid response team can improve survival and reduce cardiac arrest rates.Reference McNeill and Bryden 2 Although developed for use in admitted patients, recent evidence suggests that EWSs may be useful in the emergency department (ED).Reference Brabrand, Folkestad and Clausen 3
Researchers have begun investigating the utility of an EWS calculated from triage vitals to predict the risk of subsequent patient deterioration. In a prospective observational cohort study, a higher EWS at triage predicted increased likelihood of requiring ICU admission and greater risk of in-hospital mortality.Reference Alam, Vegting and Houben 4 , Reference Groarke, Gallagher and Stack 5 A recent study showed that an EWS using prehospital vitals also predicted similar outcomes.Reference Silcock, Corfield and Gowens 6 EWSs may be useful in the ED, but work is needed to determine how these scores may be optimally used. Earlier recognition of sepsis may be one benefit of using an EWS in the ED.
Sepsis is a common ED presentation and a major cause of morbidity and mortality in Canada.Reference Martin, Priestap and Fisher 7 Sepsis recognition is time-sensitive because early identification and treatment are associated with improved outcomes.Reference Gaieski, Mikkelsen and Band 8 Unfortunately, sepsis is sometimes not identified at ED triage,Reference Chamberlain, Willis and Clark 9 and delayed identification is a potentially preventable cause of death.Reference Goulet, Guerand and Bloom 10 EWSs may help identify patients with occult sepsis. In a recent retrospective study conducted in a British ED, an EWS was able to identify patients at risk for septic shock with a 92% sensitivity and 77% specificity.Reference Keep, Messmer and Sladden 11 Furthermore, among septic patients, a higher EWS at triage predicts an increased likelihood of ICU admission and 30-day mortality.Reference Corfield, Lees and Zealley 12 These results suggest that the use of EWSs in the ED merits further research.
The Hamilton Early Warning Score (HEWS) is a novel EWS developed as part of a quality improvement and patient safety process at our centre. A committee consisting of physician leads, clinical managers, and rapid response team personnel conducted an extensive literature review of EWSs. The HEWS was developed based upon this review with modifications to existing scores based upon committee consensus. These modifications included a scoring trigger for delirium, gradation of the neurological score depending on the severity of depression in level of consciousness, and gradation in score depending on the amount of supplemental oxygen required. The details of the HEWS are provided in Table 1. In an initial study, the HEWS was found to be a significant predictor of critical inpatient events. Furthermore, the combination of HEWS and a rapid response team reduced the incidence of cardiac arrest and death.Reference Tam, Xu and Fox-Robichaud 13
Table 1 The Hamilton Early Warning Score (HEWS)Reference Tam, Xu and Fox-Robichaud 13
a If agitated, then the central nervous system (CNS) status is assessed using confusion assessment method (CAM). If level of consciousness is decreased, then assessed according to the AVPU scale whereby the patient’s consciousness is assessed.
A=alert; CAM+ve=Confusion assessment method (CAM) screening positive for potential delerium; P=pain; U=unresponsive; V=voice.
The current study is a retrospective review examining HEWS at ED triage as a predictor of the occurrence of critical events during admission. A secondary goal was to examine HEWS as a predictor of sepsis and the occurrence of critical events among septic patients. The most widely used and validated EWS is the National Early Warning Score (NEWS), which is used within the British National Health Service.Reference Smith, Prytherch and Meredith 1 The details of the NEWS are described in Table 2. For comparative purposes, NEWS at triage was also calculated and analysed as a predictor of outcomes.
Table 2 The National Early Warning Score (NEWS)Reference Smith, Prytherch and Meredith 1
a If agitated, then the central nervous system (CNS) status is assessed using confusion assessment method (CAM). If level of consciousness is decreased, then assessed according to the AVPU scale whereby the patient’s consciousness is assessed.
A=alert; V=voice; P=pain; U=unresponsive.
METHODS
Study setting
Approval was obtained from our local research ethics boards. This study was conducted at two academic, tertiary care, adult hospitals affiliated with Hamilton Health Sciences with a combined total annual ED census of 86,000. This study is a secondary analysis of a larger prospectively identified database of consecutive patients admitted to medical and surgical floors during the 6-month implementation of HEWS on the wards, which began in January 2014.Reference Tam, Xu and Fox-Robichaud 13 , Reference Tam and Fox-Robichaud 14 The current study included only patients who were admitted through the ED.
Data collection methods
Data were collected using techniques recommended for ED medical record reviews.Reference Worster, Bledsoe and Cleve 15 Two investigators (LM and AV) were trained to abstract data from medical charts. These investigators were blinded to the study purpose during the period of data abstraction. A standardized protocol with pre-specified variables definitions was created for data abstraction. Data were entered into an electronic spreadsheet created using Microsoft Access 2010 (Microsoft Corp., Redmond, WA). A random sample of 10% of records was reviewed by a second abstractor to allow calculation of interrater agreement using the kappa statistic (κ). Disagreements or uncertainties in coding were resolved by consensus between the data abstractors and the senior resident principal investigator (SS). In all cases, the faculty principal investigator (AFR) was available to review any cases where consensus could not be reached.
Identification of cases and controls
Charts were reviewed for a critical event identified as an in-hospital death, cardiac arrest, or ICU transfer. Cardiac arrest included pulseless ventricular tachycardia, ventricular fibrillation, pulseless electrical activity/asystole, or respiratory arrests. ICU transfers were specifically unexpected ICU transfers and excluded patients who were admitted directly to the ICU from the ED or transferred to the ICU for postoperative monitoring. Cases were patients who experienced a critical event during their hospitalization. A cohort of patients who did not experience a critical event were selected as controls. Controls were selected randomly to match cases in a two-to-one fashion. To match cases and controls in terms of severity of comorbid illness, the Charlson Comorbidity Index (CCI) was used. The CCI is designed to predict a patient’s 1-year mortality based upon his or her burden of comorbid disease and is commonly in health records research.Reference Needham, Scales and Laupacis 16 A CCI score was calculated for each patient based upon review of his or her electronic medical record. A random selection of controls was weighted to match cases on the patients’ CCI scores to ensure a similar baseline burden of disease in the groups.
Abstraction of emergency department data
Demographic variables were abstracted, including age, gender, mode of ED arrival, and Canadian Triage and Acuity Scale (CTAS) level. HEWS was calculated based upon triage vitals. A standardized process was used to score neurological status based on nursing documentation of level of consciousness or the presence of delirium. HEWS was documented as missing if neurological assessment could not be interpreted or if any of the vital signs were incomplete. The presence of the systemic inflammatory response syndrome (SIRS) was determined based upon initial triage vitals and ED blood work using the American College of Chest Physicians – Society of Critical Care Medicine criteria for SIRS.Reference Bone, Balk and Cerra 17 The presence of an infection-related diagnoses was determined based upon recorded admission diagnosis using diagnostic codes from the Acute Physiology and Chronic Health Evaluation (APACHE) III prognostic system. A similar technique has been used in contemporary sepsis studies.Reference Kaukonen, Bailey and Pilcher 18 Patients who met SIRS criteria and were admitted with an infection-related diagnoses were considered septic.
Data analysis
Analyses were performed using SPSS Statistics Version 23 (IBM, Chicago, IL) and were conducted by study investigators (SS and MX) with graduate-level training in statistics. Differences between cases and controls were examined using t-tests for continuous variables and a chi-square test analysis for categorical variable. The primary outcome of interest was the ability of HEWS at triage to predict patients who experienced a critical event during their admission. This was evaluated by plotting the sensitivity (true positives) against one-specificity (false positives) using receiver operating characteristic (ROC) curves and calculating the area under the curve (AUC) to evaluate the overall performance of HEWS. Sensitivity, specificity, positive likelihood ratios, and negative likelihood ratios for various HEWS cut-offs were also calculated. In line with cut-offs used with admitted ward patients, comparisons were made across three groups of patients: moderate-risk (HEWS≥3), high-risk (HEWS≥6), and very high-risk (HEWS≥9).Reference Tam and Fox-Robichaud 14 A secondary study goal was to examine HEWS as a predictor of sepsis. As such, a similar ROC curve analysis was performed to examine HEWS as a predictor of meeting criteria for sepsis among both the total sample and among only the patients who experienced a critical event during their admission. Finally, ROC curves were calculated using NEWS to provide a comparison of the HEWS with a previously validated score.
RESULTS
Patient characteristics
The flow of patients in the final sample is summarized in Figure 1. Patients totalling 845 were included in the final sample, including 270 cases who experienced a critical event. These index critical events included 23 inpatient arrests (9%), 95 unplanned ICU transfers (36%), and 149 deaths (56%); 89 patients had missing triage vitals and could not have a HEWS calculated. Missing vital signs included fever (72%), oxygen saturation (12%), respiratory rate (3%), neurological status (1%), blood pressure (1%), and 10% missing multiple vital signs. Cases (n=35) were no more likely than controls (n=54) to have missing triage vital (χ2 (1)=2.49, p=0.12). The remaining analysis included only patients with complete triage vitals in whom a HEWS could be determined. Table 3 presents the demographic characteristics for cases and controls. Cases were significantly more likely to be male, to arrive by ambulance, to have an infection-related diagnoses, to be SIRS positive, and to be classified as septic.
Figure 1 Flow diagram of included and excluded participants. CCI=Charlson Comorbidity Index.
Table 3 Cases and control baseline characteristics
*Significant at p<0.05.
CTAS=Canadian Triage and Acuity Scale; SIRS=systemic inflammatory response syndrome.
Interrater agreement
Interrater agreement was very good for most variables, including: 1) triage HEWS 0.89; 2) presence of SIRS criteria 1.00; 3) presence of an infection-related diagnoses 0.85; and 4) occurrence of a critical event 0.89. Agreement was good for the determination of a CCI score with a kappa of 0.64.
ROC curve analysis
The ROC curve for HEWS at triage as a predictor of the primary outcome, occurrence of a critical event during admission, is illustrated in Figure 2. The AUC for HEWS to predict the occurrence of a critical event during admission is 0.62 (95% confidence interval [CI] 0.58-0.66). Table 4 presents diagnostic test characteristics of various pre-specified HEWS cut-offs for predicting the likelihood of experiencing a critical event. The ROC curves for HEWS as a predictor of the secondary outcomes, meeting criteria for sepsis, and the occurrence of a critical event among septic patients are presented in Figures 3 and 4, respectively. The AUC for HEWS to predict the likelihood of sepsis is 0.77 (95% CI 0.72-0.82), and among the cases the AUC is 0.82 (95% CI 0.75-0.90). Table 5 presents diagnostic test characteristics of HEWS at predicting the sepsis-related outcomes.
Figure 2 ROC curve for HEWS at triage and NEWS at triage as a predictor of the occurrence of a critical event during admission as defined by the combination of arrest, unplanned ICU transfer, or death. HEWS=Hamilton Early Warning Score; NEWS=National Early Warning Score.
Figure 3 ROC curve for HEWS at triage and NEWS at triage as a predictor of the occurrence of a sepsis in the total sample. HEWS=Hamilton Early Warning Score; NEWS=National Early Warning Score.
Table 4 Diagnostic test characteristics of different HEWS cut-offs for predicting likelihood of experiencing a critical event during admission
+LR=positive likelihood ratio; −LR=negative likelihood ratio.
Table 5 Diagnostic test characteristics of different HEWS cut-offs for predicting likelihood of meeting criteria for sepsis and for meeting criteria for sepsis and experiencing a critical event during admission
+LR=positive likelihood ratio; - LR=negative likelihood ratio.
Additional exploratory analyses were conducted to examine the utility of HEWS in identifying septic patients who were assigned an initial CTAS level of III or lower. This cut-off was selected because experts have recommended that septic patients should be assigned a minimum CTAS level of II.Reference Sweet, Marsden and Ho 19 In our sample, 47 patients meeting sepsis criteria were assigned a CTAS level of III or IV. The mean HEWS score for these patients was 2.8 (SD=2.1), which was significantly greater than non-septic CTAS III or IV patients (M=1.4, SD=1.7), t(433)=5.3, p<0.01.
National Early Warning Score
The ROC curve for NEWS as a predictor of the occurrence of a critical event is illustrated in Figure 2. The AUC for NEWS to predict the occurrence of a critical event during admission is 0.61 (95% CI 0.57-0.65). ROC curves for NEWS as a predictor of the secondary outcomes, meeting criteria for sepsis and the occurrence of a critical event among septic patients, is presented in Figures 3 and 4, respectively. The AUC for NEWS to predict likelihood of sepsis is 0.73 (95% CI 0.67-0.79), and among cases the AUC is 0.79 (95% CI 0.72-0.85). Comparing the ROC curves for the HEWS and NEWS shows that 95% CI of the AUC overlaps for each outcome studied.
DISCUSSION
This study is the first evaluation of an EWS at triage in a Canadian ED. The use of a retrospective cohort of admitted patients allowed the inclusion of a relatively large number of patients who experienced a critical event in comparison with previous studies examining EWSs in the ED.Reference Alam, Vegting and Houben 4 - Reference Silcock, Corfield and Gowens 6 ROC analysis indicates that, although HEWS at triage is a statistically significant predictor of the likelihood of experiencing a critical event among admitted patients, it lacks sufficient discriminative power to be clinically useful for this outcome. Further discussion on ROC interpretationReference Fan, Upadhye and Worster 20 is available. The occurrence of critical events was chosen as the primary outcome because it has been evaluated in previous studies of EWSs among ward and ED patients.Reference Smith, Prytherch and Meredith 1 - Reference Silcock, Corfield and Gowens 6 However, this outcome may not be ideal to evaluate the utility of an EWS in the ED. Many additional variables beyond initial physiological derangement at ED presentation are likely to impact upon risk for the occurrence of a critical event during admission. Future ED EWS studies should focus on outcomes more likely to be related to ED presentation such as early unplanned ICU admission, mortality, and the development of new organ dysfunction.
A comparison of the HEWS with the NEWS reveals that the performance of the scores was not significantly different in our sample. The two scores evaluate similar variables but differ in their relative weighting and scoring. In particular, the two scores differ in their evaluation of mental status alterations. The NEWS uses the alert-voice-pain-unresponsive (AVPU) scale with any alteration in level of consciousness receiving a maximum score. In contrast, the HEWS scores the AVPU scale in gradated fashion and assesses for the presence of delirium using the confusion assessment method (CAM). In our study, the neurological component of each score was retrospectively calculated based upon triage nurse documentation. As such, we cannot assess the relative ease of using each score in the ED. Prospective research is needed to assess if delirium and level of consciousness can be effectively assessed during a brief triage assessment. Previous research has demonstrated that the nurses can successfully apply the CAM to identify delirious ED patients.Reference Wei, Fearing and Sternberg 21 Future studies should be conducted to determine the optimal methods of deriving and validating the components of EWSs that maximize their predicative ability in a given clinical environment.Reference Xu, Tam and Thabane 22
In line with previous research, HEWS at ED triage was useful as a predictor of sepsis and the occurrence of critical events among septic patients. This is notable because septic patients can deteriorate if they do not receive early recognition and treatment.Reference Dellinger, Levy and Rhodes 23 HEWS may be a useful adjunct to triage systems such as the CTAS by identifying potentially septic patients within a CTAS level who need to be seen more urgently. In our sample, septic patients assigned a CTAS level of III or IV had a higher HEWS score than non-septic patients with a similar CTAS level. An elevated HEWS could also be used to trigger a medical directive for triage nurse initiated bloodwork to detect markers of organ dysfunction such as elevated lactate and creatinine. Before implementation of such a proposal, prospective studies examining the diagnostic test characteristics of the HEWS among an unselected sample of ED patients are needed. The ROC values for HEWS in our sample may not generalize to all ED patients, given that our sample included admitted patients only. Such a study is needed to determine whether the additional costs of screening and testing using HEWS are justified by the number of cases detected.Reference Romero-Brufau, Huddleston and Escobar 24
The definition of sepsis and the optimal method of identifying septic patients is an active area of research. Novel sepsis definitions were recently published that include a proposed bedside clinical score to identify septic patients in the ED setting who are at increased risk of in-hospital mortality.Reference Singer, Deutschman and Seymour 25 This measure, called qSOFA, is positive if patients have at least two of the following clinical criteria: altered mentation as defined by a Glasgow coma scale score of<15, a respiratory rate of 22 breaths per minute or greater, or a systolic blood pressure of 100 mm Hg or less. Research is needed to examine the utility of qSOFA in comparison to measures such as HEWS. Such research would be valuable, given that EWSs are currently used as part of patient monitoring in many wards and EDs.
LIMITATIONS
The current study has several limitations. First, this was a retrospective study using clinical data not originally collected for this research purpose. We attempted to maximize the reliability of our data abstraction using recommended strategies to improve methodological quality.Reference Worster, Bledsoe and Cleve 15 Future studies should prospectively record HEWS to improve the validity of recorded data and examine relevant variables such as time from ED admission to the occurrence of critical events. Second, our sample was restricted to patients admitted to hospital wards through the ED. Future research is needed to determine whether the results generalize to all ED patients. Third, our study was conducted within a single hospital system, at two tertiary academic centres. It is unclear whether our results would generalize to other types of centres. Fourth, HEWS cut-offs were based upon those used for ward patients. Future research is needed to evaluate optimal HEWS cut-off levels among ED patients. Fifth, 89 patients (10.5%) had missing triage vital signs and were excluded from analysis. The most common missing vital sign was fever, which is likely to be important in the early detection of infection-related diagnoses. Previous research has documented even higher rates of missing vital signs at ED triage.Reference di Martino, Leoli and Cinotti 26 Implementation of an EWS at triage may improve the consistency of vital sign documentation, and this should be evaluated in future studies.
CONCLUSIONS
The HEWS has potential utility in the ED to allow earlier identification of patients with sepsis. Future research is needed to determine whether the HEWS can be incorporated as part of existing triage systems and used to implement earlier screening and intervention and thereby prevent deterioration of these patients. Investigators should also incorporate qualitative research methods to determine whether hospital-wide adoption of an EWS such as HEWS improves continuity of care for critically ill patients .
Figure 4 ROC curve for HEWS at triage and NEWS at triage as a predictor of the occurrence of a sepsis among patient experiencing a critical event. HEWS=Hamilton Early Warning Score; NEWS=National Early Warning Score.
Acknowledgements
The authors thank the dedicated staff of Hamilton Health Sciences Decision Support Services who assisted with case identification for database creation.
Financial support: This project received funding from the Hamilton Health Sciences Resident Research Grant in Patient Safety as well as the Hamilton Health Sciences Department Quality and Patient Safety Award awarded to Benjamin Tam. Funding parties were not involved in the design of the study.
Competing interests: None declared.
