Economic feasibility of a novel tool to assist extubation decision-making: an early health economic modeling

Abstract

Objectives

To estimate the minimum percent change in failed extubation to make a tool designed to reduce extubation failure (Extubation Advisor [EA]) economically viable.

Methods

We conducted an early return on investment (ROI) analysis using data from intubated intensive care unit (ICU) patients at a large Canadian tertiary care hospital. We obtained input parameters from the hospital database and published literature. We ran generalized linear models to estimate the attributable length of stay, total hospital cost, and time to subsequent extubation attempt following failure. We developed a Markov model to estimate the expected ROI and performed probabilistic sensitivity analyses to assess the robustness of findings. Costs were presented in 2020 Canadian dollars (C$).

Results

The model estimated a 1 percent reduction in failed extubation could save the hospital C$289 per intubated patient (95 percent CI: 197, 459). A large center seeing 2,500 intubated ICU patients per year could save C$723,124/year/percent reduction in failed extubation. At the current annual price of C$164,221, the EA tool must reduce extubation failure by at least 0.24 percent (95 percent CI: .14, .41) to make the tool cost-effective at our site.

Conclusions

Clinical decision-support tools like the EA may play an important role in reducing healthcare costs by reducing the rate of extubation failure, a costly event in the ICU.

In the decades since the inception of critical care, advances in medical technology and in the understanding of pathogenesis and pathophysiology of critically ill patients have transpired, leading to unavoidable increase in the costs of providing medical care. In Canada, use of intensive care units (ICU) is increasing faster than acute care hospitalizations (1). The daily cost of each ICU hospitalization is up to three times the cost of a general hospital ward, due to the increased resource use in personnel, equipment, and medications (1). Patients on mechanical ventilation (MV), a defining, life-sustaining therapy in critically ill patients, accrue significantly greater costs than nonventilated patients (2). The incremental cost of MV was estimated at $1,522 USD per patient day shown by a 2005 US study (2). The already extraordinary costs of ICU care requiring MV are expected to increase in the next few years by 2.3 percent annually, outpacing population growth (3).

Across Canada, there are 286 hospitals with intensive care units capable of providing invasive MV, totalling 3,170 beds designed for this purpose (4). From 2013–14, 33 percent of Canadian ICU patients received MV, up from 28 percent in 2007–08 which is in line with rates of 19–39 percent reported internationally (1). In addition to the equipment and personnel costs related to ventilation, prolonged ventilation is also associated with increased length of stay (LOS), and complications including greater risk of nosocomial infection, pneumothorax, ventilator-associated pneumonia, and mortality (5). Thus, hastening the weaning and liberation (i.e., extubation) from MV are critically important.

Extubation remains a high stakes clinical decision, as extubation failure and reintubation has detrimental effects on patient outcomes and cost (6;7). Extubation failure is typically defined as the need for reintubation within 48–72 hour (8) although it is occasionally defined up to 1 week (9). Despite advances in critical care, failed extubation occurs at a rate of approximately 10–20 percent (6;9–11). Previous studies across Europe, Asia and the United States on extubation failure found significant increases in morbidity (6–8;12–14), LOS (6;7;13;15;16), ICU stay (6–8;12–14;16), and costs (6;8;15;16). Complications include the need for tracheostomy, transfer to long-term care or development of infection. There is also an associated increase in mortality (6–8;12;15;16) with rates of 25–50 percent (9), independent of underlying disease severity (17;18).

The Extubation Advisor (EA) was designed as a novel extubation decision support tool to assist in the assessment of extubation decision-making and prediction of extubation failure risk, with the aim of reducing the rate of failure. EA utilized a predictive model based on respiratory rate variability (RRV) (19). The EA thus has the potential to reduce adverse outcomes and costs by optimizing the timing and enabling both earlier and safer extubation. Although several studies have examined the economics of MV in the ICU, the literature is sparse on the effectiveness and cost-effectiveness of the EA tool. This early economic analysis is therefore conducted to estimate the minimum reduction in failed extubation needed to make the EA tool economically attractive. This information can help developers refine the tool to meet the needs of health care systems.

Methods

We performed an early return on investment (ROI) analysis to estimate the minimum reduction in extubation failure rates that an EA decision tool needed to achieve to make the tool cost-effective from the hospital’s perspective. We used a Markov model (Figure 1) to combine evidence regarding the existing standard of care, that is, without the implementation of the EA tool, and compare it to a hypothetical care pathway that incorporates the tool. This information can inform the tool’s value for money in its intended context and the associated risk, which helps guide further research and development (20). We estimated the cost of EA tool as the sum of maintenance costs, upfront training and material costs. We defined the effectiveness of the EA tool as the reduction in failed extubation in ICU patients as extubation failure has been shown to be associated with morbidity and mortality (6–8;12–14), prolonged hospital and ICU LOS (6;7;13;15;16), and costs (6;8;15;16) in ICU patients.

Setting and Population

Our analysis focused on intubated patients in two ICUs at the Ottawa Hospital (TOH) in Ottawa, ON, Canada, from November 2015 to November 2019. TOH is one of Canada’s largest teaching hospitals, with 1,271 hospital beds and 56,029 patient admissions annually.

Data Sources

Risk of Failed Extubations

We determined whether an extubated patient was or was not successfully extubated from the ICU database from TOH Data Warehouse, a relational database containing the operational information of each of TOH’s campuses. Failed extubation was defined as the patient being reintubated within 48 hour of being extubated. All patient data was anonymized prior to extraction for analysis. Our study was based on 434 patients who had complete baseline and hospital cost data; of these, eighty-two patients had daily records indicating intubation status from which we could assess whether or not their extubation was successful.

Hospital Costs

In this study, hospital costs for each inpatient encounter were identified within the case-costing system of TOH Data Warehouse, a standardized case-costing methodology developed by the Ontario Case Costing Initiative (21) based on the Canadian Institute for Health Information Management Information Systems guidelines (22). The case-costing system links financial, clinical, and patient activity information stored within the Data Warehouse to define intermediate products, such as nursing time, medications, and laboratory tests. The data collected by TOH is broken down by day, to allow us to observe the time to given events based on fee codes we identified as being related to extubation and intubation. The total hospital costs were equal to the sum of the direct and indirect hospital costs for each intermediate product used during an encounter for each patient.

Costs of the EA Tool

The annual institutional cost of the EA tool was calculated as the sum of annual personnel costs and annualized costs of infrastructure. Personnel costs included costs of staff time for coordination, technical support, and opportunity costs of attending training and entering patient data into the EA tool. Infrastructure costs included annual licensing costs for the EA, costs of a virtual server and server storage. Costs were estimated by multiplying the type and frequency of resource use with their unit costs. Data on resource use, that is, staff time on various activities, requirements for virtual server and storage was provided by the project staff. Information on annual licensing costs was provided by the tool developer. Staff salaries were extracted from administrative documents. Additionally, cost per patient for implementing the EA application was estimated by dividing the annual institutional cost by the average number of ventilated patients in a year at the hospital, which was obtained from the TOH data warehouse. All costs were presented in 2020 Canadian Dollars (C$).

Analysis

Based on TOH data, we developed a Markov model (Figure 1) that followed patients over time, where we could observe transitions by intubation status, discharge status, and mortality according to daily transition probabilities. The model was run as Monte Carlo simulations using 5,000 iterations, in which all parameters were varied according to their variance and distribution derived from the data set. Table 1 presents the model input parameters used for the decision model. We reported both the average trajectories as well as 95 percent confidence intervals (CI) based on the observed variance in outcomes in the ICU population over the 5-year retrospective data period. We estimated the annual institutional ROI for the EA tool based on the following formula:

$$ \mathrm{ROI}=\frac{\begin{array}{c}\left(\left|\mathrm{Annual}\ \mathrm{hospital}\ \mathrm{cost}\ \mathrm{savings}\ \mathrm{attributable}\ \mathrm{to}\ \mathrm{the}\ \mathrm{EA}\ \mathrm{to}\mathrm{ol}\right|\right)-\\ {}\left(\mathrm{Annual}\ \mathrm{cost}\ \mathrm{of}\ \mathrm{the}\hskip0.5em \mathrm{EA}\hskip0.5em \mathrm{tool}\right)\end{array}}{\left(\mathrm{Annual}\ \mathrm{cost}\ \mathrm{of}\hskip0.5em \mathrm{EA}\hskip0.5em \mathrm{tool}\right)}. $$

Figure 1. Schematic diagram for a decision analytical (Markov) model.

Table 1. Model Input Parameters

ICU, intensive care unit.

If ROI is greater than 0, the EA tool is deemed to offer good ROI. We used the simulation model to estimate the minimum percent change in failed extubation is necessary to make the ROI greater than 0. Annual hospital cost savings attributable to the EA tool was estimated from hospital costs associated with failed extubations derived from a generalized linear model with an identity distribution and a log-link function. The model adjusted for patient age, sex, and Charlson comorbidity index.

We performed a series of scenario analysis to assess the robustness of our study findings. In the first scenario, we replaced the total hospital costs with direct variable hospital costs, which include items, such as fluids and medications that could be saved if the hospital beds are not used. Another scenario used alternative assumptions for the EA’s licensing fees. We estimated the annual institutional cost of EA and its ROI by varying the EA’s licensing costs between C$2,500 (US$1,965) and C$12,500 (US$9,823) per bed per year.

Results

A descriptive analysis revealed that the average LOS in the ICU for an intubated patient was 10.70 days (95 percent CI: 8.9–12.4), during which time they costed the hospital, on average, a total of C$54,647 (US$42,943) (95 percent CI: 46,171, 63,122; US$36,282, US$49,603). We also found that 13 percent of intubated ICU patients had a failed extubation. Results from the regression analysis showed that failed extubation patients were associated with double the LOS (an additional 8.98 days, 95 percent CI: 6.15, 11.80) and similarly scaled added total cost C$43,464 (US$34,155) (95 percent CI: 28,242, 58,686; US$22,193, US$46,117). Of the total cost for patients with a failed extubation, direct variable costs accounted for 70.44 percent, followed by indirect variable costs (13.52 percent). Fixed costs accounted for 16.04 percent of total costs. Failed extubation patients were also nearly twice as likely to die in hospital (5.30 percent absolute value increase) even after adjusting for age, sex, and comorbidity index.

Figure 2 shows patient transitions under the standard of care derived from our decision model, starting from the day on which patients are intubated and admitted in the ICU. This model parameterizes the risk of failed extubation as a daily hazard function, where 13 percent of patients will fail their extubation. Parameterizing failure risk allowed us to change this single parameter to consider alternate scenarios in which we could set the new average failure rate to any value and see how it would change the average patient outcomes, all else being equal.

Figure 2. Patient transitions between health states under the standard of care of patients from the index intubation.

Our model results match the actual data, estimating the average length of ICU stay for an intubated patient at 10.4 (95 percent CI: 9.7, 11.0) days and a total cost of C$60,037 (US$47,178) (95 percent CI: 47,911, 75,197; US$37,649, US$59,091). The model estimated that, on average, a 1 percent reduction in failed extubation rates could save the hospital C$289 (US$227) per intubated patient (95 percent CI 197, 459) (Table 2). At TOH, the hospital sees approximately 2,500 intubated ICU patients per year, it is therefore estimated that the hospital would save C$723,124 (US$568,246) per year per percent reduction in failed extubations.

Table 2. Base Case Results

EA, Extubation Advisor; ROI, return on investment.

The annual institutional cost for the EA application was C$164,221 (US$129,048) or C$65.69 (US$51.62) per patient (Supplementary Table 1); of these, 57 percent (C$93,488; US$73,465) accounted for the infrastructure cost and 43 percent (C$70,734; US$55,584) was spent on personnel cost. Even at the lowest cost level (C$2,500/bed/year; US$1,965/bed/year), the software licensing cost was found to be the largest cost driver, accounting for 55 percent of the total costs of EA tool, as training and set up costs are low. Supplementary Table 1 presents a breakdown of the mean annual institutional cost for EA by resource category.

Our model estimated that at the current annual price of C$164,221 (US$129,048), the EA tool must reduce the failed extubation by at least 0.24 percent (95 percent CI: .14, .41) to make the tool a good investment from TOH’s perspective, that is, savings due to the EA tool is greater than its cost. If we only considered hospital direct variable costs, the EA tool must reduce the extubation failure rate by at least 0.34 percent (95 percent CI: .12, .39) to be cost-effective (Table 3, Scenario Analysis A). In addition, the scenario analysis that varied the annual licensing costs for EA indicated that the minimum threshold for the effectiveness of EA increased with higher licensing costs (Table 3, Scenario Analysis B). Varying the costs associated with the EA tool implementation of the base case values had minimal impact on the minimum percentage change required to make the EA tool economically attractive.

Table 3. Results from Scenario Analyses for Licensing Costs

^a The minimum reduction of extubation failure rate above which the savings due to the EA tool is greater than its cost.

EA, Extubation Advisor.

Discussion

Our study demonstrates that in a Canadian tertiary care hospital network, the attributable cost of a single failed extubation at C$43,464 (US$34,155) is significant, nearly doubling the average cost of hospitalization compared to a successfully extubated patient. In comparison, reports from the United States have shown costs of similarly large magnitude. Seymour et al. (6) showed added costs of failed extubation totalling US$33,926 in a sixteen ICU bed community hospital, while Menon et al. (16) work in an eighty-eight ICU bed Level 1 Trauma Center had even higher incremental hospital charges totalling US$126,400 (17). Pronovost et al. (15) showed that reintubation increased total hospital costs by approximately 20 percent or US$9,510 (1996 dollars) in abdominal aortic surgery patients. Although no comparable Canadian data was found, these values underscore the significant potential of clinical decision-making tools such as the EA tool for reducing hospital costs if the tool is able to reduce extubation failure rates. This economic benefit is expected to be larger if the EA tool’s impact on ICU patients and their families, such as time missed from work, was considered.

This research was conducted at a large tertiary-care hospital in Canada, and the results showed that the breakeven effectiveness required from EA is a 0.24 percent absolute reduction in extubation failure assuming a cost of C$2,500/bed/year (US$1,965/bed/year). The average extubation failure rate at our institution is 13 percent, which was within the ranges of 10–20 percent reported in others (6;9–11), highlighting how a miniscule absolute reduction of 0.24 percent (proportionally < 5 percent) will cover the costs of C$165,000 (US$129,660) on an annual basis. The breakeven effectiveness was 0.34 percent when a smaller fraction of hospital (direct variable) costs were considered. The breakeven effectiveness required from EA ranged between 0.24 and 0.76 percent absolute reduction in failure rates when licensing costs for EA were varied between $2,500 (US$1,965) and C$12,500 (US$9,823) per bed per year. This indicates that even at the highest cost of C$12,500 (US$9,823) per bed, positive ROI is achieved if extubation failure is reduced by at least 0.76 percent.

Based on the analysis of the EA and the WAVE study (19), the EA tool was safe, and it is likely that the potential for the EA to reduce the rate of extubation failure would exceed the threshold required for cost-effectiveness, although this remains to be proven in a future planned interventional trial. The WAVE study concluded that a predictive model using RRV during the last spontaneous breathing trial (SBT) provided optimal accuracy of prediction in all patients, with improved accuracy when combined with clinical impression or rapid shallow breathing index (RSBI), factors which were integrated into the EA tool.

Although the values reported in this paper pertain to the context of similar sized tertiary care Canadian ICUs, our study findings can be scaled according to the size of the facility, with larger or busier centers expected to break even at a lower percent reduction in extubation failure and benefit from an even greater effect on cost reduction. In addition, the opportunity cost of data entry, currently estimated at 10 min per SBT, is expected to decrease with time and experience as respiratory therapists integrate the EA’s standardized SBT assessment into their regular workflow.

Limitations

The main limitation of this study was the limited ICU data. Of 434 patients for which we had complete time-to-event information, only 82 patients could be used to assess failed extubation and associated mortality rates. The hospital data was well described when patients needed to be reintubated the day (or more) after their extubation, but both ICU data input and shift data did not adequately reflect changes within a single day. Data indicating an extubation or reintubation attempts were not coded. Since most failed extubations and subsequent reintubation happen within 15–22 hour (16;23;24), this meant we were likely missing a significant amount of data. Consequently, we linked the TOH with the WAVE trial data to determine failed extubation events (19). Due to data limitation, we could not perform a time to event analysis and estimate how mortality rates might vary by days from intubation. We therefore assumed a constant daily mortality rate in the model. Because this assumption was applied to the EA and usual care scenarios, it was unlikely to affect the estimated break-even-point of the EA tool. Last, it is clear that the impact of an extubation clinical decision support tool on clinician decision making may be variable between clinicians and centers. While the EA tool incorporates measures obtained during the SBT that are important to the prediction of extubation outcomes such as respiratory rate (RR), tidal volume (TV), RSBI (=RR/TV) (25), RRV (19;26;27), and cough strength, among others (25;28;29), techniques used to conduct and assess SBTs vary between institutions (30;31). The variable impact of clinical decision support tools on decision making remain to be studied. Regardless, this study serves to identify minimal thresholds of reduction in overall extubation failure rates to make a tool cost effective.

Conclusion

With rising healthcare costs, there is an increasing need for clinical decision support tools to inform care in an evidence-based and cost-effective fashion. Our early economic modeling suggests that the EA is one such tool, which has high economic feasibility as it could reduce the hospital cost of MV by providing earlier extubation and reduce the rate of extubation failure and reintubation events.