Difference between days of therapy and days of antibiotic spectrum coverage in an inpatient antimicrobial stewardship program: Vector autoregressive models for time-series analysis

Abstract

Objective:

The days of therapy (DOT) metric, used to estimate antimicrobial consumption, has some limitations. Days of antibiotic spectrum coverage (DASC), a novel metric, overcomes these limitations. We examined the difference between these 2 metrics of inpatient intravenous antimicrobial consumption in assessing antimicrobial stewardship efficacy and antimicrobial resistance using vector autoregressive (VAR) models with time-series analysis.

Methods:

Differences between DOT and DASC were investigated at a tertiary-care center over 8 years using VAR models with 3 variables in the following order: (1) the monthly proportion of prospective audit and feedback (PAF) acceptance as an index of antimicrobial stewardship efficacy; (2) monthly DOT and DASC adjusted by 1,000 days present as indices of antimicrobial consumption; and (3) the monthly incidence of 5 organisms as an index of antimicrobial resistance.

Results:

The Granger causality test, which evaluates whether incorporating lagged variables can help predict other variables, showed that PAF activity contributed to DOT and DASC, which, in turn, contributed to the incidence of drug-resistant P. aeruginosa. Notably, only DASC helped predict the incidence of drug-resistant Enterobacterales. Another VAR analysis demonstrated that a high proportion of PAF acceptance was accompanied by decreased DASC in a given month, whereas increased DASC was accompanied by an increased incidence of drug-resistant Enterobacterales, unlike with DOT.

Conclusions:

The VAR models of PAF activity, antimicrobial consumption, and antimicrobial resistance suggested that DASC may more accurately reflect the impact of PAF on antimicrobial consumption and be superior to DOT for predicting the incidence of drug-resistant Enterobacterales.

Tracking antimicrobial consumption is crucial for building an effective antimicrobial stewardship program (ASP). However, precisely estimating antimicrobial consumption is difficult despite the various metrics available for this purpose. In the inpatient setting, the days of therapy (DOT) metric, adjusted by patient days (PD) to overcome the disadvantages of defined daily dose, is often used to estimate antimicrobial consumption. ^1–3 Recently, because PD may exclude partial days spent in hospital (eg, day of admission or discharge), the denominator was changed from PD to days present (DP) to avoid underestimating patient time at risk. ⁴ The latest US National Healthcare Safety Network (NHSN) antimicrobial use and resistance (AUR) module recommends using DOT as the numerator and DP as the denominator to assess intravenous antimicrobial consumption. ⁵

Although DOT is a useful metric for estimating antimicrobial consumption, it has some important limitations; for example, it does not account for antimicrobial spectrum information. To address these issues, the antibiotic spectrum index (ASI) and days of antibiotic spectrum coverage (DASC) were developed to combine data relating to DOT and the antibiotic spectrum. ^6,7 Although the ASI is unclear with respect to its inclusion criteria and the types of pathogen targeted, DASC is superior to other new metrics because it divides the antimicrobial spectrum into intrinsic resistance and acquired resistance based on the latest research findings and expert opinion. This feature minimizes the need to review antibiotic spectrum coverage (ASC) frequently.

However, antimicrobial consumption based on DASC has not been fully assessed for hospital benchmarking. Understanding the long-term trends in antimicrobial consumption is important for assessing ASP efficacy, which may also reduce the likelihood of the emergence of antimicrobial resistance. Meanwhile, the complicated, causal relationship between antimicrobial consumption and antimicrobial resistance has not been fully elucidated. Moreover, traditional regression models do not evaluate these relationships appropriately because they have a fixed cause-and-effect relationship and assume no serial correlation among the data. Time-series analysis without fixed causality is more appropriate for assessing the relationships between these variables, considering the influence of past variables and the time-series context in which they may be associated with each other. The vector autoregressive (VAR) model as time-series analysis, which has been used to understand the dynamic behavior of multivariate systems in macroeconomics, ^8,9 may overcome the limitations of traditional regression models and work effectively in the time-series context. Although the VAR model was previously used to demonstrate a relationship only between antimicrobial consumption and resistance, ¹⁰ it may nonetheless enable analysis of the relationship among ASP, antimicrobial consumption, and microbiological data within a single framework.

The present study used the VAR model to analyze the differences between DOT and DASC in terms of intravenous antimicrobial consumption to assess ASP efficacy and the incidence of drug-resistant pathogens at a tertiary-care center over 8 years.

Methods

Study setting

The present, retrospective, observational study was conducted at Tokyo Metropolitan Tama Medical Center, a 790-bed tertiary-care center in Japan. All inpatients who received at least 1 intravenous antimicrobial agent were included, and facility-wide, intravenous antimicrobial consumption data were obtained from monthly barcode medication administration records from April 2014 to March 2022. Monthly DOT and DP were calculated and DOT was expressed per 1,000 DP using the NHSN AUR module definition. ⁵ Monthly DASC was calculated by summing the ASC score for all the categories in a previous study. ⁷ Antimicrobials with no established ASC score in the previous literature (ie, cefoperazone-sulbactam, cefmetazole, flomoxef, arbekacin, teicoplanin, and pazufloxacin) were scored by an ASP clinical pharmacist (S.M.) and an infectious disease physician (H.H.) after discussion. ^11–16 The ASC score for amikacin was also revised from 8 to 9 by modifying the CRE section using the process described above (Supplementary Table 1 online).

Antimicrobial stewardship at the study center

At the study center, once-weekly prospective audit and feedback (PAF) was implemented in April 2014 as part of an ASP targeting inpatients receiving carbapenems and piperacillin-tazobactam for >72 hours. It was managed by a multidisciplinary team and continued throughout the study period, including during the COVID-19 pandemic. Monthly data on the proportion of patients for whom ASP recommendations were accepted for inappropriate antimicrobial use were collected as a representative index of ASP efficacy. During the PAF encounter, the appropriateness of carbapenems and piperacillin-tazobactam for inpatients receiving these antimicrobials was evaluated, and the primary care teams were contacted via telephone to modify or discontinue antimicrobial therapy in case inappropriate use was detected. An ASP-related practice was considered accepted if the primary care teams made the necessary modifications within 72 hours. A clinical pharmacist (S.M.) prospectively recorded this information on a data collection form. ¹⁷

Data collection

Data on monthly incidence density, including Clostridioides difficile infections (CDI) per 10,000 PD, extended-spectrum β-lactamase (ESBL)–producing Enterobacterales detected by phenotypic testing (disk diffusion or broth microdilution) per 1,000 PD, and methicillin-resistant Staphylococcus aureus (MRSA) per 1,000 PD, were collected as microbiological data from all cultures. Moreover, because few species of multidrug-resistant bacteria have been detected at the study center, guidelines were used to determine the incidence density of 2 species of drug-resistant, gram-negative bacteria from all the cultures according to the following definitions. ^18,19 First, drug-resistant Pseudomonas aeruginosa was defined by nonsusceptibility to at least 1 antimicrobial in 2 or more of the following categories: (1) antipseudomonal cephalosporins (ceftazidime and cefepime), (2) piperacillin and piperacillin/tazobactam, (3) aztreonam, (4) fluoroquinolones (ciprofloxacin and levofloxacin), (5) aminoglycosides (amikacin, tobramycin, and gentamicin), and (6) carbapenems (imipenem and meropenem). Second, certain drug-resistant Enterobacterales, excluding ESBL and carbapenem-resistant organisms, were defined by nonsusceptibility to at least 1 antimicrobial in 2 or more of the following categories: (1) extended-spectrum cephalosporins (third- and fourth-generation cephalosporins: ceftriaxone, cefotaxime, ceftazidime and cefepime), (2) piperacillin-tazobactam, (3) aztreonam, (4) fluoroquinolones (ciprofloxacin and levofloxacin), and (5) aminoglycosides (amikacin, tobramycin, and gentamicin).

Vector autoregressive (VAR) analysis

With the VAR analysis, we assessed the dynamic correlation between multiple variables using the Granger causality test, impulse-response functions (IRFs), and forecast error variance decompositions (FEVDs). ^8,9,20 The Granger causality test qualitatively determines whether incorporating lagged values of one variable improves the predictive accuracy of other variables of interest without relying on the order of the variables. IRFs quantitatively demonstrate how a 1-unit shock in 1 variable influences the dynamic impact of each variable over time. FEVDs quantify the contribution of each variable to the specific shock of 1 variable.

The first VAR model included DASC per DOT and DOT per 1,000 DP. Then, 10 VAR models incorporated 3 variables: PAF acceptance, antimicrobial consumption (DOT or DASC), and microbiological data (CDI, ESBL, drug-resistant P. aeruginosa, drug-resistant Enterobacterales, or MRSA). Recursive VAR models were adopted using Cholesky decomposition. ^8,20 In a bivariate equation with lag-1 autocorrelation (equation 1.1) based on this method, the order of variables is crucial and should be arranged so that y_t is causally prior to z_t, especially when evaluating IRFs and FEVDs. ^8,20 In this situation, while the impulse of y_t in the same month influences z_t, the reverse is not true. This model can be expanded into a multivariate and multilag model. Therefore, the variables were placed in the 2 patterns in the first VAR model incorporating DASC per DOT and DOT per 1,000 DP, because this relationship was unclear. In the next 10 VAR models, the variables were placed in the order of PAF acceptance, antimicrobial consumption, and microbiological data because PAF affects antimicrobial consumption and because antimicrobial consumption may result in the emergence of resistance (Fig. 1). ^3,10,21,22

(1.1) $$\left\{ {\matrix{ {{{\rm{y}}_{\rm{t}}} = {{\rm{a}}_{10}} + {{\rm{a}}_{11}}{{\rm{y}}_{{\rm{t}} - 1}} + {{\rm{a}}_{12}}{{\rm{z}}_{{\rm{t}} - 1}} + {{\rm{e}}_{1{\rm{t}}}}} \cr {{{\rm{z}}_{\rm{t}}} = {{\rm{a}}_{20}} + {{\rm{a}}_{21}}{{\rm{y}}_{{\rm{t}} - 1}} + {{\rm{a}}_{22}}{{\rm{z}}_{{\rm{t}} - 1}} + {{\rm{e}}_{2{\rm{t}}}}} \cr } } \right. \cdots $$

(1.2) $$\matrix{ { \Leftrightarrow \left[ {\matrix{ {{{\rm{y}}_{\rm{t}}}} \cr {{{\rm{z}}_{\rm{t}}}} \cr } } \right] = \left[ {\matrix{ {{{\rm{a}}_{10}}} \cr {{{\rm{a}}_{20}}} \cr } } \right] + \left[ {\matrix{ {{{\rm{a}}_{11}}} & {{{\rm{a}}_{12}}} \cr {{{\rm{a}}_{21}}} & {{{\rm{a}}_{22}}} \cr } } \right]\left[ {\matrix{ {{{\rm{y}}_{{\rm{t}} - 1}}} \cr {{{\rm{z}}_{{\rm{t}} - 1}}} \cr } } \right] + \left[ {\matrix{ {{{\rm{e}}_{1{\rm{t}}}}} \cr {{{\rm{e}}_{2{\rm{t}}}}} \cr } } \right]} \hfill \cr {\quad \quad \quad \,\,\, \Leftrightarrow {{\bf{x}}_{\bf{t}}} = {{\bf{A}}_0} + {{\bf{A}}_1}{{\bf{x}}_{{\bf{t}} - 1}} + {{\bf{e}}_{\bf{t}}} \cdots } \hfill \cr } $$

Figure 1. A diagram illustrating the causality assumed to exist between prospective audit and feedback (PAF) acceptance, antimicrobial consumption, and microbiological data in this study.

The Akaike information criterion (AIC) was used to estimate the length of lags in the VAR models and to assess whether each variable had a stationarity process using these lags. A diagnostic test of the VAR models, including residual autocorrelation and stability tests, was performed. After confirming the adequacy of the recursive VAR models, the Granger causality test, IRFs, and FEVDs were performed. In the VAR models incorporating the above 3 variables, the relationship between 2 adjacent, unidirectional variables (ie, PAF acceptance to antimicrobial consumption and antimicrobial consumption to microbiological data) was evaluated to avoid investigating reverse and distant causation. Moreover, to establish whether DOT or DASC was the better indicator, these 2 metrics in each VAR model were compared to determine which had a greater number of confirmed Granger-causality relationships and more appropriate IRF and FEVD dynamics.

Other statistical analyses

DASC per DOT indicates the average ASC score of the antimicrobials used in hospitals. ⁷ To investigate how DASC per DOT and DOT per 1,000 DP affect each other over time, Spearman rank-order correlation analysis was used to assess their correlation with each other. These results were compared with those of the VAR analysis. For all statistical analyses, P < .05 was considered to indicate statistical significance. All the data were analyzed using Stata/SE version 17.0 software (StataCorp, College Station, TX). The Institutional Review Board of Tokyo Metropolitan Tama Medical Center approved this study (approval no. 3-190).

Results

Trend in antimicrobial consumption, PAF acceptance, and microbiological data

Figure 2 shows the changes in antimicrobial consumption using DOT per 1,000 DP and DASC per 1,000 DP. Supplementary Figure 1 shows the trends in PAF acceptance and the incidence of the 5 organisms.

Figure 2. Changes in antimicrobial consumption using days of therapy (DOT) and Days of antibiotic spectrum coverage (DASC) per 1,000 days present (DP).

Note. The dotted line represents a linear approximation.

DASC per DOT trend and the relationship between DASC per DOT and DOT per 1,000 DP

Figure 3 shows an upward trend for DASC per DOT between March 2019 and January 2020. For each VAR model with a different ordering of DASC per DOT and DOT per 1,000 DP, AIC preferred 5 lags, and adequacy was confirmed. Although we detected a negative correlation between DASC per DOT and DOT per 1,000 DP (Spearman rank-order correlation test, ρ = −0.58; P < .001), the Granger causality tests confirmed the effect of DASC per DOT on DOT per 1,000 DP (P = .006) but not vice versa (P = 0.38).

Figure 3. Changes in days of antibiotic spectrum coverage (DASC) per days of therapy (DOT).

Note. The dotted line represents a linear approximation.

Relationship between PAF acceptance, antimicrobial consumption, and microbiological data on the VAR models

The 10 VAR models used the 3 variables of PAF acceptance, antimicrobial consumption, and microbiological data in the given order. We adopted 2 lags for the VAR model consisting of PAF acceptance, DOT per 1,000 DP, and drug-resistant Enterobacterales and 1 lag for the other VAR models as a reasonable length according to the AIC. The adequacy of all 10 VAR models was confirmed.

Table 1 shows Granger-causality test results. In the VAR model using PAF acceptance, DOT or DASC, and microbiological data, VAR models 1 and 6, 2 and 7, 3 and 8, 4 and 9, and 5 and 10 reflect the interchanging of DOT and DASC. All of the models indicated that PAF acceptance helped predict antimicrobial consumption. Although both DOT per 1,000 DP and DASC per 1,000 DP contributed to predicting drug-resistant P. aeruginosa, only DASC per 1,000 DP contributed to predicting drug-resistant Enterobacterales.

Table 1. Results of Granger Causality Tests for All VAR Models Using PAF Acceptance, DOT or DASC, and Microbiological Data

Note. VAR, vector autoregressive; PAF, proportion of prospective audit and feedback acceptance; DOT, days of therapy per 1,000 days present; DASC, days of antibiotic spectrum coverage per 1,000 days present; CDI, incidence of Clostridioides difficile infection per 10,000 patient days; ESBL, incidence of extended-spectrum β-lactamase–producing Enterobacterales per 1,000 patient days; MRSA, incidence of methicillin-resistant Staphylococcus aureus per 1,000 patient days; resistant P. aeruginosa, incidence of drug-resistant Pseudomonas aeruginosa per 1,000 patient days; resistant Enterobacterales, incidence of drug-resistant Enterobacterales per 1,000 patient days. The results of the Granger-causality tests demonstrated the effect of the left variable on the right variable. For example, in VAR model 9, the P value of the effect of DASC on drug-resistant Enterobacterales was 0.027, and DASC was a good predictor for drug-resistant Enterobacterales. Meanwhile, for VAR model 4, the P-value of the effect of DOT on drug-resistant Enterobacterales was 0.23, and DOT was not found to be a predictor for drug-resistant Enterobacterales.

Figure 4 shows the IRFs of VAR models 1–10, for which Granger causality was confirmed (Table 1). The IRF of VAR model 1 represents the effect of PAF on DOT per 1,000 DP, which was consistent across VAR models 1 through 5. Similarly, the IRF of VAR model 6 represents the effect of PAF on DASC per 1,000 DP. A comparison of VAR models 1 and 6 demonstrated that a 1-unit shock (increase) in PAF acceptance resulted in an initial decrease of 10.4 in DASC per 1,000 DP at time 0, whereas DOT per 1,000 DP remained unchanged. After 1 month, both DASC and DOT increased, then decreased before converging to 0 after 5 months. A comparison of VAR models 3 and 8 demonstrated that the effect of both DOT and DASC per 1,000 DP on drug-resistant P. aeruginosa was similar. Specifically, a 1-unit shock (increase) in DOT or DASC resulted in a decrease in drug-resistant P. aeruginosa after 1 month, contrary to the assumed dynamics. Only DASC per 1,000 DP served as a predictor of drug-resistant Enterobacterales; thus, we focused on VAR model 9. A 1-unit shock (increase) in DASC resulted in an increase of 0.033 in drug-resistant Enterobacterales after 1 month, followed by a decrease before convergence to 0 after 9 months.

Figure 4. Impulse-response functions (IRFs) for the VAR models with the order of prospective audit and feedback (PAF) acceptance, days of therapy (DOT) or days of antibiotic spectrum coverage (DASC), and microbiological data for which Granger causality was confirmed.

Note. The x-axis shows the monthly time-series (the scale is in months), and the y-axis shows the effect of the left variable on the right variable in each VAR model in Table 1 (the scales differ). For example, in the graph in the top left panel, the IRF shows the effect of PAF on DOT in VAR model 1, and the scale of the y-axis shows DOT per 1,000 DP.

The FEVDs in Supplementary Figure 2 demonstrate that the contribution of the other variables began to increase 1 month after the shock, stabilized after 3 months, and remained at the same level thereafter. In most of the models, the contribution of the other variables was <10%.

Discussion

In this study, VAR models incorporating PAF acceptance, antimicrobial consumption, and microbiological data in the given order indicated that DASC may be a more suitable benchmark for antimicrobial consumption than DOT, according to the described methods. The Granger causality tests revealed an association between the time-series trends of both DOT and DASC with PAF acceptance, but only DASC proved to be a good predictor of incidence of drug-resistant Enterobacterales. Moreover, we detected an initial decrease in DASC reflected by high PAF acceptance, whereas the incidence of drug-resistant Enterobacterales influenced by DASC increased for several months, unlike with DOT.

In this study, we uniquely used VAR to examine the dynamic interplay among key variables, such as antimicrobial consumption, ASP, and microbiological data while retaining temporal information. Previous studies have suggested that broad-spectrum antimicrobial consumption lead to the emergence of multidrug-resistant organisms. ^23,24 Another study using VAR analysis that focused on antimicrobial consumption and antimicrobial resistance demonstrated that the use of a specific antimicrobial fueled the emergence of antibiotic-resistant bacteria. This finding led to an increase in the use of broad-spectrum antimicrobials. ¹⁰ However, there was room for improvement in the analysis method because these studies ^10,23,24 used outcome data without considering time-series trends, and they assessed the chronological relationship only between antimicrobial consumption and antimicrobial resistance.

In contrast to yet another study, ⁷ our study demonstrated a correlation between DOT and DASC per DOT using the Spearman rank-order correlation test. However, VAR analysis revealed Granger causality solely in the direction of DASC per DOT to DOT, suggesting that monitoring DASC per DOT was also useful for predicting DOT and was more informative than DOT alone. Meanwhile, the DASC per DOT metric decreased, even though DASC per DOT increased in 2019. This temporary spike in DASC per DOT was likely caused by a national shortage of cefazolin in Japan, which necessitated ceftriaxone use at the study center. ²⁵

VAR models with variables in the order of PAF acceptance, antimicrobial consumption, and microbiological data suggested that DASC is a more informative index than DOT. DASC better accounts for the antimicrobial spectrum, more accurately reflects PAF efficacy in promoting changes in or discontinuing antimicrobial therapy, and may provide more information about facility-wide outcome measures, such as antimicrobial resistance. Given these findings and the limitations of DOT (eg, inadequate consideration of de-escalation from broad-spectrum monotherapy to narrow-spectrum combination therapy), ⁷ conventional, DOT-based monitoring may be insufficient for evaluating outcomes related to ASP. Our findings have demonstrated that DASC may serve as a better predictor of drug-resistant Enterobacterales, in line with previous studies. ^10,26 For instance, some studies demonstrated that increased consumption of cephalosporins and quinolones was associated with higher incidence of drug-resistant Enterobacterales. In contrast, in our study, the incidence of drug-resistant P. aeruginosa decreased with increasing DASC, and the effect of antimicrobial consumption (DOT or DASC) on CDI, ESBL, and MRSA was negative. These results are inconsistent with the mechanism assumed to exist between antimicrobial consumption and the emergence of antimicrobial resistance. Although the acquisition of antimicrobial resistance is multifactorial and not fully understood, other studies have demonstrated that exposure to specific antibiotic classes, such as broad-spectrum antimicrobials, may be a risk factor. ^27–36 Neglecting this specific effect when calculating facility-wide antimicrobial consumption may lead to the finding of a fluctuating and negative relationship between DOT or DASC and the incidence of organisms, such as drug-resistant P. aeruginosa, CDI, ESBL, and MRSA. Moreover, in the recursive VAR models used in the present study, the relationship of the variables in reverse order (eg, ESBL and DOT) or variables that are distant from each other (eg, PAF and ESBL) might find opposite and distant causation. In this situation, the results require cautious interpretation because Granger causality assumes that the cause precedes the outcome and contains unique information about the outcome. ³⁷ Therefore, these relationships were not explicitly examined, and only the relationship between 2 adjacent, unidirectional variables was evaluated.

This study had several limitations. First, because it was conducted at a single tertiary-care hospital in Japan, these results may have limited generalizability. Second, the study focused on facility-wide antimicrobial consumption without analyzing differences in patient background or antimicrobial exposure density across different units, such as the intensive care unit. Third, although the ASC score is relatively robust, the DASC metric may not be the optimal metric for antimicrobial consumption if new, multidrug-resistant organisms emerge. Also, importantly, antimicrobials are distinguished from each other only by the ASC score, which does not consider differences in the effect of selective pressure of exposure to specific antimicrobials. Fourth, adequacy was assessed when constructing the VAR models, and the behavior of all but 3 variables not incorporated into the model was evaluated in the dynamics of the disturbance terms (see equations 1.1 and 1.2). However, the selection of variables, their ordering, and VAR modeling are often challenging. ⁹ Moreover, resistance may occur even in the absence of antimicrobial exposure, ³⁸ and no current theory has adequately explained the mechanism of resistance development. The incidence of multidrug-resistant organisms is confounded by various factors, such as adherence to hand hygiene, adherence to standard precautions, education, and environmental hygiene. ³⁹ Finally, COVID-19 may have influenced some unmeasured variables in addition to the variables used in the VAR models. ⁴⁰

In conclusion, DASC, including antimicrobial spectrum information, may provide more detailed insight than DOT through the application of VAR analysis, which accounts for time-series data on process measures, such as PAF activity, and outcome measures, such as microbiological data. VAR analysis with appropriate procedures may be generalizable to a wide range of other facilities and may help identify key aspects of intervention necessary for optimizing ASP. Because monitoring antimicrobial consumption accurately is difficult, more studies are needed to determine the method of assessing antimicrobial consumption that best reflects PAF activity and is most strongly associated with the occurrence of multidrug-resistant pathogens.