Implementation and outcomes of a clinician-directed intervention to improve antibiotic prescribing for acute respiratory tract infections within the Veterans’ Affairs Healthcare System

Abstract

Objective:

To determine whether a clinician-directed acute respiratory tract infection (ARI) intervention was associated with improved antibiotic prescribing and patient outcomes across a large US healthcare system.

Design:

Multicenter retrospective quasi-experimental analysis of outpatient visits with a diagnosis of uncomplicated ARI over a 7-year period.

Participants:

Outpatients with ARI diagnoses: sinusitis, pharyngitis, bronchitis, and unspecified upper respiratory tract infection (URI-NOS). Outpatients with concurrent infection or select comorbid conditions were excluded.

Intervention(s):

Audit and feedback with peer comparison of antibiotic prescribing rates and academic detailing of clinicians with frequent ARI visits. Antimicrobial stewards and academic detailing personnel delivered the intervention; facility and clinician participation were voluntary.

Measure(s):

We calculated the probability to receive antibiotics for an ARI before and after implementation. Secondary outcomes included probability for a return clinic visits or infection-related hospitalization, before and after implementation. Intervention effects were assessed with logistic generalized estimating equation models. Facility participation was tracked, and results were stratified by quartile of facility intervention intensity.

Results:

We reviewed 1,003,509 and 323,023 uncomplicated ARI visits before and after the implementation of the intervention, respectively. The probability to receive antibiotics for ARI decreased after implementation (odds ratio [OR], 0.82; 95% confidence interval [CI], 0.78–0.86). Facilities with the highest quartile of intervention intensity demonstrated larger reductions in antibiotic prescribing (OR, 0.69; 95% CI, 0.59–0.80) compared to nonparticipating facilities (OR, 0.89; 95% CI, 0.73–1.09). Return visits (OR, 1.00; 95% CI, 0.94–1.07) and infection-related hospitalizations (OR, 1.21; 95% CI, 0.92–1.59) were not different before and after implementation within facilities that performed intensive implementation.

Conclusions:

Implementation of a nationwide ARI management intervention (ie, audit and feedback with academic detailing) was associated with improved ARI management in an intervention intensity–dependent manner. No impact on ARI-related clinical outcomes was observed.

Outpatient acute respiratory tract infections (ARIs) are commonly treated with antibiotics.¹ Diagnostic and treatment recommendations to facilitate appropriate management of ARIs exist; however, opportunity to improve prescribing remains ample. Estimates suggest that ∼30% of antibiotic prescriptions are unnecessary, and ARI diagnoses constitute a major source of misuse.² In Veterans’ Healthcare Administration (VHA) analyses conducted between 2005 and 2012 and in 2016, most patients with ARI received antibiotics despite educational campaigns and guidelines recommending prudent prescribing.^3,4

Interventional approaches including audit and feedback with peer comparison, academic detailing, clinician communication training, and clinician public commitments to use antibiotics appropriately have reduced antibiotic overprescribing for ARIs.^5–12 Behavioral interventions, such as audit and feedback with peer comparison, have demonstrated robust improvement in prescribing and are perceived by clinicians as an acceptable approach to improve practice.^5,7,8,12 Large systemwide audit-and-feedback interventions delivered in written or electronic format demonstrate modest results, suggesting that additional interaction with clinicians may be needed to facilitate behavior change.^13,14 Academic detailing or noncommercial interactive education that uses reinforcement techniques individually delivered to clinicians has demonstrated improvements in antibiotic prescribing.^6,15 Previously, we demonstrated that augmentation of audit and feedback of clinician ARI treatment patterns coupled with academic detailing improved prescribing for ARIs without negative clinical consequences in 10 VHA clinics.⁷ However, limited data describe the impact of health system–wide outpatient antibiotic stewardship interventions on patient outcomes.

The VHA is the largest healthcare system in the United States, providing integrated inpatient and ambulatory care to 9 million veterans through 130 VHA medical centers (VAMCs) (Appendix 1 online). VHA-wide antimicrobial stewardship activities are coordinated by the Antimicrobial Stewardship Taskforce (ASTF). Similar to the entire US healthcare system, the majority of antibiotics prescribed within the VHA are dispensed in outpatient settings.¹⁷ In 2016, ASTF and the VHA National Academic Detailing Service (ADS) developed an outpatient antimicrobial stewardship intervention to improve ARI prescribing (ie, the ARI Campaign). The ARI Campaign encourages local VAMCs to engage high-prescribing clinicians with 2 strategies: audit and feedback with peer comparison of ARI antibiotic prescribing rates, and individualized academic detailing of ARI management. The ARI Campaign was initiated in October 2017. Here, we describe the ARI Campaign and its association with antibiotic treatment and patient outcomes across the VHA.

Methods

Patients

A multicenter, retrospective cohort of outpatients with visits for uncomplicated ARI between October 2012 and April 2019 was developed. Uncomplicated ARI visits were identified by International Classification of Diseases, Tenth Revision (ICD-10) procedure coding system (PCS) or ICD-9 PCS equivalent for visits with diagnosis of acute rhinosinusitis, pharyngitis, bronchitis, or unspecified upper respiratory tract infection (URI-NOS) (Appendix 2 online).⁷ Visits for patients with ARIs who had ICD-10 PCS codes for immunosuppression, dialysis, advanced malignancy, and/or chronic pulmonary disease within the past 2 years were excluded (Appendix 2 online). To enhance complicated case identification, patients with prescriptions for inhaled anticholinergics, monoclonal antibody or anti–tumor necrosis factor agents, or recent chemotherapy, with a concurrent ICD-10 PCS codes for infectious disease requiring antibiotics, or an ARI diagnosed in the prior 30 days were excluded.^3,4,7

Intervention

ASTF and VHA ADS initially defined ARI Campaign Key Messages (Table 1), created an inventory of resources, and developed an implementation guide (ie, step-by-step protocol) as a resource for local facility personnel (Appendix 3 online).

Table 1. Key Messages for the VHA ARI Campaign

Note. VHA, Veterans’ Health Administration.

The campaign was developed to align with the Centers for Disease Control and Prevention (CDC) Core Elements of Outpatient Antimicrobial Stewardship: Leadership Commitment, Action, Tracking and Reporting, and Education and Expertise.¹⁸ A VHA-wide ASTF webinar to kick off the ARI Campaign was broadcast in October 2017. Local antimicrobial stewards were encouraged to engage stakeholders prior to campaign initiation and to obtain commitment from local leaders (ie, emergency department (ED) and ambulatory care directors) key to implementation. Along with local antimicrobial stewards, VHA ADS personnel are embedded within a VAMC or a VHA regional geographical network of facilities (ie, Veterans’ Integrated Service Network or VISN). VHA ADS personnel are trained to provide academic detailing to clinicians on a variety of clinical topics. Local intervention personnel (facility antimicrobial stewards plus ADS personnel) were encouraged to coordinate intervention activities within each VAMC. Actions included clinician audit and feedback with peer-group comparison and individualized academic detailing of clinicians who frequently diagnosed ARIs.

Clinicians practicing in the ED, urgent or primary-care settings were identified through a facility-level report obtained through an electronic medical record interface (ie, ARI dashboard) that tracked clinician ARI visit totals and antibiotic prescribing. The ARI dashboard allowed personnel delivering the intervention to filter aggregated ARI visits by date, clinician, and peer group (ie, ED or primary care). Peer comparison of measures on audit-and-feedback reports were compared to average values of the peer group. Intervention personnel could print the audit-and-feedback reports for in-person distribution or e-mail the report to clinicians. Stewards were encouraged to disseminate baseline audit-and-feedback reports to clinicians with ≥15 uncomplicated ARI visits during the prior year to coincide with the beginning of the ARI season, then at least quarterly through spring and as needed until the following ARI season. Intervention personnel were encouraged to reinitiate the campaign each ARI season. Additional campaign components included enablers to support the CDC Core Elements including sample commitment letters for administrative and clinic champions, sample electronic medical record ARI disease management menus, training for antimicrobial stewards on how to perform academic detailing, printed ARI-specific academic detailing materials, clinician-focused video clips on patient communication strategies for ARIs, patient educational materials, and an ARI Campaign kickoff slide set. These materials could be accessed (or ordered) online through the ADS SharePoint site free of charge.

Local antibiotic stewards were encouraged to track facility performance on ARI-related metrics as part of their stewardship program and to report performance to appropriate local facility governing committees. In addition to audit-and-feedback reports for individual clinicians, other ARI dashboards provided local stewardship personnel with clinic-, facility-, or VISN-level performance on 5 ARI–related antibiotic metrics (Appendix 4 online). The metrics were based on professional guideline recommendations for diagnosis and treatment of ARI.^19–22

Data and outcomes

Antibiotic prescriptions filled between 2 days before and 3 days after the index visit were attributed to the ARI visit.^3,7 Because some medications prescribed within the VHA are filled by non-VHA pharmacies, dispensing data were supplemented with natural language processing (NLP) algorithm-generated data that identified additional antibiotic prescriptions documented in clinician progress notes.^3,7

Additional data obtained for analysis included patient variables (ie, sex, age, diagnosis, maximal temperature recorded on visit date), clinician variables (ie, age, sex, degree), and facility-related variables (ie, clinic type and VISN) in addition to calendar month of study.

As VAMC participation in the ARI Campaign was voluntary, the number of times local intervention personnel accessed the ARI dashboard per month for audit-and-feedback reports and identifying high-prescribing clinicians was tracked as a measure of facility intervention intensity.

The primary outcome was the probability to receive an antibiotic for an uncomplicated ARI visit before and after campaign initiation. Secondary outcomes included the probability to receive an antibiotic for acute bronchitis or URI-NOS, to receive appropriate therapy for pharyngitis or sinusitis, or to be diagnosed with sinusitis relative to other uncomplicated ARI diagnoses. Patient outcome measures included the probability of a return visit with an ARI diagnosis coded (ie, return visit) and hospitalization with an infectious-related diagnosis 2–30 days after ARI index visit before and after campaign initiation (Appendix 2 online).

Analysis

Intervention effects were assessed with logistic generalized estimating equation (GEE) models for binary outcomes (antibiotic prescribing or outcomes) with clustering by facility. The preintervention period was from October 2012 to September 2017 and the postintervention period was from October 2017 to March 2019. The logistic GEE models pre- and postintervention effects adjusted for time trend, month, patient age, patient temperature, and provider type. Assessment of intervention intensity was conducted utilizing similar logistic GEE models, except the intervention was represented in 5 strata based on the number of times the ARI dashboard was accessed. The lowest stratum comprised facilities that never accessed the ARI dashboard, indicating nonparticipation in the campaign. The remaining facilities were stratified into quartiles of ARI dashboard access intensity. Results were expressed as odds ratios (ORs) with 95% confidence intervals (CIs) for all pre- and postimplementation effects. To aid in interpretation, antibiotic and patient outcomes were also expressed as events per 1,000 uncomplicated ARI visits before and after implementation.

The ARI campaign was conducted as an operational activity; however, the analysis activities constitute research (VHA Policy Handbook guideline 1058.05). The research activities were granted IRB approval and comply with all federal guidelines and policies relative to human-subjects research.

Results

In total, 2,554,472 visits with ARI diagnoses occurred during the 7-year study period. Among them, 1,227,940 (48.1%) were excluded due to complicated conditions (Fig. 1). The final cohort included 1,003,509 visits before implementation and 323,023 visits after implementation. Most patients were male, middle-aged, and afebrile upon presentation. Most visits occurred in primary care, and most care was provided by physicians (Table 2).

Fig. 1. Study flow diagram for the VHA ARI Campaign. Note. VHA, Veterans’ Healthcare Administration; ARI, acute respiratory tract infection; COPD, chronic obstructive pulmonary disease; SSTI, skin and soft-tissue infection. ^AVisits may have met >1 exclusion criteria.

Table 2. Characteristics of Patients With Uncomplicated Acute Respiratory Tract Infection (ARI) Before and After ARI Campaign Implementation

Note. URI-NOS, unspecified upper respiratory tract infection; IQR, interquartile range; PA, physician assistant; NP, nurse practitioner; ED, emergency department; UC, urgent care.

^a Missing values: temperature (n = 73,865), systolic blood pressure (n = 55,522), provider type (n = 3).

Antibiotic prescription for uncomplicated ARI decreased after implementation (OR, 0.82; 95% CI, 0.78–0.86) (Fig. 2 and Table 3). Facilities in the upper third and fourth quartiles of intervention intensity exhibited significant reduction in antibiotic prescribing, with mean facility absolute decreases of 34 and 78 prescriptions per 1,000 ARI visits, respectively. Reductions (OR, 0.84; 95% CI, 0.80–0.88) in antibiotic prescribing for acute bronchitis and UTI-NOS paralleled those of the primary endpoint with significant reductions of antibiotic prescribing for the third quartile (−51 per 1,000) and fourth quartile (−92 per 1,000) of intervention intensity, respectively. Appropriate management of acute pharyngitis increased after implementation (OR, 1.20; 95% CI, 1.13–1.27). Improvement was driven by facilities in the fourth quartile (OR, 1.45; 95% CI, 1.24–1.70) of intervention intensity, where appropriate therapy increased by 43 per 1,000 after implementation. The proportion of sinusitis visits with preferred antibiotic therapy prescribed remained unchanged (OR, 0.97; 95% CI, 0.91–1.03). High absolute rates of prescribing preferred antibiotic therapy for sinusitis before implementation (731 per 1,000) and after implementation (756 per 1,000). The proportion of uncomplicated ARI visits diagnosed as sinusitis decreased after implementation (OR, 0.79; 95% CI, 0.75–0.82). The reductions in sinusitis diagnoses were similar across all levels of intervention intensity.

Fig. 2. Observed (2a) and predicted (2b) antibiotic prescription (%) for uncomplicated acute respiratory tract infection (ARI) diagnoses. (a) Observed monthly percentage of antibiotic prescribing was calculated for the whole cohort (overall) for the whole study period and by facility dashboard access quartiles (no access, Q1–Q4) for the 18-month postimplementation period. The probability of antibiotics prescribed for each individual was predicted using the generalized estimating equation (GEE) model as described in the Methods. (b) Predicted monthly percentage of antibiotics prescribing was estimated as the mean of the predicted probabilities for the whole cohort (overall) for the whole study period and by the facility dashboard access quartiles (no access, Q1–Q4) for the 18-month postimplementation period.

Table 3. Changes in Campaign Metrics Before and After ARI Campaign Implementation Across the VHA System and Based on Intensity of ARI Dashboard Utilization

Note. ARI, acute respiratory infection; VHA, Veterans’ Health Administration; CI, confidence interval.

^a The percentage (%) of patient visits with antibiotics prescribed for individual ARI diagnoses pre/post implementation were bronchitis (83.0 vs 78.4), sinusitis (86.6 vs 86.5), pharyngitis (68.4 vs 63.4), URI-NOS (48.8 vs 42.3).

^b ARI Dashboard hits was stratified into facilities that never accessed the dashboard indicated by 0 and for facilities that did access the dashboard into quartiles based on the number of times they accessed the dashboard over the 18-mo postimplementation period, reported as a range: first quartile (1–14), second quartile (≥14–49), third quartile (≥49–173), fourth quartile (≥173–1,300).

^c Appropriate prescribing for acute pharyngitis was defined as no antibiotic for cases with a negative group A rapid antigen detection test or throat culture (or test not performed), penicillin or amoxicillin for cases with a positive test, or cephalexin, or clindamycin for cases with a positive test and penicillin allergy.

^d Appropriate antibiotic selection was defined as defined as prescription for an aminopenicillin (±clavulanate) or, in case of penicillin allergy, doxycycline or a respiratory fluoroquinolone in patients visits with an antibiotic prescribed.

Across the VHA, the probability of a return ARI visit increased (OR, 1.04; 95% CI, 1.00–1.08) (Table 4). This effect was most pronounced for the first quartile in facilities that minimally participated in the campaign (OR, 1.15; 95% CI, 1.01–1.31). Infection-related hospitalization within 30 days (OR, 1.16; 95% CI, 0.99–1.35) was not significantly different; however, point estimates for infection-related hospitalization in the third quartile (OR, 1.30; 95% CI, 0.98–1.74) and fourth quartile (OR, 1.21; 95% CI, 0.92–1.59) of intervention intensity were observed (Table 4). The overall rate of infection-related hospitalization was low (0.22%), and the most common admitting diagnosis for infection-related hospitalization was pneumonia (22.0%) (Appendix 5 online).

Table 4. Patient Outcomes Before and After ARI Campaign Implementation Across the VHA System and Based on Intensity of ARI Dashboard Utilization

Note. ARI, acute respiratory infection; VHA, Veterans’ Health Administration; CI, confidence interval; ICD-10, International Classification of Diseases, Tenth Revision; PCS, procedure coding system.

^a ARI dashboard hits was stratified into facilities that never accessed the dashboard indicated by 0 and for facilities that did access the dashboard into quartiles based on the number of times they accessed the dashboard over the 18- mo postimplementation period reported as a range: first quartile (1–14), second quartile (≥14–49), third quartile (≥49–173), fourth quartile (≥173–1,300).

^b ARI-related visits were defined as any physical visit to the VHA with an ARI diagnostic code assigned to the visit that occurred 2–30 d after the index ARI visit.

^c Infection-related hospitalization was defined as an admission to a Medical-Surgical ward in which a primary discharge diagnosis that included an infection-related ICD-10 PCS or ICD-9 PCS equivalent code (Appendix 2) that occurred within 2–30 d after the index visit.

Discussion

This large health-system intervention was associated with improved ARI management and reduction in outpatient antibiotic prescribing for uncomplicated ARIs after implementation. Participation was voluntary, and the frequency that local personnel conducting the intervention utilized the ARI dashboards served as measure of intervention intensity. Reduction in antibiotic prescribing was related to the intensity of intervention uptake. Facilities above the median of ARI dashboard access exhibited significant reduction in antibiotic prescription. Reduction in antibiotic prescription for acute bronchitis and URI-NOS, and improvements in pharyngitis management, were also observed within intensive-intervention facilities. The proportion of ARI visits diagnosed as acute sinusitis over the 7-year period decreased across the VHA irrespective of intervention intensity. The findings suggest that implementation of the clinician-directed intervention was associated with improvements in guideline-concordant ARI management.

Outpatient return visits for ARIs increased slightly after implementation. However, the increase was limited to facilities within the lowest intensity of ARI dashboard utilization. This finding is unlikely to be related to intervention implementation because utilization of the ARI dashboard in this quartile was exceptionally limited after implementation. Infection-related hospitalization after an ARI visit was not significantly different after implementation. Point estimates for the highest intervention intensity quartiles were higher, but not significantly different than facilities that did not participate in the intervention. The absolute differences in the postimplementation infection-related admission rate between facilities that never participated and those in the third and fourth quartiles of intervention intensity were 0.05% and 0.03%, respectively. This finding indicates that intervention implementation was not associated with harm at the health-system level.

A study strength was the real-world application of the intervention across a large healthcare system through voluntary participation, which may contribute to sustainability beyond the research setting. The VHA infrastructure allowed for intervention development and facilitation by the ASTF and ADS across the healthcare system. The centralized VHA Corporate Date Warehouse was utilized to generate both the ARI dashboard data used to deliver the intervention and to conduct an integrated analyses of outpatient ARI management and patient outcomes.²³ Although many US healthcare systems do not possess data repositories or utilize academic detailing, the utilization of dashboards to track healthcare performance and academic detailing programs to improve medication prescribing is increasing among other large healthcare systems (ie, Kaiser Permanente and state health departments).²⁴ Analysis strengths include the large sample size, which provided statistical power to examine clinical outcomes with a high degree of precision and evaluation over multiple ARI seasons after implementation. Although the use of NLP to supplement antibiotic prescribing data for the analysis was a strength, it was not feasible to include these data in real time within the ARI dashboards. This finding may have resulted in underreporting of antibiotic use for audit and feedback in VHA practice settings (community-based outpatient clinics) where outsourcing of prescriptions is common. As a result, some VAMCs chose to exclude these clinics from intervention and reporting, whereas others performed academic detailing and included them.

This study had several limitations. As facility participation was voluntary, a quasi-experimental design was utilized. Facilities with higher-intensity intervention may have been more broadly engaged in quality improvement or may have employed additional or alternative interventions during the study period that we were unable to identify.^25,26 Facilities could have opted to deliver audit and feedback or academic detailing to clinicians without combining both interventions, and we were unable to track which interventions individual clinicians received. The VHA population is predominantly male and older than the overall US population, and approximately half of the ARI visits were excluded as potentially complicated due to chronic pulmonary disease and immunosuppression, which may have limited the potential reduction of overall antibiotic prescribing.^25,27 Furthermore, ARI cases were identified based on administrative coding, and the analysis did not adjust for patient-level comorbidity. Finally, the analysis did not consider potential benefits of reduced antibiotic prescribing on adverse events, antibiotic resistance, or cost.

Antibiotic prescriptions in US outpatient settings decreased during the approximate time frame of the study; however, inappropriate antibiotic prescription for adults decreased minimally.² Most inappropriate antibiotic prescribing is for ARI-related conditions and visits for ARIs have declined since 2010–2011.² Reasons for the decline in ARI visits is unclear.

In a pilot study conducted in 10 VHA clinics that utilized a similar protocol to the ARI Campaign under more controlled conditions, we observed a reduction in antibiotic prescribing for uncomplicated ARIs that was comparable to the highest intervention quartile in the ARI Campaign. We also observed no increase in ARI return visits as well as a small decrease in all-cause hospitalization.⁷ Based on the pilot-study findings and other similar studies, the interventional approach appears to be cost-effective.^28,29 To date, the VHA ARI Campaign is one of the largest US outpatient stewardship interventions using a clinician-directed approach. Prescription feedback interventions of similar focus and scale have been conducted in Europe with mixed results.^13,14 Many of these interventions focused on electronic or written dissemination of feedback coupled with dissemination or reference to prescribing guidelines. In a randomized trial of 2,900 high-prescribing primary-care clinicians in Switzerland in which feedback was provided by mail or electronically and was supplemented with national guidelines, minimal reduction in antibiotic prescription to adults was observed (between-group difference, −4.6%).¹³ In a cluster-randomized trial of 79 general practices in the United Kingdom where electronic prescription feedback was facilitated by recruitment of a local champion, a similar reduction (adjusted rate ratio, 0.84) in antibiotic prescription for treatment of respiratory tract infections without increase in secondary infection was observed.¹⁴ A large study of academic detailing to improve antibiotic prescribing of general practitioners in Norway indicated a 13% reduction in potentially inappropriate antibiotic prescribing.³⁰ To our knowledge, the ARI campaign is the only large-scale intervention to combine an audit-and-feedback approach with academic detailing targeting reduction in inappropriate antibiotic prescribing.

Future work should consider the impact of coronavirus disease 2019 (COVID-19) on outpatient antibiotic prescribing, the increasing use of telehealth for mild-to-moderate illness, the impact of diagnostic shifting, and approaches to optimizing clinician-directed intervention delivery. Although our study concluded prior to the arrival of COVID-19, large reductions in outpatient antibiotic prescription have been observed since the pandemic began.³¹ Reasons for the reductions include reduced transmission of non–COVID-19 viral illnesses such as influenza, a reduced volume of office visits, and a change in treatment modalities with large increases in telehealth.^31–33 Many VHA facilities placed the ARI campaign on hold during the pandemic; however, the ARI Campaign was recently revised with anticipation of a return to historical practice patterns once COVID-19 subsides. The increased use of telehealth to diagnose and treat ARIs will require further study, especially the role of physical assessment and diagnostic test ordering in ARI management. Altering diagnosis or coding practices in response to diagnosis-based interventions (ie, diagnostic shifting) has been observed in audit-and-feedback interventions.^7,34 Additional work is needed to develop metrics to assess a broader array of conditions for which antibiotics are inappropriately prescribed and to address diagnostic shifting. In 2022, the National Committee for Quality Assurance released the Antibiotic Utilization for Respiratory Conditions HEDIS measure, which was designed to assess a broader array of respiratory diagnoses for which antibiotics are inappropriately prescribed.³⁵ Despite the requirement that all VAMCs have a stewardship program, a number of facilities did not participate or participated minimally, and reasons for the lack of engagement requires study. Finally, further work is needed to optimize the behavioral approaches utilized in delivering audit-and-feedback and academic-detailing interventions.

In conclusion, implementation of a voluntary systemwide clinician-focused intervention involving audit and feedback of antibiotic prescribing rates coupled with academic detailing was associated a meaningful reduction in antibiotic prescribing for ARIs within facilities that intensively participated. Minimal impact on ARI-related clinical outcomes was observed. Healthcare systems implementing similar interventions should follow the CDC Core Elements in intervention design. Further work is needed to maximize reduction of unnecessary antibiotics while identifying patients for whom antibiotic therapy is appropriate.