Inappropriate antibiotic prescribing is common and has widespread public health implications, including increasing drug-resistant bacterial infections and associated morbidity and healthcare costs. Reference Durkin, Jafarzadeh and Hsueh1,2 Most antibiotics are prescribed in the ambulatory setting and an estimated 30% are unnecessary. Reference Hersh, King, Shapiro, Hicks and Fleming-Dutra3 For upper respiratory infections, such as bronchitis and the common cold, up to 50% of all antibiotic prescriptions are inappropriate. Reference Fleming-Dutra, Hersh and Shapiro4 Given the established public health threat due to inappropriate antibiotic use, coupled with recent data demonstrating only modest improvement in contemporary prescribing rates, there is an urgent need for interventions that effectively address persistent high rates of antibiotic prescribing in ambulatory settings.
Antimicrobial stewardship interventions have improved antibiotic use in hospitals, but they have not been widely implemented in outpatient practices. Furthermore, previously published studies of outpatient antimicrobial stewardship have focused on specific practice settings or provider types and on effectiveness of different individual intervention strategies like electronic health record (EHR) alerts, provider education, or provider data feedback. Reference Zahlanie, Mang, Lin, Hynan and Prokesch5–Reference Gulliford, Juszczyk and Prevost10 Few studies to date have examined how to best integrate all of these features with ongoing measurement of prescribing rates within the context of a larger, multiple-specialty, integrated healthcare network (where most provider groups now function). Reference Gulliford, Juszczyk and Prevost10–Reference Madaras-Kelly, Hostler and Townsend13 In the Carolinas Healthcare Outpatient Antimicrobial Stewardship Empowerment Network (CHOSEN) program, we applied a stakeholder-centered design to develop and implement a bundle of key intervention strategies in diverse ambulatory practices within a large integrated network, including (1) patient and provider education tailored to practice-specific needs and (2) easily accessible, timely data on prescribing patterns for providers and practice managers to track and compare antimicrobial prescribing rates. Reference Davis, Liu and Taylor14,Reference Yates, Davis and Taylor15 The current study tests the hypothesis that implementation of CHOSEN reduces inappropriate outpatient antibiotic prescribing for upper respiratory tract infections by 20% over 2 years. This target was based on published literature and national recommendations published in 2015. Reference Fleming-Dutra, Hersh and Shapiro4,16
Methods
Study design and setting
A prospective study using an interrupted time series design was conducted to evaluate the impact of the CHOSEN initiative on rates of inappropriate outpatient antibiotic prescribing. The study occurred at 162 ambulatory family medicine, internal medicine, pediatric medicine, and urgent care primary-care practices within Atrium Health (formerly Carolina’s Healthcare System), the largest integrated health system in the Carolinas. The design consisted of (1) a pre-intervention baseline period (April 2016–November 2017) followed by (2) an implementation wash-in period during which all providers were oriented to CHOSEN educational materials and were trained to use the intervention dashboard to obtain prescribing data (December 2017–March 2018) and (3) an intervention period, post-CHOSEN implementation (April 2018–March 2020). The study was approved by the Atrium Health Institutional Review Board (no. 03-17-08E).
Intervention
Findings from our preliminary mixed-methods studies conducted with patients and providers, Reference Davis, Liu and Taylor14,Reference Yates, Davis and Taylor15 along with baseline prescribing data, Reference Schmidt, Spencer and Davidson17 were used to inform and adapt intervention development through a stakeholder-centered design process (Fig. 1 and Supplementary Figs. 1–7 online), in alignment with the Centers for Disease Control and Prevention (CDC) Core Elements for Outpatient Antibiotic Stewardship Programs (ASPs). Reference Sanchez, Fleming-Dutra, Roberts and Hicks18 Finalized intervention components included (1) an antimicrobial stewardship health education campaign for patients and providers and (2) an interactive, provider-facing reporting dashboard for comparing antibiotic prescribing behaviors among providers, practices and organizational groupings. The education campaign occurred over a 6-month period from November 2017 through March 2018 to introduce the CHOSEN educational tools to all practices in the 4 primary-care service lines: internal medicine, family medicine, urgent care, and pediatric medicine (Fig. 1 and Supplementary Figs. 1–7 online). A centralized education team led by the ASP medical director and lead pharmacist coordinated with service-line leaders and performance improvement teams to rollout presentations and webinars on CHOSEN tools, education, training, and evaluation of metrics (Fig. 1). Targeted education provided at leadership and practice meetings focused on how to incorporate the tools into daily practice. The multimedia campaign for patients included social media, mass media, and printed materials, as well as a website (www.atriumhealth.org/germs) that contained patient-specific resources. All materials for patients and providers were also available on Atrium Health’s internal website.
Fig. 1. CHOSEN intervention development process, including detailed components of the antibiotic education campaign.
The CHOSEN prescribing dashboard was developed in Microsoft Power BI, a business analytics tool used regularly to track care delivery metrics at Atrium Health (Supplementary Fig. 8 online). Beginning in March 2018, EHR data were integrated monthly into the dashboard for practice and provider reporting; only providers who had 10 or more encounters for an indication were included. Physicians, advanced practice providers, quality and performance improvement coordinators, and primary-care administrators had dashboard access. The dashboard user interface provided interactive visualizations of prescribing data compared year-to-year and rolling 12 months. Data were viewable by indication, antibiotics class, and at the levels of provider, practice site, specialty medical director, and administrator. The dashboard remains part of continuous, ongoing assessment of feedback from users and leadership.
Data collection
Demographic, clinical, and outcomes data were collected directly from the EHR and administrative data sources via the enterprise data warehouse. Patient, provider, and practice characteristics were collected at the encounter level. In encounters with multiple indications for antibiotics, each indication was included. If >1 antibiotic was prescribed for an encounter, both were included in the data set.
Measures
All ambulatory visits to participating primary care practices between April 2016 and March 2020 with at least 1 upper respiratory infection diagnosis were examined. Diagnoses included acute sinusitis, otitis media (nonsuppurative), acute bronchitis, pharyngitis (nonbacterial), cough, URI, common cold, allergic rhinitis, and influenza—conditions for which antibiotics are typically not indicated (Supplementary Table 1 online). Diagnoses were obtained from the final billed record for each encounter using International Classification of Disease, Tenth Revision (ICD-10) codes.
Table 1. Characteristics of Patient Population and Distribution of Indications
Note: NS, not significant; URI, upper respiratory infection. Indication encounter percentages do not sum to 100 due to encounters with multiple indications.
Antibiotic prescriptions prescribed within 72 hours of the encounter were included. Patients were excluded if they had any concomitant diagnosis for which an antibiotic would appropriately be indicated, such as cellulitis, pneumonia, urinary tract infection, or a URI for which an antibiotic would be indicated in accordance with system guidelines (ie, bacterial pharyngitis, acute suppurative otitis media). All outpatient antibiotics were included in analysis with a specific focus on aminopenicillins, cephalosporins, macrolides, penicillins, fluoroquinolones, and tetracyclines.
The primary outcome of interest was monthly antibiotic prescribing rates by primary care service line. Monthly prescription rates were calculated as the number of encounters with an antibiotic prescription ordered, compared to the total number of eligible encounters (ie, visits with relevant ICD-10 codes) (Supplementary Table 1 online). ICD-10 codes were based on previously published literature Reference Fleming-Dutra, Hersh and Shapiro4 and were cross validated with older ICD-9 codes. Reference Chua, Fischer and Linder19 We reviewed all ICD codes to ensure that all appropriate codes were included or excluded. We performed initial validation prior to launch of database to ensure appropriate codes were included and excluded. Finally, we performed random chart validation at onset of data base to confirm appropriate data capture and thereafter on ongoing basis at request of provider.
Statistical analysis
We used an interrupted time series (ITS) analysis to evaluate the impact of the CHOSEN multimodal interventions on inappropriate antibiotic prescribing for upper respiratory infections at the encounter level within each primary-care service line. For the primary analysis, segmented regression models were constructed to compare level changes (ie, abrupt) and slope changes (ie, gradual) in antibiotic prescribing between preintervention and intervention periods (ie, once providers had access to the full scope of educational materials and dashboard data on prescribing rates). Because timing of education rollout and introduction of dashboard data to providers occurred over a 6-month period, we removed this transitional, implementation wash-in period from the primary analysis. Separate models were fit for each primary-care service line (ie, family medicine, internal medicine, pediatric medicine, and urgent care). Cumby-Huizinga and Durban-Watson test statistics were calculated to assess for autocorrelation in the error distribution, and ordinary least-squares time-series regression models were fit with Newey-West standard errors to account for the correct autocorrelation structure (ie, maximum lag set to 2 based on diagnostic information) and potential heteroskedasticity. In each primary-care service line, regression coefficients and 95% confidence intervals were estimated to test intervention changes in the level and slope, relative to preintervention prescribing trends.
We completed sensitivity analyses to examine the influence of selecting different preintervention and intervention comparison months, with and without an implementation wash-in period (Supplementary Table 4 online for details and results). We chose the final model as the best reflection of the data and program rollout, while adjusting for seasonality in the data. Finally, we calculated descriptive statistics and used the χ 2 to compare changes in the distribution of different antibiotic classes prescribed during the preintervention and intervention periods. We have reported relative changes in rates, as is standard when comparing rates across time or disease categories. Reference Low20 All statistical tests were 2-tailed, with P < .05 considered statistically significant. Analyses were performed using STATA version 15.1 software (StataCorp, College Station, TX).
Results
During the study period, there were 691,545 unique patients contributing 1,656,472 primary-care encounters with 746 unique providers. Overall, the proportion of encounters with antibiotics inappropriately prescribed decreased from 47.5% in the preintervention period (April 2016–October 2017; 286,580 antibiotics prescriptions in 704,248 encounters) to 38.7% during the intervention period (April 2018–March 2020; 277,177 antibiotics prescriptions in 832,200 encounters. The relative difference in prescribing rates was −18.5%. (P < .01). Preintervention and intervention encounter-level characteristics of patients, diagnosis indications, and prescribed antibiotic classes are shown in Table 1. There were no statistically significant differences observed in the proportion of preintervention and intervention encounters among patients by age, sex, race, ethnicity, and diagnosis indications. We detected a modest relative decrease in macrolide prescribing from preintervention (26.9% of antibiotic prescriptions) to intervention (22.1% of antibiotic prescriptions). Our fluoroquinolone rates were significantly lower than the rates for other classes of antibiotics. This finding may reflect trends in decreased fluoroquinolone prescribing nationally, with increased black-box warnings from the FDA. Additionally, pediatricians tend to use fewer fluoroquinolones, which may be reflected in our overall antibiotic distribution. Reference Tran, Antonelli, Hincapie-Castillo and Winterstein21–Reference Etminan, Guo and Carleton23
Comparisons of monthly inappropriate antibiotic prescribing rates over time are shown for each primary-care service line in Table 2 and Figure 2. During the preintervention period, encounters to urgent care had the highest proportion of visits with inappropriate prescribing (51.8%), followed by family medicine (49.6%), internal medicine (47.8%), and pediatric medicine (40.7%). After CHOSEN implementation, encounters to family medicine showed the largest relative decrease in the proportion of visits with inappropriate prescribing (−20.4%), followed by internal medicine (−19.5%), pediatric medicine (−17.2%), and urgent care (−16.6%; all relative differences P < .01).
Table 2. Change in Prescribing Rates Before and After Intervention by Practice Type
Note. CI, confidence interval.
a All relative differences were significant at P < .01
Fig. 2. Interrupted time series before and after intervention antibiotic prescribing rates over time with washout period. Month 0 is April 2016. The vertical lines indicate the washout period from month 20, October 2017 to May 2018.
In segmented regression analysis, different level changes and slope changes were observed across each primary-care service lines. Family medicine and internal medicine had similar patterns of change, with each showing a statistically significant level change (ie, immediate decrease) from preintervention to intervention (family medicine, −7.95; 95% CI, −11.05 to 4.85; internal medicine, −4.73; 95% CI, −7.75 to −1.71) but similar month-to-month changes in the intervention period relative to preintervention trends. Conversely, both urgent care and pediatric medicine did not show a statistically significant level change but did show significant decreases in the monthly trends of inappropriate prescribing in the intervention period (urgent care, −.78; 95% CI, −1.57 to .01; pediatric medicine, −.15; 95% CI, −.28 to −.03). Sensitivity analyses confirmed that our final model best fit the data, allowed for adjustments for seasonality, and did not differ significantly in the results compared to the alternative models tested (Supplementary Table 2 online).
Discussion
Reducing antibiotic resistance in the United States is an urgent priority. Reference Durkin, Jafarzadeh and Hsueh1,2,Reference Sanchez, Fleming-Dutra, Roberts and Hicks18 The CDC and US regulatory agencies, such as the Joint Commission and the Centers for Medicare and Medicaid Services, have created guidance, Healthcare Effectiveness Data and Information Set (HEDIS) metrics, and Merit-Based Incentive Payment System (MIPS) modules to encourage provider adherence to quality and safety measures for prescribing. Reference Sanchez, Fleming-Dutra, Roberts and Hicks18,24–26 Nevertheless, few providers and health networks have achieved and sustained significant reductions in inappropriate antibiotic prescribing. 27 Most antibiotic prescriptions occur at ambulatory care and urgent-care practices. High patient volumes create time-pressured care decisions, and providers often face demands to prescribe antibiotics from patients who may not understand their appropriate use. Reference Yates, Davis and Taylor15 The CHOSEN initiative was created to address such barriers that are unique to ambulatory care settings. The goals were to decrease inappropriate antibiotic utilization 20% by 2020 across Atrium Health primary-care practices and to increase stakeholder awareness and engagement in solutions. To ensure thorough adoption, education, and training, the program was implemented over a 6-month period and included 162 geographically dispersed practices.
Our study analysis evaluated antibiotic prescribing for acute URIs across 1,424,729 million visits occurring over 4 years. Few studies have previously evaluated an intervention across a large integrated health system. Reference Gulliford, Juszczyk and Prevost10–Reference Madaras-Kelly, Hostler and Townsend13 Our results demonstrated that this combination of interventions decreased prescribing for URIs by 18.5% across 4 primary-care service lines (ie, internal medicine, family medicine, pediatric medicine, urgent care). Overall, family medicine reached the 20% reduction goal (−20.4%), with the other service lines reaching close to 20% (ie, internal medicine, −19.5%; pediatric medicine, −17.2%; urgent care, −16.6%).
Although decreases in antibiotic prescribing occurred across all practice types, patterns of improvement differed. Initial rapid declines after intervention launch at family and internal medicine practices remained constant over the intervention period, which is likely attributable to widespread early adoption of tools and dashboard feedback. Similarly, pediatric practices had a sustained response. Urgent-care providers had higher rates of prescribing in the first 6 months, then continued to improve throughout the study. Our overall reduction in antibiotic prescribing compares favorably to results published on similar interventions in outpatient prescribing ranging from 4.8% to 23%. Reference Chung, Nailon and Ashraf8,Reference Dutcher, Degnan and Adu-Gyamfi11,Reference Schwartz, Ivers and Langford28
Previous studies have evaluated educational interventions, EHR-based interventions, or evaluation of peer-to-peer feedback individually. Reference Chung, Nailon and Ashraf8–Reference Gulliford, Juszczyk and Prevost10,Reference Burns, Johnson, Pham, Egwuatu and Dumkow29–Reference Wattengel, Sellick and Mergenhagen34 In a recent Canadian trial, sending a single peer-comparison letter to primary-care physicians in the highest quartile of antibiotic prescribers decreased overall prescribing and prolonged-duration prescribing. Reference Schwartz, Ivers and Langford28 Those letters contained recommendations and education, signed by medical leaders. Another Canadian study showed that education alone was insufficient to change prescribing behaviors. Reference Rolf von den Baumen, Crosby, Tadrous, Schwartz and Gomes35 Our findings highlight the additional benefit of integrating stewardship tools.
The CHOSEN program results suggest key factors that led to intervention effectiveness: (1) diverse stakeholder engagement in design and implementation, (2) unified messaging and tools across all service lines and practices, (3) senior leadership support, and (4) timely performance feedback and data transparency. Stakeholders were involved throughout program inception, design, implementation, and data analysis (Fig. 1). Their varied experience and expertise were key to driving program adoption and provider engagement. Consistent materials and messaging across practice types provided a common patient experience regardless of practice or region. The standardized reporting platform held all providers and practices to the same metrics through data transparency. Finally, practice, division, and system leadership emphasized the importance of appropriate prescribing through regular messaging and unified providers across a wide range of practices.
Despite these strengths, the program had several limitations. Our decision to use encounter-level billing data differs from other studies that utilized claims data to examine antibiotic prescribing. Reference Hersh, King, Shapiro, Hicks and Fleming-Dutra3,Reference Fleming-Dutra, Hersh and Shapiro4,Reference King, Bartoces, Fleming-Dutra, Roberts and Hick36 As a result, we could not verify that prescriptions were filled, or include prescriptions that occurred outside of a patient encounter. Nevertheless, we determined that EHR data aligned to the point of care was most consistent with CHOSEN’s focus on provider prescribing behavior. The provider dashboard reinforced this by displaying timely prescribing feedback. Furthermore, using encounter data avoided delays in claim data allowing for monthly provider updates, regardless of specific payor types.
The focus of CHOSEN focus on URI across patient groups differed from prior studies that used a tiered system or HEDIS metrics. Although tiered intervention systems [ie, that describe appropriate antibiotic prescribing as always (tier 1), sometimes (tier 2), or never (Tier 3)] offer categorization across different antibiotic prescribing appropriateness, our stakeholders preferred a simpler structure to promote implementation and provider adoption. Prescribing indications used in CHOSEN were similar to diagnoses from tiers 2 and 3 in other studies. Unlike HEDIS measures, our inclusion criteria enabled providers to focus interventions on all age ranges despite underlying comorbidities. In presentation of acute nonsuppurative otitis media and sinusitis, it is often difficult to determine bacterial or viral etiology. Previously published data have demonstrated that despite these conditions being most caused by viral infections, high levels of inappropriate prescribing remain. Reference Jaume, Valls-Mateus and Mullol37–Reference Yano, Okitsu and Hori39 Therefore, we elected to include these conditions in our data analysis.
Our choice to bundle performance feedback, patient and provider education, and media communications into a single, multimodal program limited our ability to measure the effectiveness of any single intervention. Although we studied both awareness and effects of antibiotic education and communications of providers and patients, Reference Davis, Liu and Taylor14,Reference Yates, Davis and Taylor15 we could not separate their impacts. Likewise, performance feedback effects could not be separated from other interventions. The literature has shown that a multimodal solution is likely to have more benefit than a singular focused intervention. Reference St. Louis and Okere33,Reference Buehrle, Shively, Wagener, Clancy and Decker40,Reference Linder, Meeker and Fox41 We also could not directly measure prescribing dashboard usage based on frequency of provider access; however, all dashboard data were transparent, and providers were encouraged to review and compare their utilization with other providers.
Our evaluation of the CHOSEN program highlights lessons learned. In the initial stages of CHOSEN, we identified the need for ongoing engagement of system leaders, as well as visible participation from front line providers. Although we routinely educated primary-care medical and practice directors on evidence-based recommendations and practice prescribing rates, providers emphasized the importance of incorporating education of the entire practice staff. As a result, we engaged quality improvements teams to spread education to nurses, medical assistants, and other staff. In addition, we incorporated guidance on appropriate antibiotic prescribing into new employee onboarding. These decisions were consistent with our intention to integrate effective interventions into practice at the outset. CHOSEN was expanded in 2020 to the emergency division, school-based care practices, and virtual visits. Further analysis will determine how our interventions affected prescribing practices in these diverse care settings. We structured our analyses to maintain a consistent and focused message on the importance of intervention integration to success in reducing inappropriate antibiotic prescribing. Detailed evaluations of changes in prescribing by indications, provider characteristics, practice type, and antibiotic classes, along with their interactions, are planned for future analysis.
A multidisciplinary stakeholder approach utilizing an innovative prescribing dashboard with targeted patient and provider education successfully decreased inappropriate outpatient antibiotic prescribing in a large ambulatory network. CHOSEN, using this approach, effectively designed and implemented education resources and tools to meet identified needs among both patients and providers for improved understanding and experiences. Overall, CHOSEN demonstrated significant decreases in inappropriate outpatient antibiotic prescribing for upper respiratory tract infections by nearly 20%.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/ice.2022.83
Acknowledgments
We recognize the contributions of the CHOSEN steering committee, without whom this work would not have been possible:
Leigh Ann Medaris, Chloe Sweeney, Cliff Collins, Dan Alter, Marion Davis, Beth Schwenzfeier, Jason Durham, Carmen Stuckey, Steven Jarrett, Sarah Davis, Elizabeth Handy, and Ryan Burns.
Financial support
The CHOSEN initiative was supported by a 2-year grant from The Duke Endowment. The CHOSEN initiative was supported by a diverse group of stakeholders (Supplementary Fig. 9 online).
Conflicts of interest
The authors have no conflicts of interest to report.
