Combined laparoscopic and open colon surgery rankings fail to accurately rank hospitals by surgical-site infection rate

Abstract

Objective:

To compare strategies for hospital ranking based on colon surgical-site infection (SSI) rate by combining all colon procedures versus stratifying by surgical approach (ie, laparoscopic vs open).

Design:

Retrospective cohort study.

Methods:

We identified SSIs among Medicare beneficiaries undergoing colon surgery from 2009 through 2013 using previously validated methods. We created a risk prediction model for SSI using age, sex, race, comorbidities, surgical approach (laparoscopy vs open), and concomitant colon and noncolon procedures. Adjusted SSI rates were used to rank hospitals. Subanalyses were performed for common colon procedures and procedure types for which there were both open and laparoscopic procedures. We generated ranks using only open and only laparoscopic procedures, overall and for each subanalysis. Rankings were compared using a Spearman correlation coefficient.

Results:

In total, 694,813 colon procedures were identified among 508,135 Medicare beneficiaries. The overall SSI rate was 7.6%. The laparoscopic approach was associated with lower SSI risk (OR, 0.5; 95% CI, 0.4–0.5), and higher SSI risk was associated with concomitant abdominal surgeries (OR, 1.4; 95% CI, 1.4–1.5) and higher Elixhauser score (OR, 1.1; 95% CI, 1.0–1.1). Hospital rankings for laparascopic procedures were poorly correlated with rankings for open procedures (r = 0.23).

Conclusions:

Hospital rankings based on total colon procedures fail to account for differences in SSI risk from laparoscopic vs open procedures. Stratifying rankings by surgical approach yields a more equitable comparison of surgical performance.

Surgical site infections (SSIs) cause substantial morbidity, often incurring reoperation and unanticipated hospitalization, with estimated attributable costs of >$3 billion annually. ^1,2 Preventing infection after colon surgery is a national priority for the >300,000 procedures performed annually in the United States, ³ and their associated SSI rates are as high as 14%–25%. ^4,5 The high-volume, high-risk impact of these procedures has led to mandatory reporting of colon SSIs by the Centers for Medicare & Medicaid Services (CMS) to facilitate improvements in surgical processes and outcomes. ^6,7 Hospital performance is tied to financial penalties through both the Hospital Value-Based Purchasing (VBP) Program and the Hospital-Acquired Condition (HAC) Reduction Program. ^8–10

Diagnostic claims codes have proven to be an accurate and efficient proxy for SSIs at the hospital level. Prior studies have shown that these codes can be used to accurately rank hospitals by SSI rates based upon national validation efforts involving review of full-text medical records using Centers for Disease Control and Prevention (CDC) SSI criteria. ^11–15 In fact, these codes identify a large proportion of SSI cases that would be missed by hospital surveillance. ¹⁶ As a result, these codes have been adopted by the CMS Hospital Inpatient Quality Reporting Program to validate hospital SSI reporting. ¹⁷

The CMS adjusts hospital-specific rates of colon SSIs for several risk factors for SSI in an effort to create a fair playing field for hospitals serving more complex surgical populations. However, this adjustment lacks several relevant variables. Neither the CMS nor the CDC account for concomitant intra-abdominal procedures as a potential risk factor for SSI. These procedures increase the procedure time and complexity and have been associated with increased colon SSI risk. ^18,19 Similarly, other comorbidities, such as tobacco use, ^20–24 renal disease, ²⁵ vascular disease, ²⁶ cirrhosis, ²⁷ and malignancy, ²⁷ have been shown to affect SSI as well, but these are not included in adjustment models. Electronically available comorbidities derived from International Classification of Disease (ICD) codes also have been shown to improve risk adjustment and significantly change hospital SSI rankings. ^18,28

Additionally, surgical approach (laparoscopic or open) is not included in the CMS model despite clear evidence of its benefit to equitable ranking of hospitals by SSI risk. Data from the National Surgical Quality Improvement Program (NSQIP) show that laparoscopic colon surgery carries only half the SSI risk of open colon surgery and that laparoscopic patients tend to be younger and fitter and have significantly fewer comorbidities. ⁵ This pattern has been shown repeatedly in other studies. ^4,29–31 Nevertheless, laparoscopic and open colon procedures are grouped together by CMS to generate hospital-specific SSI rates and rankings. This CMS method combines a number of different colon procedures with widely varying patient populations, SSI rates, and uptake among hospitals, which raises questions about its ability to properly adjust for these variations and mitigate the downstream impact on hospital rankings. In this study, we evaluated the impact of separately ranking hospitals using open versus laparoscopic approach, and we explored more informative and equitable ways of comparing colon SSI rates.

Methods

We evaluated hospital SSI rates and generated rankings following colon surgery in fee-for-service Medicare beneficiaries between January 1, 2009, and November 30, 2013, based on Medicare Part A inpatient claims data. ^16,32 We identified colon surgeries using the 37 ICD-9 procedure codes used by the CDC and CMS at that time for national SSI surveillance and reporting, including 8 laparoscopic and 29 open procedures. ³³ When a laparoscopic and an open procedure were coded on the same date, the procedure was considered to be an open approach.

For each colon procedure, we identified patient descriptors, including age, sex, race, and comorbidity score (based on Elixhauser score ³⁴ ) at the time of the procedure. The number of procedures and associated patient characteristics were aggregated at the hospital level. The frequency at which each procedure was performed with other concomitant colon and noncolon intra-abdominal procedures was assessed overall and by hospital. We used χ² tests to assess crude associations, unadjusted for risk factors or for correlation within hospital between patient characteritics and surgical approach.

SSI determinations were made using previously validated administrative claims codes indicating postsurgical infection. ¹⁶ This method has been shown to accurately rank hospitals by their colon SSI rates and is used by the CMS Hospital Inpatient Quality Reporting Program to validate hospital SSI reporting. ^11–16 We assessed claims made within 30 days of the surgical procedure using admininistrative claims codes suggestive of a deep or organ-space SSI: ICD-9 procedure codes 54.0, 54.11, 54.19, 86.04, 86.22, and 86.28, and ICD-9 diagnostic codes 567.21, 567.22, 567.29, 567.38, 569.5, 569.61, 569.81, 682.2, 879.9, 998.31, 998.59, and 998.6. SSI rates were calculated within hospital for each colon procedure type, among all colon procedures, and according to laparoscopic versus open status.

To exclude pre-existing infections, we excluded procedures for which an SSI claims code was designated as present on admission during the index surgical hospitalization. For patients who underwent another major surgery in the 30-day postoperative surveillance window, we censored our surveillance at the time of the subsequent surgery.

Our main analyses used generalized linear mixed model logistic regression with age, sex, race, Elixhauser comorbidity score, concomitant colon and noncolon intra-abdominal procedures, and laparoscopic status to predict SSI risk for each procedure type, accounting for clustering at the hospital level. We calculated an adjusted SSI rate for each hospital based on the population SSI rate times the hospital-specific odds ratio from the model. We ranked hospitals by their adjusted SSI rate.

Our analyses were replicated for 3 categories of procedures: (1) all colon procedures (ie, the 37 ICD-9 procedure codes listed above); (2) open procedures only (29 procedure codes); and (3) laparoscopic procedures only (8 procedure codes). A plot of adjusted hospital rankings comparing SSI rates for open versus laparoscopic procedures was generated and compared using a test of correlation. We also created 2 subgroups for sensitivity analyses by (1) excluding rare procedures that were performed by <25% of hospitals and by (2) limiting the analysis to the 5 colon procedures for which both analogous open and laparoscopic versions exist. ³⁶ (Supplementary Table 1 online) Models were rebuilt and resulting hospital rankings compared within each of these subgroups.

Results

In total, 694,813 colon procedures were performed at 4,093 hospitals involving 508,135 Medicare patients during the study period. These Medicare patients had a mean age of 74.3 years (standard deviation [SD], 9.7); 42.5% were male; 32.3% had other concomitant colon procedures, and their mean comorbidity score was 4.7 (SD, 7.8).

We detected large differences between those undergoing open versus laparoscopic procedures. Overall, the unadjusted hospital mean SSI rate was 9.8% (SD, 11.2) for all procedures, but it was 11.4% (SD, 11.7; range, 0–38.5) for open procedures and 5.8% (SD, 12.4; range, 3.9–10.5) for laparoscopic procedures. The proportion with a concomitant colon procedure was 39.5% versus 23.1% among open versus laparoscopic procedures (P < .001), and the proportion with a concomitant noncolon intra-abdominal procedure was 37.9% versus 20.6%, respectively (P < .001). Among those receiving open procedures, the mean hospital Elixhauser score was 5.1 (SD, 3.1) compared to 2.7 (SD, 2.8) among those receiving laparoscopic procedures.

The proportion of hospitals performing specific procedures varied widely, with a range of 0.1% to 92.8% across procedure codes (Table 1). The mean numbers of colon procedures per hospital were 119.5 (SD, 139.9) for open procedures and 65.2 (SD, 90.8) for laparoscopic procedures. Moreover, 811 hospitals (19.8%) used only an open approach, and among the 3,282 hospitals (80.2%) that used any laparoscopy, proportions of laparoscopic procedures varied widely (range, 0.6%–100%). Few hospitals (n = 70, 1.7%) used only laparoscopy. Mean hospital-specific Elixhauser scores varied by procedure from 2.1 to 8.3, and the mean Elixhauser score across all hospitals was 4.3 (SD, 2.8). Multivariable models showed that increasing Elixhauser score was associated with increased SSI risk: OR, 1.06 per unit Elixhauser score (95% CI, 1.04–1.07) for all procedures; OR, 1.04 (95% CI, 1.02–1.05) for open procedures only; and OR, 1.18 (95% CI, 1.14–1.22) for laparoscopic procedures only. Concomitant intra-abdominal noncolon procedures also showed increased SSI risk (Table 2). Laparoscopic surgery was strongly associated with reduced SSI risk (OR, 0.5; 95% CI, 0.4–0.5) relative to open surgery.

Table 1. Baseline Characteristics By Procedure Code

Table 2. Characteristics Associated with Surgical Site Infection (SSI) for All Colon Procedures, Open Procedures, and Laparoscopic Procedures

Note. OR, odds ratio; CI, confidence interval.

Hispanic and North American Native race were associated with increased SSI risk, although in the laparoscopic-only group, the confidence limits included the possibility of no effect. For all procedures, the OR for Hispanic race was 1.2 (95% CI, 1.1–1.3) and for Native Americans, the OR was 1.4 (95% CI, 1.2–1.6). For open procedures only, the OR for Hispanic race was 1.3 (95% CI, 1.2–1.4) and for Native Americans the OR was 1.4 (95% CI, 1.2–1.6). For laparoscopic procedures only, the OR for Hispanic race was 1.1 (95% CI, 1.0–1.4) and the OR for Native Americans was 1.3 (95% CI, 0.9–1.8) (Table 2). Similar results were obtained when we excluded procedures performed by <25% of hospitals and when we included only procedures with analogous open and laparoscopic versions.

Hospital rankings based on adjusted SSI rates differed significantly when comparing overall procedures, open procedures only, and laparoscopic procedures only (Table 3). When hospital ranks based only on open procedures were compared to ranks based only on laparoscopic procedures, 2,125 hospitals (66.2%) changed rank by 1 or more quartiles and 334 hospitals (10.4%) moved 3 quartiles, from best- to worst-performing quartiles or vice versa. Hospital rank based on overall versus open SSI rates differed but to a lesser degree: 800 hospitals (19.9%) changed rank by 1 or more quartiles, and 3 (0.1%) hospitals moved 3 quartiles. Similar results were obtained when excluding procedures performed by <25% of hospitals and when including only procedures with analogous open and laparoscopic versions (Supplementary Tables 2 and 3 online).

Table 3. Changes in Hospital Surgical Site Infection Quartile Ranks By All Colon Procedures, Laparoscopic Procedures, and Open Procedures

Note: SSI, surgical site infection; N/A, values in these columns or rows represent hospitals that did not perform the designated open or laparoscopic procedures. 1, best-performing quartile; 4, worst-performing quartile. Values in bold represent agreement between each ranking approach.

Scatterplots of hospital rank by open-only versus laparoscopic-only SSI rates revealed a strikingly poor visual correspondence. The Spearman rank correlation coefficient was 0.2 for individual rank (Fig. 1). Similar results were obtained when excluding procedures performed by <25% of hospitals and when including only procedures with analogous open and laparoscopic versions (Supplementary Figs. 1 and 2 online).

Fig. 1. Scatterplot of hospital rankings by laparoscopic and open colon surgical-site infection rates.

Discussion

Although colon surgeries are grouped by the CDC and CMS to determine hospital ranking and, in turn, reimbursement, these procedures are diverse and differ significantly in how often they are performed, how many hospitals perform them, and how often they result in SSI. In particular, open and laparoscopic procedures confer a meaningfully different risk of SSI, and lead to markedly different hospital rankings when considered separately. Properly adjusting for these differences in a single measure will be technically challenging and may not be possible or desirable.

Currently, the CMS adjusts colon SSI rates for age, sex, diabetes, American Society of Anesthesiologists (ASA) score, body mass index, closure status, and whether the procedure occurred in an oncology hospital, but the CMS fails to account for additional key differences listed below that are critical to making fair comparisons. Accounting for additional patient comorbidities, concomitant procedures, and laparoscopic versus open surgical approach affected expected SSI rates. Insufficient adjustment for complexity of illness places hospitals that care for a more complex patient population at a disadvantage because the patients they treat more often require an open surgical approach, a higher-risk procedure, or multiple simultaneous procedures. In fact, some evidence shows that the CMS HAC Reduction Program disproportionately penalizes major teaching hospitals, safety net hospitals, and hospitals with a more complex case mix. ³⁵

Even with optimal risk-adjustment, it is doubtful that inherent differences between open and laparoscopic procedures can be reconciled when these procedures are grouped together. In this study, hospital rankings based upon open procedures were very poorly correlated with rankings based upon laparoscopic procedures. For many hospitals, SSI rankings differed by 3 quartiles when their performance involving open procedures was compared to laparoscopic procedures. This finding suggests that these 2 fundamentally different types of procedures should not be combined into a single ranking schema on which hospital quality and performance penalties are based. Subgroup analysis of only commonly performed procedures or only procedures that can be performed by either laparoscopic or open approach reinforced these results. Ranking open and laparoscopic procedures separately would allow more appropriate adjustment for patient-level variables. A stratified ranking format would lessen the disadvantage faced by hospitals whose patients more often require an open approach out of clinical necessity by no longer forcing a comparison with lower-risk laparoscopic procedures at other centers.

Separately, because laparoscopic surgery ought to be encouraged and more widely used for its benefits in reducing SSI risk, even in certain medically complex patients, ³⁶ it may be important to address the adoption of laparoscopic surgery in quality measures because separate rankings for open and laparoscopic procedures could reduce a hospital’s or surgeon’s incentive to optimize the use of laparoscopy. More likely, the overt differences in SSI risk by approach would increase laparoscopic use over time and ultimately enable separate studies for SSI prevention for those patients who have no alternative to an open approach.

Notably, Hispanic and North American Native race were associated with increased SSI risk. This effect was statistically significant among open procedures but not laparoscopic procedures. Although unmeasured confounders may explain this association, a direct effect would further support use of laparoscopy.

Altogether, these findings suggest an unequal playing field on which US hospitals must compete for CMS reimbursement, with a large minority of hospitals at great disadvantage because they disproportionately serve the most ill population and perform high-risk colon procedures, including those that require an open approach, those that are not offered by most hospitals, and those that involve multiple concomitant procedures or patients with higher comorbidity scores.

This study had several limitations. We did not have access to all variables used by the CMS for risk adjustment, specifically body mass index and dedicated oncology hospital status. We also did not have information on the skill or experience of the surgeon. However, these additional variables are unlikely to sufficiently explain differences found in SSI from an open versus laparoscopic surgical approach. Although the data were derived from colon procedures performed from 2009 to 2013, the fundamental differences between laparoscopic and open procedures and the respective patient populations are unlikely to have changed significantly. In addition, the sensitivity and predictive value of claims codes for identification of SSI surveillance have been nationally validated for use as an effective ranking approach which allows for large populations to be evaluated and compared.

In conclusion, current CMS SSI hospital rankings fail to adequately account for key patient and surgical differences that would greatly enhance fair interhospital comparisons. These rankings currently impact financial disincentives applied to hospitals. CMS SSI adjustment models and hospital rankings should be generated separately for open and laparoscopic procedures and should additionally account for comorbidity scores and concomitant intraabdominal procedures that are readily available from routinely collected administrative data.