Is AGREE II a counsel of perfection? A letter commenting on Lytvyn et al¹

To the Editor—We read the systematic survey (review) of Clostridium difficile (CD) guidelines (August 2016) with interest. We suggest that Lytvyn et al are proposing a counsel of perfection, ignoring the realities of producing practical guidelines to address rising infection levels. In particular, we question their data extraction from the UK guidelines and their views that (1) a systematic review is a pre-requisite of guideline writing; (2) relatively weak evidence should not result in strong recommendations; (3) ecological studies are grade 5 evidence; and (4) probiotics have the highest level of evidence for any CD prevention intervention.

Guidelines are produced in response to emergence of new diseases, new evidence on management, or rising levels of existing disease. The latter circumstance, plus a high-profile political drive to reduce CD, prompted revision of the UK guidelines. As we explained “A formal systematic review with grading of the level of evidence (in) each study was not done … (we) did not consider that evidence had changed sufficiently to … warrant extra time and resources.” We would like to ask Lytvyn et al which studies, published up to the end of 2007, were excluded from our review that would have materially affected the final recommendations? We suggest that AGREE II amend their standard to allow the guideline authors to justify why systematic reviews were not done.

Lytvyn et al state that the UK guidelines did not update prevention-related information. However, our introduction explicitly stated, “This guidance updates and replaces the 1994 report … outlines newer evidence and approaches to good infection control and environmental hygiene … taking into account national clinical governance frameworks which did not exist in 1994.” The example of antibiotic stewardship illustrates the difference in prevention-related information between 1994 and 2008. The former simply recommend, “adoption of an antibiotic policy” and “use of narrow spectrum antibiotics whenever the causative pathogen is known.” The 2008 guidelines recommend “restrictive antibiotic guidelines using narrow-spectrum agents … for empirical and definitive treatment … .” They specify which antibiotics to avoid, and they recommend the formation of antimicrobial management teams including antimicrobial pharmacists and information technology specialists to facilitate feedback of antibiotic and CD data. In 1994, neither antimicrobial teams, antimicrobial pharmacists, information technologists, nor data feedback existed.

Lytvyn et al are concerned that guidelines fail to explain how low-quality evidence leads to strong recommendations. The UK guidelines were explicit that lower-quality evidence could result in strong recommendations if “supported by non-RCT studies and/or by clinical governance reports and/or the Code.” Maybe they overlooked this statement or did not appreciate that the Hygiene Code has legal status, with hospitals and individuals facing legal sanctions for noncompliance. Governance reports based on public inquiries or investigations by regulatory authorities cannot be ignored and must be considered alongside the scientific evidence. Do Lytvyn et al suggest that a lack of RCTs or systematic reviews of RCTs of the effects of restrictive antibiotic policies on CD levels means that antibiotic restriction should not be strongly recommended? How well do they think clinicians would follow a “moderate” recommendation”? The proof of the pudding is in the eating. Since the United Kingdom adopted these guidelines, fluoroquinolone, cephalosporin, and other broad-spectrum antibiotic prescriptions have declined markedly, with the levels of CDI falling by 70%–80%. No other country has achieved this result. ^{2 , 3}

Why do Lytvyn et al regard ecological studies as “grade 5 evidence”? Such studies recognize the environmental determinants of disease, are a well-accepted pragmatic design for evaluating public health interventions, use widely available data, and provide a wider range of exposures than a trial, thus increasing the generalizability of findings. Potential limitations are minimized if measurement, analysis, and interpretation are performed at group level, if data are available and reliable, and if inference from group to individual is avoided. ⁴ One such study, using 4 years of data from all English acute-care hospitals, showed a very strong independent association between soap use and CD rates. ⁵ Would Lytvyn et al really not consider such evidence that supports a strong recommendation for hand-hygiene with soap to prevent CD?

Lytvyn et al took all guidelines to task for not recommending probiotics, which they regard as “the prevention strategy with highest-level evidence.” All but 1 of these guidelines, however, was prepared or published before the Cochrane review of probiotics was published. The single guideline published afterward rightly regards the case for probiotics as unproven, due to few large robustly designed studies, pooling of data on different probiotics, and large amounts of missing CD data. The Cochrane review’s own reanalysis using complete data sets revealed no effect of probiotics.

We do not agree that clinical guidelines should address resource implications. Their development may require substantial extra work and research, delaying guidance needed to address rising infection levels. For example, calculating the additional isolation capacity required for CD, its cost, opportunity costs, and their potential offset by reduced CD may be less important than getting the guidelines published. Insisting that both systematic reviews and cost estimations are done, although ideal, may risk accusations of “fiddling while Rome burns.” As authors of guidelines ourselves, ^{6 , 7} we consider them to be guidance from which authors may depart when there are good grounds or circumstances to do so. ⁸ Clinical guideline writing is neither an isolated academic exercise nor so difficult that it requires a “methodologist” lead. It requires experienced clinicians and researchers to apply its principles wisely in the clinical and national contexts where the guideline will be applied. The United Kingdom was experiencing an epidemic of CDI. The publication of the 2008 guidelines was part of a concerted ultimately successful public health campaign to reduce the CDI epidemic.