OP50 IQWiG And GRADE – An Exemplary Comparison Of Methods

Abstract

Introduction

Efforts to harmonize health technology assessment (HTA) processes and methods across Europe are currently intensified. In this context, the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach has been proposed as a “common ground” in joint HTAs. However, GRADE has been primarily developed to support authors of clinical guidelines. Therefore, it is unclear whether HTA reports based on GRADE are compatible with the methods currently applied by European HTA organizations.

Methods

We contrasted IQWiG's methods paper and publications by the GRADE Working Group with regard to the following domains: 1) risk of bias (RoB) assessment 2) prerequisites for “greater benefit” (assuming that IQWiG's “greater benefit” corresponds to a GRADE assessment of at least low certainty and a small important effect) and 3) consideration of non-randomized studies (NRS). We present illustrative differences and highlight similarities.

Results

Overall, RoB assessments are very similar under both approaches. However, we identified several important differences. In case of very severe publication bias, IQWiG methods preclude drawing a conclusion, whereas GRADE requires only downgrading the certainty of evidence while still allowing for a conclusion on effect sizes. Secondly, IQWiG generally requires a statistically significant effect for a “greater benefit”, while GRADE does not (statistically non-significant effects would only necessitate downgrading the certainty of results for imprecision). Another difference is that in general, NRS are not included in IQWiG assessments when randomized studies (RS) are available and thus possible. In contrast, preliminary GRADE guidance recommends considering NRS in addition to RS when the RS evidence is of low or very low certainty.

Conclusions

While GRADE and IQWiG's method share some similarities, our exemplary analysis shows that there are some notable differences. Therefore, GRADE should not be used “out of the box” for European HTAs. To foster further discussion, more research (including a comprehensive comparison of methods and an analysis of resources for adaptation) is needed.