Response to “UNCERTAINTY MANAGEMENT IN REGULATORY AND HEALTH TECHNOLOGY ASSESSMENT DECISION-MAKING ON DRUGS: GUIDANCE OF THE HTAi-DIA WORKING GROUP”

With great interest, we read the article entitled “Uncertainty Management in Regulatory and Health Technology Assessment Decision-Making on Drugs: Guidance of the HTAi-DIA Working Group” by Hogervorst et al. (1). We wish to commend HTAi, DIA, and the Working Group for selecting this important topic.

To our surprise, the guidance only references a small subset of the extensive work on the topic of uncertainty in and outside of health technology assessment (HTA). Not referenced were articles on considerations around uncertainty in health (2–7), classifications of uncertainty in HTA (8–11) and outside HTA (12–16), and methods for uncertainty assessment (17–22), among others. For a scientific article in a scientific journal, methods and results of the scoping review are not described in sufficient detail. It remains unclear if and how the state of the art on uncertainty in HTA was used to develop the guidance.

Specifically, the part on “building blocks comprising decision-making uncertainty” bears non-negligible similarity to published work that is identified in the authors’ scoping review but not cited – the TRUST tool 2020 (11). TRUST considers the same uncertainty factors as outlined in the present article, including origin (location in TRUST), type (source in TRUST), impact/risk (same in TRUST), and relevance/judgment (appraisal in TRUST). The types of actionable uncertainty considered are also very similar: inaccurate (separated into imprecision, bias, and indirectness in TRUST); unavailable (same in TRUST); and non-understandable (transparency in TRUST). In line with existing classifications of uncertainty (8;16), TRUST also considers uncertainty stemming from methodological issues. TRUST does not include uncertainty from conflicting information, as this was considered to be reflected through imprecision or bias (4). TRUST is readily available, validated, practical, and used in practice (e.g., in Dutch Healthcare Institute reports). It is unclear how the presented guidance improves upon this.

There is an opportunity to build upon the challenges other researchers in the area of uncertainty assessment in and outside of HTA have identified and the methods proposed to address these. The progress made on the following topics has not been sufficiently covered in the guidance, including but not limited to:

• • uncertainty identification, for example, using the TRUST tool (11) and other methods (6;18);

• • uncertainty analysis methods including Bayesian methods (23), value of information (24–26), structured expert elicitation (27;28), and incorporation of difficult to quantify uncertainty (29–33);

• • uncertainty communication (17;34–36);

• • link between uncertainty and evaluation of managed entry agreement (MEA) schemes (36–39);

• • uncertainty (in)tolerance in regulatory and HTA decision-making (3;40–42).

As a next step, the Working Group refers to the link of their proposed framework with mitigation strategies. Importantly, there are existing frameworks and tools covering this topic including frameworks for classifications of different MEA schemes (43;44), and approaches for assessing MEAs (36;39;45). We urge the Working Group to consider and transparently build upon these, where relevant.

To conclude, we agree with the HTAi-DIA Working Group that uncertainty is a fundamental component of decision-making. We argue that collaboration with experts in the abovementioned topics and thorough, transparent reviews of the literature to build upon the wealth of existing knowledge will make the resulting guidance stronger.