OP46 The Decision Uncertainty Toolkit: Risk Measures And Visual Outputs To Support Health Technology Decision-making During Public Health Crises

Abstract

Introduction

During public health crises such as the COVID-19 pandemic, decision-makers have relied on infectious disease models to predict and estimate the impact of various health technologies. The difficulties associated with capturing and representing uncertainty using infectious disease models leads to a high risk of making decisions that are misaligned to policy objectives. Even when uncertainty is adequately captured in the analysis, the tools for communicating the risks and harms of making wrong decisions have proved inadequate, which can lead to the suboptimal adoption of critical health technologies including vaccines and antivirals. We aim to adapt and extend health economic methods for the characterization, estimation, and communication of uncertainty to infectious disease modeling.

Methods

Economic and infectious disease models share many features, including the comparison of policy alternatives on outcomes important to decision-makers (such as hospital census, total infections), but each takes a different approach to analysis of uncertainty. We extend best practices from health economics to infectious disease modeling and develop a suite of tools and visualization techniques which represent parameter uncertainty and the risk these unknowns present to decision-makers.

Results

In consultation with decision-makers and infectious disease modeling experts we developed the ‘Decision Uncertainty Toolkit’ of model outputs and visuals. Visual tools for uncertainty are developed to: (i) accurately capture uncertainty in key infectious disease model outputs, and (ii) support intuitive and direct interpretation by infectious disease modelers and decision-makers. We also developed quantitative measures for the downside risk of policy alternatives, specified to capture both the probability and magnitude of losses relative to policy targets for a range of infectious disease model outputs. Together, these outputs can support decision-making by quantifying outcome uncertainty and the risks associated with policy alternatives.

Conclusions

We developed the toolkit visuals and risk measures alongside infectious disease modelers and decision makers. The toolkit is designed to improve decision-maker understanding of decision risk in order to improve outcomes during future public health crises.