OP103 What To Include In A Health Technology Assessment Of Artificial Intelligence-Based Technologies: Results Of A Delphi Expert Survey

Abstract

Introduction

Clinicians are increasingly relying on artificial intelligence (AI) generated technologies for support in diagnosis, therapeutic decision-making, and prediction. Despite the increased focus on AI in health, an agreed HTA model for AI technologies, including consensus on new domains and topics to be assessed, is lacking.

Methods

A Delphi survey was sent to a multidisciplinary expert panel asking about the importance of including the nine domains and associated topics presented in the EUnetHTA Core Model, as well as 20 additional topics identified through literature reviews, when assessing AI-supported health technologies. The Delphi survey was repeated twice among the same panelists and a nine-point Likert scale was used to identify the perceived relevance of each domain and topic.

Results

The survey was sent to 87 various experts, with a total 47 of experts completing both Delphi rounds. The majority of panelists was knowledgeable of HTA (80%), familiar with the EUnetHTA Core Model (61%), and had adequate or high-level knowledge of AI (65%). The EUnetHTA domains most often indicated as “critical to include” were clinical effectiveness (82%), ethical aspects (81%), and cost effectiveness (77%), whereas organizational (59%) and social aspects (63%) were less often perceived as critical to assess. For the additional 20 topics identified through literature reviews, bias in data, accuracy in the AI model, appropriateness, and trustworthiness emerged as some of the new topics deemed critical to include in HTAs (all above 85%), whereas there was a lack of agreement on the relevance of including environmental (51%) and social sustainability (55%).

Conclusions

The study investigated in detail which issues should be included in an AI HTA core model. Current models need some adjustment and revision. At the same time, it is essential to open the discussion on including new domains and topics.