379 Using the Delphi method to strategize about health AI

Abstract

Objectives/Goals: Our goal was to determine whether a consensus exists around 1) what the main barriers to innovation in Health artificial intelligence (AI) are 2) where there are gaps in education and training in Health AI and 3) where in their workflows organizations should implement AI to see the most immediate impact on productivity. Methods/Study Population: We employed a three-round Delphi method survey to stakeholders with health and/or engineering expertise. The first round was open-ended to generate responses to the three research questions. The second round asked participants to rank the responses and provide feedback as to their reasoning. The third round provided aggregated results and feedback and asked participants to re-rank the responses. Participants were attendees at a conference that brought people with health and/or engineering backgrounds together to discuss innovation in Health AI. 55 people in total participated across the three rounds. Results/Anticipated Results: Consensus emerged on all three questions: lack of trust was seen as the single greatest barrier to innovation, experience with implementation as the greatest gap in training, and automating health documentation as the point of most immediate impact. Consensus also emerged as to which of the 10–15 responses to each question were top priorities, which were somewhat significant, and which were not that important. Some of the rankings (such as implementation) seemed to reflect hot topics of discussion at the conference, but others (such as documentation) only emerged as significant in the surveys. Discussion/Significance of Impact: We successfully employed the Delphi method to discover what stakeholders think about three important questions in Health AI. Interestingly, although we polled experts from both health and engineering backgrounds, their answers converged on all three questions.