LO82: Exploring eye-tracking technology to assess competency in point-of-care ultrasound

Abstract

Introduction: Assessment of point-of-care ultrasound (POCUS) competency has been reliant on practical, visual and written examinations performed 1-on-1 with an examiner. These tools attempt to assess competency through subjective ratings, checklists and multiple-choice questions that are labour intensive using surrogate measures. Eye-tracking has been used on a limited basis in various fields of medicine for training and assessment. This technology explores visual processing and holds great promise as a tool to monitor training progress towards the development of expertise. We hypothesize that eye-tracking may differentiate novices and experts as they progress to become competent in interpretation of POCUS images and provide an objective measure in assessment of competency. Methods: Medical students, residents and attending physicians working in an academic emergency department were recruited. Participants viewed a series of 16 ultrasound video clips in a POCUS protocol for Focused Assessment using Sonography in Trauma (FAST). The gaze pattern of the participants was recorded using a commercially available eye-tracking device. The primary outcome was the gaze parameters including total gaze time in the area of interest (AOI), average time to fixation on the AOI, number of fixations in the AOI and average duration of first fixation on the AOI. Secondary outcome was the accuracy on the interpretation of the FAST scan. Results: Four novices and eight experts completed this study. The total gaze time in the AOI (mean +/- SD) was 76.72 +/- 18.84s among experts vs 53.64 +/- 10.33s among novices (p = 0.048), average time to fixation on the AOI was 0.561 +/- 0.319s vs 1.048 +/- 0.280s (p = 0.027), number of fixations in the AOI was 158.9 +/- 29.0 vs 121.8 +/- 17.5 (p = 0.042) and average duration of first fixation was 0.444 +/- 0.119s vs 0.390 +/- 0.024s (p = 0.402). The accuracy of the answers was 79.7 +/- 14.1% vs 45.3 +/- 21.9% (p = 0.007). Conclusion: In this pilot study, eye tracking shows potential to differentiate between POCUS experts and novices by their gaze patterns. Gaze patterns captured by eye tracking may not necessarily translate to cognitive processing. However, it allows educators to visualise the thought processes of the learner by their gaze patterns and provide insight on how to guide them towards competency. Future studies are needed to further validate the metrics for competency in POCUS applications.