P.140 Development and predictive validation of an intelligent surgical bimanual skills continuous assessment system

Abstract

Background: Surgeons’ bimanual dexterity may correlate with the surgical outcome. Continuous assessment of psychomotor performance enables action-oriented feedback and error avoidance guidance. We outline an artificial intelligence (AI) application to continuously assess surgical bimanual skills and its predictive validation on surgical trainee performance throughout a neurosurgery residency program. Methods: Participants (n=50, 14 experts/neurosurgeons, 14 senior residents, 10 junior residents, 12 novices/medical students) performed two simulated subpial tumour resections a total of 300 times. A deep neural network was developed using expert/neurosurgeon and novice/medical student data to score bimanual performance at 0.2-second intervals between a score of 1.00 and -1.00. An average score was calculated for each task. Results: The average performance score differentiated among four expertise levels, p<.001. Neurosurgeons scored significantly higher than senior residents (p=.045) and junior residents scored significantly higher than medical students (p=.04). The intelligent system also differentiated between senior and junior trainee levels (p=.004). The performance score linearly correlated with resident year of neurosurgical training (adjusted R2=27.7%). Conclusions: The AI-powered intelligent system outlined is the first expert surgeon-data-based technical skills continuous assessment system, with predictive validity throughout a neurosurgical residency program. Intelligent systems may aid in the competency-based approach in surgery by accurately assessing trainee skills.