To the editor: We read with interest the recent article by McGraw et al.Reference McGraw, Chaplin and McKaigney 1 in which they developed a simulation-based curriculum for residents to learn ultrasound-guided central venous catheter (CVC) insertion. They assessed the residents’ progress during training using a Global Rating Scale (GRS) and hand motion analysis.
Hand motion analysis through specific devices has been successfully used for years in the surgical field.Reference Datta, Mackay, Mandalia and Darzi 2 - Reference Mason, Ansell, Warren and Torkington 5 More recently, they have been used in anesthesia as assessment tools for procedural skills, as well.
Two different devices using electromagnetic fields have been described in the literature. The Imperial College Surgical Assessment Device (ICSAD)Reference Hayter, Friedman and Bould 6 is a device that tracks operator’s hand motion. It uses an electromagnetic tracking system (Isotrak Il; Polhemus Inc., Colchester, VT, USA) consisting of an electromagnetic field generator and sensors placed on the back of the operator’s hands. Three dexterity scores can be measured: total distance travelled by each hand, number of movements, and total time.Reference Reznick and MacRae 7 The ICSAD has demonstrated construct validity in many surgical procedures, including open, laparoscopic, and microsurgery.Reference Aggarwal, Dosis, Bello and Darzi 3 Additionally, in the anesthesia field, its construct and concurrent validity has been established in labor epidural placement,Reference Hayter, Friedman and Bould 6 ultrasound-guided supraclavicular block,Reference Chin, Tse and Chan 8 and jugular CVC placement.Reference Varas, Achurra and Leon 9
The hand motion analysis (HMA) hardware consisted of a driveBAY electromagnetic field generator and control box (Ascension, VT, USA), one reference sensor, and two hand sensors (Model 800, 7.9 mm, 6-DOF). Three-dimensional position data from the electromagnetic sensors are registered using an open-source software. Metrics used to evaluate motion efficiency are the same: total time of procedure, total path length (distance travelled), and number of translational motions.
This device is the one used by the authors in the present study, and it has been previously validated by Clinkard for ultrasound-guided jugular CVC placementReference Clinkard, Holden and Ungi 10 but not by Chin who used ICSAD.Reference Chin, Tse and Chan 8
Both systems collect the x, y, z Cartesian coordinate information from each sensor at a determined resolution and frequency. Most reports of ICSAD use an accuracy of 1 mm at 20 Hz.Reference Hayter, Friedman and Bould 6 , Reference Chin, Tse and Chan 8 On the other hand, driveBAY device reports an accuracy of 1.4 mm at 50 Hz.Reference Clinkard, Holden and Ungi 10
Additionally, the number of hand movements is determined based on a calibration process of translational and rotational velocity thresholds. Therefore, the number of movements registered is highly dependent upon the thresholds that the researchers have pre-defined. In the present study, McGraw registered the total number of hand motions when translational or rotational velocity exceeded 50 mm/second and 50 degrees/second, respectively.Reference McGraw, Chaplin and McKaigney 1 Chin used a velocity tolerance threshold of 20 mm/secondReference Chin, Tse and Chan 8 in supraclavicular blocks, and Hayter used 7.5 mm/secondReference Hayter, Friedman and Bould 6 in lumbar epidurals.
Clearly, evidence supports that tracking motion devices are valid assessment tools for procedural skills. Nevertheless, given those technical calibration processes, careful interpretation should be taken in consideration while extrapolating these types of data. Both accuracy and movement thresholds should be the same if we want to compare numbers, such as number of movements or meters travelled (total path length). These considerations are of paramount importance if these devices are becoming part of routine assessment tools in residency programs.
The three parameters delivered by these devices have proved validity (i.e., time, total path length, and number of hand movements).Reference Mason, Ansell, Warren and Torkington 5 Specifically for CVC placement, ICSAD was validated because total path length discriminates between expert and novices and correlates with a previous, validated GRS.Reference Varas, Achurra and Leon 9 The number of movements was not reported in this study. On the other hand, the driveBAY device was validated because motion parameters discriminate between expert and novice and correlate to a previously published modified GRS.Reference Clinkard, Holden and Ungi 10 Total path length was not reported in this study. Although both devices have been validated for this procedure, the parameter used to evaluate validity was different.
Finally, the use of this motion device in the evaluation of motor skills allows obtaining quantitative data complementing previous validated visual scales. Having as many instruments as possible for evaluating motor skills could improve the learning process. In the future, if we want to set up metrics or cut-off scores to be achieved with motor skills training, a previous standardization of both parameters to be used and calibration thresholds should be established for each setting.
