INTRODUCTION
Central venous catheter (CVC) insertion is a key procedural competency for residents in Emergency Medicine and Anesthesiology under the Medical Expert Role in the CanMEDS Physician Competency Framework.Reference Frank and Jabbour 1 In recent years there has been widespread adoption of real-time ultrasound guidance (USG) during CVC insertion with the goal of improving patient safety.Reference Troianos, Hartman and Glas 2 Insertion of a CVC under USG is made up of multiple component skills, including ultrasound (US) scanning, identification of relevant anatomic structures, USG needle tracking, and vascular cannulation.Reference Chapman, Johnson and Bodenham 3 Incorporating all of these skills requires training and practice in order to achieve technical proficiency.Reference Troianos, Hartman and Glas 2 , Reference Gale and Alan 4 Any competency-based approach to teaching this skill to residents would require sufficient opportunities for training, practice, and assessment, ideally in a setting that avoided unnecessary risk to patients while optimizing learning conditions for residents.
There is an emerging consensus that simulation-based education has a substantial role to play in teaching procedural skills in general, and CVC insertion in particular, to novices prior to attempts on real patients.Reference Barsuk, McGaghie and Cohen 5 - Reference McGaghie, Issenberg and Cohen 8 Simulation allows novices to engage in repetitive practice without any risk to patients, and supports development of carefully structured curricula designed around established principles of learning and improvement of performance.Reference Kneebone 9 With respect to skills such as USG CVC insertion there is, however, a paucity of evidence for the specific type and volume of simulator practice required to achieve technical proficiency.Reference Troianos, Hartman and Glas 2 , Reference Nguyen, Prat and Vincent 10 Residency programs would benefit from evidence of this nature to guide the design of competency-based curricula, as it could inform such things as the number of practice sessions to schedule and the most effective forms of practice.
With this in mind, a simulation-based curriculum was designed with careful consideration of the conditions that are associated with optimal learning and improvement of performance, specifically: motivation of the learner, consideration of learner’s pre-existing knowledge, immediate formative feedback, repetition of the same or similar tasks, and distribution of complex skills practice over several sessions.Reference Kneebone 9 , Reference Ericsson, Krampe and Tesch-Romer 11 The curriculum was designed around a model of deliberate practice of the more challenging aspects of the procedure, specifically: simultaneous US probe manipulation, needle manipulation, and needle tip visualization on the US screen. Deliberate practice can be defined as a highly structured activity that has as its main goal the improvement of performance.Reference Ericsson, Krampe and Tesch-Romer 11 It is worth emphasizing that while deliberate practice is essential in developing expertise in procedures, it is difficult in the clinical setting, where practice is limited by patient availability and opportunities for immediate repetition of difficult aspects of the procedure are rare.Reference Kneebone 9 , Reference Swing 12
In this study, we describe the development, implementation, and evaluation of a curriculum for teaching USG CVC insertion to residents from our Emergency Medicine and Anesthesiology programs beginning their second post-graduate year. Our primary objective was to design a curriculum that would enable novice learners to achieve technical proficiency in the skills of USG CVC insertion on a simulator in advance of real patient encounters. Our secondary objective was to monitor the progress of the learners with close observation and regular hand motion analysis (HMA) of USG femoral CVC insertion on a simulator, to study the volume and type of practice necessary for novices to achieve technical proficiency in these skills.
METHODS
Study participants
Five of the six post-graduate year two residents from the Department of Emergency Medicine, and all five post-graduate year two residents from the Department of Anesthesiology of Queen’s University, were recruited. Second year residents from these programs were chosen because they are expected, with increasing independence, to perform USG CVC insertion in the clinical setting at this stage of their training. We also recruited six experts for the study who all perform and teach USG CVC insertion on a regular basis as part of their practice. Four of the experts were recruited to establish performance benchmarks, including one Royal College–certified Emergency Physician, two Royal College–certified Critical Care Specialists and a Fellow in Critical Care Medicine. Two experts, both Royal College–certified Critical Care Specialists, were recruited to review blinded videos at the end of the study.
Study protocol
The participating residents completed four training sessions of two hours each, at two week intervals. Based on availability and scheduling, four residents were assigned to group one, and six residents to group two. The groups attended the lab on alternate weeks in the first two months of their second year of training. A strategy of close observation by the instructors and regular HMA of residents performing USG femoral CVC insertion was used to monitor progress during the study. On completion of training, a global rating scale (GRS) was used by two independent blinded experts to assess videotaped performances of each resident inserting a femoral CVC with USG. The expert reviewers assessed the videos alone and submitted their results independent of each other. To ensure blinding, no sound was recorded, the participants were gowned and gloved after removing personal items such as watches and rings, and the screen only displayed the operator’s hands and torso.
Curriculum
The study protocol and curriculum is outlined in Figure 1, and included advance preparation through online videos, independent reading modules and pre-tests prior to the first two sessions to ensure understanding of the requisite background knowledge. Basic US skills such as probe orientation, image interpretation, and depth adjustment were taught during session one. Drills were designed specifically to engage the residents in deliberate practice of simultaneous US probe manipulation, needle manipulation, and needle tip visualization on the US screen. As they mastered these fundamental skills, the remainder of the procedure was taught in a stepwise fashion. All practice and assessment was done using Blue PhantomTM (CAE Healthcare) femoral and internal jugular training phantoms.
Figure 1 Study protocol and curriculum.
Hand motion analysis
HMA is a tool that provides a valid, reliable, and objective assessment of technical performance in skill-based training programs by capturing the number of hand movements and total procedure time.Reference Datta, Mackay and Mandalia 13 , Reference Grober, Roberts and Shin 14 It has been shown to be a feasible and valid means of measuring technical proficiency in USG CVC insertion and USG peripheral nerve blockade.Reference Clinkard, Holden and Ungi 15 , Reference Chin, Tse and Chan 16 As novices become more proficient, they display greater economy of motion with their hands, and HMA is able to objectively quantify such improvement. As such, serial measures using HMA have potential for use in establishing the ideal or minimum number of procedural exposures required to achieve technical competence.Reference Grober, Roberts and Shin 14
The HMA hardware consisted of a DriveBay electromagnetic field generator and control box (Ascension, VT, USA), one reference sensor, and two hand sensors (Model 800, 7.9 mm, 6-DOF). Three-dimensional (3D) position data from the electromagnetic sensors was acquired using PLUS (www.PlusToolkit.org) open-source software. The Transform Recorder module of the Perk Tutor extension (www.perktutor.org) in 3D Slicer (www.slicer.org) was used to record the location of all sensors. The Perk Evaluator module was used to analyze the recorded data and compute the HMA metrics.Reference Ungi, Sargent and Moult 17 All of the above described software for data acquisition, analysis, and calculation is open-source and freely available to any centre wishing to replicate our motion analysis approach.
Global rating scale
A previously validated GRS for CVC insertion was modified to include only the items targeting specific components of technical ability for the US-dependent portion of the procedure.Reference Ma, Zalunardo and Pachev 18 The terminology for overall ability to perform procedure was unchanged and included: 1, not competent to perform independently; 2, borderline competence to perform independently; 3, competent to perform independently; and 4, 5, 6, above average competence to perform independently. The component of aseptic technique was changed from a scale of 1 to 6 to a dichotomous variable of yes or no, meaning any break in sterile technique would be a fail for that component.
Hand motion analysis benchmarks
We decided a priori that each of the experts had to receive an overall GRS rating of at least three on a blinded video assessment of their performance in order to include their HMA metrics in the calculation of benchmarks for use in the data analysis. This was to avoid an unexpected poor performance by an expert resulting in benchmarks that were too easy to achieve. In our final analysis after video review, one of the experts did not meet this standard and was excluded. Therefore, the mean values for experts reported in the results for procedure time, total left hand motion, and total right hand motion are from the three experts who achieved the GRS standard on review of their video performances.
Data collection
Demographic data, as well as previous experience with USG CVC insertion, were collected at the beginning of session one. At the beginning of each subsequent session residents were asked to report any exposure to the procedure between sessions.
HMA data collection started with the phantom draped, all of the equipment prepared, and the resident holding the US probe ready to visualize the vessels. All participants chose to hold the US probe with their left hand, and the needle with their right hand. Data collection stopped when the CVC was in the vein and the guidewire removed. The total number of hand motions for each hand was determined as the number of distinct time periods in which the hand’s translational or rotational velocity exceeded 50 mm/second or 50 degrees/second, respectively.
The sequence of steps was standardized for all participants to allow comparison of hand motion metrics. Baseline HMA data were collected for all residents at the start of session one, and for the experts when they were available to attend the lab. The sequence of steps was demonstrated on a femoral phantom once, following which participants practiced the sequence once, and this was followed by HMA of a second attempt. Subsequent measures of HMA were made at the end of session two, and at the beginning and end of sessions three and four.
At baseline the participants performed the sequence once and measurements were recorded for total procedure time, total left hand motion and total right hand motion. For each subsequent measure the participants performed the procedure twice and the HMA results were averaged. Individual means from each training session were then pooled to generate a group mean.
The GRS was used to score blinded videos of two performances per participant of USG CVC insertion on a femoral vein phantom for nine of the residents at the end of session four. One of the residents who performed consistently well from the beginning of training was scored using their video from the end of session three.
Throughout the study the principal investigator solicited feedback from the instructors and participants on any difficulties the residents were having, and for ways to modify the practice sessions to improve performance.
Data analysis
HMA data were collected at regular intervals to determine the immediate effect of training and determine if there was significant erosion of skill between sessions. At baseline, left and right hand motion count and procedure time of residents were compared to experts using a two-sample T-test for unequal variances. The Wilcoxon signed-rank test for paired observations was used to determine if changes in HMA metrics from one data collection to the next were statistically significant. In order to assess the trend of group performance, individual averages from each training session were pooled to generate a group average for participants for that session. At the end of training, each participant was videotaped twice and the two expert reviewers scored each video using the GRS. This generated four overall GRS scores that were averaged to determine the final score for each participant on a scale of 1 to 6. The reviewers were specifically instructed to use the GRS to determine if they would be comfortable allowing the participant to perform their next insertion on a simulator independently.
The study was approved by the Queen’s University Health Sciences and Affiliated Teaching Hospitals Research Ethics Board. Signed consent of voluntary enrollment was obtained from each participant. The study took place at the Queen’s University Clinical Simulation Centre.
RESULTS
A pre-teaching survey of the participants revealed a range of prior experiences with CVC insertion in the real and simulated settings (Table 1). Over the course of training, three residents reported one to two supervised USG internal jugular CVC insertions in the clinical setting, and one resident reported one supervised USG femoral CVC insertion. The remaining six residents reported no exposures to USG CVC insertion outside of training.
Table 1 Baseline experience with CVC insertion
HMA results
Table 2 shows median and interquartile ranges of assessment scores for trainees at each measurement session. At baseline, mean time required for trainees to complete a femoral CVC insertion (t=331 seconds; range 192-700), and mean number of left (L=75; range 34-194) and right hand motions (R=86; range 51-180) were significantly greater (p<0.01) than expert benchmarks (t=87 seconds, range 80-93; L=33, range 30-37; R=39, range 31-43). None of the residents met any of the expert benchmarks at baseline. The results for individual residents, as well as the group mean for total procedure time, left hand motion, and right hand motion, are illustrated in Figures 2, 3, and 4, and suggest steady improvement occurred towards the expert means over the course of training. HMA data at the end of sessions 2, 3, and 4 indicated significant interval improvement in performance on all three measures when compared to the preceding performance. By the end of the fourth session (S4E), six of 10 residents had faster procedure times than the mean expert benchmark, and nine of 10 residents had more efficient left and right hand motions than the mean expert benchmarks.
Figure 2 Total procedure time (seconds). B=baseline; S2E=end of session 2; S3B=beginning of session 3; S3E=end of session 3; S4B=beginning of session 4; S4E=end of session 4.
Figure 3 Total left hand motion count (rotational and translational movements) . B=baseline; S2E=end of session 2; S3B=beginning of session 3; S3E=end of session 3; S4B=beginning of session 4; S4E=end of session 4.
Figure 4 Total right hand motion count (rotational and translational movements) . B=baseline; S2E=end of session 2; S3B=beginning of session 3; S3E=end of session 3; S4B=beginning of session 4; S4E=end of session 4.
Table 2 Median and interquartile ranges of assessment scores for trainees at each measurement sessionFootnote *
* p values for the Wilcoxon signed-rank test for comparison of whether the change from the previous measurement session was significantly different for each of time, left hand motions, and right hand motions
** Differences were not statistically significant (α=0.05).
GRS results
In the final videotaped assessments, all of the residents were successful in inserting a femoral CVC under USG, and there were no arterial punctures or arterial catheterizations. Nine of the residents received a mean overall score of 3 out of 6 (competent to perform independently) or better. One of the residents received a mean score of 2.25 (borderline competence to perform independently). This resident also had the weakest metrics on HMA at baseline and consistently throughout training.
DISCUSSION
Our primary objective was to design a curriculum that would enable novice resident learners to achieve technical proficiency in the skills of USG CVC insertion on a simulator in preparation for real patient encounters. The curriculum was developed with careful consideration of the conditions associated with optimal learning and improvement of performance.Reference Kneebone 9 , Reference Ericsson, Krampe and Tesch-Romer 11
We believe the participants in this study were highly motivated by the fact that they were practicing clearly defined tasks alongside their peers, were regularly assessed with HMA, and believed the training would help prepare them for the responsibility of inserting CVCs in upcoming clinical rotations. We used a part practice approach, where the skills of USG CVC insertion were broken down into readily achievable component skills that were practiced to the point of mastery before adding additional skills. An instructor to student ratio of 1:2 allowed close monitoring of performance and immediate feedback as necessary. Our curriculum was designed with an emphasis on deliberate practice, with the greatest volume of practice assigned to the more difficult aspects of the procedure.
It is known that complex skills are easier to learn if broken down into component skills, and if practice is distributed over several shorter sessions rather than one long continuous session.Reference Kneebone 9 , Reference Fairbrother 19 The curriculum was designed to gradually increase the complexity of the task, and to distribute practice over four sessions, each one separated by two weeks.
Our secondary objective was to monitor the progress of the learners with close observation and regular HMA of USG femoral CVC insertion on a simulator to study the volume and type of practice that may be necessary for novices to achieve technical proficiency in these skills. By the end of the third participants would have engaged in deliberate practice of USG of the introducer needle tip to the target vessel, in isolation or as part of a larger sequence, approximately 64 times. The HMA data at the end of the third session indicated the group mean appeared to approach that of the experts on time and left hand motion and the group mean appeared to surpass the expert mean on right hand motion (Figs. 2, 3, 4). At the beginning of the fourth session the HMA data were similar to the end of the third session, and we found no evidence of significant erosion of skill. These findings suggest skill retention, and are important, as delays almost inevitably exist between the end of a training session and an opportunity to perform a real clinical CVC insertion.
By the end of the fourth session, the participants would have engaged in deliberate practice of USG of the introducer needle tip to the target vessel approximately 86 times, and we again found evidence of significant improvement by individuals, with resident means appearing to surpass expert means, and the majority of individual residents exhibiting metrics better than the expert means. The apparent gains of the fourth session were admittedly small, and it is reasonable to question whether such a modest gain is worth the effort and expense. An argument in favor of the fourth session is that with additional practice, component skills become increasingly automatic, thus decreasing the cognitive load of performing the skill. As they transition to the clinical setting, this may make it easier for residents to simultaneously deal with other challenges, such as an unstable patient or distorted anatomy.Reference Reznick and MacRae 20 Furthermore, overlearning, or learning beyond achievement of initial proficiency, is considered the single most important determinant of skill retention.Reference Arthur, Bennet and Stanush 21
The benefits of USG in any percutaneous procedure relate to real-time visualization of target anatomy and precise needle tip visualization to avoid injury to adjacent structures.Reference Chapman, Johnson and Bodenham 3 In a clinical study of USG peripheral nerve blockade, Sites and colleagues found long learning curves and a consistent pattern of errors, including unintentional probe movement, awkward needle holding, and most importantly, poor needle tip visualization.Reference Sites, Spence and Gallagher 22 We observed a similar pattern of errors early in training, and believe this was also reflected in the baseline and early HMA metrics. The deliberate practice stations were therefore modified to include drills that required precise localization of the tip of the needle (Table 3).
Table 3 Drills to improve introducer needle tip visualization and localization
These types of errors amongst novices should not be surprising, given the challenges associated with a dynamic psychomotor skill that involves simultaneous US probe manipulation, needle manipulation, and needle tip visualization on an US screen. Ideally, as individual residents are learning the procedure and encountering difficulties, they will receive specific feedback and the opportunity for immediate repetition of that part of the procedure until proficient. This highlights a central argument for moving this early learning of the fundamental skills of USG CVC insertion away from the bedside and into the simulation lab. In the clinical setting there are really no opportunities for sustained deliberate practice of this skill, or for pausing and repeating specific difficult aspects of the procedure. Furthermore, we believe it is inappropriate to expose patients to a predictable pattern of errors by novices if this learning can be better accomplished in a simulation lab. In fact, for any procedure like CVC insertion where there is potential for significant patient harm, it is arguable that there is an ethical imperative to use simulation where appropriate, and as much as possible, to best prepare residents for real patient encounters.Reference Ziv, Wolpe and Small 23
Curricular planning in competency-based medical education begins with identifying the abilities needed of a graduate, and is followed by development of educational programs that provide sufficient experience, practice opportunities, and assessment, to ensure achievement of desired outcomes.Reference Frank, Snell and Cate 24 At our centre, residents entering their second year of training in Emergency Medicine and Anesthesiology are expected to have the ability, with increasing independence, to perform USG CVC insertions during rotations in the Intensive Care Unit, the Operating Room and the Emergency Department. Traditionally, these residents would have performed their first USG CVC insertion on a patient, following, at best, a single simulation session. Along the continuum of achieving competency in this skill, we believe that residents at this level will benefit from achieving a high level of technical proficiency in a simulation lab prior to attempts on real patients, and others have reached similar conclusions.Reference Troianos, Hartman and Glas 2 , Reference McGaghie, Issenberg and Cohen 8 Our results suggest that residents can achieve a level of technical proficiency on simulators that approaches or exceeds benchmarks set by experts, if given the opportunity to engage in three to four training sessions that incorporate deliberate practice of fundamental USG skills with close monitoring and immediate feedback.
LIMITATIONS
This study has a small sample size, and included only residents from specialties that have a strong procedural bias. It is therefore not clear that these results would be generalizable across a larger group of residents, or residents from other specialties. Future work should include a larger sample size with broader representation from other specialties that require competency in USG CVC insertion as part of training.
Procedural competency integrates multiple components, such as knowledge, skill, and attitude. In this study we focused on technical proficiency alone, which is essential, but only part of overall competency. In addition, this study was conducted exclusively in a simulated setting, and thus does not necessarily predict performance in the real clinical environment. In fact, there are a number of differences between a simulator and real patients, such as tissue resistance, vessel compressibility, and variable tissue echogenicity that result in challenges in making the transition to real patients. As such, we expect that residents trained in our program would still initially require careful supervision as they moved on to USG CVC insertion in the clinical setting. In future work, we plan to study how well residents trained using our curriculum make the transition from the simulator to real patients, and how long it takes them to achieve a level of competence sufficient for independent practice. We acknowledge that this curriculum requires a large commitment of resources, including faculty and resident time, technical support, and supplies. However, if it can be shown that this initial investment of resources can shorten the time required to achieve competency in the clinical setting, while avoiding some of the mistakes commonly made by novices, we believe this will be worth the effort and expense. Additionally, based on our experience, we have identified opportunities for efficiencies, such as reducing the number of instructors at later sessions when residents are able to practice more independently. Finally, there may also be a role for training paramedical personnel to assist with teaching, which would further reduce the need for one-on-one faculty time.
CONCLUSION
We successfully developed a simulation-based curriculum for residents learning the skills of USG CVC insertion. Our results suggest that engaging residents in three to four distributed sessions of deliberate practice of the fundamental skills of USG CVC insertion leads to steady and marked improvement in technical proficiency, with individuals approaching or exceeding expert level benchmarks.
Competing Interests: The study was supported by an Innovation Fund Grant from the Southeastern Ontario Academic Medical Organization.
