Paediatric resident identification of cardiac emergencies

Abstract

Objectives:

Critical CHD is associated with morbidity and mortality, worsened by delayed diagnosis. Paediatric residents are front-line clinicians, yet identification of congenital CHD remains challenging. Current exposure to cardiology is limited in paediatric resident education. We evaluated the impact of rapid cycle deliberate practice simulation on paediatric residents’ skills, knowledge, and perceived competence to recognise and manage infants with congenital CHD.

Methods:

We conducted a 6-month pilot study. Interns rotating in paediatric cardiology completed a case scenario assessment during weeks 1 and 4 and participated in paired simulations (traditional debrief and rapid cycle deliberate practice) in weeks 2–4. We assessed interns’ skills during the simulation using a checklist of “cannot miss” tasks. In week 4, they completed a retrospective pre-post knowledge-based survey. We analysed the data using summary statistics and mixed effect linear regression.

Results:

A total of 26 interns participated. There was a significant increase in case scenario assessment scores between weeks 1 and 4 (4, interquartile range 3–6 versus 8, interquartile range 6–10; p-value < 0.0001). The percentage of “cannot miss” tasks on the simulation checklist increased from weeks 2 to 3 (73% versus 83%, p-value 0.0263) and from weeks 2–4 (73% versus 92%, p-value 0.0025). The retrospective pre-post survey scores also increased (1.67, interquartile range 1.33–2.17 versus 3.83, interquartile range 3.17–4; p-value < 0.0001).

Conclusion:

Rapid cycle deliberate practice simulations resulted in improved recognition and initiation of treatment of simulated infants with congenital CHD among paediatric interns. Future studies will include full implementation of the curriculum and knowledge retention work.

Delayed or missed diagnosis of critical congenital CHD can result in significant morbidity and mortality. ^1–3 Early identification of congenital CHD remains a challenge despite advances in fetal imaging and widely implemented pulse oximetry screening protocols. While prenatal detection through imaging is ideal, this is not always possible for a variety of reasons including inequitable access to care. While congenital CHD pulse oximetry screening can be used to detect hypoxia or saturation differentials in an infant with underling pathology – this does not negate the need for clinicians to consider the diagnosis of congenital CHD in an infant who may not yet be in extremis. Thus, the development of physician clinical assessment skills and inclusion of congenital CHD on the differential as a cause of clinical change remains critical.

Current exposure to training on congenital CHD is limited. This is especially true for those clinicians transitioning from residency training to work in newborn nurseries, emergency departments, and hospitals – settings in which they will likely be the frontline provider that could make the difference in early recognition of undiagnosed congenital CHD. ⁵ While previous education interventions have focused on enhancing specific resident skills in reading electrocardiograms and learning cardiac auscultation, ^6–11 these skills do not address the fundamental gap in considering congenital CHD as an aetiology for decompensation followed by enacting initial diagnostic and management steps. In addition to the lack of directed training while on cardiology rotations, paediatric residents have very few opportunities to learn these skills on other rotations.

Simulation is an effective strategy for teaching clinicians, including paediatric residents, resuscitation in a safe, and standardised environment. ^12–14 High-fidelity simulation has demonstrated feasibility and effectiveness in improving paediatric resident knowledge, comfort, and confidence when caring for patients with cardiac disease. ^15–17 Prior studies have mainly used traditional debrief simulation to teach residents recognition and management strategies. ^15,16 Rapid cycle deliberate practice is a simulation method in which feedback is given in real time to allow for participants to stop and immediately correct their mistakes. Rapid cycle deliberate practice has been used to reinforce correct behaviour in paediatric emergencies and resuscitation, though has not been applied to training on congenital CHD. ^18–20 In this study, we aimed to assess the impact of a rapid cycle deliberate practice simulation curriculum at enhancing paediatric resident recognition of congenital CHD and competency at the initiation of early diagnostic and therapeutic interventions.

Methods

We performed a single-centre prospective pilot study at a large, urban, quaternary care paediatric hospital. Approximately, 52 categorical and combined paediatric interns complete a 4-week, mandatory paediatric cardiology rotation, and split their time between the outpatient and inpatient settings. The outpatient time consists of paediatric cardiology consults and clinic. During the inpatient portion, the interns are front-line providers alongside advanced practice providers on a 29-bed acute care cardiology unit, supervised by a cardiology fellow and attending.

Prior to our work, the curriculum for first year paediatric residents on the cardiology rotation consisted of a series of lectures that rotated monthly with no specific focus on the recognition and early management of congenital CHD. Didactics did cover the main aetiologies of congenital CHD, including one lecture on single ventricle physiology and one on acyanotic heart lesions that included an overview of coarctation of the aorta.

Curriculum development

Using Kern’s six-step approach to curriculum development, we conducted an informal targeted needs assessment which revealed an opportunity to use simulation-based education to fill an experiential learning gap. ²¹ We anchored our curriculum within the conceptual framework of Ericsson’s deliberate practice – the theory that learning occurs through an intentional experience partnered with expert-level feedback on performance, followed by the opportunity to apply what was learned. ^22,23 Rapid cycle deliberate practice is a simulation-based modality that aligns with this conceptual framework – consisting of learners performing a task in a simulated environment, being stopped to correct mistakes, followed by immediately allowing for practice of those learned skills or behaviours. ²⁴

We designed the curriculum to span 4 weeks, aligned with the paediatric cardiology rotation (Table 1). The paediatric interns were all participants in the curriculum but had the option of opting out by not answering the assessments or surveys. During weeks 2, 3, and 4 interns participated in one-hour sessions during which 2 simulations occurred: an assessment simulation followed by a training simulation. During the assessment simulation, interns were given a case prompt and then were allowed to navigate the simulation uninterrupted. Following the conclusion of the simulation, a structured debrief occurred through which the interns were provided feedback and core areas for improvement were identified. During the training simulation, rapid cycle deliberate practice was used to identify and mitigate gaps in knowledge, skills, or behaviour in real time. The week two assessment simulation was a case of neonatal sepsis, while the training simulation was a case of critical coarctation. This was done to demonstrate the similarity in presentation between sepsis and congenital CHD. The week three simulations were both cases of critical pulmonary stenosis while the week four simulations were both cases of hypoplastic left heart syndrome. During the assessment simulation, a checklist of observed behaviours by the intern team was completed in real time by one of the trained facilitators. At the end of week 4, interns repeated the online case-based assessment as well as a retrospective pre-post survey assessing perceived change in knowledge.

Table 1. PRICE curriculum assessments.

Survey development and outcome measures

We developed all survey instruments de novo by a multidisciplinary team of paediatric cardiologists, medical educators, and simulation experts. Our primary outcome was measured skills observed via the simulation checklists. Our secondary outcomes were perceived skill ascertainment and measured knowledge via the retrospective pre-post survey and online case scenario assessments, respectively.

Online case scenario assessment

During week 1, the interns took an online case-based assessment that consisted clinical scenarios followed by free response and multiple-choice questions (MCQs) (Supplementary figure 1). We scored the free-response questions using a checklist of expected statements adapted from the simulation checklists (Supplementary figure 3). The same case-based assessment was repeated during week 4 following completion of all simulation sessions. Participants completed all assessments independently through an online REDCap survey which ensured de-identified data, but linked data across assessments and assured secure storage. ²⁵ Two members of the study team reviewed a total of 10 cases (5 pairs of week 1 and week 4 assessments) using the developed rubric and were found to have an interrater reliability across all items of 0.8. All remaining assessments were reviewed by a single study team member.

Simulation checklist

We developed a checklist of expected observable behaviours for completion by a facilitator during the assessment simulations in weeks 2, 3, and 4. We placed emphasis on actions that were deemed “cannot miss” by our multidisciplinary investigator team (Supplementary figure 3). Items ranged from recognition of distress, to asking for help, to obtaining 4-extremity blood pressures, and checking pre and post-ductal pulse oximetry. All checklists were completed directly within a secure REDCap database.

Retrospective pre/post survey

We developed a retrospective pre-post survey to characterise intern perceptions of the change in their ability to recognise and treat congenital CHD post-curriculum compared to pre-curriculum. We used this question design to minimise the impact of changes in intern self-assessment standards over time as they experience a shift in their frame of reference following an intervention. ²⁶ Paediatric interns completed the survey in week 4 following completion of all simulations, with responses stored within a secure REDCap database.

Data analysis

We used summary statistics including medians with interquartile range and frequencies with percentages to describe participant demographic characteristics and pre- and post-intervention case scenario assessment scores. We analysed changes of case scenario assessment scores from week 1 to week 4 using a paired samples t-test. Furthermore, we analysed differences in case scenario assessment scores between participant groups and simulation checklist scores at weeks 2, 3, and 4 using mixed effect linear regression with random intercept to account for within-subject correlation. P-values < 0.05 were considered statistically significant. We utilised SAS 9.4 for all statistical analysis.

This study was approved by the CCHMC Institutional Review Board.

Results

A total of 26 interns (100% of those eligible) participated in the curriculum between August and January 2022. The majority of participants were women (75%), with 22% having previous exposure to a paediatric cardiology rotation as medical students and 9% expressing interest in specialising in paediatric cardiology (Table 2).

Table 2. Participant demographics and background.

Online case scenario assessment

Twenty-one participants (84%) completed both the pre and post case scenario assessments; the remaining five participants were excluded due to incomplete assessments. Overall, interns’ individual median scores for the online case scenario assessment increased from week 1 to week 4 (4/18, interquartile range 3–6 versus 8/18, interquartile range 6–10, p-value < 0.0001) (Fig. 1a). There was no difference in group performance in the case scenario assessment based on the time at which they completed their paediatric cardiology rotation relative to other intern year clinical rotations and education (Fig. 1a).

Figure 1. a, b: Panel a shows box and whisker plot depicting aggregate median online case scenario assessment scores from week 1 to week 4 and then group scores for week 1 and week 4. Panel B shows retrospective pre-post aggregate survey scores between week 1 and.

Simulation checklist assessment

A total of 6 groups completed all three simulation sessions. Group A was not included due to not participating in all three simulation sessions. There was a statistical increase in the percentage of observed completed tasks from weeks 2–3 (73% versus 83%, p-value 0.0263), as well as from weeks 2 to 4 (73% versus 92%, p-value of 0.0025). There was no statistical difference between weeks 3 and 4 performance (p-value 0.193) (Fig. 2a). There was no statistical difference observed between groups (Fig. 2b).

Figure 2. A shows a box and whisker plots for simulation checklist scores by percentage complete from week 2, 3 and 4, difference between week 2 and 3 p-value 0.0115, difference between week 2 and 4 p-value 0.0006. B shows group scores for groups B through G for weeks 2, 3, and 4. This shows each intern groups’ progress over the course of the curriculum highlighting improvement in all groups.

Retrospective pre-post survey

Retrospective pre-post survey scores increased (1.67 out of 5, interquartile range 1.33–2.17 versus 3.83 out of 5, interquartile range 3.17–4; p-value < 0.0001) (Fig. 1b). No differences were observed across participants in different groups (Fig. 1b).

Discussion

We conducted an education pilot study that assessed the impact of a rapid cycle deliberate practice-based simulation curriculum on paediatric interns’ knowledge, observed behaviours, and self-assessed knowledge related to recognition and treatment of paediatric clinical decompensation from undiagnosed congenital CHD. Our intervention resulted in a statistically significant increase in case scenario scores, number of behaviours performed during assessment simulations, and self-assessed knowledge. Our study adds to the literature by demonstrating the utility and impact of simulation on paediatric residents’ clinical knowledge and skill sets when evaluating a child with CHD, while specifically employing rapid cycle deliberate practice as the main intervention. ^15–17,22

Previous studies have shown that rapid cycle deliberate practice is more effective than traditional debrief simulation in improving paediatric resuscitation skills as well as paediatric critical care skills. ^18,19,24 Rapid cycle deliberate practice is intrinsically linked to the conceptual framework of deliberate practice and our study showed that with consistent practice, paediatric interns can improve their identification and treatment of a child with congenital CHD. By conducting the study in our unit, interns were able to attend without significant interruption to their clinical responsibilities and practice in a familiar setting. Using rapid cycle deliberate practice allowed for immediate correction such that interns could focus on the simulation, how to intervene, and what actions corresponded to a clinical improvement. Over the course of the month, the paediatric interns were able to apply skills they learned in each session to subsequent simulations demonstrating statistical improvement by week 3 of the rotation, without additional improvement between weeks 3 and 4. Our pilot study may have been underpowered to observe a smaller difference in the final simulation; alternatively, 3 simulations may be sufficient to achieve our objectives. Eliminating one simulation session would reduce the needed resources and infrastructure, which may promote sustainability. Our findings are consistent with a prior paediatric resuscitation study, which found a significant improvement in resident performance of key resuscitation skills using rapid cycle deliberate practice in comparison to standard simulation debrief models. ²⁷ These studies suggest that rapid cycle deliberate practice could have broader utility in other subspecialty settings where residents have limited exposure, yet early recognition and initiation of therapy are critical for positive patient outcomes.

In addition to rapidly teaching resident-specific skills, interns showed improvement in overall knowledge and self-assessed knowledge after participating in the simulations. We noted no difference in week 1 and week 4 case scenario assessments between the groups, despite later groups having been exposed to 5 additional months of clinical training during their first year of residency. The group comprised of the least experienced interns, group A, and the most experienced interns, group G, scored similarly suggesting that despite other skills attained throughout intern year, the identification and treatment of patients with congenital CHD required deliberate practice and targeted education. Residents are less likely to attain these skills via incidental encounters or informal and unintended contextual learning (situated learning) alone. ^23,28 Interestingly, knowledge increase was not associated with interns’ interest in paediatric cardiology. Since postnatal congenital CHD will not often be diagnosed by paediatric cardiologists, but rather by newborn nursery clinicians, emergency medicine physicians, and primary care physicians, we believe that training all paediatric residents is important to prepare them for future practice. These front-line clinicians must have a practical learning environment to recognise congenital CHD and learn initial treatment.

Limitations

This study has several limitations. First, our study was conducted at a single paediatric institution, so may not be generalisable to other training programmes. However, our paediatric interns’ performance likely reflects that of other large academic medical centres. Second, our institution benefits from a well-resourced simulation team. Our curriculum, which incorporated several facilitated computerised manikin-based simulations during each rotation block within a clinical unit, may not be reproducible at other institutions. Third, as a pilot study, our sample size was small which may have impacted our ability to detect significant differences between simulations in weeks 3 and 4. Fourth, although this pilot was integrated into the interns’ standard curriculum, they were allowed to opt out of completing the online pre and post case-based assessments and the retrospective pre-post survey, which could bias the results, despite our high participation rate. Fifth, we did not include a control group to evaluate if the changes seen in overall performance came solely from simulation curriculum that included rapid cycle deliberate practice or the standard didactic curriculum and bedside teaching interns received during the rotation. Sixth, all our assessments and surveys were developed de novo, as there were no preexisting validated tools to use for this curriculum. Finally, we did not assess knowledge retention and application in actual patient care, which could be important areas for future work.

Conclusion

A pilot rapid cycle deliberate practice simulation curriculum focused on recognition and management of congenital CHD can improve paediatric interns’ knowledge and skills observed in simulation encounters. The pilot was also feasible in our busy academic paediatric residency training programme. Future plans include evaluating knowledge retention, application to patient care, and spreading the curriculum to other institutions to assess the generalisability of our approach and findings. We also plan to consider how this approach may be utilised to improve recognition of clinical deterioration and initiation of interventions in children with other low incidence, high-risk conditions.