The development of entrustable professional activities reference cards to support the implementation of Competence by Design in emergency medicine

Abstract

We designed two practical, user-friendly, low-cost, aesthetically pleasing resources, with the goal of introducing residents and observers to a new Competence by Design assessment system based on entrustable professional activities. They included a set of rotation- and stage-specific entrustable professional activities reference cards for bedside use by residents and observers and a curriculum board to organize the entrustable professional activities reference cards by stages of training based on our program's curriculum map. A survey of 14 emergency medicine residents evaluated the utilization and helpfulness of these resources. They had a positive impact on our program's transition to Competence by Design and could be successfully incorporated into other residency programs to support the introduction of entrustable professional activities-based Competence by Design assessment systems.

Résumé

Le groupe a conçu deux documents à la fois pratiques, conviviaux, peu coûteux et agréables à l’œil, dans le but de présenter aux résidents et aux observateurs un nouveau système d’évaluation du modèle d'acquisition des compétences par conception, fondé sur des activités professionnelles confiables. Les documents comprenaient un ensemble de fiches de référence illustrant des activités professionnelles confiables liées aux étapes de formation et aux stages cliniques, à utiliser au chevet par les résidents et les observateurs ainsi qu'un tableau cartonné représentatif du curriculum visant à répartir les fiches de référence liées aux activités professionnelles confiables selon les étapes de formation fondées sur la carte du programme. Quatorze résidents en médecine d'urgence ont évalué l'utilisation et l'utilité de ces documents. L'enquête a révélé que ces derniers avaient facilité la transition vers le modèle d'acquisition des compétences par conception, et se prêteraient bien à d'autres programmes de résidence afin d'aider à la mise en œuvre de systèmes d’évaluation de la nouvelle approche de formation, fondés sur des activités professionnelles confiables.

BACKGROUND

The Royal College Competence by Design assessment system was implemented nationally for the emergency medicine (EM) resident cohort beginning postgraduate training in 2018.¹ The goal of Competence by Design is to ensure that residents are competent upon completion of residency through the assessment of entrustable professional activities representing the key tasks of our discipline.² EM has 28 entrustable professional activities spanning four stages of training, some of which require observation of specific clinical presentations.³ To be successful in this assessment program, Competence by Design residents will need to be familiar with their new assessment requirements, but the number and complexity of entrustable professional activities make this challenging. We sought to design educational resources that would clarify the expectations of residents on each rotation and stage of training.

PURPOSE

We designed a curriculum board and practical, aesthetically pleasing reference cards to facilitate and simplify the completion of required entrustable professional activities assessments through each rotation and stage of residency.

INNOVATION DESCRIPTION

We mapped the EM entrustable professional activities to rotations within our curriculum and developed stage- and rotation-specific entrustable professional activities reference cards, as well as a curriculum board to organize them by stage of training (Figure 1).

Figure 1. Representation of a curriculum board with enlarged stage- and rotation-specific reference cards. EPA = entrustable professional activities; CP = clinical presentations.

Reference cards

The cards were printed on lightweight plastic and designed to hang behind residents’ identification badges. The text was sized to allow for easy visibility at arm's length, and each one was uniquely coloured. All cards contain the entrustable professional activities for the specified stage/rotation, an icon representing each activity, and the suggested number of observations for each clinical presentation (see Figure 1).

Rotation- and stage-specific cards differ slightly. Rotation-specific cards contain only activities that residents would experience on a non-EM clinical rotation. Stage-specific cards contain all entrustable professional activities for that stage and were designed for EM rotations. Some stages (Core and Transition to Practice) contained too many entrustable professional activities to fit on one card and were printed double-sided to decrease the number of cards. Twenty unique cards were developed.

Curriculum board

The curriculum board was developed to organize the cards within our program's curriculum map. The cards were organized by stage of training, allowing residents to view the map and the activities associated with each stage at a glance.

Innovation evaluation

The resource evaluation was deemed exempt from ethical review by the University of Saskatchewan Behavioral Ethics Board (BEH #988). After 36 weeks within the Competence by Design assessment program, all 14 Royal College EM residents were invited to complete a survey investigating the helpfulness and utilization of the resources (questions outlined in Tables 1 and 2). A narrative explanation was requested after each response. The results were analysed using descriptive statistics and a qualitative content analysis.

Table 1. Resident survey responses regarding the utilization of the reference cards and curriculum board

Table 2. Resident survey responses regarding the helpfulness of the reference cards and curriculum board

RESULTS

All 14 residents in our program completed the survey. Survey responses regarding utilization and helpfulness are presented in Tables 1 and 2, respectively. The qualitative content analysis described how each resource was used.

Rotation-specific cards were found to be most helpful at the beginning of off-service rotations and as a quick reference for the staff. One resident stated, I use them to remember which [entrustable professional activity] to focus on. I show the cards to my staff so they see a quick summary of what I'm focusing on.

Stage-specific cards were referenced more during EM shifts as an overview of the different entrustable professional activities. One resident noted, It's helpful in the ED when I have 15 [entrustable professional activities] in core and it can be difficult to remember all of them.

The curriculum board provided an overview of the program and where each entrustable professional activity fit. One resident stated, It is great as an overview of which [entrustable professional activity] fit into which rotations during a given stage. [It] can help you see which more difficult [to get] ones you can get later.

A common theme seen in comments on all of the resources was that they were believed to be more helpful as the residents started new rotations and stages.

DISCUSSION

We developed resources that clarify the rotation- and stage-specific expectations of our new entrustable professional activities-based assessment system for our residents. The narrative comments suggest that each was used by the participants as intended. An examination of why the resources were helpful for some and not for all residents is beyond the scope of this study. However, principles of modern educational theory were incorporated into their design which likely improved their utilization and helpfulness.

Cognitive load theory suggests that extraneous load can be decreased by cognitively offloading a task to accessible external resources.⁴ The reference cards were designed to fit behind the residents’ hospital identification so that they would be readily accessible. This may have decreased the extraneous cognitive load created by residents needing to recall entrustable professional activities. Extraneous load would have been most prominent shortly after the introduction of the entrustable professional activities when the resources were believed to be most helpful.

Multimedia learning theory posits that we possess limited cognitive capacity and have visual and verbal processing channels.⁵ The reference cards were designed to spatially align text and pictures to ensure that both pathways were leveraged. This may have facilitated familiarity with the entrustable professional activities faster than what would have been seen with resources leveraging only a single channel (e.g., a list of entrustable professional activities).

Our program will continue to make the reference cards available to residents, particularly when they enter new stages and rotations. Other residency programs at our institution are interested in developing similar cards, which will provide new opportunities for study of this innovation. Additional research could investigate their impact on cognitive load and/or compare their impact with similar text-only resources. To facilitate the use of this innovation in EM, the resources that we developed will be freely downloadable from CanadiEM.org.⁶

Limitations

Though these innovations show promise in supporting a transition to Competence by Design, we did not demonstrate an impact on the culture of assessment or investigate the perspective of faculty observers. The small sample size (14 residents), implementation within a single centre, and short period of resource utilization limit our ability to draw strong conclusions on the ultimate role that these resources could play in the introduction of Competence by Design.

SUMMARY

Reference cards and a curriculum board were designed to facilitate the transition to entrustable professional activities-based assessment within Competence by Design by clarifying the expectations of residents within each stage and rotation of training. Most residents felt that they were used and helpful. Similar resources could be developed by other programs.