INTRODUCTION
The Royal College of Physicians and Surgeons of Canada certified its first specialist emergency physician (EP) in 1983.Reference Elyas and Stapleton1 Over the past 4 decades, the specialty has matured with 14 residency training programs across Canada offered in French or English. The design of specialist emergency medicine (EM) training programs followed a model organized by time, manifested in a set of standardized clinical rotations that exposed trainees to (assumed) clinical content. All of this changed on July 1, 2018.
On this date, all Canadian EM specialist training programs migrated to a competency-based medical education model. This transition was part of a 7-year, system-wide change to all Canadian specialty training programs.2 EM was in the second cohort of Royal College disciplines, following an initial pilot of two specialties.
A form of outcomes-based education, competency-based medical education focuses on the abilities required of specialist EPs for successful entry into practice.Reference Carraccio and Englander3,Reference Frenk, Chen and Bhutta4 Rather than a simple recitation of relevant knowledge, competency-based medical education requires the demonstration of relevant competencies, where a competency is “an observable ability of a health professional, integrating multiple components such as knowledge, skills, values, and attitudes. Since competencies are observable, they can be assessed to ensure their acquisition. Competencies can be assembled like building blocks to facilitate progressive development.”Reference Frank, Snell and Cate5
The purpose of this study was to describe the rationale, process, and redesign of the most significant change in specialist EM training since the inception of the specialty in Canada.
RATIONALE
There are four compelling arguments supporting the transition of specialist EM training to a competency-based medical education model.Reference Holmboe, Sherbino, Englander, Snell and Frank6 Firstly, in exchange for serving the public good, the profession enjoys self-regulation via the determination of the standards for practice certification and accreditation for residency programs. However, the public trust is eroding. The CEO of the Canadian Medical Protective Association indicates that “college matters and hospital matters – based on complaints either attributed to the physician's behaviour or to the competency of the physician – have risen globally…We do track … all articles pertaining to doctors and the vast, vast majority of them are negative…”Reference Colin7
Accountability to the public requires transparency and rigour in training programs. Specific, systematic articulation of all of the specialist EM competencies and the associated assessment processes provides the public reassurance that Canadian specialist EPs are competent to provide emergency care to any patient with any acute health concern on the first day after graduation.8 The transformation of EM programs to a competency-based medical education model is an attempt to provide such transparency and rigour.
Secondly, in an age of fiscal accountability, government seeks evidence of healthcare resource stewardship. Justification for 5 years of specialty training has been supported by a collaborative working group that included representation from the Canadian Association of Emergency Physicians (CAEP), the College of Family Physicians of Canada (CFPC), and the Royal College of Physicians and Surgeons of Canada. “[Specialist EP] graduates, due to their longer dedicated training in EM, have additional abilities, including but not limited to: critical care experience, advanced resuscitation skills, advanced toxicology, pediatric EM, research and a higher level of experience with the management of critically ill patients…. Due to curriculum differences, [specialist EPs] have additional training in some areas, including EM administration, research, and pre-hospital care.”8 The competency-based medical education transition maintains the high specialist EM standards of practice, while developing education system efficiencies. The complete re-examination of all components of EM specialist training mapped to the complexities of contemporary EM practice identified inefficiencies. Moreover, traditional curricula organized around time, where the calendar dictates training progression and allows only a single date of exit from training, is inefficient. Competency-based medical education allows learners to progress when they demonstrate the ability to do so. While most learners will require 5 years to demonstrate competence, competency-based medical education opens up the possibility of shorter training time. Eight years of training data from a University of Toronto pilot orthopedic surgery competency-based medical education residency training program identified a 15 percent decrease in required training time.Reference Nousiainen, Mironova and Hynes9
Thirdly, faculty struggle with ineffective assessment systems that cannot provide useful data on resident performance to assist with educational diagnoses and prescriptions or effective curriculum planning. Traditional curricula over-emphasize end-of-training high stakes knowledge exams with little attention to authentic performance. A competency-based medical education model grounds assessment at the bedside, using multiple, low stakes, biopsies of longitudinal performance mapped to a mastery learning model.Reference Iobst, Sherbino and Ten10 Immediate feedback on performance allows the resident to efficiently acquire skills, while aggregation of hundreds of these data points guide global resident performance decisions, the functioning of rotations and curricula.
Fourthly, adopting a competency-based medical education model serves resident needs. EM is highly competitive in Canada with less than 50 percent capacity to accept medical students who rank specialist EM as their first choice for residency. Successful applicants have a track record of academic success and effective learning processes. Yet, traditional EM curricula assume that residents learn at the same pace, where advancement proceeds as a cohort dependent upon the crude determination of the change of the academic year. EM resident learning trajectories indicate that these assumptions of uniform progress are false.Reference Chan, Sherbino and Mercuri11 By organizing training around the needs of the individual, mastery learning models allow tailored education and variable progress for residents, optimizing their time and limiting calendar watching.
Further, by clearly articulating the progressive stages of training, competency-based medical education addresses the differences in abilities among entering residents. The transition to discipline (i.e., residency) stage ensures that fundamental competencies are acquired before more complex ones. Similarly, the transition to practice stage allows for meaningful acquisition of the tacit knowledge required of practice.
PROCESS
Two years prior to launch, a working group was convened to develop the Canadian specialist EM competency-based medical education model. Membership included the Royal College EM Specialty Committee with democratic representation from the five established regions (Atlantic; Quebec; Ontario and Nunavut; Manitoba and Saskatchewan; Alberta, British Columbia, Yukon, and North West Territories), all residency program directors, representatives from pediatric EM, critical care, clinical pharmacology and toxicology, the national specialty society (CAEP), plus a chair and vice chair. Additional representation included all assistant residency program directors, CFPC representation, two resident representatives, and an educational consultant supplied by the Royal College. Project support was provided by the Royal College, including an administrator and policy writer.
A consensus process was adopted with all final decisions collectively endorsed by every member of the committee. Standard Royal College policies regarding the structure, governance, and accreditation process for residency training were observed. However, the design and content of the competency-based medical education curriculum were constructed de novo. No working group member identified a conflict that influenced the consensus process.
Three in-person retreats (each 3 days in length) were held over an 18-month period. Four subcommittees were struck to address specific issues related to the four stages of residency training. Bimonthly webinar conference calls were held with the whole special working group, and monthly webinar conference calls were held for each subcommittee during periods of intense deadlines. Draft policies and memos were securely distributed for revision and comment electronically. Document version control was maintained by administrative staff.
Queen's University's specialty EM residency program, through a special Royal College approval process, launched their competency-based medical education curriculum on July 1, 2017. A planned iterative program evaluation was conducted at 3, 6, and 12 months to provide pilot data informing the national launch.Reference Hall, Rich and Dagnone12 A change management initiative for residency program directors and EM faculty was developed in the 9 months leading up to the 2018 launch. Educational resources, project Gantt charts, and faculty development resources were supplied. Topics covered included the rationale for change, principles of competency-based medical education, new stages of training, new assessment processes (e.g., entrustable professional activities, competence committees, exam timing), and the new area of concentrated expertise, among others.
Prior to launch, a meeting of all postgraduate medical education deans representing every medical school with a specialty EM residency training program convened to indicate the readiness of their home program to launch. EM was the only discipline with unanimous university-level endorsement among the first three cohorts of transition to competency-based medical education, indicating the EM residency programs had appropriate resources, faculty development, and change management strategies to support the transition.
PRINCIPLES OF CBME
There are no commonly agreed upon principles of competency-based medical education published in the literature to ensure fidelity of implementation.Reference Van Melle, Frank, Holmboe, Stockley and Sherbino13 The working group adopted the following four principles to ensure common implementation across all 14 programs.
Outcomes defined
The CanMEDS 2015 FrameworkReference Frank, Snell and Sherbino14 is a generic physician competency framework adopted in more than 43 educational jurisdictions around the world.15 While some competencies are common across disciplines, it was essential to tailor the framework to the required competencies of a Canadian specialist EP. Box 1 includes the definition of a specialist EP. Supplemental Appendix 1 contains the key and enabling competencies that describe the practice of a specialist EP.
Box 1. Definition of a Canadian specialist emergency physician
“A Royal College Emergency Medicine specialist is a physician who uses highly developed clinical reasoning skills to care for patients with acute and often undifferentiated health problems, across a broad spectrum of illnesses and injury in all age groups, frequently before complete clinical or diagnostic information is available. An Emergency Medicine specialist is able to determine which conditions require immediate care and which conditions can be investigated and managed in a different setting. They assume a consultant's role in the specialty, providing comprehensive emergency adult and pediatric care in academic/teaching, community, or regional hospital settings.
Royal College EM specialists are resuscitation specialists for patients with undifferentiated presentations, possessing expertise in the anatomy, physiology, pathophysiology, pharmacology, toxicology, and management of all acute presentations. EM specialists use their comprehensive knowledge of related fields at the interface between emergency care provision and the healthcare system, including toxicology, traumatology, prehospital care, environmental medicine, and disaster medicine.
Royal College EM specialists are an academic and community resource, providing advanced clinical patient care; support to other physicians and healthcare professionals in an emergency setting; leadership in the administration of emergency departments, emergency medical systems, healthcare institutions, and related programs; and the conduct of relevant research and education with the goal of advancing knowledge and improving individual and/or community health outcomes.”Reference Dreyfus and Dreyfus16
The working group achieved consensus and endorsed these EM competencies. They serve as the destination to guide trainees through residency. For program directors and faculty members teaching in residency programs, the EM competencies provide a common mental model. Using this framework, a residency training program can be designed to achieve these competencies.
For society and government, this document serves as a specific articulation of the practice of a specialist EP. This is of particular importance in a period where the term emergency physician is often used as a function of geography (i.e., the work environment of the physician).8 Recognizing the tremendous heterogeneity between emergency departments and patient populations across Canada, using competency to define a physician allows for greater clarity with respect to health human resources needs.
Staged progression
With competency-based medical education, time is no longer the determinant of resident progression, where prescribed dwell times (e.g., rotation attendance) serve as the surrogate for the acquisition of experience. Rather, time is used as an independent variable, where a resident actively navigates at an individualized rate based on their demonstration of advancing ability.Reference Iobst, Sherbino and Ten10 While the designation of postgraduate years 1 to 5 has been preserved, this nomenclature is only used by provincial governments with respect to funding positions and resident salaries. The new educational design describes four stages: transition to discipline (i.e., residency), foundations, core, and transition to practice (Figure 1).
Figure 1. Stages of training.
Each of these stages includes specific milestones that describe the knowledge, skills, and attitudes to be acquired at this stage. (See Supplemental Appendix 2.) Specific assessments are tied to each stage. Advancement from one stage to the next requires the demonstration of the abilities specific to each stage.
This developmental design allows the sequencing of foundational to advanced abilities.Reference Ericsson17,Reference Chan, Sebok-Syer and Thoma18 Regular checkpoints requiring a comprehensive review of performance ensure that trainees on failing learning trajectories are identified early with the possibility of less intensive remediation of performance. For trainees on excelling learning trajectories, an earlier increase in responsibility accelerates residency completion.Reference Harris, Bhanji and Topps19
For both trainees and faculty, the stages of learning make transparent the progression of ability required to become a specialist EP. In essence, an explicit educational roadmap is provided. This common mental model relegates expectations of junior versus senior performance that was inconsistent between trainees, faculty, and programs.
Tailored learning
The traditional model of specialist EM training used a common curriculum among all residency programs, where specific rotations, often in the service of other specialties, were required. The new competency-based medical education model returns flexibility to learners and programs. Rather than prescribing a specific rotation and specific specialty service, the new model indicates the type of patient experiences required for the resident to develop competence in managing key EM presentations. The competence by design model does specify training experiences, but these are framed as exposure to specific patient subpopulations, rather than specific discipline rotations. For example, a prescribed 4-week rotation on an orthopedic service has been replaced with a requirement for a resident to develop competence in managing patients with acute musculoskeletal injuries. This allows a program director to effectively design a locally based curriculum.
For residents, an emphasis on tailored learning is reflected in the lack of specific dwell time with various required patient experiences. This introduces some flexibility into individualized training. Of course, institutional (e.g., university, hospital) logistics must be negotiated. (For example, most required training experiences will be structured in increments of 4 weeks to smooth transitions in service requirements.) Residents play a key health human resource in the delivery of healthcare in our teaching institutions. Also, the funding envelope for training is specific to a 5-year period. However, within this envelope of funding, residents, in conjunction with their program directors, have the ability to tailor more experience with one patient population and less with another, depending on their specific needs.
Finally, a specific portion of training is dedicated to an area of concentrated expertise. This training experience allows a resident to pursue deeper expertise in a clinical or academic domain relevant to his or her future practice as a specialist EP. This required learning experience is unique to EM. It ensures that the specialty is continually advancing clinical, administrative, research, and educational domains. In turn, this improves the quality of Canadian emergency health systems.
Supplemental Appendix 3 describes the clinical training experiences, simulated training experiences, and topics requiring formal instruction, mapped to each stage of training.
Programmatic assessment
Programmatic assessment is the systematic, longitudinal acquisition of multiple assessments from multiple assessors addressing multiple competencies to inform a group-based global judgement of a trainee's progression towards a competent practice.Reference Sherbino, Bandiera and Frank20 Programmatic assessment emphasizes authentic performance. For trainees and frontline faculty, programmatic assessment will be the most notable change in the transition to a specialist EM competency-based medical education model.
Traditionally, residency training programs rely heavily on an end-of-training high stakes knowledge exam, ad hoc in-training oral and written exams, and poorly designed and executed end-of-rotation reports (e.g., ITERs).Reference Chan and Sherbino21 EM is well situated to adopt programmatic assessment, because many programs currently employ direct-observation assessment instruments that capture authentic performance.Reference Englander, Frank and Carraccio22
The two most notable additions to assessment processes are entrustable professional activities and competence committees. Entrustable professional activites are key tasks (i.e., work) of the discipline that can be observed and entrusted to a trainee and are assessed via direct observation instruments.Reference Acai, Li, Sherbino and Chan23 A renewed emphasis on direct observation of authentic performance is a key principle of programmatic assessment. Entrustable professional activities describe an element of work of specialist EP practice, integrating competencies from multiple CanMEDS roles. Each activity is supported by 10 to 15 competencies (i.e., milestones). Entrustable professional activities are mapped to a specific stage of training. There are 28 EM entrustable professional activities, representing critical and sentinel, but not exhaustive, specialist EP work. (See Supplemental Appendix 4.)
Unique to entrustable professional activities is an entrustment scale (Table 1) anchored by descriptors of the degree of supervision required for completion. Complementary to the entrustment score is a required narrative included for the completion of each entrustable professional activity. On average, one entrustable professional activity is completed during each emergency department shift. Ideally, the entrustable professional activity to be observed is negotiated between resident and observer at the start of the shift and completed midway through the shift. Once complete, the observer should share his or her observations and entrustment score with the resident. Both the entrustment scale and the narrative feedback are recorded via a web-based platform. The Royal College e-portfolio or university-specific proprietary platforms are used to facilitate the aggregation of data and the provision of personalized assessment dashboards available to residents and residency programs.Reference Chan, Sherbino and Mercuri11
Table 1. Entrustment scaleReference Li, Sherbino and Chan24
While entrustable professional activities are a key part of the resident's overall assessment portfolio, they are complemented by in-training exams and other locally implemented assessments to provide a multi-modality perspective on resident progress. Included in Supplemental Appendix 4 are nine additional special assessment instruments.
The data assembled in each resident portfolio, including entrustable professional activity assessments, special assessments, in-training exams, and other local instruments, are adjudicated on a regular basis by a competence committee.Reference Hauer, Chesluk and Iobst25 This committee, comprising the program director and faculty, and, in some cases, residents or non-physician community members, depending on local practice, systematically reviews the aggregated portfolio data to make a group judgement about a resident's performance towards attaining stage-specific and finally global competence.Reference Hauer, Cate and Boscardin26,Reference Cooney, Chan and Gottlieb27 The competence committee is not bound by accounting practice, where aggregated mean scores drive a decision. Rather, emulating scientific peer review or jurisprudence, competence committees weigh all evidence to determine a summary decision. Competence committees require deliberate structure and systematic practices to avoid group think and mitigate bias.Reference Norman, Norcini and Bordage28 Competence committees indicate when an entrustable professional activity is achieved, and a resident can progress to the next stage. Most notably, the final step in completing specialist EM residency training requires the competence committee to endorse that the resident is ready for practice.
The high-stakes written and oral exam is no longer at the end of training; it is situated between the core and transition to practice stages. Placing the exam at the end of training has led to inappropriate social stressors on trainees, absenteeism from clinical and educational responsibilities, and a de-emphasis on the clinical practice of EM with an overt attention to rare trivia. The final stage of training, transition to practice, requires indirect supervision of the residents to allow them to demonstrate competence in managing a section of an emergency department independently, including teaching junior learners and attending to relevant clinical administrative issues.
FUTURE STEPS
The designed versus the delivered versus the experienced curriculum is not a direct correlation. The conceptual frameworks and theories informing this new model of specialist EM training have many assumptions.Reference Van Melle, Gruppen and Holmboe29 Additionally, many unintended consequences of certain design features (including gaming of the system) have yet to be realized. Moreover, while competence by design is informed by education theory, there are a number of untested assumptions. For example, will the cost savings of a more efficient residency design offset the costs required for increased education infrastructure, simulation training, and so forth? Will entrustable professional activity assessments adequately demonstrate global specialist EP competence? Also, will competence by design improve the quality of education and, more significantly, the care of emergency patients? These representative questions are only a portion of the assumptions that require testing.30 Future steps for EM educators and researchers include program evaluation of local and national curricula. Challenging the assumptions that inform this design is critical to refine future iterations and understand areas of educational success and failure.Reference Gofton, Dudek, Wood, Balaa and HAmstra31
CONCLUSION
The transition of Canadian specialist EM residency training to a competency-based medical education model is not a perfect design. The future of EM education will see refinements and new evolutions in curricula. However, competency-based EM residency education attempts to maintain the social contract, steward government resources, facilitate best education practices for faculty, and improve learning efficiency for residents. The refinement of the competencies required for specialist EM practice, the implementation of stages of residency with associated educational roadmaps, the adoption of tailored learning, and the design of programmatic assessment mark improvements upon traditional residency education. The lessons we learn from this national transition will inform the next generation of specialist EM physicians.
Competing interests
None declared.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/cem.2019.417.
Acknowledgements
The authors thank Brian Au, Isabelle Bertrand, Stephen Choi, Warren Cheung, Brian Chung, Marie-Maud Couture, Jaelyn Caudle, Damon Dagnone, Matt Davis, Pierre Désaulniers, Sandy Dong, Cheryl French, Emerson Genius, David Ha, Carine Haggar, Andrew Hall, April Kam, Janet MacIntyre, Jill McEwen, Sarah McPherson, Nazanin Meshkat, Christine Meyers, Avik Nath, Gina Neto, Wesley Palatnick, Alim Pardhan, Sarah Reid, Helen Ross, Robert Sedran, Samir Sherif, Marco Sivilotti, Brent Thoma, Lisa Thurgur, Caroline Tyson, Kristen Weersink, David White, Rob Woods, and Joel Yaphe for their contributions toward this study.
