CLINICIAN'S CAPSULE
What is known about the topic?
Simulation plays an important role in Canadian emergency medicine (EM) with applications in quality improvement, systems development, and medical education.
What did this study ask?
Within EM, what simulation-based research is currently taking place, and what are the priority research themes for future study?
What did this study find?
Simulation in competency-based medical education, simulation for interdisciplinary and inter-professional learning, and simulation for summative assessment are the top priority research themes.
Why does this study matter to clinicians?
A focused research agenda, specific to Canadian EM, will ensure that the growth of simulation is both effective and efficient.
INTRODUCTION
Simulation plays an integral role in the Canadian healthcare system with applications in quality improvement, systems development, and medical education. Simulation is “a tool, device, or environment that mimics an aspect of clinical care,”Reference McGaghie, Issenberg, Cohen, Barsuk and Wayne1 and it has been embraced by the specialty of emergency medicine (EM) at every stage of medical training from undergraduate medical education to continuing professional development.Reference Ilgen, Sherbino and Cook2,Reference Russell, Hall and Hagel3 Coincident with its increasing role in EM, simulation-based research has proliferatedReference Bradley4 and evolved from studies demonstrating that simulation is an effective training methodology to exploring the translational outcomes of simulation-based curricula and programs.Reference McGaghie, Issenberg, Cohen, Barsuk and Wayne1,Reference Ilgen, Sherbino and Cook2,Reference McGaghie, Issenberg, Petrusa and Scalese5–Reference Cheng, Auerbach and Hunt7 High-quality research is required to ensure the effective and efficient use of simulation in diverse contexts, and to discover and inform novel applications.Reference Bradley4,Reference Cheng, Auerbach and Hunt7,Reference Cook8
Bond et al.Reference Bond, Lammers and Spillane9 described a simulation-based research agenda within EM in 2007, highlighting the importance of patient-centred initiatives, continuing medical education, and multicentre efforts to address skill transfer, the validity of simulation for assessment, debriefing techniques, and outcomes related to team performance. Cook et al.Reference Cook8 subsequently called for simulation-based research using comparative studies, validity studies, and qualitative methods. Further, Ilgen et al.Reference Ilgen, Sherbino and Cook2 identified the need to question optimal educational design and evaluate the cost-effectiveness of simulation interventions. Most recently, proceedings from the 2017 Academic Emergency Medicine consensus conference identified simulation-based research priorities that relate to patient safety and outcomes at the systems level.Reference Bond, Hui and Fernandez10
Although these research agendas provide broad direction, they are not specific to Canadian EM, and it has been difficult to relate their priorities to tangible outcomes. The Canadian context is particularly unique given the current national implementation of competency-based medical education across all postgraduate training programsReference Frank, Snell and Sherbino11,Reference Oandasan12 and the concomitant call for the increased use of simulation for training and assessment.Reference Holmboe, Sherbino, Long, Swing and Frank13 Our objective was to summarize simulation-based research activity in Canada, identify its facilitators and barriers, and establish consensus for future simulation-based research priorities in Canadian EM.
METHODS
We conducted a survey-based study consisting of questionnaires and final consensus generation between February and May 2018. Simulation-leads from all 14 academic departments/divisions of EM associated with an adult Fellow of the Royal College of Physicians and Surgeons of Canada (FRCPC) residency training program were invited to participate. A simulation lead was defined as an EM faculty with either a formal simulation-related title (i.e., Simulation Education Lead) or an individual with significant simulation expertise identified at his or her institution. Simulation-based research was defined as any scholarshipReference Glassick14 that incorporates simulation as the study objective, content, or investigative method.Reference Cheng, Auerbach and Hunt7 This study was approved by the Health Sciences Research Ethics Board at Queen's University (REB #6023280).
Questionnaire 1 (Q1) – SBR activities, barriers, and facilitators
Q1 (Supplemental Appendix A) was aimed to capture all current simulation-based research activity and describe the perceived facilitators, barriers, and successful strategies for conducting simulation-based research in Canadian EM. After piloting, simulation-leads administered Q1 at their institution to all EM faculty involved in scholarship related to simulation. Results were collated by two authors, and simulation-based research activity was grouped using a previously described framework.Reference Bond, Lammers and Spillane9 Perceived facilitators, barriers, and strategies were compiled and grouped into themes via an inductive thematic analysis.
Questionnaires 2 (Q2) and 3 (Q3) – SBR priorities
Q2 and Q3 were administered electronically to all simulation-leads. Q2 reflected the results of Q1 back to simulation-leads and asked them to generate a list of priorities for simulation-based research. Simulation-leads were encouraged to consider the goal of advancing the use of simulation in EM when generating simulation-based research priorities and asked to solicit the perspectives of other faculty and stakeholders within their respective simulation programs. Responses to Q2 were collated, reviewed for duplication, and summarized as priority research themes, using an inductive thematic analysis. Q3 reflected these themes back to simulation-leads and asked them to assign a “low priority” or a “high priority” to each, with an aim to prioritize future work in potentially limited resource environments. Simulation-leads distributed the list of themes to senior EM simulation-based educators within their institution, who were also asked to assign priority and recommend any additional themes.
Consensus generation
Simulation-leads met in-person and via the electronic platform Google Hangouts (Alphabet, California, USA) in May 2018 at the Canadian Association of Emergency Physicians (CAEP) Annual Conference with the purpose of achieving consensus on the themes and priority ranking. Consensus was defined a priori as unanimous agreement. If unanimous agreement could not be reached after discussion, that particular theme would be removed. Comments from senior simulation educators were reviewed, and suggestions for additional simulation-based research themes were incorporated if endorsed by consensus. In addition, example research questions were generated for each simulation-based research priority.
RESULTS
Twenty simulation-leads representing all 14 Canadian emergency departments or divisions with an FRCPC-EM residency training program participated. Eight institutions had a single simulation-lead representative, and six institutions had two simulation-leads who worked together. All were practising EM faculty with FRCPC designation apart from one senior FRCPC-EM resident. Mean simulation-lead age was 36.7 (median 36) years, with a mean of 5.9 (median 4) years in practice. Fourteen of 20 simulation-leads had a fellowship in simulation, an advanced degree in medical education, or both.
Q1 identified 62 active simulation-based research projects from 14 institutions (median per institution = 4.5, IQR = 4) that are listed in Supplemental Appendix B. Table 1 presents the distribution of these projects by categories and subcategories. Table 2 presents the summary themes from 34 facilitators, 41 barriers, and 28 strategies identified by simulation leads.
Table 1. Active simulation-based research (SBR) projects in Canada by category* and subcategory
* Bond WF, Lammers RL, Spillane LL, et al. The use of simulation in emergency medicine: a research agenda. Acad Emerg Med 2007;14(4):353–63.
Table 2. Perceived facilitators and barriers to simulation-based research (SBR) and strategies to overcome barriers
All simulation-leads completed Q2 (response rate 100%), generating 49 priorities for future simulation-based research, summarized into seven themes. Q3 (response rate 100%) reflected these themes back to simulation-leads for prioritization and identified the following three as having the most “high priority” categorizations: simulation in competency-based medical education, simulation for interdisciplinary and inter-professional learning, and simulation for summative assessment, receiving 12, 10, and 7 “high priority” categorizations, respectively. Eleven senior simulation educators from five academic institutions reviewed the priority simulation-based research themes. Their prioritization aligned with that of the simulation-leads. One additional research priority was suggested by a senior educator: simulation as an investigative methodology.
Fourteen of 20 (70%) simulation-leads representing 12 of 14 (86%) institutions attended the consensus meeting. There was unanimous agreement with the categorization of simulation-based research priorities from Q3 and with the decision to add the research theme identified by the senior educator. The final list of eight simulation-based research, priority themes and example questions are presented in Table 3 in rank order based on prioritization assignment.
Table 3. Priorities for simulation-based research (SBR) in Canada
DISCUSSION
This study summarizes simulation-based research activity in Canadian EM and presents consensus priorities for scholarship from Canadian EM simulation educators. Simulation-based research is occurring in all but one FRCPC-EM training centre, with most activity focused on education and training, and a minority addressing evaluation and assessment. The reported barriers and facilitators of simulation-based research are reviewed in the following texts, followed by a contextualization of the eight priority themes for simulation-based research in EM in Canada (see Table 3).
Barriers and facilitators of simulation-based research
Lack of funding and/or protected time from clinical service were two commonly cited barriers to simulation-based research activity across the country. This is consistent with prior literature identifying barriers to simulation-based research.Reference Cheng, Auerbach and Hunt7,Reference Savoldelli, Naik, Hamstra and Morgan15 If simulation-based research is to be a priority within EM, then funding in the form of specific grants, access to “seed” money to encourage new projects, and both departmental and institutional level support will be necessary. Comments from several simulation-leads also spoke to the importance of clarity within departments on deliverables and metrics specific to simulation-based research. A lack of mentorship with interest and expertise in simulation-based research was identified as an additional barrier to scholarship with a large variation between institutions. Simulation-based research “champions” within an institution create an environment that fosters further scholarship. Academic emergency departments might consider investing in an individual simulation-based research champion to motivate and focus others on a shared vision.
While all FRCPC-EM training programs report access to high-fidelity patient simulators and task-trainers,Reference Russell, Hall and Hagel3 lack of access to additional infrastructure (e.g., reliable videotaping equipment, paid confederates, access to simulation laboratories for long periods, and supportive colleagues) required to conduct simulation-based research was indicated as a significant barrier. This speaks to the importance of articulating simulation-based research as a priority both within departments and simulation facilities. Further, collaboration at the departmental, institutional, and national level is needed to engage in simulation-based research. This can be facilitated through supporting a local champion, as discussed previously, developing a national position statement that supports simulation-based research as a priority in EM, and creating a platform at the national level that would enable inter-institutional sharing of ideas, resources, and data.
Simulation in competency-based medical education
As postgraduate EM training in Canada transitions to competency-based medical education, simulation will play an increasingly important role in both the delivery of high-quality training experiences and the assessment of entrustable professional activities.16 Competency-based medical education requires direct observation of learners; however, the clinical environment in EM is unpredictable and certain high-acuity events are rarely encountered, making them challenging to observe.Reference Wang, Quinones and Fitch17 Simulation can assist with these challenges by providing safe and reproducible experiences while also allowing expert observation, focused feedback, and deliberate practice.Reference Issenberg, McGaghie, Petrusa, Lee Gordon and Scalese18
Simulation for interdisciplinary and interprofessional learning
EM is practised within a social context involving interactions within teams and across disciplines and specialties. In situ simulation conducted within the actual workplace represents a unique opportunity to observe teams within their clinical environmentReference Rosen, Salas and Wu19 and to evaluate team function within the broader healthcare system.Reference Petrosoniak, Brydges, Nemoy and Campbell20 Multiple studies demonstrate improvements in team performance following simulation-based training.Reference Armenia, Thangamathesvaran and Caine21 Less is known about the translational impact of simulation-based team training on patient outcomes, though recent data are promising.Reference Josey, Smith and Kayani22,Reference Andreatta, Saxton, Thompson and Annich23 Simulation-leads agree that future simulation-based research should seek to establish the optimal role of both laboratory-based and in situ simulation, and how simulation can best serve team-based learning objectives and the assessment of “collective competencies.”
Simulation for high-stakes assessment
The standardization, fidelity, and reproducibility of simulation make it well-suited for the assessment of clinical competence,Reference McGaghie, Issenberg, Petrusa and Scalese5 and it will likely take on a greater role within competency-based medical education programs of assessment. In EM, this process is well underway, with most postgraduate program directors indicating they would be comfortable incorporating simulation-based assessmentsReference Russell, Hall and Hagel3 and several programs already using simulation-based examinations.Reference Hagel, Hall and Dagnone24,Reference Dagnone, Hall and Sebok-Syer25 Postgraduate training in anesthesia now includes a national standardized mid-training, simulation-based examination,Reference Chiu, Tarshis and Antoniou26 and a similar examination for EM residents may complement current high-stakes assessment processes. Despite the potential benefits of using simulation for assessment, caution should be exercised given the current limited evidence for validity in higher-stakes decisionsReference Cook, Brydges, Zendejas, Hamstra and Hatala27 and the potential threat that the introduction of assessment may pose to the paradigm that the simulation suite is a “safe-space” for practice and failure.Reference Rudolph, Raemer and Simon28
Simulation for continuing professional development
Simulation has been recognized as a powerful tool to facilitate learning beyond residency training.Reference Jensen and Torsher29,Reference Hobgood, Mulligan and Bodiwala30 There is emerging evidence for procedural task-training, theatre-based simulation, and in situ simulation in continuing professional development to enhance both individual and team performance in critical situations.Reference Petrosoniak, Auerbach, Wong and Hicks31 Recent culture shifts emphasize inter-professional collaboration and enhanced patient safety by developing more learner-driven and problem-based curricula, often delivered in a simulated environment.Reference Kilian, Binder and Marsden32 As competency-based medical education moves beyond postgraduate training, simulation will assume a more prominent role in the maintenance of skills and development of new competencies. However, best practices and strategies for continuing professional development using simulation have not been described, creating a significant opportunity for innovation and scholarship.
National curricular development
Postgraduate EM training programs in Canada have embraced simulation-based education; however, there is great variation in its quantity and curricular delivery due to local differences in funding, resources, barriers, and clinical contexts.Reference Russell, Hall and Hagel3 There is increasing pressure on postgraduate training programs to improve efficiency in training and to ensure that all training experiences are optimized for effective learning.Reference Nousiainen, Caverzagie, Ferguson, Frank and Collaborators33 A national simulation-based curriculum, similar to that derived for pediatric EMReference Bank, Cheng, McLeod and Bhanji34 or the Nightmares CourseReference McMurray, Hall, Rich, Merchant and Chaplin35,Reference Chaplin, Egan and Cofie36 , would support the development of more ambitious simulation programs.
Best practices in simulation-based education
A previous systematic review identified 12 features and best practices of simulation-based medical education.Reference McGaghie, Issenberg, Petrusa and Scalese5 Many of the research gaps identified therein were highlighted again in our study, including questions pertaining to the optimization of feedback for learning and how to best integrate simulation within multiple other teaching modalities. There is also merit in examining the use of standardized templates, the sharing of case content or entire curricula,Reference Chaplin, Egan and Cofie36 and the development of valid and reliable simulation assessment tools for the EM-specific context. Similar to other pedagogical methods, simulation-based education is context-dependent, and best practices should be reflexive to allow for differing implementations and environments.
Simulation-based education outcomes
In order to justify the substantial costs associated with simulation, simulation-based research is needed to demonstrate improved educational outcomes. While there are data emerging to support this, studying the impact of simulation-based education has been a longstanding priority in the literature.Reference Bradley4,Reference McGaghie, Issenberg, Petrusa and Scalese5,Reference Okuda, Bryson and DeMaria37,Reference McGaghie, Issenberg, Petrusa and Scalese38 Determining a return-on-investment of simulation in EM training will be important to assist administrators in the allocation of resources. Recently, Cook et al.Reference Cook, Andersen, Combes, Feldman and Sachdeva39 argued that modern simulation-based research should focus on the “value proposition” of simulation-based education, with a goal to guide future research that focuses on outcomes and costs, measuring resource requirements, provider performance, patient outcomes, and impact on the healthcare organization. Assigning monetary value to simulation-based education is riddled with difficulties, but simulation-leads underscored the importance of this theme in order to provide a concrete justification.
Simulation as an investigative methodology
Simulation can be used as a research platform with several benefits over clinical-based research.Reference Cheng, Auerbach and Hunt7 The safety and ethical considerations related to clinical research are less relevant in the simulation lab. Furthermore, the research environment in a simulated setting can be controlled and reproduced in order to mitigate confounding influences and focus on the research variable in question.Reference Cheng, Auerbach and Hunt7 These features make it an attractive methodology to explore clinical and non-clinical research questions. For example, simulation has been used to identify latent safety threats in trauma resuscitations,Reference Fan, Petrosoniak and Pinkney40 inform staffing workload and responsibilities in a new emergency department,Reference Geis, Pio, Pendergrass, Moyer and Patterson41 and develop and implement novel technologies.Reference McGraw, Chaplin and McKaigney42,Reference Szulewski, Braund and Egan43
Limitations
This project has several limitations. Firstly, as a survey-based study coordinated by a self-selected group of simulation-leads, the perspectives of other stakeholders with respect to simulation-based research, although solicited, may not have been represented. In addition, the questionnaires were not anonymous. The resultant bias is a potential threat to the validity of the results. Secondly, the response rate to Q1 cannot be reported. We relied on individual simulation-leads to administer Q1 within their institution to all EM faculty involved in simulation scholarship, but the denominator was not collected. Thirdly, this project only surveyed departments of EM with an active FRCPC postgraduate program, so any important simulation-based research activity outside of these centres is not represented. Finally, we have not included the pediatric EM community, which has a well-established simulation-based research network.Reference Cheng, Auerbach and Calhoun44
CONCLUSION
This project summarized the current state of simulation-based research activity in Canadian EM and outlined a set of research priorities for the future. This represents the first step in the development of a cohesive, focused simulation-based research agenda specific to Canadian EM. We aim to facilitate a national conversation that will foster collaboration and lead to the next wave of innovation in simulation-based education and scholarship.
Acknowledgements
The authors would like to thank the Kingston Resuscitation Institute for supporting the creation and initial meetings of the Emergency Medicine Simulation Educators Research Collaborative group.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/cem.2019.416.
