Development of a national, standardized simulation case template

Abstract

Emergency medicine (EM) training programs incorporate simulation for teaching as well as formative and summative assessment. The development of a simulation curriculum for Canadian postgraduate EM programs is underway and would be facilitated by a standardized, user-friendly, nationally endorsed simulation template. We convened a nationally representative group of simulation educators to participate in a three-phase process to develop and refine a simulation case template for Canadian EM educators. Participants provided feedback by means of free text comments and focus groups which were analyzed to inform modification of the template. We anticipate that this template will facilitate the sharing of cases across sites and the development of standardized cases for simulation-based assessment.

Résumé

On a recours à la simulation tant dans l'enseignement que dans les évaluations formatives et sommatives, dans les programmes de formation en médecine d'urgence (MU). Un curriculum de simulation est en cours d’élaboration dans les programmes de formation postdoctorale en MU au Canada, et la tâche serait facilitée par la conception d'un modèle normalisé de simulation, convivial et applicable à l’échelle du pays. Un groupe représentatif de formateurs en simulation au pays a donc été formé pour participer à un processus en trois phases d’élaboration et d'amélioration d'un modèle de simulation conçu à l'intention des enseignants en MU au Canada. Les participants ont fait part de leurs observations dans des commentaires sous forme de texte libre et dans des groupes de réflexion, puis, après analyse, le modèle de cas a été amélioré en conséquence. L’équipe de recherche est d'avis que ce modèle facilitera l’échange de cas dans tous les centres de formation et l’élaboration de cas normalisés pour les évaluations fondées sur la simulation.

INTRODUCTION

Nearly all Royal College of Physicians and Surgeons of Canada Emergency Medicine (RCPSC-EM) postgraduate training programs have a simulation-based education curriculum.¹ The transition to a national competency-based assessment system² will require an increase in the use of simulation to provide standardized training experiences and another avenue for assessment.^3,4 Unfortunately, high-quality simulation cases are resource-intensive to develop, and there is no formal mechanism within Canada's EM training programs to promote the collaborative sharing of cases.

A standardized national scenario template could facilitate the development and sharing of high-quality simulation cases by providing a common framework. Simulation scenario templates guide the structure and delivery of simulation cases.⁵ The use of unfamiliar, external case templates is challenging for educators and simulation technicians (simulation center employees who assist with simulation cases)⁶ who must frequently revise them before use. Furthermore, national scholarly collaboration has been identified as fundamental to the advancement of simulation-based education in EM in Canada⁷ and a common template could assist with this goal.

We partnered with the EM Simulation Education Researchers Collaborative (EM-SERC), a group of simulation educators committed to advancing EM simulation-based education in Canada by means of collaborative research and knowledge translation, to develop a national simulation template.

PURPOSE/RATIONALE

Well-developed simulation templates have the potential to facilitate the delivery of simulation-based education by assisting users in organizing scenarios and identifying potential challenges before implementation.⁵ Herein, we describe the development and refinement of a national EM simulation scenario template.

DESCRIPTION OF INNOVATION

Several university-based simulation centers were already using the EM Sim Cases (https://emsimcases.com) template; therefore, we chose to use it as our first draft for the template derivation process. We generated feedback on the template using three phases. We consulted members of the EM-SERC mailing list, which included 20 EM simulation educators representing every RCPSC-EM training site. We resolved conflicting comments using a democratic process.

In phase 1, we sought free-text feedback on the EM Sim Cases template by means of email. We received 65 comments from 11 respondents. Two authors (J.B. and K.C.) performed an inductive thematic analysis which identified four major themes (formatting, objectives, debriefing, and assessment tools). Fifty-one of 65 comments were incorporated into the revised template.

In phase 2, we sought free-text feedback on the revised template by means of email. Two authors (J.B. and K.C.) conducted a second thematic analysis of 40 comments from 12 respondents and identified three broad themes (formatting, objectives, and debriefing). We incorporated 30 out of 40 comments into the penultimate template.

In phase 3, we sought feedback on the penultimate template by means of focus groups with simulation educators and technologists at six Canadian university-affiliated simulation programs across four provinces. Several sites also included nurses, respiratory therapists, and physicians from specialties other than EM. An investigator at each site led their local focus group using guided questions (online Supplemental Appendix A) soliciting participant feedback for the overall template and each individual section. The feedback was collected using either audio recordings or written notes. The feedback was transcribed and e-mailed to J.B. and K.C. This phase generated 98 specific comments which we grouped by template section and served to inform the final template (Supplemental Appendix B).

DISCUSSION

We describe a structured process, endorsed by Canadian simulation educators, to develop an iteratively refined simulation case template. The incorporation of feedback from key EM simulation educators across the country will improve the successful and wide-ranging adoption of the template. We anticipate this product will facilitate the collaboration of simulation initiatives nationwide. Specifically, this template will support: (1) a reduction in the duplication of efforts across sites by facilitating the sharing of simulation cases; (2) a national collaboration for the development of both simulation cases and curricula; and (3) multicenter simulation-based research by removing confounders related to the local adoption of an unfamiliar case template. This will be important to improve the rigour and validity of data by reducing inter-site variability.

The template will be particularly important for the use of simulation in national high-stakes assessment and multisite research. Simulations used for high-stakes assessment and high-quality research must be reproducible to strengthen their validity.⁸ The standardization of cases is imperative to meet these objectives⁹ and the consistent use of a template is likely to lead to comfort among faculty and technicians running the cases.¹⁰

Several training sites plan to convert their simulation cases to our final template. To ensure that it serves the needs of the EM community we will conduct a usability analysis and survey among sites who adopt this new template. We plan to increase the adoption of this template using several steps: (1) publication in free-open-access form on https://emsimcases.com; (2) a post outlining its release disseminated by means of Twitter; and (3) a simulation case-series from multiple Canadian postgraduate training programs using the new template published at https://emsimcases.com.

Our project has several limitations. First, most EM-SERC members are from RCPSC-EM training sites, as such, Canadian College of Family Physicians-EM training sites may be underrepresented, thus limiting its generalizability within the Canadian academic EM simulation community. Second, although we solicited feedback from a national group of simulation educators (EM-SERC), simulation technologists, and experienced simulation educators, the perspectives of our allied health colleagues were not methodically pursued, and it may be less applicable to non–physician-based simulation education. Finally, we did not anonymize feedback which may have introduced bias. We mitigated these limitations by obtaining national buy-in and increased generalizability by including simulation leaders from multiple sites in the development process.

SUMMARY

We derived a national simulation case template to encourage the sharing of cases between programs, to facilitate further simulation research, and aid in the development of standardized simulation scenarios for assessment purposes.