Quality improvement primer part 2: executing a quality improvement project in the emergency department

Abstract

The topics of quality improvement (QI) and patient safety have become important themes in health care in recent years, particularly in the emergency department setting, which is a frequent point of contact with the health care system for patients. In the first of three articles in this series meant as a QI primer for emergency medicine clinicians, we introduced the strategic planning required to develop an effective QI project using a fictional case study as an example. In this second article we continue with our example of improving time to antibiotics for patients with sepsis, and introduce the Model for Improvement. We will review what makes a good aim statement, the various categories of measures that can be tracked during a QI project, and the relative merits and challenges of potential change concepts and ideas. We will also present the Model for Improvement’s rapid-cycle change methodology, the Plan-Do-Study-Act (PDSA) cycle. The final article in this series will focus on the evaluation and sustainability of QI projects.

Résumé

Les sujets liés à l’amélioration de la qualité (AQ) et à la sécurité des patients ont gagné de l’importance dans les milieux de soins de santé au cours des dernières années, tout particulièrement au service des urgences (SU), qui est souvent le porte d’entrée du système de soins de santé. Dans la premier article d’une série de trois, conçue comme une introduction à l’AQ à l’intention des cliniciens qui travaillent au SU, il a été question de la nécessité d’élaborer un plan stratégique afin de concevoir un projet d’AQ qui soit efficace, et ce, à l’aide d’une étude de cas fictive. Dans le deuxième article, nous poursuivrons la démarche avec le même exemple en vue de réduire le temps écoulé avant l’administration d’antibiotiques chez les patients atteints de sepsie et nous présenterons le Model for Improvement. Seront ainsi examinés les caractéristiques d’un bon énoncé du but visé, les différents types de mesures susceptibles de vérification durant la réalisation des projets d’AQ ainsi que les avantages et les difficultés que comportent chacun des concepts et idées possibles de changement. Nous présenterons également la méthode de changements à cycles rapprochés du modèle d’amélioration, soit le cycle Planifier – Exécuter – Étudier – Agir (PEÉA). Enfin, le dernier article de la série portera sur l’évaluation et la durabilité des projets d’AQ.

INTRODUCTION

After the Institute of Medicine published two important reports in 2000 and 2001 that brought the topics of quality improvement (QI) and patient safety (PS) to the forefront of health care, the number of projects framed as QI and PS has skyrocketed in all medical disciplines. ^{1 - 3} These topics are especially relevant to the high-volume fast-paced emergency department (ED) setting, where adverse events may be more likely and consequential. ^{4 , 5}

This article is the second of three in a series designed as a QI primer for emergency medicine (EM) clinicians. In the first article, we introduced QI methodology and strategic planning tools such as stakeholder analysis and engagement, Ishikawa diagrams, Pareto charts, and process mapping. ⁶ We used a fictional example of a project in your ED intended to improve the timeliness of antibiotics administration for patients admitted with septic conditions, and a quality gap was identified through chart audits. Key challenges that your core change team identified included the under-recognition of sepsis, communication problems between health care providers, and delays in patient assessment. In this article, we will build on this information and guide your team through the steps required to plan and execute a successful QI project.

The model for improvement

There are many frameworks available to guide QI work, including the well-known Lean and Six Sigma models and others developed by large organizations such as the Veterans Health Administration in the United States. ^{7 - 10} One of the most frequently used QI models in health care is the Model for Improvement (MFI), which the Institute for Healthcare Improvement popularized. ^{7 , 11} The MFI is the authors’ preferred model because of its simplicity, effectiveness, and clarity. The MFI revolves around three basic questions and four steps, as shown in Figure 1 and described below.

Figure 1 The Model for Improvement. Reproduced with permission from Langley et al. ⁷

AIM

The first of the three questions in the MFI is “What are we trying to accomplish?” In other words, what is the overarching goal of your project? A QI project without an aim would be akin to a research project without a hypothesis—it is highly unlikely to be successful. Your core change team must choose an aim that strikes a balance between setting an overambitious target, which may lead to disappointment if the project is unrealistic and fails, and setting such a low target that it fails to inspire stakeholders or achieve meaningful results. It is also crucial to limit the scope of the project to the level of influence of the project proponents (e.g., within the ED) as it is extremely challenging (and often poorly received) to change the way others deliver care. The good news is there is never a shortage of care processes to be improved within our own scope of practice.

The aim of your project must be clearly articulated and detailed, which is often referred to as a SMART aim—Specific, Measurable, Actionable, Realistic, and Time-defined. ¹² For your sepsis project, your team picks the following aim statement: “In six months, we will decrease the median time from triage to receipt of antibiotics for patients admitted with a diagnosis of sepsis from the current six hours to three hours.” This stretch goal (a 50% improvement) is likely realistic given the success of institutions in complying with this Surviving Sepsis Campaign guideline target and includes all components of a SMART aim. ¹³

MEASURES

The second question in the MFI is “How will we know that a change is an improvement?” Tracking select metrics before, during, and after your project is key to determining whether any progress has been made and if modifications are required along the way. It is important to determine who will collect these data and how, and to outline how the metrics will be reported to the core change team and front-line providers. The MFI includes three types of measures: outcome, process, and balancing measures. Patient-centred measures are those that are directly relevant to patients and should be prioritized whenever possible. They are not a separate category per se but can be a characteristic of any measure from the three other categories. One last type of measure that you may consider collecting is patient-reported measures (i.e., measures that patients report to the health care team). It is important to note that not all projects need to track all types of measures.

Outcome measures

Outcome measures report the impact of care on the health status of patients and populations, and these measures represent what truly matters to patients. For example, the mortality rate of patients with sepsis admitted from your ED would be an outcome measure (Table 1). Due to technical, logistical, and/or financial limitations, it is often difficult to obtain outcome measures in a reliable and timely way such that they can be used to inform ongoing QI projects.

Table 1 Measures and data collection methods chosen for your sepsis project

CPOE=computerized physician order entry; ED=emergency department; IV=intravenous.

Process measures

Process measures reflect what is done while providing and receiving care. Process measures, in general, are more feasible to collect than outcome measures, particularly within the timeframe required for rapid-cycle changes. Ideally, the process measures selected have evidence of a link to patient outcomes. In sepsis, for example, evidence supports that the timely administration of antibiotics (i.e., a process) is linked to a reduction in mortality (i.e., an outcome). ^{14 , 15} As such, your core change team selects the median time to antibiotics as a process measure of interest (Table 1). Another example would be the time from triage to bed placement that is directly linked to a challenging process of care that your team had previously identified.

Balancing measures

Balancing measures represent checkpoints that allow you to ensure there are no potential unintended consequences resulting from your change initiative. They represent what else has changed in your system that potentially could be associated with worse off care for some patients. For example, the rate of inappropriate antibiotic therapy in your ED would be a good balancing measure for your project because a focus on early antibiotic therapy may lead to inappropriate and widespread prescribing of antibiotics to patients with non-infectious or viral conditions (Table 1).

Patient-reported measures

Patient-reported measures—including patient-reported outcome measures and experience measures—are important, yet challenging, metrics to capture and interpret. ^{16 , 17} Research suggests positive associations exist among patient experience, clinical effectiveness, and PS. ¹⁸ Through follow-up surveys or interviews, it can be valuable to capture patients’ perspectives on the process or outcome of their care and to consider how this feedback should inform change ideas. If your core change team has enlisted the help of patient advisers, discussing this feedback with them can be of tremendous value.

CHANGE IDEAS

The third and final question in the MFI is “What change can we make that will result in improvement?” For most EM clinicians, who are often action-oriented individuals, this focus on ideas may seem to come too late in the whole process. However, as we discussed in the first article of this series, it is crucial to understand the current status of the system first, what the underlying issues are, and where we want to go before deciding how to get there. QI projects sometimes fall prey to a “let’s do something now” mentality, which results in well-meaning but often inadequate solutions that target the wrong problems.

Change concepts (e.g., reminders) are useful categories of approaches that can help your team identify specific change ideas suitable to your project (e.g., put an “Antibiotics for sepsis” sticker on all computers in the ED). Your team can look online for repositories of change concepts, which can trigger creative change ideas you would not have otherwise thought about. Both peer-reviewed publications and grey literature may inform change ideas that have been shown to work in other similar settings. Further, not all interventions are created equal, as described in the hierarchy of intervention effectiveness framework devised by the Institute for Safe Medication Practices. ¹⁹ Changes that are people-focused and rely on all individuals completing the same task the same way every time (e.g., education about a process), although important and often necessary, are less likely to be effective than changes that are system-focused and remove human variability from the equation (e.g., a computerized trigger). ^{20 , 21} Table 2 presents, in decreasing order of effectiveness, the various types of change concepts with their definitions and examples that your core change team could consider for your sepsis project. It is important to understand that the strategies at the top, while potentially the most impactful because they cannot be easily circumvented, are also the bluntest and therefore the most likely to have unintended consequences.

Table 2 Change concepts and change ideas, ranked in decreasing order of effectiveness

Note: Modified with permission from the Institute for Safe Medication Practices. ¹⁹

CPOE=computerized physician order entry; ED=emergency department; EHR=electronic health record.

THE PLAN-DO-STUDY-ACT CYCLE

Now that your core change team has determined the aim of its QI project, the measures that will be collected, and the initial changes that will be tested, you are ready to test your team’s hypotheses to learn how your change ideas impact the local environment. The MFI uses a four-step cycle to achieve this that is commonly referred to as the Plan-Do-Study-Act (PDSA) cycle. ⁷ The PDSA cycle revolves around small tests of change whereby you trial different change ideas and improve them iteratively until the desired change is achieved and sustained in your system.

Plan

During the planning stage, your team will refine the change ideas/interventions and develop a theory as to why they should work in your local environment. As with any research method, in QI, you need to test your hypothesis empirically to understand what did and did not work (the difference being in clinical epidemiologic trials, you often have one fixed hypothesis for an entire project, whereas in QI, you develop a new hypothesis for each new PDSA cycle). Your team needs to plan carefully the specific modalities of the test of change (i.e., what, when, where, who, and how), the indicators you will measure to identify success and failure (which may differ from your project measures identified above), and for how long you will run the test. Health Quality Ontario provides a useful worksheet template that can help your team think of the practical steps necessary to test your idea. ²²

Do

Teams often want to plan tests of change down to the most intricate detail, which can delay action substantially. It is often more effective to try out something imperfect in the real world instead of trying to design the perfect plan. The learning opportunity afforded by a practical test with suboptimal results will often lead to more constructive improvements than spending additional time planning an intervention. The front-line workers must conduct the tests of change because it is the end-users who will inform design, testing, and sustainability. It is also important to document any problems and unintended consequences that arise, as this will help team members reflect on what happened over time.

Study

The third step of the PDSA cycle is to study. This entails analyzing the data you collected and interpreting the results with your core change team. It is important to compare your results against your hypotheses and reflect on if and why they differ. A discussion in which all team members are encouraged to challenge the status quo and ask hard questions—such as whether any given change idea should even proceed based on the resources required to test it—will help inform subsequent steps and improve the project. In the third article of this series, we will discuss measurement tools such as run charts that can help you track the results of your QI project.

Act

The fourth step in the PDSA cycle is to act. Based on the information gathered as part of the first PDSA cycle, your team must determine how the test of change will be refined for the next PDSA cycle, which you should start as soon as possible to maintain momentum. In some cases, this could mean scaling a project from one doctor, one nurse, and one patient to five of each, then to 20 of each, and then to everyone, all the time. After a few tests of change, your intervention usually starts to feel mature; you may be ready to implement your change idea permanently. Sustainability becomes your next challenge, and ongoing data collection and monitoring can help you see whether your success lasts.

YOUR SEPSIS PROJECT IN ACTION

Before engaging in its first PDSA cycle and refining the change ideas, your core change team researches similar projects on antibiotics and effective interventions in sepsis. You then rank the interventions based on a hierarchy of effectiveness and the likelihood of success in your ED. For some of the most important elements contributing to delayed antibiotic therapy for patients with sepsis previously identified in your Pareto chart (see the first article of this series), you identify the following interventions:

1. 1. Education intervention: Improve nurses’ and physicians’ recognition of sepsis through education sessions at nursing huddles and physician rounds.

2. 2. Standardization intervention: Accelerate the processing of stat physician orders for patients with sepsis using a new standardized flagging system for such orders.

3. 3. Policy intervention: Engage the ED leadership in developing a policy to improve communication between nurses and physicians about which patients to see next and to mandate routine communication between nurses and physicians about sicker patients.

4. 4. Computerization intervention: Partner with the information technology (IT) department to create a computerized intervention that flags and preferentially places patients with severe sepsis in the next available stretcher.

Practical example of a PDSA approach

Planning a full PDSA cycle requires a good amount of work and commitment, but the same approach can be used to solve smaller questions that inevitably arise in various change initiatives. Using the above standardization intervention example regarding a flagging system to identify STAT sepsis orders, your group begins with its first change idea: you ask the physicians to place a yellow Post-it^® note on the top of charts with STAT orders (Plan #1). One week into this trial (Do #1), you notice the Post-its^® are being used all over the department to leave clinical notes to colleagues (Study #1), thereby nullifying the effect of the intervention (Act #1). Your team, therefore, creates an orders box for STAT orders that is placed next to the regular orders box (Plan #2). However, another week into this test of change (Do #2), an anonymous audit of the ED reveals that most physicians use the STAT orders box for any orders they deem important, despite not being urgent or pertaining to sepsis (Study #2). After debating the next best steps with your core change team (Act #2), you decide to provide education on the appropriate uses of the STAT orders box to the physician group given that it had never been made clear to them initially (Plan and Do #3). Upon surveying the nurses the following week, you learn they have seen a significant improvement in the appropriate utilization of the STAT orders box by the physicians, but they themselves feel conflicted about the prioritization of STAT orders when other important orders have been waiting for much longer (Study #3). Your team therefore decides to engage the nursing leadership more formally in clarifying expectations from a nursing perspective (Act #3). As a result of your iterative PDSA cycles, your team is encouraged by the improvements seen in the turnaround time of sepsis orders and plans to build on this success for its next steps.

CONCLUSIONS

The MFI provides a simple and useful framework for tackling QI issues. The first three questions pertain to the aim of the project that should be Specific, Measurable, Actionable, Realistic, and Time-defined (SMART); to the measures that will be collected (outcome, process, and balancing); and to the change concepts and ideas that can help drive the project, keeping in mind the hierarchy of intervention effectiveness. The MFI also uses PDSA cycles to help improve small tests of change iteratively until they are ready for long-term implementation. It is always important to define the theory behind the change, which often provides a first test of likely success, allows for comparison with actual results, and helps inform next steps. In the next and final article of this three-part series, we will present tools that can be used for evaluating and monitoring QI projects and methods used to sustain improvement.