Validity of root cause analysis in investigating adverse events in psychiatry

Summary

Root cause analysis (RCA), imported from high-reliability industries into health two decades ago, is the mandated methodology to investigate adverse events in most health systems. In this analysis, we argue that the validity of RCA in health and in psychiatry must be established, given the impact of these investigations on mental health policy and practice.

Safety in healthcare has been an area of much interest and debate ever since the publication of the Institute of Medicine's report To Err is Human: Building a Safer Health System ¹ in the USA. This was followed by establishment of the National Patient Safety Agency in England in 2001.² The World Health Organization³ declared patient safety in healthcare to be a global priority. There has also been an increasing interest in understanding patient safety in mental healthcare.^4,5

A key area of focus in the above was the need to learn from individual adverse events in healthcare by carrying out detailed structured investigations. Root cause analysis (RCA), widely used in industry as the investigative methodology of choice to examine safety events, was adopted in healthcare in the 1990s⁶ to investigate patient safety events in nearly all major healthcare systems, and is mandated in many, including in the UK,^7,8 USA⁶ and Australia.⁹

RCA was originally developed in systems engineering and psychology to identify causal factors underlying variation in performance.^10–12 It is described as a range of approaches and tools used to identify how and why an adverse event occurred, in order to understand how to reduce recurrence of that type of event in future,^13,14 and is intended to be a general approach to uncovering system-level causes and contributory factors behind an adverse event or a near-miss.¹⁵ A typical RCA investigation leads to the identification of root causes and contributory factors that are expected to have either directly caused the adverse event or made a substantial contribution to it.^16,17

This article summarises the existing literature on the use of RCA in health and in psychiatry, and examines the importance of establishing the validity of this investigative methodology.

Use of RCA in health

A review of the existing literature has identified no papers directly addressing the validity of RCA in adverse-event investigation in health or in psychiatry. Most authors have focused on the lack of evaluation of effectiveness of RCA in health, especially with reference to recommendations and action plans,^12,17–22 which are the output of RCAs. Several authors have considered whether RCA is the best method to investigate adverse events in health.

One of the earliest reviews questioning the widespread and somewhat uncritical adoption of RCA as the investigative methodology of choice for adverse events in healthcare was by Wu et al (2008).¹⁸ They argued that although RCA had been adopted widely in the USA, there had not been an evaluation of its effectiveness in healthcare. Their focus was on acute hospital-based medicine and they noted that at the time there were no studies in the peer-reviewed literature of the effectiveness of RCA to reduce risk or improve patient safety, nor were there evaluations of cost-effectiveness compared with other tools to mitigate hazards. The authors noted that there were no nationally agreed definitions of adverse events, reporting requirements, guidance for conducting these investigations, or a central process for quality assurance of reports or monitoring of the implementation of recommendations or actions.

They also noted the resource-intensive nature of RCA, with each investigation taking 20–90 person-hours to complete. They acknowledged that the investigation of an adverse event in medicine based on RCA is best suited to incidents related to medical devices and manufactured products, such as medicines, and less so for problems related to performance and communication. They concluded that there was lack of evidence of effectiveness, utility and cost-effectiveness of RCA, and recommended that RCA be evaluated for effectiveness and utility in healthcare. However, the authors did not address the issue of validity, nor the use of RCA in mental health services.

In a review of use of RCA in acute hospital-based medicine, Trbovich et al (2017)²³ contended that because of wide variation in the quality of investigations and recommendations of RCAs, they inevitably failed to explore ‘deep system problems’, resulting in recommendations and corrective actions primarily aimed at changing human behaviour rather than system-based changes.

The authors noted that critical incidents arise from the interplay between active failures and latent conditions, and investigations and corrective actions should focus on latent conditions that make active failures more likely. They noted that RCA teams tend to focus on the first causal factor identified, rather than considering these factors as part of a sociotechnical system (i.e. the interaction of people and technology embedded in an organisational structure), thus leading to recommendations and corrective actions that do not address fundamental problems. They emphasised the importance of identifying effective corrective actions that are aligned to causal factors, acknowledging the reluctance to identify solutions requiring substantial financial resource.

Referencing James Reason, who developed the ‘Swiss Cheese model’ of error causation,²⁴ they characterised the goal of error investigation as ‘draining the swamp, not swatting mosquitoes’. The authors concluded that making system improvements is difficult and expensive, but to realise the promise of RCA, health systems had to be prepared to fund teams with expertise in human factors and safety signs.

This review did not address the issue of validity or the differing issues in mental health services. The authors acknowledged the inherent limitations of a retrospective analysis of this type, including hindsight bias; the difficulty of ascertaining to what extent contributing factors actually caused the adverse event; and other factors that may have been missed, especially those that shape behaviour, such as task complexity and flawed workflow, which enabled the adverse event to occur. They also noted the lack of evidence of effectiveness and recognised that recommendations and actions do not seem to improve safety.

Kellogg et al (2017)²⁵ carried out a quantitative and qualitative retrospective analysis at a single acute hospital in the USA. They examined 302 RCA investigation reports completed over the previous 8 years, specifically the summary description of the incident, recommendations and action plans to determine a taxonomy of solution-types proposed by RCA teams. They found that the most common proposed solution types were training, process change and policy reinforcement, all of which are potentially weak solution types, as these failed to address the most basic premise of safety engineering involving sociotechnical systems, which is that human errors will always be repeated, and it is therefore the system that needs to adapt.^11,26 Solution types such as technology changes and institutional-level changes, which are more effective in bringing about sustainable change, were not proposed as frequently. The authors argued that this was evidence of the lack of effectiveness of RCA.

They concluded that the current approach of healthcare systems to RCA was contributing to failures in improving patient safety, and that RCA teams need guidance on solution types in order to improve the effectiveness of RCA. However, the authors did not include mental health services in their review and focused on identifying ways of improving the effectiveness of RCA by providing guidance to RCA teams on developing better recommendations and action plans. They did not comment on the validity of RCA in healthcare.

Vrklevski et al (2018)²⁷ evaluated the use of RCA for investigating and responding to adverse events in a large metropolitan mental health service in Australia. Using a mixed-method approach, the authors examined 26 RCAs (linked to 21 suicides, 3 homicides and 2 deaths from other causes), focusing on the type of ensuing recommendations and the time to implementation. They found that recommendations were ‘weak’ in 71% of cases. Weak recommendations are those that require a change in human behaviour, which when used alone are unlikely to affect a sustained improvement in safety.²⁸ By contrast, strong recommendations in this context rely less on human behaviour and memory, and are likely to be effective and sustainable. The review found that only 65% of recommendations were implemented within 12 months, and no root cause was identified in 70% of incidents. They noted a lack of fidelity to the original RCA model, with much variability in how investigations were completed.

Semi-structured interviews with mental health staff who had completed RCA reviews were undertaken, to identify ways in which RCA processes could be strengthened: these found low engagement of clinicians in RCA, although participants felt that RCA was useful as it provided a formal, structured process to review adverse events. The authors noted that RCA is better suited to medical incidents that occur with moderate to high frequency, but less so for rare events or those that involve complex interactions of human variables that are difficult to control, such as homicides and suicides. They concluded that this investigation enhanced understanding of why recommendations from investigations on adverse events in mental health services are not always implemented, while acknowledging that although the RCA model offers a formal and systematic approach to the review of serious critical incidents in mental health, it is not the model of best fit. However, the authors did not comment directly on concerns relating to validity.

A review by Peerally et al (2017)¹³ identified eight challenges with the use of RCA in healthcare. The authors felt that RCA does have face validity in health, but did not specifically refer to validity in mental health services. They reflected that the name encouraged a flawed reductionist view by suggesting the existence of a single root cause or a small number of root causes. Like other authors, they too noted the significant variation in the quality of RCA investigations, and felt the lack of independence, need to preserve reputation, and resource and time constraints meant that the process was vulnerable to political highjack.

They noted the lack of effective recommendations arising from RCA investigations, leading to weaker solution types and failure to reduce recurrence and increase safety. Poor feedback to clinical teams involved was found to contribute to lack of double-loop learning methods needed for sustained improvement in safety. They noted the tendency for RCA investigations to remain within organisations, leading to lack of system learning; the temptation to assign blame to individuals or teams for the adverse events, and the difficulty of accounting for the role of many actors (human, process and environmental) that lead to an outcome. The authors concluded that RCA does have benefits in healthcare and considerable face validity, and made recommendations to improve its use.

Hagley et al (2019)²⁹ examined a number of alternatives to RCA. They identified seven methodologies, which included:

1. (i) after-action review;

2. (ii) adverse event debriefing and huddles;

3. (iii) learn from defect tool;

4. (iv) SWARM;

5. (v) concise incident analysis;

6. (vi) comprehensive frameworks for incident report investigation and analysis; and

7. (vii) aggregate RCAs and multi-incident analysis.

The authors provided a comparative analysis of these approaches with reference to the time taken for the tool to be administered and resultant improvement in safety.

The authors noted that the best methods to investigate adverse events remain unclear, as all tools have trade-offs, and that the appropriate investigative framework may vary depending on the context of the healthcare organisation.

This review²⁹ did not comment on the validity of any of these tools, which still fundamentally rely on some variation of RCA to identify why an adverse event occurred, and devise recommendations and actions based on identified causes. It was not specific to mental health services, and it is not clear whether any of the seven tools examined were used in mental health services.

Conclusion

The retrospective investigation of adverse patient safety events to understand causation and reduce recurrence is accepted as an important way of improving patient safety in health services. Despite differences in the architecture and delivery of healthcare systems in different continents there are striking similarities in approaches to undertaking such investigations. RCA has been adopted wholesale by most health services, including mental health services, across the world as the investigative methodology of choice.

Most research has focused on the effectiveness of recommendations and corrective actions identified by the RCA and found that these do not result in improved safety for patients, as similar adverse events continue to occur. This observation has led authors to conclude that investigators have not used RCA correctly, or that they need more guidance to formulate the right recommendations and corrective actions. Some alternatives have been implemented, but each of these approaches invariably defaults to using a simplified version of RCA.

This literature search shows there has been no specific examination of the validity of RCA in healthcare. The use of RCA is likely to be especially problematic in mental health services where adverse events occur as a result of complex interplay of people, including the patient, the local clinical and domestic environments, and the wider healthcare system. Retrospective investigation of such incidents using RCA typically assumes linear causality, where one event is assumed to lead to another, and where the agency exercised by patients, the role of counterfactuals and the impact of systemic factors such as resource are neglected. Such investigations typically identify one or more root causes and a number of contributory causes, but their actual role in causing the adverse event cannot be tested empirically. Investigations themselves are complex and often do not yield lasting improvements to patient and practitioner safety. This has implications for policy and practice.

As several authors^30–33 have noted, such investigations serve many different explicit and implicit functions, including reaching an understanding of why the adverse event occurred to reduce future recurrence; providing a narrative to those involved, especially the patient and/or the family; attributing responsibility, including blame; providing assurance to provider organisations, commissioners and regulators; and perhaps, above all, showing that the health system learns from such adverse events. In addition, there is a wish for learning that is generally applicable from such an examination of individual events. But can all these functions realistically and coherently be discharged by a methodology such as RCA, especially in mental health, where safety relies little on equipment and process and much more so on human interaction and behaviour? This review would suggest not.

This review is the first stage of an in-depth examination of methodology used in investigations of adverse events in mental health services. It indicates that the validity of RCA in mental health investigations has received little attention, and this is necessary to ensure that we focus our efforts on identifying the true causes of adverse events and thereby developing appropriate responses.