A clinical audit of patient safety in an Irish radiotherapy department

Abstract

Introduction:

Radiotherapy is an ever-changing field with constant technological advances. It is for this reason that risk management strategies are regularly updated in order to remain optimal.

Methodology:

A retrospective audit of all reported incidents and near misses in the audited department between 1 November 2020 and 30 April 2021 was performed. The root cause of each radiotherapy error (RTE), safety barrier (SB) and the causative factor (CF) would be defined by the Public Health England (PHE) coding system. The data will then be analysed to determine if there are any frequently occurring errors and if there are any existing relationships between multiple error.

Results:

670 patients were treated during the study period along with 35 reports generated. 77·1% (n = 27) were incidents, and 22·9% (n = 8) were near misses. 2·8% (n = 1) were reportable incidents. The ratio of RTEs to prescriptions was 0·052:1 (5·2%). 37% of RTEs were associated with image production. Slips and lapses were involved in 54·2%. Adherence to procedures/protocols was a factor in 48·5% (n = 17). Communication was a factor in 11·4% (n = 4).

Discussion:

The proportion of Level 1 incidents was higher in this department (2·8%) than in the PHE report (0·9%). Almost one-third, 31·4% (n = 11) of errors stemmed from one technical fault in image production. SB breaches were prevalent at the pre-treatment planning stage of the pathway. A relationship between slips/lapses and non-conformance to protocols was identified.

Conclusion:

The rate of reported radiotherapy incidents in the UK is lower when compared with this department; this could be improved with the implementation of the quality improvement plan outlined above.

Introduction

Radiation oncology is one of the pillars of cancer treatment. Ionising radiation is utilised to treat tumours in both radical and palliative settings. Radiotherapy is an ever-changing field with constant technological advances. It is for this reason that risk management is frequently reviewed in order to remain optimal. Guidelines and regulations have been recently advanced by the European Commission since the completion of the ACCIRAD project.

The ACCIRAD project set out to assess the position of radiotherapy safety and its regulation within European countries along with the Level of compliance with regulation. This project aims to promote harmonisation between different European centres and essentially improve patient safety. ¹ The recommendations put forward by the ACCIRAD project are as follows.

• First, legislation governing radiotherapy safety and risk assessment should be updated in accordance with the latest scientific evidence (including RP-181).

• Second, a specific methodology (i.e., designed for use in radiotherapy) should be used to assess quality and manage risk.

• Third, a protocol should be established to ensure wide dissemination of all relevant risk management information.

• Fourth, all employees who work with radiotherapy should receive specialised training in risk management and safety.

• Fifth, public reporting of adverse events and near misses is essential. Sixth, clinical audits should be performed routinely to promote safety.

• Finally, regulatory bodies should perform regular inspections of radiotherapy facilities to ensure adherence to all relevant regulations.

As part of the EU, Ireland’s legislation governing radiotherapy safety falls under RP-181· ² A common taxonomy for specific radiotherapy-related events is key in enabling an accurate comparison to be created between departments. Radiotherapy errors (RTEs) have the potential to cause serious harm to patients therefore reporting accuracy is of paramount importance. Platforms such as SAFRON and ROSIS provide a common method of public reporting and risk management along with learning systems. ^3,4 The Health Information and Quality Authority is the independent regulatory agency responsible for adherence to regulations in Ireland, and they perform regular inspections of departments throughout the country. ⁵

The ACCIRAD project provides a general classification for RTEs (Figure 1) and guidelines on improving reporting within a department (Figure 2).

Figure 1. ACCIRAD classification of RTE level.

Figure 2. ACCIRAD recommendations on improving reporting.

Relevance

This project is a clinical audit, specifically it is a reactive analysis of errors that occurred in an Irish hospital’s radiotherapy department over a 6-month period. By carrying out a clinical audit, we are fulfilling the 6^th recommendation from the ACCIRAD project and therefore promoting patient safety. Furthermore, it is well documented that a clinical audit is the most appropriate method of assessing events and overall improving safety within a department. ⁵

The results of this audit may highlight stages of concern with the radiotherapy process and potentially lead to improvements in risk management within the department. Nevertheless, the audit will serve as an additional safety net for risk management within the department and potentially promote event reporting and the safety culture within the department.

Aims and objectives

• To gather all incident/near miss reports within the department over a 6-month period.

• To determine the percentage of reportable incidents and the ratio of RTEs to prescriptions during the studied period.

• To determine the cause of each RTE and categorise the cause appropriately using the same taxonomy as Public Health England (PHE).

• To correlate and categorise the data using Microsoft Excel to assess if any trends or patterns are present.

• To develop a Quality Improvement Plan to rectify any issues and create a plan for a follow-up audit.

Methodology

This is a retrospective clinical audit of reported incidents and near misses over a random 6-month period in one radiotherapy department. The studied period was between 1 November 2020 and 30 April 2021.

This purpose of this audit is to answer the question; ‘How does patient safety in an Irish radiotherapy department compare with those in the UK?’

The guiding principle with regard to radiation exposure is ‘As low as reasonably achievable’ or ALARA. ⁶ There is no acceptable standard on radiation incidents defined by HIQA.

PHE carry out national audits of radiotherapy errors (RTEs) and near misses over 2-year intervals in the UK and compares them with their previously reported results. Their most recent report entitled ‘The Public Health England Biennial Radiotherapy error data analysis and learning report: January 2018–December 2019. (Report No. 6)’ will be used as a standard to compare the results of this audit. Specifically, the percentage of patients who had a reportable ‘Level 1’ incident occur during their treatment will be used as a standard for this audit. 0·9% of patients involved in the PHE study had a reportable incident occur; therefore, this is the standard that this audit will be compared against.

As per HIQA, under Regulation 17(1)(e), significant events are defined by the competent authority. ⁷ In the UK and as per the PHE report, the incident Levels can be defined as the following:

• A reportable incident is any incident that falls under the reportable category of any of the Ionising Radiation Medical Exposure Guidelines 2017 (2)R). ⁸ These are generally clinically significant (e.g. an overdose of 20%); however, they may be correctable by reducing doses of future fractions.

• A non-reportable incident (Level 2) may be clinically significant but in the form of an underdose, which is not reportable under IR(ME)R.

• A minor incident (Level 3) is an incident in the technical sense but of no actual clinical significance.

• A near miss (Level 4) is a potential radiation incident that was detected and prevented before treatment delivery.

• A non-conformance (Level 5) is a non-compliance with any aspect of a documented procedure but having no direct effect on radiotherapy delivery. ⁹

An illustration of the levels of RTE can be found in Figure 3.

Figure 3. Classification of level of RTE.

Data collection

The deputy manager of the department made available the necessary data on the patients treated within the study period. These reports were anonymised and were only accessible on a computer located within the department. The files were also password-protected. The patients involved and their hospital numbers were not made available to the auditor. The data available on each report included the date and time of the incident, the type of incident (e.g. over dose/under dose), the patient diagnosis, and a description of the error written by a clinical specialist at the time of the incident/near miss.

The taxonomy system used to code each RTE is the Development of Learning (DOL) taxonomy, developed by The Institute of Physics and Engineering in Medicine, The Royal College of Radiologists and The College of Radiographers. ¹⁰

The DOL taxonomy is a refined coding pathway that has been developed to reflect radiotherapy practice. The codes include safety barriers (SBs), which are also known as critical control points, and their primary function is to detect and prevent RTEs. There is agreement that 40% of workflow steps in radiotherapy are SBs. This means that of all the tasks involved in the radiotherapy process, 40% are specifically focussed on the detection and prevention of RTEs. ¹¹ There is a complexity to assessing errors as the chain of events preceding an RTE, which may also influence an event occurring. This is therefore the argument for having a depth of defence where multiple SBs are in place for critical steps. ¹² The DOL approach focuses on identifying where along the radiotherapy pathway the RTE took place along with the SBs, which failed to prevent the RTE from occurring. This is a modified version of the taxonomy by Ford et al· ¹³

The DOL taxonomy also includes causative factor (CF) taxonomy which enables finding the root cause of a variety of different issues. CFs associated with the individual are the determining factor of 70%–80% of errors in medicine. ¹⁴ Procedural factors are due to failing to follow protocols/procedures or having an inadequate protocol in place. Technical factors relate to equipment used which directly leads to an error such as IT network or equipment failure. Patient-related factors relate to individual actions or circumstances of the patient which directly led or contributed to the error. Teamwork or managerial factors transcend all levels of management, including all those at an operational level up to senior management. Elements of this include inadequate leadership, staffing and resources, along with clarity on responsibilities. Finally, there are always factors that may not directly fit the taxonomy of the above-mentioned areas; however, these are rare.

Data analysis

Each report was reviewed independently. All quantifiable data included in the reports were entered into a Microsoft Excel spreadsheet which was located on the same computer and password-protected. The description section of each report was reviewed by a radiation therapist with experience within the department and was used to identify every code in the DOL taxonomy which applied to the report. This was further checked by the deputy manager to ensure accuracy. Microsoft Excel was used to collate the results and to highlight the patterns of SB breaches and relationships between SB breaches and CFs.

Ethics

Ethical approval was granted by the Clinical Research Ethics Committee of the Cork Teaching Hospitals. ¹⁵ CREC Review Reference Number: ECM 4 (k) 11/1/2022.

Results

This audit included 670 patients who were prescribed and treated with radiotherapy during the 6-month study period which ran between 1 November 2020 and 30 April 2021. During the course of their treatment, 35 incident/near-miss reports were generated. Of the 35, 77·1% (n = 27) were incidents and 22·9% (n = 8) were near misses. Of the 35 reported, 80% (n = 28) involved radical patients and 20% (n = 7) involved palliative patients. On further classification of the incidents, 2·8% (n = 1) were reportable incidents, 71·4% (n = 25) were minor radiation incidents (Level 3) and 25·7% (n = 9) were near misses (Level 4). There were no Level 2 or Level 5 reports.

The treatment unit was the most common place for an incident/near miss to be detected with 82·8% (n = 29) incidents/near misses detected there. This is followed by ‘Radiation Therapist Checks’ which accounted for 14·2% (n = 5) and ‘CT Simulation’ with 2·8% (n = 1). With regard to the types of reports that were generated, 51·4% (n = 18) of the reports involved the patient being overdosed. 22·8% (n = 8) were set-up errors, and the remaining 25·7% (n = 9) were classified as ‘other’ and were mostly comprised of near misses. These results are illustrated in Figures 4–7.

Figure 4. Incidents per classification.

Figure 5. Incidents per intent.

Figure 6. Incidents per level.

Figure 7. Incidents per type.

The most common source of minor incidents was during the treatment unit process, specifically in relation to onset image production (37·1%) (n = 13). Of the image production incidents, 84·6% (n = 11) were due specifically to the Cone Beam CT (CBCT) kV panel drop error, while 15·4% (n = 2) were due to patient set-up errors requiring an additional CBCT to be acquired.

There was a high proportion of SB breaches within the pre-treatment planning process with 20% (n = 7) of the reports involving a SB breach in this part of the pathway. Of these 7, 42·9% (n = 3) led to incidents on the treatment unit, and 57·1% (n = 4) were near misses picked up at the treatment unit process. Furthermore, there was a pattern of SB breaches within the pre-treatment planning phase with 14·2% (n = 5) of the reports involving two or more SB breaches within the pre-treatment planning phase. Figure 8 illustrates the frequency of individual errors including SB breaches.

CFs were primarily ‘slips and lapses’ along with ‘adherence to procedures/protocols’. There was also a strong correlation between these factors. This can be explained as a lapse in concentration leading to a failure to complete a check, therefore not adhering to a protocol. Slips and lapses were involved in 54·2% (n = 19) of RTEs. Adherence to procedures/protocols was a factor in 48·5% (n = 17) of RTEs. Additionally, equipment failure was a factor in 37·1% (n = 13) of incidents. Communication was a factor for 11·4% (n = 4). Failure to recognise hazards, device/product design and inadequate training were all factors in single incidents 2·9% (n = 1). Figure 9 illustrates the frequency of individual CFs.

With regard to the single reportable incident involved in this study, there were three SB breaches involved spanning the referral for treatment phase, the pre-treatment planning phase and the treatment data entry process. The CFs involved were down to individual and procedural factors.

Figure 8. Radiotherapy pathway codes including safety barriers.

Figure 9. Causative factors.

Discussion

With regard to the PHE Biennial Radiotherapy error data analysis and learning report: January 2018–December 2019 (Report No. 6), there were a total of 18,853 reports available for analysis as this comprised 98·3% of UK radiotherapy providers over a 2-year interval. This allows for a very accurate interpretation of the results. For the purpose of this audit, the relevant figure from the study is the percentage of reportable radiation incidents (Level 1) reported, which was 0·9%. The study also described the percentage of Level 2 reports which were also 0·9%, Level 3 reports, 35·9%, Level 4 reports, 24·4% and Level 5 reports, 37·9%.

With regard to the results of this audit, 2·8% of the reports were reportable incidents. The percentage of Level 1 reports could be higher due to the smaller sample size in this study. However, this may also be due to the lack of non-conformance (Level 5) reporting. If there had been more non-conformances reported, then this figure would lower the ratio of Level 1 reports.

The PHE report also states that they had an estimated 45 RTEs per 1,000 prescriptions. This gives a ratio of 0·045 (4·5%) which is slightly lower than that of this audit which had 35 RTEs/670 prescriptions, giving a ratio of 0·052 (5·2%). This indicates that there are more reports per patient treated in this department than in the PHE report despite the lack of non-conformance reports.

On inspection of the results, almost one-third, 31·4% (n = 11) stemmed from a single technical fault in relation to image production. These were all minor radiation incidents (Level 3). The root of all these errors is a CBCT kV panel drop error that interrupts CBCT scans part way through acquisition and requires a CBCT to be reacquired. This causes over-exposure of patients to radiation. The additional dose received by the patient may not have any clinical effect but subclinically it may be harmful to healthy tissue. The long-term effects of low-dose radiation on tissue are stochastic; however, it remains appropriate to address the issue to comply with the ALARA principle. ⁶ Input from the linear accelerator manufacturers and/or the physics department will be required to find the root of this issue. The PHE report also states that the majority of its Level 3 reports were in relation to image production. By solving this technical issue alone, the percentage of minor radiation incidents (Level 3) would be reduced by 44%, bringing it below that of the PHE report.

There was a pattern of SB breaches in the pre-treatment planning process. Specifically, the pattern initiates with an error during the verification of plan checks and progresses to other issues. From the results of this audit, none of the RTEs from the pre-treatment planning process were identified before proceeding to the data entry process and/or treatment unit process. There were seven reports involving a SB breach during the pre-treatment planning process, five of which were two or more SB breaches in the planning process alone. This is the only process outlined in this audit where two or more SBs were breached during the same process. Therefore, this is a key area where more emphasis on process checks is required; this process may potentially benefit from an additional SB. ¹⁶

There was a relationship between slips/lapses and failure to adhere to procedures and protocols. Slips and lapses are execution errors. They result from failures in the execution and/or storage stage of an action sequence. Slips relate to observable actions and are commonly associated with attentional or perceptual failures. Lapses are more internal events and generally involve failures of memory. ¹⁷ It is difficult to determine the cause of these more specifically due to the variety of potential factors which may precipitate a slip/lapse. Environmental factors such as distractions and workload may also play a role. It should be highlighted how a slip/lapse in concentration at the point of administering treatment has no further SBs to prevent an RTE. Preventing human errors is a huge aspect of safety in medicine. Treatment information systems (TIS) such as MOSAIQ and Aria have a purpose to minimise the potential for human error. One aspect on which the departmentʼs TIS could be improved is where there is a couch rotation of 1 degree off the set point. The TIS will not currently highlight this during a CBCT scan but will before treatment. This causes a set-up error and requires another CBCT. These errors could be prevented by the TIS.

Communication was a factor involved in 11·4% (n = 4) of incidents; this is another element of human error. The majority of communication in a radiotherapy department is in the form of written notes in patient charts on the treatment information system. With regard to the RTE in this audit, these were mostly in relation to the failure to create the prompt known as a quality checklist (QCL) to trigger the next part of the patient pathway. When no QCL is created, a check can be missed, leading to delays for a patient starting treatment or a gap in treatment between two phases of treatment, for example. The treatment information system automatically generates the next QCL; there are occasionally checks which vary from the norm and therefore rely on staff to input. This is where these incidents have arisen. As mentioned earlier, it is difficult to determine exactly what the root cause of individual error is due to the vast amount of potential causes.

Device/product design was also a factor, specifically in Surface Guided Radiotherapy (SGRT) DICOM selection. The error of positioning a patient to an incorrect DICOM is a human error currently, as the preselected DICOM defaults to the previously used DICOM, which may be related to a new treatment plan opened by the planning staff. The common error is using a DICOM intended for a future phase of treatment for a current one, leading to a CBCT scan which will have to be re-acquired after using the correct DICOM. Automatically linking the treatment plan on the treatment information system to the SGRT DICOM would be a potential way of minimising this error. This requires an updated version of the SGRT system, which has been implemented since the data included in this audit was collected.

Reassuringly, there were no non-reportable radiation incidents (Level 2). However, there were no non-conformances (Level 5) reported either, yet this is unlikely to reflect reality. Reporting of non-conformances should be looked upon as a positive action with emphasis placed upon anonymity if necessary. Without data on non-conformances, it is impossible to determine commonly occurring issues that ultimately may lead to a more serious error. Using Heinrich’s triangle theory, for every one serious event, there should be 30 minor events and 300 unreported occurrences. ¹⁸ While these numbers may not reflect radiotherapy incidents, they do show that there should be a higher proportion of minor incidents such as non-conformances than minor radiation incidents. This is reflected in the PHE report also and in the Medical Exposure Radiation Units report in 2016· ¹⁹ Therefore, more effort should be placed into detecting and reporting non-conformances within the department. Figure 10 illustrates the frequency of each level of incedent comparing between this department and the PHE report.

Figure 10. Report level by percentage.

Quality improvement plan

• Resolve issue causing CBCT kV panel drop error.

• Improve reporting of Non-Conformances (Level 5 errors) by using ACCIRAD recommendations (Figure 2).

• Improve pre-treatment planning safety checks +/− additional SB in this process.

• Resolve issue with SGRT DICOM selection by altering protocols/procedures.

• Improve conformance with regard to QCLs.

• Resolve issue with TIS causing errors in couch angle positioning.

Re-Audit

The quality improvement plan should be implemented as soon as possible as at least 6 months will have to pass after this in order to compile enough data to re-audit. A longer audit period would also be recommended in order to allow a more accurate interpretation of the results.

Conclusion

In conclusion, 2·8% (n = 1) of the reports generated were reportable incidents, higher than that of the PHE report (0·9%). The ratio of RTEs to prescriptions was also higher 0·052:1 (5·2%) in this study compared to 0·045:1 (4·5%) in the PHE study. Therefore, patient safety appears slightly better in the UK compared with this department; however, this could be improved using the quality improvement plan outlined previously.