Can the use of knowledge-based planning systems improve stereotactic radiotherapy planning? A systematic review

Abstract

Introduction:

This study aimed to systematically review the literature to synthesise and summarise whether using knowledge-based planning (KBP) can improve the planning of stereotactic radiotherapy treatments.

Methods:

A systematic literature search was carried out using Medline, Scopus and Cochrane databases to evaluate the use of KBP planning in stereotactic radiotherapy. Three hundred twenty-five potential studies were identified and screened to find 25 relevant studies.

Results:

Twenty-five studies met the inclusion criteria. Where a commercial KBP was used, 72.7% of studies reported a quality improvement, and 45.5% reported a reduction in planning time. There is evidence that when used as a quality control tool, KBP can highlight stereotactic plans that need revision. In studies that use KBP as the starting point for radiotherapy planning optimisation, the radiotherapy plans generated are typically equal to or superior to those planned manually.

Conclusions:

There is evidence that KBP has the potential to improve the quality and speed of stereotactic radiotherapy planning. Further research is required to accurately quantify such systems’ quality improvements and time savings. Notably, there has been little research into their use for prostate, spinal or liver stereotactic radiotherapy, and research in these areas would be desirable. It is recommended that future studies use the ICRU 91 level 2 reporting format and that blinded physician review could add a qualitative assessment of KBP system performance.

Introduction

Stereotactic radiotherapy is a technique that allows the ablation of both primary and metastatic disease using planning, immobilisation and delivery techniques that enable precise and accurate radiation delivery. It can be categorised into the following ¹ :

• Stereotactic radiosurgery (SRS) as a treatment of malignant (e.g., metastatic tumours) or benign tumours (e.g., meningioma) intracranially, as well as functional or vascular disorders (e.g., arteriovenous malformations) with a single fraction of radiotherapy.

• Fractionated stereotactic radiotherapy of intracranial malignant or benign tumours and functional or vascular disorders.

• Stereotactic body radiotherapy (SBRT or SABR) of extracranial malignant or benign tumours and functional or vascular disorders.

Stereotactic radiotherapy could be delivered on traditional or specialist linear accelerators (Linacs) or via dedicated stereotactic platforms such as CyberKnife™ (Accuray Inc., Sunnyvale, CA) or Gamma Knife (Elekta AB, Stockholm, Sweden). On Linacs, the delivery may be by intensity-modulated radiotherapy (IMRT) or volumetric-modulated arc radiotherapy (VMAT).

Stereotactic radiotherapy is an area of growth and is likely to be especially useful with introducing more complex planning techniques, such as adaptive planning, which is becoming possible with the latest radiotherapy equipment. However, stereotactic radiotherapy is acknowledged as a highly technical technique, ² with additional recommendations for the workforce to deliver safely. ³ As such, there are likely to be fewer staff trained in the area and even fewer who are very experienced.

Radiotherapy planning has traditionally been a manual process and has gone through iterations of complexity as technology has developed. Radiotherapy planning aims to create a treatment that ensures adequate tumour volume irradiation to deliver the highest expected therapeutic ratio while limiting the dose to organs at risk (OARs) to maintain an acceptable toxicity profile. ⁴ There are secondary aims of ensuring a highly conformal prescription dose to the PTV and limiting the amount of radiation fall off away from the PTV.

Stereotactic radiotherapy planning often involves the use of inverse planning techniques. Complex computations perform inverse planning optimisation, but the result can be heavily influenced by the optimisation parameters entered by treatment planners at the beginning of the process and can be difficult and time-consuming. ⁵ These parameters consist of beam configuration (size, angle, number, etc.) and weightings used by the optimiser to assess the priority of where to deposit dose (i.e., maximise to the tumour and minimise to normal tissues). This leads to an iterative planning process as the underlying algorithm is attempting to minimise a cost function based on the optimisation parameters entered and may generate solutions that are not clinically suitable. ⁵ With experience, treatment planners could reduce the number of iterations required to achieve a clinically acceptable plan, but the starting optimisation parameters could still be subjective. The length of time to perform an iteration could vary dramatically depending on the size of the site being treated, the number of optimisation parameters, the resolution of the CT scan and the specific planning system being used.

Methods to standardise and streamline radiotherapy planning are available in many commercial treatment planning systems utilising templates and scripted processes. Automated planning techniques aim to reduce inter-planner variability and improve the overall quality and efficiency of the process. ⁶ Another growing area is the use of knowledge-based planning (KBP) systems. These systems utilise previous radiotherapy treatments to build models that can predict a likely achievable dose in a new patient with a similarly located tumour or to help derive a better starting point for treatment planning optimisation, ⁷ with some systems integrated into commercial treatment planning software, allowing automated plan generation after they have been used.

Ge et al. classify KBP systems into six variables that the KBP systems aim to predict: (1) dose-volume histogram; (2) one or more specific dose metrics; (3) voxel-level doses; (4) objective function weights; (5) beam-related parameters; and (6) quality assurance metrics, using two overarching methods: (a) case and atlas-based methods and (b) statistical modelling and machine learning methods. ⁸ More detailed explanations and reviews of KBP systems are available in published literature. ^7–12

The benefits of using KBP systems in non-stereotactic treatments have already been established. These include comparable or improved plan quality, reduced planning time and a reduction in the variation of plan quality between planners. ⁸

While IMRT and VMAT can deliver stereotactic radiotherapy, most research has focussed on KBP’s non-stereotactic applications. This is likely due to a combination of factors. The number of patients in non-stereotactic groups is larger than those in stereotactic groups leading to the prioritisation of research in this area. Additionally, the characteristics of stereotactic radiotherapy treatments differ from non-stereotactic treatments with increased heterogeneity, potentially making it difficult to build robust KBP models.

The potential applications of KBP in stereotactic radiotherapy would be similar to that of non-stereotactic treatments, for example, to ensure consistent treatment planning within and between centres and make the best use of available resources. In this context, this systematic review aims to assess and summarise the current evidence on using KBP in stereotactic radiotherapy planning and evaluate whether KBP is suitable for stereotactic techniques. If so, can its use improve the radiotherapy plans generated? Improvement is measured in terms of better-quality metrics and faster treatment planning and delivery.

Methods

Search strategy

A systematic review was carried out following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. ¹³ Searches were carried out using Medline and Scopus databases and checking the Cochrane database. Only full journal articles were considered, and the search was limited to English language only. Grey literature was not included as it is not clearly defined and does not necessarily meet the quality of peer-reviewed publications. ¹⁴ The search was made on 28 August 2022. The databases were searched from 2010 until August 2022 to capture current practices. Manual searches of reference lists were also performed. Database-appropriate search strategies were developed around the terms KBP and stereotactic radiotherapy. Where appropriate, MeSH headings and wild cards were used to catch variations in terminology and proximity operators. Full-search methodology, including database-specific search terms and a PRISMA checklist, is included in the Appendix. A review protocol has not been registered.

Eligibility criteria

Eligible studies were studies on the application of KBP in stereotactic radiotherapy planning. Studies that just utilised automated planning without using a priori knowledge in the form of a KBP system were excluded. No restrictions were placed on the patients involved other than stereotactic techniques needing to be used. PICO questions and complete eligibility criteria are included in the Appendix.

Study selection

Duplicate studies and conference abstracts were removed, and the titles of the remaining articles were assessed for eligibility. Two authors (AR and IG) screened the abstracts of all studies from the initial search to select articles for data extraction based on inclusion criteria. Discrepancies were discussed between authors to agree on a consensus. Critical appraisal of articles before their inclusion was performed using the Joanna Briggs Institute Checklist for Quasi-experimental studies. ^15,16

Figure 1 shows a PRISMA flow diagram and the identification and screening performed.

Figure 1. PRISMA flow diagram.

Results

Database search

A MEDLINE and Scopus databases search yielded 325 articles; 27 were identified after initial screening. After identifying studies from other sources (e.g., manual checking of references), which identify six more studies, and further screening, 25 articles met the eligibility criteria for this systematic review. Study characteristics are shown in Table 1, and the details of studies included in this systematic review are shown in Table 2.

Table 1. Summary of study characteristics

Table 2. List of studies and characteristics

Description of quality metrics used: PTV; metrics such as PTV coverage, d_min, d_max, etc. GM; gradient measures such as GI50, GI25, etc. CI; conformity indices such as Paddick Conformity Index. HI; Homogeneity index. OAR; OAR dose metrics.

Stereotactic radiotherapy sites when KBP was evaluated

The most common sites where KBP was used for stereotactic radiotherapy were intracranial (44%) and the lungs (28%). Other sites included the prostate (12%), spine (12%) and kidneys (4%). These are common sites treated with stereotactic radiotherapy. No studies identified the use of KBP in stereotactic treatments of the liver, bone or nodal metastases.

Type of KBP used

Before 2020, most research on the use of KBP in stereotactic radiotherapy planning was performed using non-commercial systems (76.9%), many utilising machine learning techniques. After 2020, most literature on the use of KBP in stereotactic radiotherapy planning (75%) has been with the commercial system RapidPlan (Varian Medical Systems, Palo Alto, USA), potentially due to the ease of implementation through their planning system ‘Eclipse’. RapidPlan is based on principal component analysis, and its basis is explained in the literature. ^17,18

Treatment platform

Most KBP systems identified were used with conventional Linacs (80%). The second commonest system was the CyberKnife radiosurgery platform (16%), and one study was Gamma Knife based (4%).

Number of cases used to train and test

Generally, more plans were used to train than validate. The training and testing models’ modal range was 21–50 cases. Using fewer cases to test than train is unlikely to be a concern, as it is more important to build a robust model that can handle different clinical scenarios (by taking into account a more significant number of cases). The ideal ratio between training/validation/test or training/test is likely to be dataset-dependent. In other areas of machine learning, the optimal training-to-testing ratio is related to how many parameters are used to describe the data. ¹⁹

Quality of KBP versus manual planning

Quality metrics employed in the studies analysed were conformity indices (CI), gradient measure (GM), PTV statistics (PTV), homogeneity index and OAR doses. Most studies used at least one OAR dose assessment (96%). The majority of intracranial studies used a GM (63.6%).

Even though a quality improvement was seen in 60% of the studies, only some explicitly compared a KBP in a clinical setting where a meaningful comparison could be made. When using a commercial KBP system, improvement was seen in 72.7% of studies.

Time taken to plan

Overall, 40% of studies reported a decrease in planning time, whereas 45.5% reported a reduction in planning time with a commercial KBP system. Not all studies were designed to assess whether there was a decrease in planning time as they were either evaluating different models, using KBP to identify suboptimal plans, or looking for planning quality improvement.

Discussion

Type of KBP system used

Before 2020, most KBP systems reported in the literature were developed in-house, requiring centres to have the necessary expertise and time to implement. However, commercial solutions are available now; since 2020, most research on using KBP in stereotactic radiotherapy has used commercial systems.

Almost half of all studies identified in this review used the commercial system RapidPlan. Other commercial systems are available, but their application to date for stereotactic radiotherapy planning appears to be limited.

Commercial systems such as RapidPlan benefit from being maintained by a third-party vendor but, consequently, can suffer from being a ‘black box’. However, the ability of commercial systems to aid the inverse planning parameters used for planning individual cases is likely to be an essential factor in their increased use. This feature may also be desirable so that the full benefits of a particular model can be realised. There may also be legislative considerations of whether an in-house created KBP system constitutes a medical device; if so, systems that are FDA/CE marked (or similar) may become more desirable.

Foy et al. reported that another advantage to commercial systems like RadidPlan is that when training a model, the system reports goodness-of-fit statistics and other tools that can be used to indicate the quality of the model. ²⁸ RapidPlan models can also be publicly shared. One study identified in this review utilised such a model, highlighting models from outside institutions that can produce high-quality plans in other centres. ³⁴ Visak et al. created their model but stated that it is likely that any centre that followed the same dose constraints (RTOG-0813) ⁴⁶ is expected to be able to adapt this model for their own centre. ⁴²

Using RapidPlan, Snyder et al. found that for lung SBRT, if sufficient variability of patient geometry and enough model training was performed, a knowledge-based model that encompasses both IMRT and VMAT techniques is achievable. ²³ This may be useful for centres that have built up experience in one technique but plan to switch to another or use both techniques routinely.

Almost a third of the KBP systems identified in this review utilised machine learning methods (such as CNN and ANN). The authors of one such KBP system acknowledged that many patients are required to train ANN models. ²⁰ This may be difficult to fulfil in stereotactic radiotherapy due to the relatively small number of cases. However, the authors used 617 lesions to train their model, which was the most observed in this review. Other studies identified in this review have achieved good results, with considerably fewer cases used to train KBP models.

In comparing a CNN KBP system called DoseNet, and alternative deep learning architectures, the CNN system was superior to the alternatives. ²⁹ The same authors of DoseNet extended this work using a generative adversarial network (GAN) called DoseGAN, a form of unsupervised machine learning. ³⁵ Their motivation for this was that CNNs use pixel-to-pixel loss to update the underlying models; however, the dose distributions in stereotactic radiotherapy are often heterogeneous, making modelling using pixel-level loss difficult. The results of their work show that DoseGAN predicts the most realistic dose volume for a given set of input anatomy rather than the best. They showed that DoseGAN achieved more realistic dose predictions than the other models it was tested against. The authors highlighted that their model could be used as a support tool to determine achievable plan dosimetry before planning starts or to aid the optimisation process.

Simpler in-house KBP systems, such as those used by Yu et al., may fall outside the category of medical devices. ³⁷ Although these systems do not automatically create a clinical plan following their use through integration with a commercial planning system, they inform the planning team of likely achievable parameters that can then be used to modify prescription dose or the number of fractions, saving time in the process from not having to construct a plan and then change it to achieve a specific plan parameter. The advantages of such systems are that they are cheap to implement, not requiring expensive software or extensive programming skills.

Heuristic KBP is particularly effective. ²⁷ In this work, the author’s KBP system automatically generated planning optimisation scripts to create clinical plans. This reduces planner variability as the scripts are generated automatically from the KBP-derived dose prediction, which means that the optimiser focuses on reducing the cost function on appropriate constraints.

Quality improvements

In the majority of studies included in this review, the use of KBP improved stereotactic radiotherapy planning by either generating superior plans or by reducing the time taken for a plan that was at least equal to that was planned manually. Some KBP systems were only used to evaluate clinical plans and prompt whether further optimisation was required. ³⁷ The ability to reduce variability between planners or using as a method to speed up the stereotactic radiotherapy planning process could be valuable to busy radiotherapy centres with increasing stereotactic patient numbers, especially in public health settings.

SRS

The most common area of research for stereotactic applications of KBP was for intracranial SRS. Most SRS studies reported improvements when using KBP (63·6%). Improvements in GM varied, and the authors of one study reported a 1·1 mm improvement on average while noting a 23% decrease in the volume of the brain receiving 10 Gy (V_{10 Gy}). ²² This may be significant as V_{10 Gy} and V_{12 Gy} are predictors of symptomatic radiation necrosis. ^47–49 In the same study, when KBP showed that the clinical plans generated could be further optimised, improvements in CI were also seen when these were replanned (a reduction from 1·12 ± 0·09 to 1·08 ± 0·11).

While OAR doses in SRS cases were the same or better across most of the studies assessed, in one study, the mean dose to the brainstem was higher in KBP plans compared to manual plans. ²⁰

As well as objective evidence of an improvement in plan quality, qualitative assessment regarding whether a physician approves a KBP-generated plan is desirable. As well as objective quality measures, some studies used blinded physician review. This is the ultimate test of a knowledge-based plan (akin to the Turing test), ⁵⁰ as plan approval is generally always performed by the treating physician or another healthcare professional working under protocol. Across three different plan types (isolated, involved and multiple brain metastases), a study by Ziemer et al. found that in a blinded physician review, physicians preferred plans generated by KBP approximately 80% of the time across the different plan types. ²⁶ In another patient cohort, the same authors found that in 78·1% of cases, a blinded review by a physician found the KBP-optimised plans to be equivalent or superior to the manually planned cases. ²⁷

O’Toole et al. identified time savings in moving to a single isocentre VMAT knowledge-based plan for multiple brain metastases compared to their existing multiple isocentre SRS technique. They required approximately 45 minutes to plan and 40 minutes to treat, regardless of the number of metastases. In contrast, in their multiple isocentre technique, each metastasis requires its own plan and treatment delivery. ⁴⁰

Where quality improvements in studies are reported, it is not necessarily the case that KBP planning performs better in every metric. In some cases, improved performance in one area might be offset by a reduction in another. For example, O’Toole et al. found that the average CI were better in KBP plans but that low-dose wash was worse in four out of five dose levels. ⁴⁰ Like all radiotherapy planning, there are trade-offs in what can be prioritised; therefore, studies that include blinded physician review are of particular interest, as discussed earlier.

Lung

Lung SBRT was the second most common KBP stereotactic treatment site investigated.

Bai et al. showed an improvement in the time taken to plan while maintaining equivalent quality to manually planned treatments; the authors noted a significant reduction in ‘hands-on’ planning time from 40–60 minutes down to 10–15 minutes. ³¹ While they did not observe a substantial improvement in plan quality, the plans were deemed acceptable by experienced physicians.

One potential complication with lung SBRT KBP is that tumours may be present in many different locations within either lung; centrally versus peripherally, upper versus lower and anterior versus posterior. Snyder et al. created four models in RapidPlan depending on where the tumour was located. Another limitation of KBP systems that they identified may be when an OAR abuts or overlaps a PTV and that manual intervention may be required to achieve acceptable plans. ²³ This limitation was also recognised by Foy et al. and is at least partially attributed to RapidPlan prioritising target structures. ²⁸

Cornell et al. found that KBP plans generated with RapidPlan were comparable or preferred to manual plans in 63·9% of lung SABR cases in their study. ³⁶ They also commented on the experience of the staff performing plans for comparison against those generated with KBP in detail, acknowledging that an inherent weakness is that conclusions can only be drawn for the specific models under investigation and that researchers should also be aware of the potential for bias to be introduced into plans performed as part of research studies.

Visak et al. found that using RapidPlan KBP to plan non-coplanar VMAT lung SBRT plans reduced planning time to less than 30 minutes, compared to an average of 129 minutes for an experienced planner. ⁴² KBP plans had a slightly longer average beam on time when treated (2·49 versus 2·15 minutes) and a higher average number of monitor units (3480 versus 3020 MU), both statistically significant. They found that KBP plans had a statistically significant higher average modulation factor, but there was no statistically significant difference in their pre-treatment quality control measurements. The authors also compared KBP against a novel dynamic conformal arc (k-DCA) technique against historical clinical plans. ³⁸ They found that knowledge-based and k-DCA plans were similar or improved over their original clinical plans. They discovered that knowledge-based and k-DCA plans had similar conformity to clinical plans, with maximal OAR doses being lowered. Again, they observed that knowledge-based plans had increased monitor units on average than the original clinical plans.

Prostate

Prostate SABR KBP studies only account for 12% of studies identified in this review. There has been an increasing trend toward hypofractionation for the treatment of low-risk prostate cancer with trials such as CHHIP. ⁵¹ The use of SBRT for low-risk prostate cancer may increase as the results of the PACE-B ⁵² trial reach maturity.

The effect of using all plans (unrefined) and selecting only high-quality (refined) plans (chosen for their superior quality metrics) to build refined and unrefined models has been performed by Shiraishi et al. ²⁵ They found that the refined model showed that some manually planned treatments had room for improvement. This indicates that although bias can be introduced by selecting which plans are used to build a model rather than choosing an entire population of previous treatments, sometimes this bias is appropriate (i.e., plans generated with higher quality metrics develop better models).

It was identified by Kearney et al. that in robotic radiosurgery (with the CyberKnife platform), the location of hotspots within the PTV was specific to each plan; this led to the KBP system generating more homogenous doses than human planners. They acknowledged the limitations of their system; it was only trained for CyberKnife treatments of the prostate, and prostate patients tend to have a homogenous anatomical environment across a patient population. ²⁹

Spine and other applications

Foy et al. found that they could reduce manual planning time from 1–1·5 hours to 10–15 minutes using KBP in spinal SBRT while improving or maintaining plan quality in terms of normal tissue objectives and PTV coverage. ²⁸

There is little research comparing KBP and first principle (FP)-based systems such as PlanIQ (Sun Nuclear Corporation). Geng et al. identified that FP techniques provide quick insight into the patient’s anatomy for spinal SRS. ⁴⁴ Since the technique ignores beam geometry, it might not be appropriate in situations with fewer IMRT fields. Systems such as PlanIQ have shown improvements over manual plans, like the improvements seen in many KBP systems.

A novel application of KBP was in radiotherapy trial quality assurance, whereby treating centres submitted plans were compared to those generated by KBP. This provided timely assurance that submitted trial plans were of sufficient quality and, in some cases, highlighted that improvement may be possible. ³⁹

Implications for future research

The use of KBP in stereotactic planning of spinal metastases has yet to be extensively published. This may be because apart from the spinal cord, cauda equina and nerve roots that are generally present or at least proximal in most cases, the other OARs will vary greatly depending on whether the spinal metastases are in the cervical, thoracic or lumbar vertebra. Even within these groupings, there can be variation in the extent of disease within the vertebra and the location of the OARs proximal to the site being treated. Liver metastases potentially have some of the same limitations as spinal metastases; peripheral tumours are potentially proximal to the chest wall, whereas centrally located tumours may be close to great vessels, the gall bladder and the digestive tract. Similarly, nodal metastases can present in a large variety of nodal chains over a large body area, which could lead to poor-quality models due to the variability of proximal OARs.

There was a notable lack of use of KBP systems with what were once considered the de facto radiosurgery platforms of Gamma Knife (Elekta) and CyberKnife (Accuray) and was commented on by some authors. ³⁷ This is noteworthy for two reasons. Firstly, the Gamma Knife platform has traditionally only performed intracranial radiosurgery, and over 1 million patients have been treated using this platform. ⁵³ The latest Gamma Knife treatment platform (Icon) can also perform treatments with fractionated regimes, owing to the ability not to require a stereotactic frame to be fitted for each fraction. Secondly, the CyberKnife radiosurgery system is a dedicated stereotactic treatment platform that can treat both intracranial and extracranial disease.

Across all of the studies assessed as part of this review, no consistent metrics were used to compare knowledge-based plans to manual plans. The authors of this study propose that future studies making comparisons use the level 2 reporting format as described in ICRU 91 ⁵⁴ :

• Dose-volume specification of PTV (D50%, D_near-max, D_near-min, etc

• Dose-volume reporting specific to OAR and PRV (volume of tissue receiving a clinically relevant dose, near maximum dose and D_mean/D_median)

• Dose homogeneity

• Dose conformity (conformity index and gradient index)

Conclusions

To the authors’ knowledge, this is the first systematic review to evaluate the use of KBP in stereotactic radiotherapy planning. This review has demonstrated that the use of KBP in stereotactic radiotherapy planning is becoming more widespread, and its use has been shown in some cases to improve the quality of stereotactic radiotherapy planning, either by providing a support tool to enable an independent check of the quality of plans being developed or by generating plans that are of superior quality to those planned manually. Evidence suggests that it may also be able to reduce the time taken to plan stereotactic radiotherapy treatments. Not all studies evaluated this aspect, but since 2020, 33.3% of studies reported a time-saving.

Currently, there are limited or no studies looking into the effects of KBP in stereotactic treatments of the prostate, spine, liver or nodal metastases. Further research in these areas would be valuable, especially as public healthcare systems expand the sites treated with stereotactic radiotherapy. This systematic review identified no consistent way of assessing the quality metrics between knowledge-based and manually planned stereotactic radiotherapy treatments. Future studies are recommended to use the ICRU 91 level 2 reporting format. While objective measures help quantify the performance of KBP systems, blinded physician review allows KBP systems to be judged as how they would be used clinically, adding an essential qualitative assessment to KBP system performance.