Advancing hospital-based health technology assessment: evaluating genomic panel contracting strategies for blood tumors through a multimethodology

Abstract

Introduction

The adoption of genomic technologies in the context of hospital-based health technology assessment presents multiple practical and organizational challenges.

Objective

This study aimed to assist the Instituto Português de Oncologia de Lisboa Francisco Gentil (IPO Lisboa) decision makers in analyzing which acute myeloid leukemia (AML) genomic panel contracting strategies had the highest value-for-money.

Methods

A tailored, three-step approach was developed, which included: mapping clinical pathways of AML patients, building a multicriteria value model using the MACBETH approach to evaluate each genomic testing contracting strategy, and estimating the cost of each strategy through Monte Carlo simulation modeling. The value-for-money of three contracting strategies – “Standard of care (S1),” “FoundationOne Heme test (S2),” and “New diagnostic test infrastructure (S3)” – was then analyzed through strategy landscape and value-for-money graphs.

Results

Implementing a larger gene panel (S2) and investing in a new diagnostic test infrastructure (S3) were shown to generate extra value, but also to entail extra costs in comparison with the standard of care, with the extra value being explained by making available additional genetic information that enables more personalized treatment and patient monitoring (S2 and S3), access to a broader range of clinical trials (S2), and more complete databases to potentiate research (S3).

Conclusion

The proposed multimethodology provided IPO Lisboa decision makers with comprehensive and insightful information regarding each strategy’s value-for-money, enabling an informed discussion on whether to move from the current Strategy S1 to other competing strategies.

Introduction

Precision medicine is becoming an indispensable approach in addressing many diseases including cancer as it adds to diagnostics, prognostics, and predictive value of therapy response and disease progression (1;2). The increasing usage of precision medicine has been propelled by the development and dissemination of genomic technologies such as next-generation sequencing (NGS) techniques which allow for high-throughput genetic sequencing (3). Such methods are particularly relevant in the oncology field, given the documented potential for tumor uniqueness (4). NGS application to a variety of cancers has helped identifying novel cancer genes and genomic architectures, characterizing mutational profiles, and understanding tumor pathways (2). NGS mutation screening has been recommended as the standard of care for hematological malignancies with proven advantages in terms of survival rates (5) and actionable clinical utility (6).

Nevertheless, there are multiple challenges related to the adoption of genomic technologies in practical settings (1–3). From the complex process of interpreting genomic data according to the latest scientific findings to all the ethical, legal, and social implications which surface from dealing with sensitive personal data, these technologies will eventually affect all healthcare stakeholders in different ways, both at an individual level and population level (7). Similarly, healthcare providers face challenges like dealing with increasing market options competing with the investment in in-house technical skills and knowledge production (7). This might prove particularly challenging in the recent field of health technology assessment (HTA) of genomic biomarkers and technologies with concomitant gaps and trials already identified in the literature (8–10). Furthermore, hospital-based HTA has challenges of its own, demanding tailor-made decision support tools adapted to the decision context and considering particular organizational and economic dimensions of analysis (11–13).

With these challenges in mind, some authors revised common practices within published genomic HTA reports (8;9), and a set of guidelines has been devised to improve the evaluation process (14). Moreover, considering the intricacy of the genomic testing field as well as the different contexts and perspectives encountered when deciding between comparable technologies, conducting a fit-for-purpose HTA at a hospital level might result in evidence-based relevant recommendations for the decision maker (DM) and the remaining stakeholders (15). The use of sound decision analysis methods and the involvement of all different of stakeholders can contribute to a more thorough and relevant assessment of alternative genomic testing strategies (16;17). This engagement is deemed essential for the harmonization of methods and tools to gauge the impact and value of these technologies, particularly for hospitals and patients (18). Multicriteria decision analysis (MCDA) has already proved successful in HTA contexts, allowing for the comparison of technologies in multiple criteria increasing transparency, efficiency, and objectivity in healthcare decision making (19).

Aim

This work aimed to assist the DM of Instituto Português de Oncologia de Lisboa Francisco Gentil (IPO Lisboa) in analyzing which strategies for genomic testing contracting had the highest value-for-money. Acute myeloid leukemia (AML) is a genetically heterogeneous disease with poor prognosis (6). In the particular setting of AML clinical pathway (CP) of care at IPO Lisboa, NGS is currently used for patient stratification/risk assessment, therapy adjustments, and disease monitoring and may be used to determine patients’ eligibility for participation in specific clinical trials. In this study, a multimethodology was designed to specifically evaluate genomic testing contracting strategies for patients suffering from AML through the eyes of real-life hospital stakeholders and experts, and within the frame of genomic biomarker and hospital-based HTA literature.

Methods

Overview of the Multimethodology

We developed and implemented a multimethodology that combines several modeling techniques within a common frame and is divided into three main steps, as illustrated in Figure 1. After undergoing a series of meetings to understand the decision context and the needs of the hospital (step 1), we built models to map the CP and to evaluate the (multicriteria) value and the cost of each selected strategy (step 2). These modeling components generated information to be combined into graphs that enable value-for-money analyses, including a three-dimensional graph (“strategy landscape”) that informs on the added value and added cost of the strategies (step 3). The three stages of the multimethodology, which were carried out between March and October of 2021, are further detailed in the next subsections.

Figure 1. Multimethodology to enable value-for-money analyses of strategies for genomic testing contracting in the context of IPO Lisboa.

Step 1: Problem Identification

Five meetings were held over a month involving four key stakeholders from IPO Lisboa, including one member of the Board of Administration, the director of the diagnostic laboratory, one hematologist oncologist (physician), and one research manager. These four stakeholders represented, as a group, the DM IPO Lisboa. These meetings were essential to understand the problem at hand and the institution’s needs, to select which genomic testing strategies should be compared in the scope of this study, and whom to directly involve in each modeling step. Three genomic testing contracting strategies were selected for this analysis: Strategy 1 (S1) – Standard of care, TruSight™ Myeloid Sequencing Panel (this strategy maintained the contracting of the current NGS panel for patients with AML by IPO Lisboa); Strategy 2 (S2) – FoundationOne Heme (Roche); and Strategy 3 (S3) – New diagnostic test infrastructure (description in Table 1). It was also decided that the value model should consider the views and knowledge of a group composed of a hematological physician and two laboratory technicians (hereby called the “evaluators”), and that cost modeling should be assisted directly by the Board member. Both the DM IPO Lisboa and the evaluators would also be responsible for validating the outputs of the different steps of the multimethodology applied.

Table 1. Brief description of the three selected genomic testing strategies: Strategy 1 (S1) – Standard of care, Strategy 2 (S2) – FoundationOne Heme test, and Strategy 3 (S3) – New equipment

Step 2: Building Models to Map the CP and to Evaluate the (Multicriteria) Value and the Cost of Each Selected Strategy

Step 2.1: CP Mapping

Mapping the CPs of AML patients was deemed a critical step due to several reasons: first, to better understand the journey of a patient with this type of cancer, and the possible variations in terms of timings, treatment, and results; second, to estimate the time point when the NGS test is performed, and its implications in the whole CP; and finally, to analyze changes to the CP in the cases of adopting other NGS contracting strategies. For this purpose, patients were divided into three groups according to the European Lymphoms Network (ELN) Risk Stratification (20), that is, whether they belonged to the favorable, intermediate, or adverse-risk group, and their CPs were described and schematized as a process flowchart. This was done in collaboration with the evaluators over the course of three meetings.

Step 2.2: Value Modeling

We used MACBETH (Measuring Attractiveness by a Categorical Based Evaluation Technique) (21) as an interactive multicriteria decision analysis approach to build a quantitative (numerical) value model to assess the value of each genome contracting strategy for IPO Lisboa. MACBETH is built on the principles of value-difference (or strength of preference) measurement, and its distinctive feature is that it is based on qualitative (nonnumerical) pairwise comparison judgments (using the semantic categories: “null,” “very weak”, “weak,” “moderate,” “strong”, “very strong,” and “extreme”) (21). MACBETH has been perceived to be a “convenient way to express value judgments by lowering cognitive load” (22) (p. 150), and it has been applied with success in multiple health settings (e.g., (23;24)). The implementation of the MACBETH method is enabled by the M-MACBETH software (25).

To build IPO Lisboa’s value model with MACBETH, we guided the group of evaluators through three phases: model structuring, model evaluation, and model validation. In model structuring, the evaluators participated in three workshops, led by an impartial facilitator, which culminated in the selection of the five criteria to be included in the model. In the first workshop, we presented the group of evaluators with a list of 34 potential evaluation aspects – relevant for the evaluation of in vitro medical devices – obtained from a Delphi survey previously created for the MEDI-VALUE project (26). Still within the first workshop, we asked the group to classify each aspect as critical, fundamental, complementary, or irrelevant, considering their relevance for the genomic technologies’ context. Afterwards, a debate was promoted for the participants to share their views and reach a compromise regarding which aspects should be included or not in the analysis. In the end, the evaluators agreed on six evaluation aspects. During the second workshop, the six previously selected evaluation aspects were structured in two sets: aspects directly relevant to the patient, and aspects related to the institution and its stakeholders. Furthermore, a detailed description of each aspect was formulated, and relevance classifications were adjusted. Finally, in the third workshop, two evaluation aspects were merged to avoid redundancy, and the final structure of the model was concluded. Furthermore, to measure the extent to which a panel contracting strategy contributes to a criterion, a descriptor of performance was associated, and each strategy was classified according to those levels of performance, as portrayed in Table 2. This structuring process is aligned with the collaborative value modeling framework (27), an integrated sociotechnical setting that combines Delphi processes with in-person decision conferencing processes (28) and sound tools for value modeling with the (multicriteria) MACBETH approach.

Table 2. Criteria selected to assess the value of different genomic testing strategies at IPO Lisboa for blood tumors in the constructed MCDA model, and associated descriptors of performance (cost is dealt with separately in the value for money analysis)

Note: Furthermore, each strategy’s performance on each criterion is presented.

Having selected the criteria and genomic testing contracting strategies, we moved into the evaluation phase of model building. This phase involved two key activities: building intracriteria value scales (which convert performance into value on each criterion) and weighting the criteria (to make the value scales commensurate). These elements were then integrated using a simple additive value model - see (1).

(1) $$ v(s)=\sum \limits_{j=1}^n{k}_j{v}_j(s), $$

where $ v(s) $ is the overall value score of a strategy s (s = S1, S2, S3), j designates a criterion (j = 1, …, n), $ {k}_j $ is the weight assigned to the criterion $ j $ (with $ {k}_j>0\hskip0.35em $ and $ \sum \limits_{j=1}^n{k}_j=1 $ ), and $ {v}_j(s) $ is the (intracriterion) partial value of $ s $ on j. The developed additive value model used as references the worst and best performance levels in each criterion, corresponding to 0 and 100 (intracriterion) value scores, respectively; and the overall (intercriteria) value score has a minimum and maximum of 0 and 100 value scores, respectively; that is, the lower bound corresponds to a hypothetical strategy with the worst performance in all criteria (worst all over), and the upper bound corresponds to a hypothetical strategy with the best performance in all criteria (best all over).

To build (intracriteria) value scales and to weight the criteria, an online survey was sent to the group of evaluators and five other IPO Lisboa professionals, as the group of evaluators deemed as relevant to consider other perspectives besides their own (those participants included a third laboratory technician, two other hematology doctors, one research manager and one Hospital Administrator with responsibilities on the area). All the adopted MACBETH protocols of questioning can be seen in Supplementary Material 1 (translated version of the Portuguese protocols). In particular, for the weighting process, participants were first asked to order the criteria using the specific procedure: participants selected the criterion where an improvement from the lowest to the highest level of performance would be most valuable, followed by the second most important, and so on. Then, they rated the attractiveness of improvements in each criterion using the (adapted) MACBETH qualitative scale, which included the options “very weak or weak,” “moderate,” “strong or very strong,” and “extreme.” After gathering individual preferences, the Borda voting system was employed to identify the most consensual order among all answers. With five criteria under consideration, each participant’s top-ranked criterion was given 4 points, the second received 3 points, and so on, with the last criterion receiving no points. Furthermore, participants’ qualitative answers from the MACBETH scale – regarding the attractiveness of improvements – were used to populate the judgment matrix for weighting. This matrix was then employed to convert qualitative judgments into a numerical weighting scale. The model and the resulting scale were discussed, adjusted, and validated during the decision conference according to the evaluators’ views.

In the last phase of value modeling, a prototype value model based on the answers from the online survey was discussed, adjusted, and validated during a decision conference where the evaluators were present to finalize the final value model.

Step 2.3: Cost Modeling

We estimated the cost of each strategy through a Monte Carlo simulation model where we only included costs which differentiate across contracting strategies, as suggested in the literature (29;30). This step was developed with the contribution of the Board member (DM IPO Lisboa). Before building the simulation model, the relevant groups of costs were identified. For the cost analysis, we considered a period of 5 years. This timeframe was chosen due to the rapid evolution of the technologies being evaluated, the limited data available, and the fact that the health system is not yet fully organized to use genomic panel data extensively. A discount rate of 4 percent was applied when calculating the present value of each group of costs, as suggested by the Portuguese official economic evaluation guidelines (31). The protocol used for cost modeling, together with the choice of the statistical distributions and the definition of the output functions, is available in Supplementary Material 2.

Step 3: Combining the Results

The results from the Monte Carlo cost simulation model were combined with the global value scores obtained with M-MACBETH and represented in two formats: as a 3D strategy landscape graph crossing the cost, cost variability and value dimensions, for more detailed visualization; and a XY plot portraying the value and the extra cost of each strategy, as produced by the M-MACBETH software. Both graphs enable analyzing how strategies S2 and S3 compared with S1, the standard of care, so that their extra costs and extra benefits can be discussed, and thus, the strategies’ value-for-money is understood. The expected impact of adopting each alternative strategy on the current CP was also performed, by considering input from stakeholders that took into account their knowledge on data, evidence, and hospital processes.

This study was approved by the Ethics Committee of Instituto Superior Técnico, University of Lisbon (reference no. 14/2019 (CE-IST)) and of IPO Lisboa (UIC/1239).

Results

CP Mapping

The CP flowcharts built for each patient risk group can be seen in Supplementary Material 3. After being referred to IPO Lisboa with suspicion of AML diagnosis, the patient is called for the first consultation with a hematologist oncologist, and several laboratory tests are carried out for diagnosis confirmation and risk stratification. This phase does not include the NGS test that might be performed later involving a small number of genes quickly analyzed for a decisive and accurate risk assessment. Since the NGS testing can take 2–4 weeks to be delivered, it is not used for the initial diagnosis or choice of treatment. NGS results are relevant for therapy adjustments and for the identification of molecular targets for disease monitoring or even to determine patients’ eligibility for inclusion in ongoing clinical trials. These results can potentially alter the individual CP of an AML patient, although the benefits of applying a larger gene panel need to be further studied.

Value Modeling

Regarding the multicriteria value model, Figure 2 shows the overall value score obtained for each strategy, which is the result of multiplying the (intracriteria) value scores by the weighting coefficients of each of the five criteria (line on the bottom of the figure), by applying equation (1). According to these results, Strategy S2 is the most attractive, with an overall score of 72, followed by S3 and S1, with respective scores of 49.8 and 35; and S2 has higher value in all criteria, except on Knowledge Improvement. The model (intracriteria) value scales can be seen in Supplementary Material 4.

Figure 2. Table portraying the intracriteria value scores for the three strategies (in each criterion) and the global value score (by applying equation (1), and with reference options worst all over, and best all over made explicit).

A sensitivity analysis was performed on the criteria weights, to understand whether their changes would significantly affect the overall value scores. Results indicated that one would need to increase the weight of the “Resource Optimization” criterion by 19.6 percentual points for S1 to surpass S3; an increase by 13.1 percentual points in the “Knowledge Improvement” weight would be sufficient for S3 to surpass S2 as the strategy with the highest overall score (for visualization of results, see Supplementary Material 4); and changes in the weight of the remaining criteria would not produce significant changes in the model results.

Cost Modeling

After running the Monte Carlo cost simulation model, a cost probability distribution was obtained for each strategy, depicting variability in its estimation. The summary of the most relevant statistics obtained for each strategy can be found in Supplementary Material 2. S1 had the lowest predicted costs, with a mean value of 237.632 (EUR), followed by S3 with a mean of 437.922 (EUR). S2 had the highest mean cost (754.313 (EUR)) but also the highest standard deviation, reflecting the uncertainty around the prices of the FoundationOne Heme test. Sensitivity analyses were performed to understand the impact of each input variable in the results of the cost model, showing that for S1 the salary of the hematology-specialized technicians was the parameter with the highest impact in the cost probability distribution, followed by the number of allocated human resources. In the case of S2, the input with the strongest effect on the mean cost was the price of each FoundationOne Heme test, followed by the expected number on annual AML reports. For S3, the salary was once again the input with the highest potential impact on cost, followed by the estimated initial investment.

Combination of the Results

Table 3 summarizes the overall score and mean cost of each strategy and describes the observed impact of each strategy on the AML patients’ current CP. The implementation of a larger gene panel (strategies S2 and S3) could potentially lead to additional genetic findings and conceptually result in even more personalized treatment and monitoring of each patient. On a patient level, the implementation of Strategy S2 may provide access to a wider range of clinical trials. Similarly, by sequencing new genes after the implementation of Strategy S3, IPO Lisboa would have more complete genetic databases which could prove to be beneficial in the future, especially in terms of research. However, according to IPO Lisboa stakeholders and experts, none of these changes would directly reflect in the general pathways which were mapped for the three risk groups of patients.

Table 3. Summary of the value score and (mean) 5-year cost of each strategy and expected impact on the current CP of AML patients at IPO Lisboa upon simulated implementation

A strategy landscape graph and an XY plot combining the overall value score of each strategy (obtained with M-MACBETH and based on the developed additive model) with its 5-year mean cost (obtained from the Monte Carlo simulation model) (Supplementary Material 5) were then generated to allow better visualization of both the value and cost modeling steps. One can see that although S2 was given the highest value score in the value model, its 5-year cost function shows a greater deviation from the mean cost when compared to the other strategies. When cost uncertainty is not considered, results show that a higher global score for strategies S2 and S3 is associated with an increased mean cost (in comparison with the ongoing standard). One can say that all strategies are in the efficient frontier, but S2 entails substantially higher cost uncertainty. IPO Lisboa is currently adopting S1 and may envisage moving toward S2 or S3.

Discussion

Key Findings for Genomic Biomarker Literature

The adopted multimethodology was designed to assist the DM of IPO Lisboa in analyzing which strategies for genomic testing contracting have the highest value-for-money. The methodology included mapping the CP of AML patients, building a MACBETH value model to evaluate each genomic testing contracting strategy in multiple criteria, and estimating each strategy’s cost with a Monte Carlo simulation model. Regarding the choice of genomic testing contracting alternatives, strategies S1 and S3 refer to in-house procedures, in which IPO Lisboa is completely responsible for collecting, processing, analyzing, interpreting, and reporting each patient’s genomic data. Although this requires more resources and time, the process can be closely monitored, and all data stored with the potential for future use in research purposes (upon ethics and regulatory appraisal). On the other hand, Strategy S2 involves commissioning an external service. While this approach reduces demands on the hospital workforce and equipment, it comes at a higher cost, even though it maintains the quality of the analysis and without compromising the quality of the analysis. However, only the final results of the test would be available to IPO Lisboa, hindering any further analysis (either confirmatory or investigational) on their part. Another topic discussed with IPO Lisboa stakeholders when selecting the alternatives was the number of genes tested in each strategy, due to the progressive spread of NGS techniques and the concomitant drop in prices (3). Even though the currently used panel already comprises the most common mutations for AML, studying a larger number of genes might not only benefit future research at IPO Lisboa (in the specific case of Strategy S3) but also potentially help find suitable clinical trials for patients to enroll. Additionally, the analysis of results should consider the generic potential benefits of off-the-shelf solutions, like FoundationOne (S2), for which implementation can offset the delays associated with developing S3. In fact, these readymade platforms can expedite the identification of suitable clinical trials for patients. However, such immediate benefits should be weighted against the future potential of S3, fostering research and potentially enabling the development of a more personalized gene panel. Such balance can be informed by the developed model and led to discussions at IPO Lisboa. Finally, although all the steps of the methodology have some level of associated uncertainty (for instance, due to imprecise or missing data), the lack of discussion about Strategy S3 at IPO Lisboa needs to be accounted for when interpreting results. This concern was properly discussed with the DM IPO Lisboa and evaluators, which at the end of this study considered it worth deepening the analysis of Strategy S3 and pondering several alternatives within this strategy, as long as it would enable exploring a larger and more personalized gene panel than the currently employed.

To the best of our knowledge, this is the first study looking into the evaluation of strategies for genomic panel contracting using MCDA.

Key Findings for Hospital-Based HTA

Although MCDA has been increasingly explored in healthcare contexts, its use requires fitting and adaptation to the context. In this study, further to cost, five criteria were carefully chosen and refined by the group of evaluators, taking into account the intricacies of the disease, the complexities surrounding the related NGS tests and the functioning and priorities of the institution. Strategy S2 was shown to have the highest score, performing well in most criteria, while Strategy S3 stood out not only for its clinical relevance but also for potential knowledge retention at the institution; Strategy S1 has been shown to be cheaper but having a lower value. These results suggest the need for IPO Lisboa to discuss its willingness to pay for generating extra value through the acquisition of genetic panels in the AML (and other) context(s); and echo the trend in the healthcare community of investing in larger gene panels, as they contribute to increasing knowledge regarding human genetics at the individual and population levels besides the discussions toward implementing whole genome sequencing as a standard diagnostic test in oncology (32;33).

Although there are some published studies in the field of cost estimation of sequencing tests (e.g., (30)), the difficulty of assessing the long-term impact of genomic findings is a recurrently referred challenge. In this case, only the short-term differential costs across strategies were considered, and a sensitivity analysis was conducted to address possible sources of uncertainty related to the collected data. The Monte Carlo simulation model results revealed Strategy S2 to be the most costly, but also the one entailing higher uncertainty surrounding the price of the commercial solution (i.e., FoundationOne Heme test), as expressed by the adopted triangular input function (i.e., we considered a min, most likely, and max value taking into account the DM knowledge and concerns). According to the DM knowledge and views, S3 entailed much less uncertainty, although several cost components required for setting a new diagnostic test infrastructure are also expected to vary.

Some considerations could be drawn regarding the impact of strategies on the patients’ CP. Despite observing no significant changes in the CPs due to different genomic testing contracting strategies – for example, the use of distinct panels impacted treatments but not the pathways themselves – it is anticipated that increased genomic testing in hospitals, along with greater data availability and enhanced evidence regarding hospital processes, will lead to more detailed modeling. Consequently, this could potentially result in changes to the clinical pathways. Therefore, all findings should be wisely and critically analyzed by the hospital (29).

All participants involved in the study afterward provided positive feedback regarding the methodology employed and the subsequent results. They considered the study highly relevant for IPO Lisboa given the ever-changing landscape of genetic diseases and constant turnover of genomic technologies; they recognized the role of the facilitator, which was seen as crucial in the whole process, working as a bridge between the clinical and the technical fields, while reaching out to different stakeholders; they found the multimethodology to be adequate and answering to their needs, enabling a sound discussion on how to move forward regarding genomic testing contracting strategies (post-assessment insights are available in Supplementary Material 6).

Strengths and Limitations of the Study

The implemented multimethodology was carefully designed and planned to contribute to the challenge placed by IPO Lisboa. By combining cost and CP modeling with value measurement, one could better understand the strengths and limitations of each of the selected strategies. This approach added an additional layer of complexity to the analysis but enabled a deeper analysis in comparison with a simpler unidimensional methodology. The combination of a technical component with some form of social interaction with the stakeholders involved brought richness and soundness to the analysis.

Several limitations should be acknowledged in this study to promote critical analysis and inspire better practices in the future. First, although NGS techniques and other technologies have helped unravel the mysteries of cancer, there have been discussions on the extent to which a larger gene panel is relevant for disease progression. Secondly, issues were perceived concerning the definition of strategies S3 and S2, which should be more matured by the hospital, and which should be considered in the analysis of results. Thirdly, several assumptions were required for cost modeling. Finally, all models were built specifically for the particular decision context and should be tested and adapted to other contexts.

The implementation of MCDA in everyday hospital-based HTA entails multiple challenges, such as the need for specialized skills in MCDA modeling and time consumption for model building. The application of MCDA involves developing a sociotechnical process that requires an understanding of the methods and techniques used and the social processes required for aligning different values and perspectives and using sound techniques. Given that MCDA concepts are still not widespread in health economics evaluation courses, adoption of MCDA might require initial training and continued refinement of these skills among staff involved in hospital-based HTA processes. Regarding time consumption, while it is true that the MCDA approach may require more time in model building, we believe that, in the long run, it could streamline the decision-making process and contribute to other gains. For instance, creating a comprehensive picture of the technology’s value upfront avoids subsequent misalignments between stakeholders which can delay the adoption of the technologies; and the development of reusable MCDA models for assessing distinct technologies in different HTA processes can entail a one-off cost, as models can then be used for some time. Related discussions to these issues have been reflected upon in (19).

Conclusion

This study aimed at helping IPO Lisboa, a renowned cancer research center and hospital in Portugal, assessing the value of implementing three different genomic testing contracting strategies for the care of AML. A multimethodology was implemented to assess strategies, involving the modeling of patients’ CP, a multicriteria model of value with hospital stakeholders and experts, and differential costs. This study demonstrates the advantages of incorporating MCDA together with cost and uncertainty modeling to understand the value-for-money of competing genomic testing contracting strategies, showing the relevance of involving stakeholders, measuring multidimensional value, and modeling cost uncertainty in the assessment of technologies in hospital settings.