Development of an multicriteria decision analysis framework for rare disease reimbursement prioritization in Malaysia

Abstract

Objectives

Rare diseases (RD)-related policies have received significant attention due to the pressing medical requirements associated with these medical conditions and the substantial impact and treatments they may have on healthcare budgets. Nevertheless, policymakers frequently encounter difficulties in managing issues concerning resource allocation and prioritization within this population. Realizing the need to address such problems, this study was conducted to develop a framework based on the multicriteria decision analysis to improve RD reimbursement prioritization in Malaysia.

Methods

Primarily, a scoping review was performed to identify the methods and criteria used for the reimbursement of RD treatment, followed by strategic stakeholder engagement and a deliberative process on determining the best approach for the framework, including criteria identification, elicitation of weights, and a pilot assessment using the framework.

Results

The findings reflected the priorities and perspectives of the stakeholders, which identified eight key criteria and their associated weights, namely effectiveness (19.6 percent), disease severity (15.6 percent), safety (14.2 percent), access to treatment (12.6 percent), economic consideration (12.2 percent), type of therapeutic treatment (11.5 percent), availability of alternatives (8.3 percent), and population group (6 percent).

Conclusions

In summary, the developed framework was well-accepted by the Rare Disease Committee, which will be applied as part of the committee deliberation for transparent and equitable decision making on fund allocation and reimbursement of orphan and RD treatment in Malaysia.

Introduction

Rare diseases (RD) impose massive clinical and economic burdens and challenges to the healthcare system upon failing to address the patient’s needs and not guaranteeing equal access to treatment (1). In general, RD patients and caregivers face uncommon, severe, debilitating conditions, often characterized by poor prognosis and limited treatment options (2). The primary challenge in RD research stems from the rarity of these diseases, which obstructs the establishment of randomized clinical trials with adequate statistical power to detect overall treatment effects and account for disease heterogeneity. As such, this limitation complicates the identification of appropriate end points and the generation of clinically relevant, measurable, and reproducible treatment outcomes (3).

In addition to the direct medical expenses linked to RD, individuals and society have to bear significant costs, including indirect expenses from productivity losses, nonmedical expenditures, such as spending on home or vehicle modifications, and certain healthcare costs not covered by insurance (4). In Malaysia, a rare disease is defined as a life-threatening and/or chronically debilitating rare condition, as listed in the Malaysian Rare Disease List, affecting fewer than 1 in 4,000 people. The needs of these patients have been recognized, with significant progress in managing RD, including the setting up of the National Rare Disease Committee (NRDC) with several subcommittees, the establishment of a National Rare Disease List, and the development of the Malaysian Orphan Medicine guidelines to facilitate the treatment access (5).

RD-related policies have gained considerable interest owing to the urgent medical need and the significant impact of RD and their treatment protocols on healthcare budgets. While each country adopts different orphan drug policies, healthcare budgets, and the level of patient access (6), the main policies that curtail a patient from receiving orphan drugs involve registration and reimbursement (7). Despite extensive efforts to promote the development of RD-related therapies in the past decades and supported by regulatory and economic incentives, most RD still lack specific treatment (8). In fact, the development of these promising therapies is a challenging task as they normally fail to deliver due to unacceptable adverse effects and/or lack of response. The limited supporting real-world evidence and low methodological quality due to the small number of patients may also provide inadequate mandatory pharmacokinetic and pharmacodynamics information needed to approve these drugs under such rare conditions. Besides, they may not reach the prerequisite threshold for peer-reviewed publications with standard trial designs. Therefore, it is crucial to develop a system that recognizes such information as a valuable contribution to the literature, and it should be considered essential to the development of future successful therapies (9).

Reimbursement and pricing systems vary among countries based on several factors, such as the size of the healthcare budget, the type of healthcare and health insurance system, patient copayment rules, reimbursement timelines, and evidence requirements (such as type, level, and presentation). Consequently, patient access is often unpredictable and restricted. The exorbitant price of many orphan drugs, frequently coupled with the limited amount of clinical evidence (mainly due to the small patient population), can inflate the Incremental Cost-effectiveness Ratios (ICER) beyond the willingness-to-pay level (10). The growing demand for reimbursement of expensive innovative therapies also raises concerns about their long-term affordability (11). Given the commonly expensive acquisition of orphan drugs and their uncertain (cost-) effectiveness (at least at the time of submission), decision makers have faced difficulties in reimbursing them through their standard assessment and subsequent appraisal processes (12).

Decision making in healthcare matters involves comparing different alternatives to seek the best treatment based on multiple factors that meet the decision-makers’ and the organization’s expectations (13). Besides, RD commonly places a heavy burden on the family and caregivers, the impact of which is usually not taken into consideration in standard cost-effectiveness analyses (14). Thus, decision makers are increasingly adjusting their reimbursement processes by considering the specific characteristics of orphan medicinal products and RD (15). Health systems may adopt novel reimbursement decision-making strategies to complement the standard assessment and mitigate the uncertainty of the clinical benefits of a new treatment that has been trialed for a relatively short duration (16).

Among the various approaches that the health system and reimbursement bodies can employ include cost-effectiveness models, budget impact analysis, multicriteria decision analysis (MCDA), and other alternative reimbursement models, such as reference pricing in pricing negotiation and managed entry agreements (17). Although waivers and reduced data requirements are often present in some form or another, there are not yet any specifically tailored health technology assessment (HTA) approaches for orphan drugs (18). Nevertheless, the framework for the appraisal of RD treatment developed by Improved methods and actionable tools for enhancing health technology assessment (IMPACT HTA) supports a consistent, flexible assessment to ensure fairness, given the unique circumstances of the disease (19).

MCDA is a potential alternative that can cater to the lack of appropriate HTA tools by incorporating benefits and costs specific to RD treatments beyond the standard cost per quality-adjusted life years (QALY), such as socioeconomic aspects. Recently, MCDA has gained increasing attention in reimbursement decisions for orphan drugs due to the belief that the traditional cost-effectiveness approach used to assess the value of orphan drugs is incapable of comprehending all the multi-dimensional factors that inform treatment benefits (20). Interestingly, MCDA can support decision-making processes by considering and weighing a range of factors of a certain intervention and generating a single composite outcome score, which can then be compared between different health technologies (21).

Although the role of HTA in policy formulation and decision making of health technologies has become more significant over the years (22), the established mechanism for assessing health technologies is still unable to provide a solid framework for the allocation and reimbursement of RD treatment. Hence, this article aims to develop a framework based on the MCDA to enhance RD reimbursement prioritization in Malaysia.

Method

A scoping review was conducted to identify the methods and criteria used for the reimbursement of RD treatment. This information was then presented in a stakeholder meeting attended by methodology experts and key stakeholders on RD, including clinicians, patients, and patient organization representatives. Policymakers were also consulted in this meeting to identify the suitable method for developing the RD assessment framework. The meeting members agreed to explore the use of MCDA in a structured workshop.

Development of the MCDA framework

Following the stakeholder meeting in November 2021, a three-day in-person workshop was organized in February 2022, which aimed to develop the MCDA process for RD through active engagement among multi-stakeholders. Around fifty personnel from various backgrounds were invited to attend this workshop, including NRDC members, which comprise clinicians from the Ministry of Health and Ministry of Education (36 percent), academicians (2 percent), government officials (24 percent), patients and representatives from patient’s organizations (12 percent), and other healthcare professionals (14 percent). Several representatives from the industrial sector (12 percent) were also invited to this workshop. The participants were briefed on the general role of HTA and the proposed framework for RD assessment (supplementary process) on the first day of the workshop, as well as the importance of scientific evidence in decision making. Figure 1 illustrates the supplementary process workflow, which includes approval by the highest level of policymakers, HTA, and the Clinical Practice Guidelines (CPG) Council. The participants were then introduced to the MCDA steps and a video presentation on the general MCDA process.

Figure 1. Workflow of the supplementary process.

The five-step methodology for the MCDA framework for RD in this paper was adopted from the International Society for Pharmacoeconomics and Outcome Research (ISPOR) MCDA Emerging Good Practices Task Force (21;23). Steps one and two involve identifying decision problems and criteria, while steps three and four assess the performance and elicitation of criteria weights. Finally, step five evaluates the aggregate scores.

Identification of Decision Problems and Criteria

On the second day of the workshop, the participants were introduced to a preidentified decision problem to aid in deciding the best treatment to be reimbursed through the Rare Disease Trust Fund. The problem was discussed intensively and agreed upon, commencing the next step of identifying relevant criteria. A list of identified criteria retrieved from relevant published literature was also presented as examples to assist participants in understanding the purpose of the workshop. Subsequently, a brainstorming session was conducted using a free version of the interactive presentation software https://www.mentimeter.com/ to foster active participation and proceeded with group work (24). Participants were asked to identify the number of criteria and select those relevant to be included in the MCDA framework. After that, each group was given the opportunity to present their selection of criteria along with their definitions, and the criteria performance was gathered and deliberated further before beginning the criteria weighting exercise.

Assessing the Performance and Elicitation of Criteria Weights

As suggested by NRDC, five interventions were considered in the MCDA framework based on prior topics:

1. 1) Propionyl-Coenzyme A (CoA) carboxylase (PCC) deficiency & methylmalonyl-CoA mutase deficiency – Carglumic acid

2. 2) Systemic Juvenile Idiopathic Arthritis – Tocilizumab

3. 3) Systemic Juvenile Idiopathic Arthritis – Anakinra

4. 4) Connective Tissue Disease-related Pulmonary Arterial Hypertension (CTD-PAH) (Adult) – Macitentan

5. 5) Connective Tissue Disease-related Pulmonary Arterial Hypertension (CTD-PAH) (Adult) – Sildenafil

Data on the alternative performance of the intervention for each criterion were gathered using systematic reviews. Meanwhile, a narrative review was prepared to describe multiple methods used in the MCDA criteria weighting, which include direct rating, Simple Multi-attribute Rating (SMART), Analytical Hierarchy Process (AHP), Discrete Choice Experiment (DCE), Categorical-based Evaluation Technique (MACBETH), Potentially All Pairwise Rankings of All Possible Alternatives (PAPRIKA), and Conjoint Analysis (CA) (25). In view of multi-stakeholder involvement, the SMART method was employed in this workshop for the criterion weight elicitation owing to its simplicity, flexibility for weight assignment either as absolute or relative, and the number of selected criteria (25;26).

Firstly, the participants were given a piece of paper to write down their preferences from the list of selected criteria that have been collectively agreed upon in order of importance. Starting from the reference criteria (either the least or most essential), all participants were required to assign weights based on the significance of the following criterion compared to the reference criteria using an online-based Google form with a 10–100 scale measurement. The least crucial criterion was assigned a minimum weight of 10, while the most vital criterion with a maximum weight of 100. An arbitrary 10 points were allocated for the least essential criterion to avoid a possible redundant zero-weight criterion. Participants were asked to assign a higher weight if the reference criterion was the least essential or a lower weight if the reference criterion was the most essential compared to the weight that was assigned to the previous one.

Value Assessment

On the last day of the workshop, the participants were presented with scientific reports based on comprehensive literature reviews by three facilitators to provide relevant information on the performance of each intervention. The presentation described five interventions to treat three RD conditions according to prior topics suggested by the NRDC. The session served as a pilot exercise to assess the feasibility of the proposed MCDA framework for RD and the suitability of the proposed criteria to capture all relevant dimensions required for the value assessment across all RD.

After each presentation, the participants were asked to apply a direct rating method using an online-based Google form with a 0–100 scale measurement to assign a score for each intervention on each criterion (0 = lowest performance and 100 = highest performance). During this exercise, the participants were given an opportunity to clarify any inquiries pertaining to the topics that had been presented. They were also encouraged to provide comments and feedback on the overall process.

Data analysis

Data were collected individually using an online-based Google form for all exercises. The data were then analyzed using Microsoft Excel, and the results were presented to the participants using Microsoft PowerPoint. Criteria weights were normalized to sum up to 1 for each participant. For the weight assignment using the 10–100 scale measurement, each weight was divided by the sum of the weights across all criteria. The value contribution was calculated by multiplying the normalized weight of each criterion and the assigned score for each intervention. The most commonly applied aggregation formula was used in the analysis, as expressed below (23).

(1) $$ {\displaystyle \begin{array}{l}{\mathrm{v}}_j=\sum \limits_{i=1}^n{\mathrm{S}}_{ij}\times {\mathrm{W}}_i\\ {}{\mathrm{v}}_j=\mathrm{Overall}\ \mathrm{value}\ \mathrm{contribution}\\ {}{\mathrm{S}}_{ij}=\mathrm{Score}\ \mathrm{for}\ \mathrm{intervention}\hskip0.3em j\hskip0.3em \mathrm{on}\hskip0.3em \mathrm{criterion}\hskip0.3em i\\ {}{\mathrm{W}}_i=\mathrm{Weight}\ \mathrm{of}\ \mathrm{criterion}\hskip0.3em i\end{array}} $$

Further analysis was conducted using the chi-square and Kruskal Wallis tests to determine variations in response between participant groups since the data were not normally distributed.

Results

Identification of Criteria and Criteria Performance

The brainstorming session was conducted to collate all criteria critical to the stakeholders, which yielded 208 individual responses that included some big word clouds, such as “effectiveness,” “safety,” “quality of life,” “disease severity,” “cost,” ‘affordability,” and “sustainability.” To further streamline and decide on the criteria deemed essential to answer the preidentified decision problem, the participants were divided into eight groups, where each group presented around five to nine criteria (half of the groups preferred eight). From the discussion, the overlapping criteria were aggregated to simplify the selection and avoid repetition. Consequently, eight significant criteria were identified, as depicted in Table 1. The definition and performance for each criterion were based on the resulting group work and discussion among the participants.

Table 1. Finalized criteria with definition and criteria performance

Elicitation of Criteria Weights

Only five participants were unable to attend the workshop due to scheduling conflicts. Therefore, the forty-five attendees who successfully participated in all planned activities recorded a 100 percent response rate. The attendees consisted of clinicians (38 percent), other health professionals (33 percent), patient representatives (16 percent), and industry representatives (13 percent). The criterion’ effectiveness’ was given the highest relative weight (median score = 100, mean score = 93.33 ± 8.53). Meanwhile, the “population group” was unanimously ranked the least essential criterion (median score = 20, mean score = 30.22 ± 21.66). Figure 2 illustrates the average score for each criterion in descending order. The final weightage allocated for each criterion was calculated by dividing the mean score by the average score for all criteria.

Figure 2. Mean (SD) of criteria weight according to the relative importance rated by participants for the MCDA framework. MCDA, multicriteria decision analysis; SD, standard deviation.

Eight criteria and their associated weights were identified as effectiveness (19.6 percent), disease severity (15.6 percent), safety (14.2 percent), access to treatment (12.6 percent), economic consideration (12.2 percent), type of therapeutic treatment (11.5 percent), availability of alternatives (8.3 percent), and population group (6 percent), as represented through a line graph in Figure 3. A subgroup analysis was carried out to assess the differences in the criteria ranking between the four main participant groups and the overall average weightings. Generally, the six criteria ranked by allied health professionals and patient representatives were similar to the overall ranking. On the other hand, only four criteria ranked by the industry representatives matched those of the overall results. As depicted in Figure 3, there were no significant differences between the groups for all criteria, although “economic consideration” (H (3) = 9.105, P = 0.028) from the industry representatives’ group recorded the highest relative weight.

Figure 3. Criteria weight by the participant groups.

Value Assessment

During the pilot assessment of the proposed MCDA framework, the participants had to set a rating according to the agreed performance of each criterion for the five drugs used in treating three types of RD, as illustrated in Figure 4. The highest score (over 70 percent) was recorded by carglumic acid for the treatment of Propionyl-CoA carboxylase deficiency and methylmalonyl-CoA mutase deficiency. The average weighted scores for other drugs were above 60 percent, with three of them scoring above 65 percent. In all cases, “effectiveness” was the main contributing criterion to the final score estimate. In comparison, carglumic acid obtained the highest weighted score for each criterion, apart from “economic consideration,” which ranked the second lowest among all drugs. Supplementary 1 provides a summary of the performance score for each intervention.

Figure 4. MCDA assessment for the five drugs used in three RD treatments. MCDA, multicriteria decision analysis; RD, rare diseases.

Discussion

A more effective approach is required to manage complex decision making in financing RD treatment to replace the conventional HTA and cost-effectiveness analysis (27). Such approaches may include utilizing MCDA, a decision-making tool that considers multi-dimensional factors and compares medical technologies by combining individual criteria into one overall appraisal (21). Previously, Mohammadshahi et al. (28) reviewed that the majority of European countries utilized MCDA as the most common method for prioritizing orphan drugs and RD. Remarkably, the present study developed an MCDA framework to aid decision making in prioritization of fund allocation and reimbursement for RD in Malaysia based on the good practices recommended by Thokala et al. (21) and Marsh et al. (23). Moreover, the framework was deliberated on and agreed upon by various stakeholders, including policymakers, clinical experts, patient advocate groups, and patients.

Meanwhile, Németh et al. (25) revealed an inverse correlation between the number and complexity of questions to be answered and the complexity of the criteria weighting methodology. Considering the trade-off between the size and heterogeneity of the stakeholders involved in the exercise and the complexity of questions, this study selected the SMART method for the criteria weighting process, which was accepted by the group over its simplicity and feasibility to use for all stakeholders involved.

Similar to the method employed by Schey et al. (17) to determine the response of the stakeholders, this study developed web-based interactive survey tools, such as Mentimeter and Google form, to gain input on the brainstorming criteria and assign weight scores to each criterion. The eight prioritized criteria were then ranked by the participants as follows: effectiveness of treatment, disease severity, safety, access to treatment, economic consideration, type of therapeutic treatment, availability of alternatives, and population group. The results agree with an Australian study, which listed clinical benefit and safety as the top prioritized criteria, although the present study did not prioritize the quality of evidence, such as in this study (29). In contrast, the most frequent criteria identified by Mohammadshahi et al. (28) were cost-effectiveness, budget impact, and disease severity after analyzing six main categories: health outcomes and clinical implications, economic aspects, disease and population characteristics, therapeutic alternatives and uniqueness of orphan technologies, evidence, and other criteria addressing social and organizational criteria. The findings illustrate the uniqueness of issues pertaining to RD in the Malaysian healthcare setting.

Economic implications were ranked fifth on the Malaysian priority list. The high cost of orphan drugs will remain a key concern in any decision making, and treatments will not be prioritized if it is the only major criterion. Besides, stakeholders were more focused on facilitating access to treatment. The different currency exchange may also favor high-income countries. Nevertheless, Campillo-Artero et al. (20) cautioned against over-dependence on MCDA in spite of its advantages and suggested appropriate involvement of stakeholders. As such, the stakeholders’ views on this matter were considered. In fact, the patients’ and patient advocate groups’ perspectives were among the strengths of the proposed framework. Since patients are regarded as the end-user of any health policy decisions, they should have the opportunity to participate in the decision-making process (30).

Limitations

Given that the participants represent different working and educational backgrounds, they provide different perspectives that may account for the variation in individual weights across the criteria. The number of representatives from each affiliated group was also uneven, which may have resulted in over- or underestimation in the scores. Some of the clinicians, pharmaceutical representatives, patient advocate groups, and patients represent a specific disease and may be unfamiliar with the treatment and diseases that were presented. Besides, the usual method, language, and information used in the traditional HTA may not have been understood and appreciated by all, especially if they were not experts or represented the disease.

Initially, the stakeholders championed different diseases and had difficulty in deciding a consensus for the scoring. Although each stakeholder had their priorities, they took a while to appreciate the MCDA and scored different criteria during the mock exercise. Hence, similar suggestions of criteria were aggregated under eight criteria. Eventually, the committee decided to use these priority scores and revise them when necessary. In view of these limitations, this study emphasized the importance of the deliberation process.

Conclusion

This study described the construction of an MCDA framework to complement the committee deliberation for transparent and equitable decision making on fund allocation and reimbursement of orphan and RD treatment in Malaysia. The brainstorming criteria for the assessment and scoring of the priority weights reflected the priorities and perspectives of the stakeholders, where the NRDC generally accepted the developed MCDA framework. A follow-up pilot study of the framework will be conducted, with more deliberation and discussion to refine further and improve the framework. Extensive research on the perception of MCDA users could also be conducted, and the impact of applying MCDA in decision making to the healthcare system would provide further beneficial outcomes.