Hostname: page-component-5cf477f64f-fcbfl Total loading time: 0 Render date: 2025-04-02T21:34:40.169Z Has data issue: false hasContentIssue false

Navigating change: a comparative analysis of health technology assessment reforms across agencies – processes, drivers, and interdependencies

Published online by Cambridge University Press:  14 March 2025

Gayathri Kumar
Affiliation:
OHE – Office of Health Economics, London, UK
Priscila Radu
Affiliation:
OHE – Office of Health Economics, London, UK
Patricia Cubi-Molla*
Affiliation:
OHE – Office of Health Economics, London, UK
Martina Garau
Affiliation:
OHE – Office of Health Economics, London, UK
Eleanor Bell
Affiliation:
UKHSA, London, UK
Jia Pan
Affiliation:
Adelphi Values, London, UK
Ramiro Gilardino
Affiliation:
MSD, Zurich, Switzerland
Julie Van Bavel
Affiliation:
MSD, Zurich, Switzerland
Agnes Brandtmüller
Affiliation:
MSD, Budapest, Hungary
Katherine Nelson
Affiliation:
Drexel University, Philadelphia, PA, USA
Melinda Goodall
Affiliation:
Goodall HTA Consulting Limited, Manchester UK
*
Corresponding author: Patricia Cubi-Molla; Email: [email protected]
Rights & Permissions [Opens in a new window]

Abstract

Objectives

Health technology assessment (HTA) is a critical part of healthcare decision making in many countries. Changes in Methods and Processes (M&P) of HTA agencies can affect the time and degree of patient access to treatments. Published literature focuses on the different M&P adopted by HTA agencies, rather than on how these have come about over time. Our study investigates key HTA reforms and explores their drivers and interdependencies in a set of HTA agencies in Europe, Asia-Pacific, and North America.

Methods

We conducted a targeted literature review on M&P guidelines and subsequent changes to those, for 14 HTA agencies. We supplemented and validated initial findings with 29 semi-structured interviews with country-specific experts. We used analytical tools to create process maps, proactivity and influence networks, and clusters of HTA agencies.

Results

We found that processes leading to M&P reforms follow similar steps across HTA agencies. The three most important drivers to reforms were HTA practice and guidelines in other countries; the healthcare policy, legal, and political context within the agency’s country; and experience of challenges in the assessment by the HTA body itself. International collaborations have the potential to accelerate the evolution of HTA systems and the implementation of reforms.

Conclusion

We identified PBAC (Australia), CDA-AMC (Canada), NICE (England), IQWiG (Germany), and ZIN (the Netherlands) as HTA agencies that are catalysts of HTA reforms as well as internationally influential. International collaborations may represent a useful route to accelerate changes as long as they ensure wide stakeholder engagement at an early stage.

Type
Method
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0), which permits unrestricted re-use, distribution and reproduction, provided the original article is properly cited.
Copyright
© The Author(s), 2025. Published by Cambridge University Press

Introduction

Health technology assessment (HTA) is a critical part of healthcare decision-making in many countries. Current HTA agencies have different methods (their preferred technical approaches and practices on how to conduct HTA) and processes (steps followed and stakeholders involved in carrying out HTA). HTA methods and processes (M&P) can significantly impact recommendations made by HTA agencies (Reference Vreman, Mantel-Teeuwisse, Hövels, Leufkens and Goettsch1) and have wide-ranging effects on patients, providers, industry, and society as a whole. HTA M&P can also influence patient access to new treatments and impact research and development (R&D) investment decisions. Therefore, HTA M&P should evolve in response to scientific advances, changes in societal preferences, methodological developments, and challenging political contexts.

Published literature compares different agencies’ M&P in a static way (international comparison of HTA M&P at a particular point in time (Reference Heintz, Gerber-Grote, Ghabri, Hamers, Rupel and Slabe-Erker2;Reference Zisis, Naoum and Athanasakis3), and generally exploring a single topic of interest (Reference Zhang and Garau4Reference Kennedy-Martin, Slaap, Herdman, van Reenen, Kennedy-Martin and Greiner6)). Cross-border dynamics of HTA M&P (how guidelines evolve over time) are less analyzed in the literature (Reference Sharma, Aggarwal, Downey and Prinja7), and usually focus on the emergence of HTA organizations, publication of their first guidelines (Reference Oortwijn, Broos, Vondeling, Banta and Todorova8Reference Liu, Wu, Ahn, Kamae, Xie and Yang10), or refer to a specific topic (Reference Wang, Qiu, Zhou, Francois and Toumi11). This article is the first attempt, to our knowledge, to document past full or partial HTA reforms, analyze drivers and processes leading to these reforms, and show how HTA agencies influenced each other in the development and reviews of their M&P guidelines. Understanding what lies behind HTA reforms is important for stakeholders to identify opportunities for engagement, inform evidence generation matching forthcoming HTA requirements, and support policy discussions.

This article aims to identify and analyze recent changes in HTA M&P, explore the processes and drivers for these changes, and analyze the dynamics between countries in terms of proactivity in implementing changes and the degree of influence between them. We considered a sample of HTA agencies in Europe, Asia-Pacific, and North America, chosen as a representative model for the breadth of approaches to HTA implementation.

Methods

We conducted a targeted literature review and analyzed documents published from 2010 to 2023 related to M&P guidelines as well as changes made to those guidelines. Our research focused on HTA programs for pharmaceuticals, including medicines and vaccines, which start when a product is selected for assessment and concludes with a recommendation on funding within the healthcare system. Other types of health technologies (e.g., devices, digital therapeutics) and other activities that may be carried out by HTA agencies, including horizon scanning, were not included in the scope. We supplemented our findings with semi-structured interviews with country-specific HTA experts.

We investigated HTA agencies in 14 countries, as described in Box 1. These countries were chosen as examples of more established HTA agencies in Europe and the Asia-Pacific region.

Box 1. Full and abbreviated name of Health Technology Agencies for each country included in the study

The pragmatic search of HTA agency websites and bibliographic databases was conducted in two stages. The first stage identified relevant documents published by the HTA agencies of interest and secondary literature relating to major changes in HTA M&P in general. In the second stage, we identified information specific to changes in the following topics: discount rates, modifiers, patient involvement in HTA (PI), real-world evidence (RWE), and surrogate end points. These topics were deemed particularly dominant in the recent HTA debate, both historically and with the aim of assessing innovative medicines. Data were extracted on: the timing of key M&P changes; qualitative descriptions of the policy changes and the agency’s positions on topics; drivers of reform; and references to other HTA agencies in the guidelines. Further details on the search strategy and data extraction protocol are shown in Supplementary Material 1. Our findings relate to changes in HTA M&P in general and specifically to the five HTA topics of interest.

Subsequently, we interviewed 29 experts with HTA experience with the agencies of interest (two experts per agency and an additional expert from the EUnetHTA collaboration). The interviews aimed to validate the literature review findings and elicit additional insights into the local context, including the interviewees’ views on the proactivity and influence of HTA agencies and opportunities and barriers to reforms. The interview guide is available in Supplementary Material 2.

We combined the findings from the literature review and interviews using several analytical tools. First, we tabulated the timings of country-specific HTA M&P updates, distinguishing between full and partial revisions. Second, we created a diagram to represent the process followed by each HTA agency to consider, discuss, and implement changes in the methods guidelines. Third, we created a framework that lists all the drivers that may trigger a review of the M&P or lead to the implementation of changes. The framework was based on the results from the literature review and was validated through expert interviews. Fourth, we provide the frequency that each driver in the framework was identified as an influence for changes in country-specific HTA M&P guidelines. Fifth, we created a network diagram representing the level of influence exerted by HTA agencies proxied by the number of times their M&P is referenced in other HTA agencies’ guidelines or related publications. We also created a heatmap of HTA agency proactivity, which presents the relative order in which countries implemented their first reform by topic. Finally, we grouped HTA agencies regarding their proactiveness to changes in M&P and the influence of those changes over other HTA agencies.

Dynamics between countries were identified by way of exploring (a) historical correlation that may occur because of the timeline; (b) historical causation (i.e., M&P changes by an HTA agency that are directly influenced by changes of another agency; and (c) prospective collaboration or agreements between countries to align on M&P and share learning.

Results

We selected 374 publications in the literature search. Supplementary Material 3 presents the publication years of guidelines and updates identified in the review. We differentiate between full revisions of HTA guidelines and partial updates (if changes are only sought for specific sections in the guidelines). Before 2000, the agencies PBAC, CDA-AMC (previously CADTH), INFARMED, and ZIN had already published their HTA M&P guidelines – with full revisions in the case of PBAC and CDA-AMC (12Reference Bos16). By 2016, all the countries in our list had their M&P guidelines published (1728). We also observe that the revisions of these guidelines become more frequent over time.

Process followed for HTA M&P reviews

Evidence on the reform process followed by PBAC, NICE, IQWiG, CDA-AMC, and DMC was found primarily in the literature, and it was more formally defined compared to the reform process followed by the other HTA agencies in scope, where interviewees’ input was key to retrieve it. As a result, there are varying opportunities and risks for stakeholder interactions throughout each agency’s process.

Our findings suggest that M&P reforms follow similar steps across HTA agencies, as depicted in the process map that reflects the process for NICE M&P updates (see Figure 1) – although timelines and the extent of stakeholder involvement may differ (2932). The process illustrated in Figure 1 most applies to changes in the methods, as changes related to the processes often occur separately.

Figure 1. Process to guidelines and methods review in NICE. *Note that NICE process for M&P updates has changed to a modular approach where large reviews will no longer occur. We use this example because of its robustness and its relevance to past reviews, which are the focus of our analysis.

The reform process is usually initiated by a review of existing methods, with emphasis on identifying evidence supporting the case for change. The next stage includes a draft “proposal of change” document informed by the review and prior informal discussions with stakeholders to gather feedback and raise issues with the previous method updates. The “proposal of change” document may be accompanied by stakeholder meetings for some agencies to discuss findings and proposed changes (e.g., INFARMED and ZIN). A public consultation usually follows in which stakeholders from industry, patient organizations, academia, and members of the public are invited to share their views on the project. The format of the stakeholder consultation may be via online questionnaires or in-person interviews, in addition to several informal feedback points throughout the process. Feedback is incorporated into the final HTA methods guideline, which is subsequently published.

Major HTA M&P updates tend to happen in a 4- to 6-year cycle; nevertheless, when the need for a method update arises, these may be initiated outside of the update cycle. For example, NICE’s single technology appraisal process was introduced off-cycle in 2006 and was motivated by industry demand; and future NICE methods updates will use a modular and iterative approach when needed, to be more agile in reviewing and introducing updates in the future (33).

Framework of HTA M&P drivers

We identified the drivers of M&P reforms and categorized these into three themes: stakeholders, country-specific context, and cross-border context. Stakeholders include HTA agencies, academics, patient representatives, healthcare professionals, industry, and society. The country-specific context refers to healthcare policy, legal, and political context. For the cross-border context, we highlighted the following drivers: scientific advances in health technologies and/or change in the R&D process; regulatory approval process and pathway changes; HTA practice or guidelines in other countries; and external shocks. Table 1 shows the complete list of drivers of changes in HTA M&P, their descriptions, and some examples identified in the literature review.

Table 1. List of drivers of changes in HTA M&P, description, and examples

Frequency of drivers by country

The frequency of mentions of drivers, both in the literature and predominantly by interviewees, in relation to a specific country is presented in Table 2. The three most important drivers are HTA practice or guidelines in other countries (18 instances across all countries); the healthcare policy, legal, and political context (16); and the HTA body itself (15). International best practices are considered by most of the HTA agencies explored in this study, except for AIFA and TLV, where evidence of such practices was not found. Updates to guidelines can also be triggered by a change in national government and subsequently policy, particularly for countries in which HTA M&P are closely intertwined with legal statutes. For example, we identified that interest rates in the country impacted the discount rate recommended by HAS and DMC (34;35).

Table 2. Frequency of drivers, as mentioned in the literature and by interviewees, of M&P reform relating to key HTA topics, by country

Stakeholders that drive M&P reforms include primarily the HTA body itself, followed jointly by patients and industry, then academics, and society and healthcare professionals. Many drivers stem from the need to improve on existing M&P for HTA, due to challenges with assessment throughout the process as well as arising from the HTA body’s internal perception on the topics of interest. Academic position, generally sought directly by the HTA agency, and updates to the methodology surrounding the topic were also shown to drive changes to HTA M&P. In general, HTA agencies seek to understand stakeholders’ needs and opinions, and evidence-based arguments from stakeholder groups can be a key driver to change.

Scientific advances in certain health technologies or changes in R&D processes may simply necessitate updated guidelines to assess the relevant intervention accurately; however, this driver was only explicitly mentioned twice across all countries. We did not find evidence that external shocks, such as the COVID-19 pandemic, and the regulatory approval process are a cause of changes for the countries in scope.

Influence exerted by other HTA agencies

The number of times an HTA agency’s M&P is referenced in another HTA agency’s guidelines or publications is used as a proxy for their level of influence. Figure 2 depicts the direction of influence in a network diagram. PBAC, CDA-AMC, and NICE were referenced as the most impactful, while AIFA, AEMPS, DMC, INFARMED, and ACE were not referred to at all by other HTA bodies in scope.

Figure 2. Network diagram of HTA agency influence. The direction of influence is represented by the direction of the arrowhead. For example, an arrow pointing from HAS to PBAC would mean that HAS mentions PBAC in its guidelines. A double-headed arrow indicates that both HTA agencies mention each other in their guidelines (e.g., CDA-AMC and PBAC). The number in brackets represents the number of times an agency is mentioned by other agencies included in the study, and the size of the nodes is proportional to that. Agencies that have no number in brackets are not mentioned by another agency (e.g., INFARMED). Likewise, agencies that neither mention nor are mentioned by another agency have no links (e.g., AIFA).

Heatmap of proactivity

Figure 3 highlights which HTA agencies were first to implement changes to their guidelines for each topic. The numbering indicates the relative position of their updates compared to the other countries. In cases where HTA agencies changed their guidelines in the same year, they were assigned the same relative position. Agencies that do not consider a topic in their guidelines or have not changed their guidelines since 2010 are represented by white and gray cells, respectively.

Figure 3. Heatmap of changes to HTA methods or processes. The heatmap depicts the relative ordering of M&P guidelines updates relating to each topic for the HTA agencies in scope. Gray cells indicate that there has been no change in the HTA agency’s stance on the topic since 2010. Non-shaded cells denote that the HTA agency does not explicitly refer to the topic in their guidelines. Health technology assessment; NL: The Netherlands.

The HTA agencies that were first, second, or third movers in most topics were PBAC (first mover in discount rates, modifiers, and surrogate end points); NICE and IQWiG (among the top three movers in four topics); and CDA-AMC (three topics). AIFA, INFARMED, CDE, and ZIN were each second or third movers in one topic but were later (fourth onward) to move across other topics, to have no change, or not consider a topic at all in their guidelines. KCE, DMC, ACE, and AEMPS were fifth or later to move, if at all; and TLV either had no change or did not consider any of the topics.

The heatmap does not convey the level of innovation in an agency toward a specific topic, suggest comparability, or express the value judgment of different agencies or countries. Some countries may have had well-established evaluation methods for some topics from the outset and, therefore, did not require any changes, such as PBAC and TLV’s stated positions on accepting surrogate outcomes in the absence of final outcomes since 1995 and 2003, respectively. The stance is comparable with current guidelines from other agencies that may have enacted multiple or only fairly recent changes to reach the same position, such as INFARMED.

Clusters of HTA agencies by proactivity and influence

We grouped HTA agencies into three clusters based on our analysis of proactivity and influence reported in previous sections, alongside insights provided by expert interviewees: catalysts (NICE, PBAC, ZIN, CDA-AMC, and IQWiG); traditionalists (HAS, TLV, and KCE); and observers (DMC, AIFA, INFARMED, ACE, AEMPS, and CDE).

Catalysts

HTA agencies defined as catalyst are proactive in implementing M&P changes, and those changes impact other HTA bodies. Our findings highlight that NICE is the most proactive HTA agency, with more than four full revisions of its initial M&P guidelines. NICE is also identified as the most influential among the included agencies, with 10 other HTA agencies (CDA-AMC (36), HAS (37), IQWiG (38), AEMPS (Reference Puig-Junoy, Oliva-Moreno, Trapero-Bertran and Abellan-Perpiñan39), KCE (Reference Cleemput and Neyt18), DMC (35), INFARMED (40), ACE (41), CDE (27), and ZIN (28)) referencing NICE in their guidelines. Besides that, NICE International provides advisory services for international health organizations, ministries, and government agencies (42) and is involved in an international collaboration spanning three continents (43). NICE has also previously been actively involved in EUnetHTA Joint Actions (44).

Similar to NICE, PBAC stands out as a high-influence HTA body in our analysis. PBAC is mentioned in the HTA guidelines of six other agencies (KCE (Reference Cleemput and Neyt18), CDA-AMC (19), HAS (37), IQWiG (45), AEMPS (Reference Puig-Junoy, Oliva-Moreno, Trapero-Bertran and Abellan-Perpiñan39), and ACE (41)). Our findings also identify PBAC as a first mover in providing M&P guidelines updates related to discount rates, modifiers, and surrogate end points.

While ZIN implemented reforms in all topics, these changes were introduced at a relatively late stage – compared to the order of the agencies in our sample. ZIN has influenced the M&P guidelines of three other agencies (HAS (34), IQWiG (38), and KCE (Reference Cleemput, Neyt, Thiry, De Laet and Leys46)). While not explicitly referenced in NICE’s 2022 M&P guidelines, ZIN’s proportional shortfall approach to capturing severity as a modifier has most likely influenced NICE’s approach to accounting for disease severity (47). ZIN also shows involvement in multiple international collaborations, such as the EUnetHTA Joint Clinical Assessment (JCA) Committee, EUnetHTA21, and EU IVDR (4850).

CDA-AMC is also highly proactive in updating M&P guidelines around discount rates, RWE, and surrogate end points. CDA-AMC is referenced in the HTA guidelines of six other agencies (PBAC (13), HAS (51), IQWiG (38), AEMPS (Reference Puig-Junoy, Oliva-Moreno, Trapero-Bertran and Abellan-Perpiñan39), INFARMED (40), and ACE (41)) and is involved in an international collaboration with eight other global HTA agencies (43). Our results also identify IQWiG as a relatively proactive and influential HTA agency, although to a lesser extent than the other catalyst agencies. HAS (34) and AEMPS (Reference Puig-Junoy, Oliva-Moreno, Trapero-Bertran and Abellan-Perpiñan39) reference IQWiG are described in their guidelines.

Traditionalists

We label HTA agencies as traditionalist if they exert some degree of influence over other HTA agencies and take a reactive approach to implementing changes in their M&P guidelines. HAS published its first M&P guidelines in 2011, and we only identified one full revision dating 2020. Topic-specific reforms were also relatively late within our sample, suggesting that HAS is generally reactive to HTA reforms. We consider HAS influential, as it is referenced in the guidelines of KCE (Reference Cleemput and Neyt18), IQWiG (38) and AEMPS (Reference Puig-Junoy, Oliva-Moreno, Trapero-Bertran and Abellan-Perpiñan39), and it contributes to international initiatives through its involvement in EUnetHTA. HAS’s early access process is often referred by other agencies, such as AIFA.

We observed the limited proactivity of TLV and KCE in instigating reform. Since the publication of the first M&P guidelines (TLV in 2003 and KCE in 2008), they were only reviewed once (partial review for TVL in 2017 and full review for KCE in 2012). TLV and KCE are considered moderately influential, as they are both referenced in HAS’ guidelines (37) and engaged in international collaborations, such as EUnetHTA and JNHB (Joint Nordin HTA-Bodies) (in the case of TLV) (52).

Observers

We consider an HTA agency to be an observer if they are generally a “late mover” in implementing reforms and have little influence on other agencies’ M&P guidelines. For example, INFARMED was one of the first European HTA agencies to formalize the HTA M&P in a written document (15). However, it has been a “late mover” in reforms to HTA topics (exception for patient involvement in HTA), and updated M&P guidelines only in 2019, being less influential among the agencies in scope.

DMC, AIFA, and AEMPS have been “late movers” in implementing topic-specific reforms, and their M&P guidelines are not referenced by other agencies. DMC has shown signs of gradual involvement in the international debate via EUnetHTA, as well as its recent entry into the JNHB collaboration, while AIFA and AEMPS are engaged in the EUnetHTA collaboration.

ACE and CDE are also “late-movers” on M&P reforms. Although ACE references CDE’s guideline, no other agencies in this study have referred to ACE’s or CDE’s guidelines.

Discussion

Our research identified variations among agencies in how formal and structured their M&P reform processes are. NICE is an example where there is a process with clear steps, including stakeholder involvement and opportunities for their input. Other agencies have less transparent or well-defined processes, which might make it challenging for external parties to anticipate, get involved, and contribute. This could represent a key priority for HTA agencies to address to ensure inclusivity and broad endorsement among local stakeholders.

Most of the drivers identified in the literature for change referred to the perspectives of different stakeholders, such as academics, patients, and HTA experts. We add that the foundation of evidence-based reforms should also include recent robust empirical evidence (including societal preference studies) and method development.

International collaborations have the potential to accelerate the evolution of HTA systems and the implementation of reforms by enabling agencies to anticipate and address common challenges in a timely and efficient manner. We observe an increase in international collaborations between HTA agencies. A recent collaboration includes eight agencies across Australia, Canada, and the United Kingdom (43), which aims to improve work sharing and collaborate on horizon scanning and methods development. Given that most agencies in this collaboration were classed as catalysts in our analysis, the authors expect that they will provide international leadership and be a crucial drive for HTA method evolution. EUnetHTA was one of the first examples of joint HTA work and information sharing among a large number of countries, and some of its principles have informed the method guidelines for the JCA, part of the Regulation of HTA in the European Union. We anticipate that the latter will have a predominant role in shaping the M&P of HTA systems in European member states in the coming years. Finally, Joint Nordic HTA Bodies (53) (previously known as FINOSE) provides an example of how neighbor countries with similar HTA systems can benefit from cooperation to promote the convergence of methods and efficient assessments. Going forward, collaborations should promote more streamlined and regular updates, similar to the modular approach that NICE is implementing, and also pool resources together to conduct initial literature reviews of HTA practices, identify emerging innovative methods, select those suitable for HTA practice, and pilot them jointly.

Finally, to be fully successful, collaborations should ensure full and early involvement of stakeholders, to increase the legitimacy of changes, improve evidence generation, and facilitate implementation of reforms at the national level.

As a limitation, our literature review only included publications in English, which might have led to the exclusion of relevant documents in local languages. Where identified as relevant by experts, additional non-English documents were added and machine-translated (Google Translate). Only a few documents related to CDE were professionally translated into English, as machine translation was not deemed appropriate. Language bias could have also impacted the reference to guidelines across HTA agencies, resulting in primarily English-speaking agencies (NICE, PBAC, and CDA-AMC) being more likely to be referenced by other agencies, and hence being considered as catalysts.

We have tried to mitigate this by validating the literature results with interviews with experts from all countries. An additional way to further limit this could be to include experts who speak the language of each country considered in the writing process. Furthermore, we encourage similar research to occur in other regions where the same languages are spoken, such as Spanish-speaking communities in South America.

We also note that, as our study focuses on more established agencies in Europe and south-east Asia, there is a risk that our choice of HTA agencies may not be representative of other areas where HTA is in development or nascent. The exclusion of emerging HTA agencies could also influence the generalizability of the conclusions.

The core literature review was conducted from January 2000 to April 2022. Additional updates published between the end dates of the searches and September 2023 were identified on an ad hoc basis. We also note that PBAC was surrounded by a policy and methods review at the time this article was written; therefore, its current reform processes and drivers might not be reflected here.

Evidence on the drivers was not extensive. However, it is important to note that documents related to past reforms are often removed from agency websites, and specific factors leading to individual reforms may only appear in agencies’ committee or board papers that were not included here. We did not make assumptions on potential interactions between drivers, only reporting on how they were mentioned within the literature and by expert interviews.

The interview process was based on a limited sample size, meaning that some experiences or views on past reforms may not have been captured adequately. We also restricted the number of interviewees to two experts per HTA agency. This impacts the analysis of those HTA agencies with less detailed or specific M&P guidelines, allowing more room for flexibility in practice. In those cases, our findings from the literature review do not entirely align with the experts’ opinions. For example, interviewees noted that TLV focuses on changing the application of methodology in practice rather than changing the documented guidelines; and this may explain the observed limited proactivity of the TLV in instigating reforms.

Finally, it is important to note that our results depict influence and proactivity in relative rather than absolute terms. Although the list of countries in scope is extensive, the relative positions can change with the addition (or exclusion) of other HTA agencies to the scope. For example, the interviews revealed that several countries in Latin America and Asia are developing their M&P based on CDA-AMC guidelines; that PBAC’s M&P guidelines serve as a model for the HTA approach in Japan; and the influence of INFARMED among the HTA agencies of Greece, Romania, and Cyprus. However, those links were out of this project’s scope and are not reflected in our clustering exercise.

Further research should explore the impact of HTA reforms on a set of quantitative metrics, including timelines to recommendations, degree of patient access to interventions, and patient outcomes; and qualitative ones, including quality of stakeholders’ submissions and of the decision-making process. Specifically, in the context of EU HTA regulation, new research can map its impact on national HTA M&P guidelines after a few years of implementation. Collaborations across agencies and, more generally, the research community should define and test optimal processes for M&P updates and their implementation. An example is provided by the framework developed by Jiu et al. (Reference Jiu, Hogervorst, Vreman, Mantel-Teeuwisse and Goettsch54) for the introduction of novel HTA methods.

Conclusion

To the best of the author’s knowledge, this article is the first attempt to document past full or partial HTA reforms and analyze the drivers and processes leading to these reforms, including how HTA agencies have influenced each other in the development and reviews of their M&P guidelines. They identified PBAC, CDA-AMC, NICE, IQWiG, and ZIN as HTA agencies that are catalysts of HTA reforms as well as internationally prominent. NICE, PBAC, and CDA-AMC are among the agencies with the most influence on the M&P guidance of other countries. International collaborations (such as the recent one between HTA agencies in Australia, Canada, and the United Kingdom, as well as the Nordic collaboration) represent a valuable route to accelerate changes and ensure comprehensive stakeholder engagement at an early stage. These alliances could create convergence between HTA guidelines and provide international leadership in methods change. This could be beneficial for those agencies with limited or no guidance on certain topics. However, their success depends on how the national legislative framework and political objectives are addressed.

Future research should assess how HTA reforms impact HTA systems, such as timelines to develop recommendations, degree of patient access to interventions, and, in the longer term, patient outcomes.

Supplementary material

The supplementary material for this article can be found at http://doi.org/10.1017/S0266462325000133.

Acknowledgments

Stina Salomonsson and Taruja Karmarkar from MSD contributed to the final review of this article.

Funding statement

This study was sponsored by Merck Sharp & Dohme Corp. through contracted research. The contract agreement guaranteed academic independence and the right to publish research results without interference from the sponsor.

Competing interest

Ramiro Gilardino, Julie Van Bavel, Agnes Brandtmüller, Katherine Nelson, and Melinda Goodall were employed by Merck Sharp & Dohme Corp at the time when the manuscript was written.

Eleanor Bell, Patricia Cubi-Molla, Martina Garau, Gayathri Kumar, Jia Pan, and Priscila Radu were employed by the Office of Health Economics at the time when the manuscript was written. The Office of Health Economics receives research grants and consulting income from a range of public and private funders including governments, nonprofits and pharmaceutical companies.

Ramiro Gilardino, Julie Van Bavel, Katherine Nelson, and Melinda Goodall hold shares/stock options in Merck Sharp & Dohme Corp.

Gayathri Kumar holds shares/stock options in Eli Lilly & Company.

Ramiro Gilardino holds honorary roles as Co-Chair of HTAi Global Policy Forum Task Force on Life Cycle Approaches for HTA and advisory board member for Americas Health Foundation, Washington, DC is a Non-Executive Director of the Board of Directors at ISPOR, in the United States, and has received consulting fees from Celgene SARL, a Bristol Myers Squib Subsidiary in Switzerland (June 2022–January 2023).

Footnotes

Authors were at OHE (Eleanor Bell and Jia Pan) or MSD (Katherine Nelson and Melinda Goodall) at the time of the study.

References

Vreman, RA, Mantel-Teeuwisse, AK, Hövels, AM, Leufkens, HGM, Goettsch, WG. Differences in health technology assessment recommendations among European jurisdictions: The role of practice variations. Value Health. 2020 Jan 1;23(1):1016.Google Scholar
Heintz, E, Gerber-Grote, A, Ghabri, S, Hamers, FF, Rupel, VP, Slabe-Erker, R, et al. Is there a European view on health economic evaluations? Results from a synopsis of methodological guidelines used in the EUnetHTA partner countries. PharmacoEconomics. 2016 Jan 1;34(1):5976.Google Scholar
Zisis, K, Naoum, P, Athanasakis, K. Qualitative comparative analysis of health economic evaluation guidelines for health technology assessment in European countries. Int J Technol Assess Health Care. 2021;10:e2.Google Scholar
Zhang, K, Garau, M. International cost-effectiveness thresholds and modifiers for HTA decision making [Internet]. Office of Health Economics; 2020. Available from: https://www.ohe.org/publications/international-cost-effectiveness-thresholds-and-modifiers-hta-decision-makingGoogle Scholar
Angelis, A, Lange, A, Kanavos, P. Using health technology assessment to assess the value of new medicines: Results of a systematic review and expert consultation across eight European countries. Eur J Health Econ. 2018 Jan;19(1):123152.Google Scholar
Kennedy-Martin, M, Slaap, B, Herdman, M, van Reenen, M, Kennedy-Martin, T, Greiner, W, et al. Which multi-attribute utility instruments are recommended for use in cost-utility analysis? A review of national health technology assessment (HTA) guidelines. Eur J Health Econ. 2020;21(8):12451257.Google Scholar
Sharma, D, Aggarwal, AK, Downey, LE, Prinja, S. National healthcare economic evaluation guidelines: A cross-country comparison. PharmacoEconomics Open. 2021 Jan 10;5(3):349364.Google Scholar
Oortwijn, W, Broos, P, Vondeling, H, Banta, D, Todorova, L. Mapping of health technology assessment in selected countries. Int J Technol Assess Health Care. 2013 Oct;29(4):424434.Google Scholar
The Access Delivery Partnership. The Emergence of Health Technology Assessment Organizations: Lessons from five countries [Internet]. 2017. Available from: https://adphealth.org/upload/resource/The_Emergence_of_HTA_Organizations_2018_ADP_003.pdfGoogle Scholar
Liu, G, Wu, EQ, Ahn, J, Kamae, I, Xie, J, Yang, H. The development of health technology assessment in Asia: Current status and future trends. Value Health Reg Issues. 2020 May 1;21:3944.Google Scholar
Wang, Y, Qiu, T, Zhou, J, Francois, C, Toumi, M. Which criteria are considered and how are they evaluated in health technology assessments? A review of methodological guidelines used in Western and Asian countries. Appl Health Econ Health Policy. 2021 May 1;19(3):281304.Google Scholar
PBAC. 1995 Guidelines for the Pharmaceutical Industry on Preparation of Submissions to the Pharmaceutical Benefits Advisory Committee: Including major submissions involving economic analyses [Internet]. Available from: https://pbac.pbs.gov.au/content/information/archived-versions/pbac-guidelines-1995.docGoogle Scholar
PBAC. Interim document to accompany the guidelines for the pharmaceutical industry on preparation of submissions to the Pharmaceutical Benefits Advisory Committee Including major submissions involving economic analyses [Internet]. 2000. Available from: https://pbac.pbs.gov.au/content/information/archived-versions/pbac-guidelines-interim-version-to-industry-2000.docGoogle Scholar
CADTH. Guidelines for the economic evaluation of health technologies: Canada. 2nd ed. [Internet]. Government of Canada; 1997. Available from: https://www.cadth.ca/media/pdf/peg_e.pdfGoogle Scholar
Bos, M. Health technology assessment in The Netherlands. Int J Technol Assess Health Care. 2000;16(2):485519.Google Scholar
PBAC. Guidelines for preparing a submission to the Pharmaceutical Benefits Advisory Committee Version 5.0 [Internet]. 2016. Available from: https://pbac.pbs.gov.au/content/information/files/pbac-guidelines-version-5.pdfGoogle Scholar
Cleemput, I, Neyt, M. Belgian guidelines for economic evaluations and budget impact analyses. 2nd ed. 2012. p. 84. https://kce.fgov.be/sites/default/files/2021-11/KCE_183_economic_evaluations_second_edition_Report_update.pdfGoogle Scholar
CADTH. Guidelines for the economic evaluation of health technologies: Canada. 3rd ed. [Internet]. Government of Canada; 2006. Available from: https://www.cadth.ca/media/pdf/186_EconomicGuidelines_e.pdfGoogle Scholar
DMC. Handbook for the Medical Council process and method regarding new drugs and indication extensions. 2021. https://filer.medicinraadet.dk/media/5eibukbr/the-danish-medicines-council-methods-guide-for-assessing-new-pharmaceuticals-version-1-3.pdfGoogle Scholar
HAS. Choices in methods for economic evaluation [Internet]. 2011. Available from: https://www.has-sante.fr/upload/docs/application/pdf/2011-11/guide_methodo_vf.pdfGoogle Scholar
IQWiG. General methods Version 4.2 [Internet]. 2015. Available from: https://www.iqwig.de/methoden/iqwig_general_methods_version_204-2.pdfGoogle Scholar
AIFA. Guidelines for the compilation of the dossier in support of the application refundability and price of a medicine [Internet]. 2020. Available from: https://www.aifa.gov.it/documents/20142/1307543/2021.01.22_estratto_linee_guida_sezione_E.pdfGoogle Scholar
López-Bastida, J, Oliva, J, Antoñanzas, F, García-Altés, A, Gisbert, R, Mar, J, et al. Spanish recommendations on economic evaluation of health technologies. Eur J Health Econ. 2010 Oct;11(5):513–20.Google Scholar
TLV. General guidelines for economic evaluations from the Pharmaceutical Benefits Board (LFNAR 2003:2). 2003. p. 3. https://www.tlv.se/download/18.2e53241415e842ce95514e9/1510316396792/Guidelines-for-economic-evaluations-LFNAR-2003-2.pdf.Google Scholar
CDE. Methodological guidelines for the cost-effectiveness analysis of healthcare technology assessment. 2013.Google Scholar
National Health Care Institute. Guideline for economic evaluations in healthcare [Internet]. Ministerie van Volksgezondheid, Welzijn en Sport; 2016. Available from: https://english.zorginstituutnederland.nl/publications/reports/2016/06/16/guideline-for-economic-evaluations-in-healthcareGoogle Scholar
NICE. Review of the health technology evaluation processes. 2021. Available upon request from the corresponding author.Google Scholar
NICE. NICE partners with international health technology assessment bodies to boost collaboration on shared opportunities and challenges. News. 2022. Available from: https://www.nice.org.uk/news/article/nice-partners-with-international-health-technology-assessment-bodiesGoogle Scholar
NICE. NICE publishes new combined methods and processes manual and topic selection manual for its health technology evaluation programmes. 2022. Available from: https://www.nice.org.uk/news/article/nice-publishes-new-combined-methods-and-processes-manual-and-topic-selection-manual-for-its-health-technology-evaluation-programmesGoogle Scholar
DMC. The Danish Medicines Council process guide for assessing new pharmaceuticals [Internet]. 2021. Available from: https://medicinraadet.dk/media/ckyg1cde/the-danish-medicines-councils-process-guide-for-assessing-new-pharmaceuticals-version-1-2.pdfGoogle Scholar
CADTH. Guidelines for the economic evaluation of health technologies: Canada. 4th ed. Government of Canada; 2018.Google Scholar
HAS. Choices in methods for economic evaluation (English version) [Internet]. 2012. Available from: https://www.has-sante.fr/upload/docs/application/pdf/2012-10/choices_in_methods_for_economic_evaluation.pdfGoogle Scholar
IQWiG. General methods version 2.0 [Internet]. 2006. Available from: https://www.iqwig.de/methoden/methods_iqwig_version_20.pdfGoogle Scholar
Puig-Junoy, J, Oliva-Moreno, J, Trapero-Bertran, M, Abellan-Perpiñan, JM. English translation: Guide and recommendations for the performance and presentation of economic evaluations and budgetary impact analaysis of medication. 2014. https://catsalut.gencat.cat/web/.content/minisite/catsalut/proveidors_professionals/medicaments_farmacia/farmaeconomica/caeip/gaeip_publica_castellano_octubre2014_catsalut.pdf.Google Scholar
NICE. About NICE International | NICE International | What we do | About. 2020. Available from: https://www.nice.org.uk/about/what-we-do/nice-international/about-nice-internationalGoogle Scholar
NICE. International health technology assessment collaboration expands. News. 2023. Available from: https://www.nice.org.uk/news/article/international-health-technology-assessment-collaboration-expandsGoogle Scholar
NICE – EUnetHTA [Internet]. 2018. Available from: https://www.eunethta.eu/nice/Google Scholar
IQWiG. General methods version 1.0 [Internet]. 2005. Available from: https://www.iqwig.de/methoden/methods_iqwig_version_10.pdfGoogle Scholar
Cleemput, I, Neyt, M, Thiry, N, De Laet, C, Leys, M. Threshold values for costeffectiveness in health care [Internet]. KCE; 2008. Available from: https://kce.fgov.be/sites/default/files/2021-11/d20081027396.pdfGoogle Scholar
Office of Health Economics. NICE’s severity modifier: A step in the right direction, but still a long way to go. 2022. Available from: https://www.ohe.org/insight/nices-severity-modifier-step-right-direction-still-long-way-go/Google Scholar
EUnetHTA. JCA – EUnetHTA [Internet]. 2021. Available from: https://www.eunethta.eu/jca/Google Scholar
EU. The European Union in vitro diagnostics regulation – regulation (EU) 2017/746 (EU IVDR) [Internet]. 2017. Available from: https://euivdr.com/Google Scholar
EUnetHTA. Home – EUnetHTA [Internet]. 2017. Available from: https://www.eunethta.eu/Google Scholar
HAS. Doctrine de la Commission d’évaluation économique et de santé publique. 2021. https://www.has-sante.fr/upload/docs/application/pdf/2021-09/doctrine_de_la_ceesp.pdf.Google Scholar
Joint Nordic HTA-Bodies [Internet]. 2024. Available from: https://jnhtabodies.org/Google Scholar
Jiu, L, Hogervorst, MA, Vreman, RA, Mantel-Teeuwisse, AK, Goettsch, WG. Understanding innovation of health technology assessment methods: The IHTAM framework. Int J Technol Assess Health Care. 2022;38(1):e16.Google Scholar
PBAC. Guidelines for preparing submissions to the Pharmaceutical Benefits Advisory Committee Version 4.3 [Internet]. 2008. Available from: https://pbac.pbs.gov.au/content/information/archived-versions/pbac-guidelines-v4-3-2008.pdfGoogle Scholar
IQWiG. General methods Version 6.0 [Internet]. 2020. Available from: https://www.iqwig.de/methoden/general-methods_version-6-0.pdfGoogle Scholar
Australian Government Department of Health. Reference Committee for the Health Technology assessment policy and methods review [Internet]. 2021. Available from: https://www.pbs.gov.au/info/news/2022/04/reference-committee-HTA-reviewGoogle Scholar
NICE. Briefing paper for methods review workshop on structured decision making [Internet]. 2011. Available from: https://www.sheffield.ac.uk/sites/default/files/2022-02/DSU_TAMethodsGuideReviewSupportingDocuments.pdfGoogle Scholar
Figure 0

Figure 1. Process to guidelines and methods review in NICE. *Note that NICE process for M&P updates has changed to a modular approach where large reviews will no longer occur. We use this example because of its robustness and its relevance to past reviews, which are the focus of our analysis.

Figure 1

Table 1. List of drivers of changes in HTA M&P, description, and examples

Figure 2

Table 2. Frequency of drivers, as mentioned in the literature and by interviewees, of M&P reform relating to key HTA topics, by country

Figure 3

Figure 2. Network diagram of HTA agency influence. The direction of influence is represented by the direction of the arrowhead. For example, an arrow pointing from HAS to PBAC would mean that HAS mentions PBAC in its guidelines. A double-headed arrow indicates that both HTA agencies mention each other in their guidelines (e.g., CDA-AMC and PBAC). The number in brackets represents the number of times an agency is mentioned by other agencies included in the study, and the size of the nodes is proportional to that. Agencies that have no number in brackets are not mentioned by another agency (e.g., INFARMED). Likewise, agencies that neither mention nor are mentioned by another agency have no links (e.g., AIFA).

Figure 4

Figure 3. Heatmap of changes to HTA methods or processes. The heatmap depicts the relative ordering of M&P guidelines updates relating to each topic for the HTA agencies in scope. Gray cells indicate that there has been no change in the HTA agency’s stance on the topic since 2010. Non-shaded cells denote that the HTA agency does not explicitly refer to the topic in their guidelines. Health technology assessment; NL: The Netherlands.

Supplementary material: File

Kumar et al. supplementary material

Kumar et al. supplementary material
Download Kumar et al. supplementary material(File)
File 380.2 KB