Healthcare systems struggle to provide access to innovative health care with limited budgets so prefer to allocate funding to technologies with demonstrated value for money (Reference Henshall and Schuller1;Reference Lakdawalla, Doshi and Garrison2). Establishing the value of a new technology, however, is a complex process. Multiple perspectives on value must be considered and reconciled as stakeholders, including payers, patients, and manufacturers, put forward an increasing number of different value definitions (Reference Lakdawalla, Doshi and Garrison2).
Health technology assessment (HTA) is designed to inform value judgements and policy making (Reference Henshall and Schuller1;Reference Akehurst, Abadie, Renaudin and Sarkozy3). HTA systematically and transparently summarizes the evidence on various properties of a healthcare technology and the likely consequences of its adoption (Reference Akehurst, Abadie, Renaudin and Sarkozy3;4). Certain aspects of HTA are context-specific as comparators, disease burden, patient preferences, resource use, and costs often vary between settings (Reference Akehurst, Abadie, Renaudin and Sarkozy3;Reference Lothgren and Ratcliffe5).
Some elements, however, can be transferred (Reference Kristensen, Lampe and Wild6). Transferring elements has the potential to reduce duplicated work and increase the efficiency of evidence development, thereby saving resources while improving the transparency and predictability of the HTA process (Reference Akehurst, Abadie, Renaudin and Sarkozy3;Reference Lampe, Mäkelä and Garrido7). The HTA Core Model® (“the model”) was designed by the European Network for HTA (EUnetHTA) to address this issue of transferability (Reference Kristensen, Lampe and Wild6–Reference Kristensen, Mäkelä and Neikter8). The model provides a standardized framework to generate and share HTA evidence while enabling collaboration and avoiding duplication of work. However, the model is designed to be flexible so model users can select assessment elements that best fit their needs and integrate the selected elements into their specific reporting formats.
Evaluations of the model have been conducted from the perspective of HTA agencies but an industry perspective is currently lacking (Reference Macpherson and Thompson9). In 2013, F. Hoffmann-La Roche (Roche) and EUnetHTA agreed on a project to investigate how the model can support HTA evidence generation within Roche. Roche wanted to explore whether the model can guide the development of HTA evidence, be a repository for sharing evidence internally and serve as a framework for discussing the value of innovative medicines with payers, HTA agencies and other HTA stakeholders. Using the model has the potential for Roche (and other HTA evidence contributors) to become more efficient in developing HTA evidence and preparing pricing and reimbursement activities at the global and local level.
METHODS
The HTA Core Model was originally developed during the EUnetHTA Project (2006–08), following detailed review of the HTA process, and was expanded and improved further in EUnetHTA Joint Actions 1 (2010–12) and 2 (2012−15) (Reference Kristensen, Lampe and Wild6–Reference Kristensen, Mäkelä and Neikter8). Model applications are available for the assessment of medical/surgical interventions, diagnostic or screening technologies, and pharmaceuticals (10–13). The model has three components: the ontology, methodological guidance on conducting the assessment, and a common reporting structure to standardize outputs. The ontology is designed to guide evidence collection and assessment, by structuring information in nine domains (Table 1). Domains are divided into topics, each of which in turn contains issues of relevance to HTA. Issues are phrased as generic questions and, combined with their respective domain and topic, form an assessment element (10;11).
Table 1. Domains in the HTA Core Model Application for Pharmaceuticals (v2.0)
Note. Table references: Model Handbook 1.1 and deliverables for model versions 2.0, 2.1, 3.0 (10–13).
aMoved to TEC domain in version 3.0.
b“Patients and social aspects” in v3.0, includes topics “Patients’ perspectives,” “Social group aspects,” and “Communication aspects.”
cSourced from (draft) version 2.1.
A survey was conducted at Roche in June 2014 to evaluate if the nine-domain model application for pharmaceuticals (version 2.0) was fit for purpose, that is, represented a useful framework for standardized evidence generation and value assessment, from an industry perspective. Usefulness was defined according to the following criteria: (i) Importance to payers: Assessment elements were classified as useful if they met payers’ needs, for example, for benefit assessment in pricing and reimbursement negotiations or formulary listing. The term “payer” was used broadly, to denote national or local payers, reimbursement decision makers or providers of guidance. (ii) Relevance for value proposition: Elements were classified as useful if they contributed, during the initial submission or in the future, to demonstrating and communicating the value of the technology
An online questionnaire was developed to appraise the importance and transferability of all 130 assessments elements (Table 2). The model was also evaluated for possible redundancy of elements and for elements missing from the model.
Table 2. HTA Core Model Evaluation Questionnaire
The questionnaire was drafted following an initial workshop of the Global Pricing and Market Access (GPMA) team. The draft was piloted with the United Kingdom country affiliate, and the questionnaire revised in line with affiliate feedback. The final questionnaire was shared with country affiliate teams who completed one online questionnaire per team. All country affiliate staff participating in the survey were directly involved in evidence development, reimbursement submissions or payer negotiations for pharmaceuticals, to reflect experiences and needs associated with production and use of HTA within Roche.
Data from completed questionnaires were analyzed using descriptive statistics and collated results were returned to country affiliates for preparation of workshops. In one workshop per country and a joint workshop attended by delegates from country affiliates and the GPMA team in September 2014, open questions and between-country differences were discussed to establish a common understanding of assessment elements. Following the workshops, country affiliates had the opportunity to revise their initial answers before the summary report was drafted based on questionnaire results and discussions at workshops. After review by country affiliates and EUnetHTA, the final report, which included the survey questions and detailed results, was published on the EUnetHTA Web site in December 2014 (Reference Hoffmann-La Roche14).
RESULTS
Teams from six country affiliates (Canada, France, Germany, Italy, the Netherlands, United Kingdom) with a total of thirty-one staff participated in the survey and local workshops (Table 3). An additional four GPMA staff participated in the joint workshop.
Table 3. Characteristics of Teams Participating in the Survey
On average, 67 percent of all assessment elements were considered important by country affiliates (Figure 1). Differences in the average percentage of important elements were observed across domains. In four of the first five domains (health problem/current use of technology, characteristics of technology, clinical effectiveness, costs, and economic evaluation), ≥70 percent of elements were judged to be important while the fifth domain (safety) was less important. In other domains, between 40 percent (legal aspects) and 62 percent (organizational aspects) of elements were considered important. Of note, as the legal domain was developed referencing European law, elements of this domain were not immediately applicable to the Canadian setting and were, therefore, considered to be of low importance by the Canadian team.
Fig. 1. Results of the online survey, averaged across all six country affiliates.
Just over half (51 percent) of elements were considered to be transferable between settings (Figure 1). The safety and clinical effectiveness domains had the highest shares of transferable elements, with 90 percent and 79 percent, respectively. In the economic, ethical, organization, social, and legal domains, <50 percent of elements were deemed transferable.
Workshop discussions expanded on survey results. Overall, the first five domains were considered crucial for reimbursement as they reflected payers’ current evidence needs and demonstrated the value of a technology. Safety, however, was of lower importance in the context of the model as safety is already covered by regulatory requirements so, for some safety elements, payers may require less evidence to be discussed in HTA than for elements in other domains.
In addition, there is often a paucity of data for some safety-related assessment elements, for example, elements covering public and environmental risk. The last four domains were judged to be less important because few payers currently require their inclusion. Workshop participants agreed that these domains could be used to demonstrate the value of a technology from the perspectives of different stakeholders. However, elements in these domains often require context-specific evidence and were, therefore, considered to have low between-context transferability. Some participants also considered these four domains to be less important due to shortcomings in how domains were implemented in the model, for example, because some elements in these domains were unclear or because some elements were considered to be missing.
In discussions on the usefulness of the model for manufacturers, several potential benefits were identified. The model was considered, from the industry perspective of Roche, to provide an exhaustive, standardized framework for scoping and generating HTA evidence. Participants agreed that such a standardized approach would improve the comparability and quality of HTA processes, by providing a shared HTA terminology for communication internally (e.g., across affiliates) and externally (e.g., with payers in different settings).
Participants also explained that the model would enable a comprehensive, checklist-style approach to generate evidence more consistently within the company. It was suggested that the model could be used as a company-wide repository for HTA submissions to reduce duplicated work by sharing transferable elements. Following this internal survey, an access evidence generation process was implemented at Roche, which relies on tools developed on the basis of the model and its assessment to provide a standardized, systematic, and consistent approach to plan and generate evidence for HTA submissions.
Limitations of the model that might reduce its usefulness were also discussed. Participants agreed that, from an industry perspective, model usability could be improved with a more targeted version of the model, for example, by excluding duplicated elements and elements considered less important in the survey. Participants expressed concerns that it was not clear if and when future versions of the model would be developed. It was also unclear if model usage should be expected to extend beyond its current use cases, for example, to use in reimbursement decision making, and would be accepted by national and sub-national decision makers.
DISCUSSION
The present study provides evidence that the HTA Core Model is fit for purpose from an industry perspective and a useful tool for evidence generation and value propositions for healthcare technologies. Despite the need for some modifications, for example removal of duplicated and unimportant elements as identified in the survey, the model was considered as a valuable tool for sharing evidence across settings and discussing value, both internally and with other HTA stakeholders such as payers and HTA agencies.
This assessment, albeit from a different stakeholder perspective, was aligned with recent evaluations conducted by HTA agencies and the European Commission (Reference Macpherson and Thompson9;Reference Pasternack, de Groot, Kleijnen and Polman15;16). In a comparison of the model with national HTA reports in the Netherlands and Luxembourg, 40 percent and 50 percent of model elements, respectively, were judged relevant for the national assessment (Reference Pasternack, de Groot, Kleijnen and Polman15;Reference Kõrge, Berndt, Hohmann, Romano and Hiligsmann17). Of relevant elements, 64 percent and 47 percent, in the Netherlands and Luxembourg, respectively, were already included in national report templates. In both studies, including interviews with representatives from European HTA agencies and surveys of national uptake of the model, some elements were considered to be redundant or of limited value. There also was some concern that decision makers might be uncomfortable with new HTA formats and question the use of the model. On balance, the model was deemed to offer an opportunity for standardized, comprehensive, and collaborative HTA and evidence generation (Reference Pasternack, de Groot, Kleijnen and Polman15;Reference Kõrge, Berndt, Hohmann, Romano and Hiligsmann17).
While the survey reported here produced results similar to those obtained by HTA agencies, the survey is not without limitations. A relatively small sample of staff assessed the model and results may not be readily generalizable to other manufacturers, for example, companies of a different size and structure or those working in different markets or therapeutic areas. It should be noted, though, that participants were experienced in evidence generation and reimbursement submissions in a wide range of contexts. Their perspective was considered to be comprehensive and likely applicable to other HTA evidence contributors. The lack of formal validation of the questionnaire may be considered another limitation. The questionnaire, however, was designed to follow closely earlier evaluations conducted by EUnetHTA and was piloted with a country affiliate team. In addition, participants’ understanding of the questionnaire was aligned and confirmed in workshops.
Since this study was performed, version 3.0 of the model has become available. Results of the survey were taken into account by the model developers, along with feedback from other sources, when designing version 3.0: Some elements considered by Roche staff to be unimportant or duplicated in version 2.0 were removed or included in other elements in version 3.0. These changes reflect the responsiveness of model developers to input from stakeholders. Together with patients, providers, and payers, manufacturers have been a key stakeholder in EUnetHTA since the EUnetHTA Project and have provided input to all model versions. From the perspective of manufacturers, inconsistent HTA requirements between settings are associated with high costs as evidence generation has to be duplicated or available evidence must be extensively adapted (Reference Lothgren and Ratcliffe5;18). A standardized framework such as the HTA Core Model, which enables sharing and re-use of evidence internally and with HTA bodies in different settings, would be useful to reduce the cost and improve the timeliness of HTA for both manufacturers and HTA bodies (Reference Lothgren and Ratcliffe5).
Different HTA requirements not only increase the cost of evidence generation, they also suggest that the same evidence may be interpreted differently and lead to different HTA outcomes (Reference Lothgren and Ratcliffe5;18;Reference Fischer, Heisser and Stargardt19). The lack of consistency and transparency in evidence interpretation and appraisal between settings makes HTA processes less predictable for patients, clinicians, and manufacturers. Again, standardizing requirements across contexts, coupled with transparent, consistent decision frameworks, improves manufacturers’ ability to plan and predict assessment and reimbursement of their products. In turn, these improvements are likely to spur research and innovation and reduce the time and cost of making healthcare technologies available to patients.
From the perspective of all HTA stakeholders, standardizing the HTA process, particularly with regard to the scope and methodology of evidence generation, also helps to make HTA better equipped for the increasing demands on HTA, such as assessing technologies earlier and over longer periods, for example, over the technology's lifecycle (Reference Husereau, Henshall, Sampietro-Colom and Thomas20). At the same time, the number of stakeholders and value definitions to be considered is likely to increase (Reference Lakdawalla, Doshi and Garrison2;Reference Husereau, Henshall, Sampietro-Colom and Thomas20). These developments will require additional resources and might lead to even larger variability in HTA processes. An agreed-upon, standardized structure and terminology for HTA, which enables collaboration and can accommodate various value definitions, is likely to help all HTA stakeholders to meet increased demands on their resources and time without comprising the quality of the HTA process.
While attempts to standardize HTA are in relatively early stages, standardization is recognized to be associated with efficiency and quality benefits in medical research and regulatory approval (Reference Simera, Moher and Hirst21;22). Similar to the HTA Core Model, these efforts are designed to improve the transparency and quality of research (Reference Simera, Moher and Hirst21). Producers and users of evidence benefit from reliable frameworks which increase the comparability and reproducibility of results.
These benefits of standardization are similar to those obtained in other industries (23;24). Standardization is associated with more effective, less costly information exchange. In addition, standards contribute to economic and productivity growth and innovation through faster dissemination of new knowledge and increased confidence of producers and consumers (23;24). It seems plausible that similar benefits can be derived from a standardized approach to HTA.
In conclusion, in an online survey and workshops conducted at Roche, staff with experience in pricing and reimbursement submissions for pharmaceuticals at the global and local level assessed the usefulness of the HTA Core Model. From the industry perspective of Roche, the HTA Core Model was found to be fit for purpose as a framework for standardized evidence generation, as a repository for efficient sharing of evidence between settings and as a common terminology for value discussions with other HTA stakeholders, including payers and HTA agencies. Domains covering burden of disease and current technology use, features, clinical effectiveness, and economic evaluation of the technology were most relevant to demonstrate the value of healthcare technologies, particularly of pharmaceuticals. The results of this analysis were used as a primer for future research on using the HTA Core Model for access evidence generation within Roche.
CONFLICTS OF INTEREST
Marlene Gyldmark and Ansgar Hebborn are employees of F. Hoffmann-La Roche. Jörg Ruof was an employee of F. Hoffmann-La Roche when the work described in this paper was performed. Johannes Pöhlmann is an employee of Ossian Health Economics and Communications, which received consulting fees from F. Hoffmann-La Roche to support the preparation of this article. Kristian Lampe was an employee of the Finnish National Institute for Health and Welfare, working for EUnetHTA Joint Action 2 (Work package 8) when this work was performed and later received consultancy fees from F. Hoffmann-La Roche. Finn Børlum Kristensen received consultancy fees from F. Hoffmann-La Roche. This research received no specific grant from any funding agency, commercial or not-for-profit sectors.
Kristian Lampe and Finn Børlum Kristensen contributed to the development and dissemination of the original and subsequent versions of the HTA Core Model®.
Open access
