This article presents an overview of the practical methods and tools to support transnational health technology assessment (HTA) that the partners in the European network for HTA (EUnetHTA) planned, developed, and pilot tested. EUnetHTA was established as a response to an expressed need of European Union (EU) Member States and the European Commission (EC) to establish a sustainable network for HTA in Europe (Reference Kristensen, Mäkelä and Allgurin Neikter14). The tools are described and discussed in detail in a series of articles coordinated for publication in this issue of the Journal (Reference Carbonneil, Quentin and Lee-Robin4;Reference Lampe, Mäkelä and Velasco-Garrido16;Reference Moharra, Espallargues and Kubesch19;Reference Pasternack, Anttila and Mäkelä21–Reference Wild, Simpson and Douw26). Another article in this issue describes the planning, development, and implementation of a sustainable network for HTA in Europe and lists the sixty-four Partners that participated (Reference Kristensen, Mäkelä and Allgurin Neikter14).
HEALTH TECHNOLOGY ASSESSMENT
The International Network of Health Technology Assessment Agencies (INAHTA) defines HTA as a multidisciplinary field of policy analysis. It studies the medical, social, ethical, and economic implications of development, diffusion, and use of health technology (12).
The EUnetHTA Project developed an explanatory definition of health technology assessment extending from the INAHTA definition (Reference Kristensen, Mäkelä and Allgurin Neikter14).
HTA Collaboration in Europe and the Need for Practical Tools
The need for practical transnational (i.e., cross-border, transcending national boundaries) collaboration to support timely, relevant, and high-quality national/regional HTA results has become evident with the increasing political support and impact that HTA has gained in Europe in recent years (Reference Kristensen, Mäkelä and Allgurin Neikter14). During 2006 to 2008, the EUnetHTA Project examined the entire process of HTA and its links to policy, addressing several key challenges (e.g., reporting standards, information sharing) for enhancing transnational collaboration.
Between 1995 and 2002 the EUR-ASSESS, HTA Europe, and the ECHTA/ECAHI projects, which were all supported by the European Commission, made important collaborative contributions to methodology and training and to understanding of the relationship between HTA and policy (Reference Banta, Kristensen and Jonsson1). These projects made recommendations on the next steps toward implementing European collaboration in HTA (Reference Banta2;Reference Banta and Oortwijn3;Reference Jonsson, Banta, Henshall and Sampietro-Colom13;Reference Liberati, Sheldon and Banta17).
INAHTA was established in 1993 and, since then, has made consistent contributions to facilitate collaboration between HTA institutions. The HTA Database, which includes references to ongoing projects and concluded HTA reports, was created in 1998 and contains information on HTAs. It is managed in collaboration with the Centre for Reviews and Dissemination in the United KIngdom (10). INAHTA developed the Checklist, first published in 2001 and revised in 2007, to be completed by agencies on each of their HTA reports as an aid to furthering a consistent and transparent approach to HTA (9). An HTA Glossary was also developed in 2006 (11). EuroScan is a collaborative network of member agencies for the exchange of information on potentially important new and emerging drugs, devices, procedures, programs, and settings in health care. EuroScan maintains a searchable Web site with links to member agency reports assessing emerging technologies (5). The international society for the promotion of HTA, Health Technology Assessment International (HTAi), includes agencies and individuals as members, and maintains a searchable Vortal of information resources (8).
Memberships in international organizations and networks in HTA overlap considerably. EUnetHTA strategically sought added value in its efforts and, where possible, built on existing tools while responding to the politically expressed need for cross-border collaboration on HTA in Europe.
OBJECTIVES
EUnetHTA set out to provide reliable, timely, transparent, and transferable information on the short- and long-term effects of health technologies as input to decision making in Member States and in the EU. By doing this, EUnetHTA shall assist the EU Member States and others to effectively plan, deliver, and monitor health services. The strategic objectives of EUnetHTA were reported elsewhere (Reference Kristensen, Mäkelä and Allgurin Neikter14).
EUnetHTA was designed to be a practical project. Hence, the specific objectives in relation to assessing technologies and producing HTA reports focused on the development of useful tools.
METHODS
The EUnetHTA Project spanned 3 years, from January 2006 to December 2008, and comprised eight Work Packages (WPs). It developed a structure for coordination, management, and governance to support the work facilitated by three WPs (Reference Kristensen, Mäkelä and Allgurin Neikter14). The organization of the work within WPs and the tools used to enhance collaboration within EUnetHTA are described in detail elsewhere (Reference Kristensen, Mäkelä and Allgurin Neikter14). The WPs developed annual work plans that were shared with other WPs in the overall coordination of the project management structure. A wide spectrum of methods were applied in the WPs, for example, literature searches, survey questionnaires, Delphi surveys, pilot and applicability testing of tools, structured reviews of drafts, and meetings among experts and other forms of collaboration to build consensus.
SIX STREAMS OF WORK TO MEET PRACTICAL RESEARCH CHALLENGES FOR HTA
The following describes the objectives, methods, and results of six streams of HTA-related work in the Project. Figure 1 depicts WPs in relation to the HTA and policy processes.
Figure 1. EUnetHTA Work Packages (WP) in relation to the processes of health technology assessment (HTA) and policy making.
Inform Decision Makers About Emerging Technologies
A substantial amount of information is being gathered by horizon scanning activities. However, this information does not find its way to nonspecialized audiences, including relevant policy makers. This situation led to the following objectives: (i) Make the information gathered within established horizon scanning activities available to a wider audience by disseminating information on new and emerging technologies beyond regional or national decision makers by means of a Europe wide newsletter; and (ii) Develop a prototype of such a newsletter and pilot test the processes of production.
The EuroScan database was used as source of information to identify potentially relevant technologies (Reference Wild, Simpson and Douw26) to be described in a newsletter. For prioritization, a panel of HTA experts applied a set of criteria to score the potential impact of the technologies and select the technologies to be included in a newsletter. Subsequently, articles were written and a pilot newsletter was published.
As the descriptions of technologies added to the EuroScan database from July through December 2007 resulted in a list of 104 technologies and 12 were selected to be described in articles of different length and depth. Potential readers who were asked for feedback on the relevance, content, timeliness, and readability of the newsletter responded mostly positively, but they asked for more information on cost-effectiveness and questioned timeliness (Reference Wild, Simpson and Douw26).
Dissemination of an EU-wide newsletter is a feasible but time-consuming activity. An EU-only newsletter based on EuroScan information was not considered to be the appropriate instrument, and EUnetHTA will avoid duplication of efforts now planned by EuroScan (5). Other options should be pursued as part of a future collaborative action, for example, providing a core set of early awareness information, or an on-demand electronic information system (Reference Wild, Simpson and Douw26).
Develop and Pilot Test a Framework and Model for Transnational HTA
Although HTA agencies worldwide share a common set of principles and methodological approaches, the structure of HTA reports varies considerably across agencies depending on their national standards, particular work processes, and context. This hampers information sharing among HTA doers. This work stream had the following objectives: (i) Develop and test a generic framework enabling international collaboration for producing and sharing results of HTA; (ii) Develop a framework and a model for medical and surgical interventions and for diagnostic technologies; (iii) Analyze and describe the process and outcomes of two pilot assessments based on the generic mode; and (iv) Discuss the applicability of the model and identify areas in need for further development.
A framework and a model were developed for the assessment of medical and surgical interventions and for diagnostic technologies. Nine domains (Health problem and current use; Description and technical characteristics; Safety; Effectiveness; Costs and economic evaluation; Ethical; Organizational; Social; and Legal aspects) based on the EUR-ASSESS Project were included in the model (Reference Lund Håheim, Imaz and Läubli Loud18).
Ten international teams constructed what was named the HTA Core Model, dividing information contained in an (ideal) HTA into standardized assessment elements. Each element contains a generic issue that must be translated into a practical research question when doing an assessment (e.g., impact on a health outcome such as mortality or function). Elements were described in detail in element cards. Two assessments, designated as Core HTAs were produced. Guidance on the use of the HTA Core Model was collected into an HTA Core Model Handbook (Reference Lampe, Anttila and Pasternack15;Reference Lampe, Mäkelä and Velasco-Garrido16;Reference Pasternack, Anttila and Mäkelä21).
The HTA Core Model consists of 133 assessment elements for medical and surgical interventions and 153 for diagnostic. In total, 121 and 143 of the elements, respectively, are designated core elements. The elementary structure of the HTA Core Model proved useful in preparing pilot HTAs. Research question framing varied between domains in the pilot Core HTAs (Reference Lampe, Mäkelä and Velasco-Garrido16;Reference Pasternack, Anttila and Mäkelä21).
The HTA Core Model is a novel approach to HTA. It enables effective national and transnational production and sharing of HTA results in a common, structured format and represents a wide range of perspectives. It can be developed into a platform that enables and encourages genuine transnational HTA collaboration between institutions and individuals in terms of work distribution and maximum utilization of a common pool of structured HTA information for national HTA reports.
The current version is usable and available for further testing in an electronic format, but should be further refined to identify overlapping issues in domains and assessed regarding its usability as a basis for conducting HTA. Core HTAs are intended to serve as a basis for local HTA reports. Consequently, core HTAs do not contain recommendations on technology use, which may be found in HTA reports targeted at specific national or regional settings (Reference Lampe, Mäkelä and Velasco-Garrido16).
The next steps to be taken within EUnetHTA Collaboration (Reference Kristensen, Mäkelä and Allgurin Neikter14) include implementation of the electronic version and more widespread, but carefully monitored, pilot application of the HTA Core Model in practice among HTA institutions.
Develop and Pilot Test Tools to Adapt HTA Reports to New Contexts
The numerous HTA agencies across Europe each produce their own HTA reports—often on the same topic—which is useful but time consuming and costly. If appropriately adapted, HTA reports could reduce the time required to make policy decisions, reduce costs, and even increase the number and quality of reports produced by individual agencies. This work stream had the following objectives: Examine and understand the process of adaptation; Investigate whether the adaptation of HTA reports could be useful to agencies across Europe, and how this might be achieved in practice; Develop an HTA adaptation toolkit for use by agencies across Europe; Undertake quality assurance testing of this toolkit; and Develop a glossary of HTA adaptation terms to help reduce misunderstanding of terms used in HTA reports from contexts other than the reader's own.
A process involving several rounds of input from twenty-eight European HTA agencies and scholarly institutions (EUnetHTA Partners) was used to identify routines on how to adapt existing HTA reports to new contexts. Several methods were used: literature searching, a survey of adaptation experience, two rounds of a Delphi survey, meetings, drawing on the expertise and experience of the partnership, and two rounds of quality assurance testing (termed applicability testing) (Reference Turner, Chase and Milne24;Reference Turner, Chase and Milne25).
Descriptions of previous examples of adaptation in the literature are sparse. Most respondents had previous experience in adapting reports, and all believed that adaptation was useful. There was strong support for the development of an adaptation toolkit, which was then developed and tested by EUnetHTA Partners. This toolkit is composed of a series of checklists and resources that identify or clarify the relevance, reliability, and transferability of data and information from existing reports. A glossary of HTA adaptation terms was developed (Reference Rosten, Chase, Hicks and Milne23).
The toolkit and glossary will be a valuable resource when reading HTA reports produced in different contexts and when adapting HTA reports produced in other countries (Reference Rosten, Chase, Hicks and Milne23–Reference Turner, Chase and Milne25). Next steps within EUnetHTA Collaboration include Web-based implementation and more field testing.
Develop and Pilot Test Tools to Generate Transnational Evidence for HTA of New Technology
Access to new, promising technologies may depend on the generation of additional evidence. Several countries have developed policy frameworks allowing timely access to promising health technologies on the condition that additional evidence is generated. Access with evidence generation (AEG) is well known in the context of marketing approval, but is a more recent concept in relation to coverage (i.e., inclusion in standard care, reimbursement by health insurance, etc.). However, an important barrier to evidence generation is the lack of structured collaboration among HTA agencies. This work had the following objectives: (i) Create an overview of known national AEG mechanisms in various countries associated with marketing approvals and funding or coverage decisions; (ii) Determine the types of structured collaboration that could facilitate evidence generation and to develop a Web-based toolkit to support such.
A search for information on the AEG mechanisms used by twenty-three countries (twenty European countries, the United States, Canada [Ontario], and Australia) included a literature review, surveys of WP Partners, meetings, and consultation of key people in the field. There is growing interest in implementing AEG policies at the coverage decision stage (Reference Carbonneil, Quentin and Lee-Robin4). Collaboration modalities were discussed at a workshop attended by all WP Partners, national experiences were analyzed, and a generally applicable policy framework was developed with key factors for its successful operation (Reference Quentin, Carbonneil and Moty-Monnereau22). Initial emphasis was on information sharing. Standardized forms for information sharing were developed and tested, and information technology developments led to the creation of a Web-based toolkit. The Web site allows access to structured and standardized forms for requesting information, posting information in response to a request, and posting information spontaneously. An online database contains all the information requested or posted.
International collaboration is particularly needed to gather a critical mass of high-quality data quickly, while ensuring timely access to promising technologies. The new Web site for sharing information on evidence generation should help managers and policy makers in facilitating robust decisions on the timely adoption of promising health technologies (http://eiffel.eunethta.has-sante.fr/). It will become fully operational as EUnetHTA Partners supply relevant, accurate, and updated information, and regularly use the Web site (Reference Carbonneil, Quentin and Lee-Robin4;Reference Quentin, Carbonneil and Moty-Monnereau22).
Support to HTA Capacity Building
The number of HTA agencies in Europe has grown rapidly in recent years. Furthermore, many countries without a formal HTA program are showing increased interest in establishing one. The experiences of institutionalizing HTA could be helpful for other countries. The objectives were the following: (i) Define the minimum components related to the scope, structure, process, and visibility of an HTA organization; (ii) Develop tools for information support to organizations/institutions implementing HTA; and (iii) Produce a handbook on HTA capacity building.
Several activities were carried out: (i) a review of HTA organizations including Web sites of existing HTA organizations to learn about their characteristics in relation to structure, setting, process, and visibility; (ii) a worldwide survey on HTA organizations to gain knowledge on the current state of HTA and its institutionalization and insights into characteristics and processes of such organizations; (iii) a survey on information units in HTA organizations; (iv) a workshop; and (v) a survey on HTA educational programs, sent to INAHTA members and Cochrane Centers, to provide an updated overview of the most relevant educational programs on HTA and HTA-related areas worldwide. The results were compiled to produce a Handbook on HTA Capacity Building (Reference Moharra20).
A Survey of HTA Educational Programs identified ten MSc programs (five MSc in HTA and five HTA-related MSc). It also identified eleven courses that were part of a postgraduate HTA-related course (Reference Moharra20).
In the establishment of a new agency, it is important not only to secure funding, but also to attract trained staff. Multidisciplinary teams composed of a wide range of specialized professionals, researchers, and administrative assistants are the ideal work units needed to produce sound HTA reports. Ultimate success also depends on the quality and relevance of HTA reports, an efficient information dissemination system, and a willingness at the policy level to integrate HTA into decision making. It is worth taking advantage of the new technologies to promote and become more active in planning dissemination strategies (Reference Moharra, Espallargues and Kubesch19).
Understand the Relation Between HTA and Health Policy Making
The aim of HTA is to support policy making and decision making at different levels in the health system. However, often the real impact of HTA in these processes has not tapped its full potential—not least due to a lack of informing potential target audiences about what HTA can offer. The objectives in this context were the following: (i) Review the relationship between HTA and policy making from several perspectives, with a special focus on Europe; (ii) Report the studies in a book in cooperation with the European Observatory on Health Systems and Policies; (iii) Promote HTA as policy input by transmitting the value of HTA to a wide public in decision making and healthcare management; and (iv) Increase the awareness of HTA activities and evidence-based decision making.
Following a workshop with healthcare policy makers and managers, a group of HTA researchers and scholars produced a book covering the different aspects of the relationship between HTA and policy making. The chapters were based on a mix of systematic literature reviews, document analyses, expert knowledge, and practical experiences from European HTA agencies. In addition, the workshop facilitated exchange of views between a group of decision makers from different levels of various European healthcare systems and WP partners. The material was compiled into a book and published in cooperation with the European Observatory on Health Systems and Policies (Reference Velasco Garrido, Kristensen, Nielsen and Busse6).
Based on empirical literature, the book on HTA and policy making demonstrates factors that might enhance or hinder the contribution of HTA to policy making. Several research utilization models were studied in terms of their potential contribution to bridge the often apparent disconnect between research and policy. Generalities of HTA and of policy and decision making in the health system were described, and current options to locate HTA institutions in the health system were discussed. The book described the rationale and possible nature of HTA impact evaluation and applied a hierarchical model to structure current knowledge on the impact of HTA reports. In addition, a systematic review of the literature on barriers and facilitators in using HTA evidence in decision making provided insight on ways to improve knowledge transfer to decision making. A summary of strategies proposed to improve HTA utilization was also presented (Reference Velasco Garrido, Kristensen, Nielsen and Busse6).
The study on HTA and policy making in Europe provided an interesting multiplicity of perspectives from both inside the HTA community and from health systems research and political science. Important issues were raised concerning, on one hand, the integration of HTA with other efforts to support and improve the quality of a health system (e.g., clinical guideline production, monitoring of healthcare quality), and, on the other hand, the interrelation between HTA and health services research. Practical considerations on the complex role of HTA are gaining a better theoretical and empirical foothold. However, the field remains immature, and more multidisciplinary research into the relationship between HTA and health policy is needed to support practical application of HTA in health systems. The EU Seventh Framework Programme (FP7) also identifies this issue (Reference Velasco Garrido, Kristensen, Nielsen and Busse6;7).
DISCUSSION
Next Phase of European Collaboration in HTA
EUnetHTA achieved its objectives and delivered tangible results complying with its project description, while involving a large group of organizations and individuals. The concrete models and tools need more piloting and adjusting, and the structures for this are now in place with the EUnetHTA Collaboration (Reference Kristensen, Mäkelä and Allgurin Neikter14). The need to coordinate the content of work streams was identified, and the intensity of participation varied across partners. To increase added value for all, the next phase of European collaboration should carefully consider the division of work among organizations according to competencies and needs. EUnetHTA Collaboration Partners should carefully consider the advice for the future provided by the internal evaluation (Reference Lund Håheim, Imaz and Läubli Loud18).
CONCLUSION
What was achieved for transnational HTA work in Europe?
The EUnetHTA Project succeeded in building practical tools for several of the key areas of HTA: setting up a new agency; informing about new technologies; facilitating new evidence generation; performing and reporting actual cross-border assessments to support timely, relevant, high-quality Core HTA information that can be used for national/regional reporting; adapting information from one setting to another; and understanding better the relation between HTA and health policy. These tools together with a sustainable network (Reference Kristensen, Mäkelä and Allgurin Neikter14) form an operational and functional basis for European collaboration in HTA. Further development and piloting of the tools is needed, and will be done within the EUnetHTA Collaboration.
The working process put into practice during the development of the Project itself, and its results, can already be considered a concrete achievement of transnational HTA work in Europe. EUnetHTA has prepared the necessary organizational framework, the collaborative working process, and the main tools to facilitate daily work. This structure will build a solid foundation for concrete European collaboration in HTA.
CONTACT INFORMATION
Finn Børlum Kristensen, MD, PhD ([email protected]), Director, Danish Centre for Health Technology Assessment, National Board of Health, Islands Brygge 67, DK-2300 Copenhagen S, Denmark; Adjunct Professor, Faculty of Health Sciences, University of Southern Denmark, Winsløwparken 19, 3, Odense C, DK 5000, Denmark
Kristian Lampe, MD ([email protected]), Senior Medical Officer, Finnish Office for Health Technology Assessment, National Institute for Health and Welfare, P.O. Box 30, 00271 Helsinki, Finland
Deborah L. Chase, BSc, PhD ([email protected]), Senior Research Fellow, Wessex Institute for Health Research and Development NETS-CC, University of Southampton, Alpha House, Enterprise Road, Southampton Science Park, Chilworth, Southampton S016 7NS, UK
Sun Hae Lee-Robin, MD, MPH ([email protected]), Head of Department, Medical and Surgical Procedures Assessment, French National Authority for Health, 2 avenue du Stade de France, Saint-Denis La Plaine CEDEX, F-93218
Claudia Wild, Dr. phil. ([email protected]), Director, Ludwig Boltzmann Institute of Health Technology Assessment, Garnisongasse 7/20, 1090 Vienna, Austria
Montse Moharra, BA ([email protected]), Research Assistant, Health Technology Assessment Unit, Catalan Agency for Health Technology Assessment and Research (CAHTA), Roc Boronat, 81–95, Barcelona, Spain, 08005
Marcial Velasco Garrido, MD, MPH ([email protected]), Researcher, Healthcare Management, Technische Universität Berlin, Straße des 17. Juni 135, Secr. H80, Berlin 10623, Germany
Camilla Palmhøj Nielsen, MA, PhD student ([email protected]), Special Advisor, Danish Centre for Health Technology Assessment, National Board of Health, Islands Brygge 67, DK-2300 Copenhagen S, Denmark; PhD Student, Department of Political Science, University of Copenhagen, Østre Farimagsgade 5, DK-1353 Copenhagen K, Denmark
John-Arne Røttingen, MD, PhD, MSc ([email protected]), Chief Executive, Norwegian Knowledge Centre for the Health Services, P.O. Box 7004, St. Olavs plass, N-0130 Oslo, Norway
Susanna Allgurin Neikter, ([email protected]), Marketing Director, Swedish Council on Technology Assessment in Health Care (SBU), Olof Palmes Gata 17, P.O. Box 3657, SE 103 59 Stockholm, Sweden
Marie Louise Bistrup, cand.arch., MPH ([email protected]), Research Assistant, Danish Centre for Health Technology Assessment, National Board of Health, Islands Brygge 67, DK-2300 Copenhagen S, Denmark
