Background: A framework for collaborative production and sharing ofHTA information, the HTA Core Model, was originally developed within EUnetHTA in2006–08. In this paper, we describe the further development of theModel to allow implementation and utilization of the Model online. The aim wasto capture a generic HTA process that would allow effective use of the HTA CoreModel and resulting HTA information while at the same time not interfering withHTA agencies' internal processes.

Methods: The work was coordinated by a development team in Finland,supported by an international expert group. Two pilot testing rounds wereorganized among EUnetHTA agencies and two extensive core HTA projects tested thetool in a real setting. The final work was also formally validated by a group ofHTA agencies.

Results: The HTA Core Model Online—available at http://www.corehta.info—is a web site hosting a) a tool toallow electronic utilization of the HTA Core Model and b) a database of producedHTA information. While access to the HTA information is free to all, theproduction features are currently available to EUnetHTA member agencies only. Apolicy was crafted to steer the use of the Model and produced information.

Conclusions: We have successfully enabled electronic use of the HTACore Model and agreed on a policy for its utilization. The system is alreadybeing used in subsequent HTA projects within EUnetHTA Joint Action 2. Identifiedshortcomings and further needs will be addressed in subsequent development.

Objectives: Many European countries perform rapid assessments of the relative effectiveness (RE) of pharmaceuticals as part of the reimbursement decision making process. Increased sharing of information on RE across countries may save costs and reduce duplication of work. The objective of this article is to describe the development of a tool for rapid assessment of RE of new pharmaceuticals that enter the market, the HTA Core Model® for Rapid Relative Effectiveness Assessment (REA) of Pharmaceuticals.

Methods: Eighteen member organisations of the European Network of Health Technology Assessment (EUnetHTA) participated in the development of the model. Different versions of the model were developed and piloted in this collaboration and adjusted accordingly based on feedback on the content and feasibility of the model.

Results: The final model deviates from the traditional HTA Core Model® used for assessing other types of technologies. This is due to the limited scope (strong focus on RE), the timing of the assessment (just after market authorisation), and strict timelines (e.g. 90 days) required for performing the assessment. The number of domains and assessment elements was limited and it was decided that the primary information sources should preferably be a submission file provided by the marketing authorisation holder and the European Public Assessment Report.

Conclusions: The HTA Core Model® for Rapid REA (version 3.0) was developed to produce standardised transparent RE information of pharmaceuticals. Further piloting can provide input for possible improvements, such as further refining the assessment elements and new methodological guidance on relevant areas.

Objectives: The aim of this study was to present the development, structure and results of a database on planned and ongoing health technology assessment (HTA) projects (POP Database) in Europe.

Methods: The POP Database (POP DB) was set up in an iterative process from a basic Excel sheet to a multifunctional electronic online database. The functionalities, such as the search terminology, the procedures to fill and update the database, the access rules to enter the database, as well as the maintenance roles, were defined in a multistep participatory feedback loop with EUnetHTA Partners.

Results: The POP Database has become an online database that hosts not only the titles and MeSH categorizations, but also some basic information on status and contact details about the listed projects of EUnetHTA Partners. Currently, it stores more than 1,200 planned, ongoing or recently published projects of forty-three EUnetHTA Partners from twenty-four countries. Because the POP Database aims to facilitate collaboration, it also provides a matching system to assist in identifying similar projects. Overall, more than 10 percent of the projects in the database are identical both in terms of pathology (indication or disease) and technology (drug, medical device, intervention). In addition, approximately 30 percent of the projects are similar, meaning that they have at least some overlap in content.

Conclusions: Although the POP DB is successful concerning regular updates of most national HTA agencies within EUnetHTA, little is known about its actual effects on collaborations in Europe. Moreover, many non-nationally nominated HTA producing agencies neither have access to the POP DB nor can share their projects.

Objectives: The European Network for Health Technology Assessment (EUnetHTA) Joint Action Project Framework, 2010–12, Work Package 7B (WP 7B), was aimed at facilitating collaboration between HTA agencies to avoid duplication of assessment efforts. A major task of WP 7B was to collect information on planned and ongoing (POP) projects by EUnetHTA agencies and to incorporate this information in a POP Database. We analyzed whether the Database served its intended purpose.

Methods: A survey was sent to all fifty-seven EUnetHTA partners, complemented by telephone interviews with the Ludwig Boltzmann Institute of HTA in Austria (lead institution). Furthermore, detailed documentation on the activities of the POP Database was provided to the research team at CAST (University of Southern Denmark) by the lead institution.

Results: Forty-two of fifty-seven agencies (74 percent) responded to the survey. Eleven collaborations initiated by agencies themselves were reported. The scope of these collaborative activities was usually limited to information exchange on for example literature search protocols. A slight reduction of duplication of effort was documented. In addition, twelve collaborations at the full report level were initiated by the lead institution.

Conclusion: While the POP Database has the potential to reduce duplication of effort, this has not been realized during the 3-year period of the EUnetHTA Joint Action Project Framework, 2010–12. Further evidence needs to be gathered to determine whether the POP Database is effective and whether the benefits outweigh the resources required to maintain it.

Objectives: Clinical endpoints are defined as valid measures of clinical benefit or harm due to treatment, that describe the impact of treatment on how a patient feels, functions, and survives. The choice of endpoints and the manner in which they are reported have a major impact on the relative effectiveness assessment (REA) of pharmaceuticals. The aim of this article is to describe the guideline development process and the key findings that set a framework for appropriate use of endpoints in REAs in Europe.

Methods: A multi-health technology assessment (HTA)-agency collaborative process in EUnetHTA JA1 was used to scope, draft, and finalize methodological guidelines for REA in Europe.

Results: Patient-relevant clinical endpoints can be broadly categorized into: mortality, morbidity and health-related quality of life. A clinical endpoint is a main symptom or sign of a disease that is clinically relevant, valid, reproducible and responsive to change. Preference is for long-term or final endpoints whenever possible. Surrogate endpoints may be used when there is compelling evidence of a clear and consistent correlation of treatment effects on the surrogate and final outcome of interest.

Conclusions: The relevance and hierarchy of the different types of clinical endpoints depend on the research question, disease, and the treatment investigated. Not only the primary endpoint, but also other relevant endpoints are assessed in comparison to adequate comparator(s). This simultaneous assessment of all relevant endpoints is a hallmark of REA.

Objectives: The aim of this study was to describe the techniques used in achieving consensus on common standards to be implemented in the EUnetHTA Information Management System (IMS); and to describe how interoperability between tools was explored.

Methods: Three face to face meetings were organized to identify and agree on common standards to the development of online tools. Two tools were created to demonstrate the added value of implementing interoperability standards at local levels. Developers of tools outside EUnetHTA were identified and contacted.

Results: Four common standards have been agreed on by consensus; and consequently all EUnetHTA tools have been modified or designed accordingly. RDF Site Summary (RSS) has demonstrated a good potential to support rapid dissemination of HTA information. Contacts outside EUnetHTA resulted in direct collaboration (HTA glossary, HTAi Vortal), evaluation of options for interoperability between tools (CRD HTA database) or a formal framework to prepare cooperation on concrete projects (INAHTA projects database).

Conclusions: While being entitled a project on IT infrastructure, the work program was also about people. When having to agree on complex topics, fostering a cohesive group dynamic and hosting face to face meetings brings added value and enhances understanding between partners. The adoption of widespread standards enhanced the homogeneity of the EUnetHTA tools and should thus contribute to their wider use, therefore, to the general objective of EUnetHTA. The initiatives on interoperability of systems need to be developed further to support a general interoperable information system that could benefit the whole HTA community.

Background: This article describes the lessons learned from an international pilot assessment using the first version of the HTA Core Model® and Guidelines for rapid Relative Effectiveness Assessment (REA) of pharmaceuticals based on input from three different perspectives: the assessors, the users (health technology assessment organisations) and the marketing authorisation holder.

Methods: A pilot assessment was performed of pazopanib for the treatment of advanced or metastatic renal cell carcinoma for which 54 individuals from 22 EUnetHTA member organisations from 16 European countries gave their contribution. The work was divided in eight domain teams. Subsequently, results of these domain teams were synthesised in one pilot report. Feedback on the outcomes of the pilot was gathered throughout the project and through structured surveys.

Results: The first version of the assessment was produced in six months and consisted of 55 question and answer pairs, 8 domain reports and a synthesis section that combined the results from the different domains. The organisation of the pilot required intense coordination. Main points of criticism on the assessment were the lengthiness of the document and overlap of information throughout the assessment.

Conclusions: A reduction in the number of authoring organisations and individuals participating is necessary to avoid information overlap and increase efficiency in undertaking the assessment. Involving several organisations (e.g. five) in an in-depth review could still ensure the benefit of broad participation from various countries. The focus of a rapid REA should be on the first four domains of the Model.

Objectives: The HTA Core Model is a framework for producing health technology assessments (HTAs) in a structured format. The Model splits the content of a HTA into assessment elements. The objective is to explore the adaptability of these assessment elements in national report production in a pilot case study comparing a national HTA report and the HTA Core Model.

Methods: An on-going Dutch HTA report on endovascular repair of abdominal aortic aneurysm (EVAR) was chosen as a typical representative of a national report on medical interventions. The author of the EVAR report assessed the relevance and comprehensiveness of the assessment elements of the HTA Core Model for her work. Another researcher annotated the Core Model specific content in the EVAR report. Matching and missing content, as well as the distribution of information in the EVAR report were tabulated and analysed in joint deliberations.

Results: Forty percent of the assessment elements of the Core Model were considered relevant for the EVAR report. Some issues relevant for EVAR but missing from the Core Model were identified: they were about re-interventions, secondary prevention, subpopulations that benefit most, and the length of the hospital stay. The distribution of information differed substantially between the Code Model and the national report.

Conclusions: The assessment elements of the HTA Core Model covered most relevant questions of the national report. In order to facilitate easy adaptation of information, the distribution of information should be more consistent in the national report and the Core model.

Objectives: The purpose of the European network for Health Technology Assessment (EUnetHTA) is to make HTA agencies collaborate sharing methods and tools thus avoiding duplication of evaluative efforts and allowing resource savings. From 2010 to 2012, the activities of the network were carried out through EUnetHTA Joint Action 1 and Work Package 4 Strand B aimed at producing two Core HTAs with two main objectives: to test the Web based Core model and the collaborative working models. Our objective in this article is to give an historical record of the Work Package activities highlighting what worked and what did not in the collaboration of researchers’ groups coming from different agencies.

Methods: A retrospective description of all the steps for the joint production of the two Core HTAs is provided starting from the first step of selecting technologies of common interest. Primary researchers’ views on the whole process have been collected through a semi-structured telephonic interview supported by a questionnaire. Coordinators views were gathered during internal meetings and validated.

Results: Majority of respondents thought topic selection procedure was not clear and well managed. About collaborative models, small groups were seen to enable more exchange, whatever the model. According to coordinators, loss of expertise and experience during the production process, different languages, and novelty of the Online Tool were main barriers.

Conclusions: Lessons learned from this first experience in Joint Action 1 paved the path for the collaboration in Joint Action 2, as it allowed enhancements and changes in models of collaborations and coordination.

Objectives: Evaluation is essential for the management of international projects or networks in health technology assessment (HTA). It extends beyond the normal process of project management by incorporating qualitative dimensions and provides information about a project's effectiveness and achievements. This article aimed to identify the factors that are important for the success of international HTA projects. The European network for Health Technology Assessment Joint Action (EUnetHTA JA) is presented as an exemplar.

Methods: Methods for the evaluation of international HTA projects include interviews, focus groups, questionnaires, observations and documentary review, and the key points of these approaches have been summarized. The impact and effectiveness of the EUnetHTA JA was evaluated by questionnaires of project participants and external stakeholders, and by documentary review.

Results: The response rate for the three annual questionnaires sent to project participants ranged from 86 percent to 88 percent and for external stakeholders ranged from 65 percent to 88 percent. Key factors for project success included production of deliverables according to the workplan, achievement of objectives, added value generated, effective communication, involvement of external stakeholders, workstream management and progress from the preceding EUnetHTA 2006–2008 project.

Conclusions: The experience of this project can inform the evaluation of future international HTA collaborations, such as the EUnetHTA 2nd Joint Action and HTAsiaLink. A high response rate was achieved to the self-completion questionnaires and the strategy followed is recommended for evaluation of international HTA projects. Future assessments of international HTA projects should strive to measure outcomes and impact, not just outputs and process.