Evidence is limited on how to synthesize and incorporate the views of stakeholders into a multisite pragmatic trial and how much academic teams change study design and protocol in response to stakeholder input. This qualitative study describes how stakeholders contributed to the design, conduct, and dissemination of findings of a multisite pragmatic clinical trial, the COMprehensive Post-Acute Stroke Services (COMPASS) Study. We engaged stakeholders as integral research partners by embedding them in study committees and community resource networks that supported local sites. Data stemmed from formal focus groups and continuous participation in working groups. Guided by Grounded Theory, we extracted themes from focus group and meeting notes. These were discussed as a team and with other stakeholder groups for feasibility. A consensus approach was used. Stakeholder input changed many aspects of the study including: the care model that treated stroke as a chronic condition after hospital discharge, training for hospital-based providers who often lacked awareness of the barriers to recovery that patients face, support for caregivers who were essential for stroke patients’ recovery, and for community-based health and social service providers whose services can support recovery yet often go underutilized. Stakeholders brought value to both pragmatic research and health service delivery. Future studies should test the impact of elements of study implementation informed by stakeholders vs those that are not.

A number of publications have discussed approaches to training the scientific workforce in comparative effectiveness research (CER) and patient-centered outcomes research (PCOR). To meet this need, funders have offered resources for developing educational materials and establishing training programs. To extend these efforts into specific researcher communities, the Agency for Healthcare Research and Quality developed an R25 Funding Opportunity Announcement that called for basic, advanced, and experiential training for a specific researcher community in collaboration with associated program partners. This paper describes the strategies developed by the 5 subsequently funded programs, their specific researcher communities and program partners, and the challenges associated with developing in-person and online programs. We focus on lessons learned that can be translated into developing training programs nationwide and on training for the special populations of interest. We also discuss the creation of a sustainable network for training and the conduct of comparative effectiveness research/patient-centered outcomes research in targeted communities.

Background: Defining relevant outcome measures for clinical trials on medical devices (MD) is complex, as there is a large variety of potentially relevant outcomes. The chosen outcomes vary widely across clinical trials making the assessment in evidence syntheses very challenging. The objective is to provide an overview on the current common procedures of health technology assessment (HTA) institutions in defining outcome measures in MD trials.

Methods: In 2012–14, the Web pages of 126 institutions involved in HTA were searched for methodological manuals written in English or German that describe methods for the predefinition process of outcome measures. Additionally, the institutions were contacted by email. Relevant information was extracted. All process steps were performed independently by two reviewers.

Results: Twenty-four manuals and ten responses from the email request were included in the analysis. Overall, 88.5 percent of the institutions describe the type of outcomes that should be considered in detail and 84.6 percent agree that the main focus should be on patient relevant outcomes. Specifically related to MD, information could be obtained in 26 percent of the included manuals and email responses. Eleven percent of the institutions report a particular consideration of MD related outcomes.

Conclusions: This detailed analysis on common procedures of HTA institutions in the context of defining relevant outcome measures for the assessment of MD shows that standardized procedures for MD from the perspective of HTA institutions are not widespread. This leads to the question if a homogenous approach should be implemented in the field of HTA on MD.

Background: The Comprehensive Post-Acute Stroke Services (COMPASS) Study is one of the first large pragmatic randomized-controlled clinical trials using comparative effectiveness research methods, funded by the Patient-Centered Outcomes Research Institute. In the COMPASS Study, we compare the effectiveness of a patient-centered, transitional care intervention versus usual care for stroke patients discharged home from acute care. Outcomes include stroke patient post-discharge functional status and caregiver strain 90 days after discharge, and hospital readmissions. A central tenet of Patient-Centered Outcomes Research Institute-funded research is stakeholder engagement throughout the research process. However, evidence on how to successfully implement a pragmatic trial that changes systems of care in combination with robust stakeholder engagement is limited. This combination is not without challenges. Methods: We present our approach for broad-based stakeholder engagement in the context of a pragmatic trial with the participation of patients, caregivers, community stakeholders, including the North Carolina Stroke Care Collaborative hospital network, and policy makers. To maximize stakeholder engagement throughout the COMPASS Study, we employed a conceptual model with the following components: (1) Patient and Other Stakeholder Identification and Selection; (2) Patient and Other Stakeholder Involvement Across the Spectrum of Research Activities; (3) Dedicated Resources for Patient and Other Stakeholder Involvement; (4) Support for Patient and Other Stakeholder Engagement Through Organizational Processes; (5) Communication with Patients and Other Stakeholders; (6) Transparent Involvement Processes; (7) Tracking of Engagement; and (8) Evaluation of Engagement. Conclusion: In this paper, we describe how each component of the model is being implemented and how this approach addresses existing gaps in the literature on strategies for engaging stakeholders in meaningful and useful ways when conducting pragmatic trials.

Objectives: Our objective was to gather perspectives from payers on how comparative effectiveness research (CER) in the United States and relative effectiveness (RE) research in Europe will impact evidentiary standards for access decisions of new drugs by 2020.

Methods: We conducted semi-structured interviews with fourteen senior officials representing public and private payers, health technology assessment groups, and pricing and reimbursement bodies in the United States and Europe. An online survey assessed current use of CER/RE evidence and potential trends that might influence its use for decision making by 2020. A semi-structured interview elicited payers' definitions of CER/RE and was structured around four hypothetical cases resembling drugs expected to be more common or poised to create policy challenges by 2020. Topics included acceptance of study designs and analytic methods associated with CER/RE. A systematic content review was done to extract relevant information.

Results: According to key informants, randomization will remain an essential component for assessing comparative or relative effectiveness. They anticipate greater use of policy levers such as conditional reimbursement or prior authorization to manage diffusion of new drugs. Case studies provided important insights into situations when certain types of CER evidence may be acceptable (e.g., observational data when differences between drugs are largely convenience).

Conclusions: Industry perceptions that CER/RE will change payers' evidentiary requirements in the future are consistent with our findings. Growing investment in payers' own data and increased reliance on policy tools to control diffusion of new drugs may also influence the type of evidence industry will be required to produce by 2020.

Objectives: Evaluation is essential for the management of international projects or networks in health technology assessment (HTA). It extends beyond the normal process of project management by incorporating qualitative dimensions and provides information about a project's effectiveness and achievements. This article aimed to identify the factors that are important for the success of international HTA projects. The European network for Health Technology Assessment Joint Action (EUnetHTA JA) is presented as an exemplar.

Methods: Methods for the evaluation of international HTA projects include interviews, focus groups, questionnaires, observations and documentary review, and the key points of these approaches have been summarized. The impact and effectiveness of the EUnetHTA JA was evaluated by questionnaires of project participants and external stakeholders, and by documentary review.

Results: The response rate for the three annual questionnaires sent to project participants ranged from 86 percent to 88 percent and for external stakeholders ranged from 65 percent to 88 percent. Key factors for project success included production of deliverables according to the workplan, achievement of objectives, added value generated, effective communication, involvement of external stakeholders, workstream management and progress from the preceding EUnetHTA 2006–2008 project.

Conclusions: The experience of this project can inform the evaluation of future international HTA collaborations, such as the EUnetHTA 2nd Joint Action and HTAsiaLink. A high response rate was achieved to the self-completion questionnaires and the strategy followed is recommended for evaluation of international HTA projects. Future assessments of international HTA projects should strive to measure outcomes and impact, not just outputs and process.

Objectives: Many European countries perform rapid assessments of the relative effectiveness (RE) of pharmaceuticals as part of the reimbursement decision making process. Increased sharing of information on RE across countries may save costs and reduce duplication of work. The objective of this article is to describe the development of a tool for rapid assessment of RE of new pharmaceuticals that enter the market, the HTA Core Model® for Rapid Relative Effectiveness Assessment (REA) of Pharmaceuticals.

Methods: Eighteen member organisations of the European Network of Health Technology Assessment (EUnetHTA) participated in the development of the model. Different versions of the model were developed and piloted in this collaboration and adjusted accordingly based on feedback on the content and feasibility of the model.

Results: The final model deviates from the traditional HTA Core Model® used for assessing other types of technologies. This is due to the limited scope (strong focus on RE), the timing of the assessment (just after market authorisation), and strict timelines (e.g. 90 days) required for performing the assessment. The number of domains and assessment elements was limited and it was decided that the primary information sources should preferably be a submission file provided by the marketing authorisation holder and the European Public Assessment Report.

Conclusions: The HTA Core Model® for Rapid REA (version 3.0) was developed to produce standardised transparent RE information of pharmaceuticals. Further piloting can provide input for possible improvements, such as further refining the assessment elements and new methodological guidance on relevant areas.

Background: This article describes the lessons learned from an international pilot assessment using the first version of the HTA Core Model® and Guidelines for rapid Relative Effectiveness Assessment (REA) of pharmaceuticals based on input from three different perspectives: the assessors, the users (health technology assessment organisations) and the marketing authorisation holder.

Methods: A pilot assessment was performed of pazopanib for the treatment of advanced or metastatic renal cell carcinoma for which 54 individuals from 22 EUnetHTA member organisations from 16 European countries gave their contribution. The work was divided in eight domain teams. Subsequently, results of these domain teams were synthesised in one pilot report. Feedback on the outcomes of the pilot was gathered throughout the project and through structured surveys.

Results: The first version of the assessment was produced in six months and consisted of 55 question and answer pairs, 8 domain reports and a synthesis section that combined the results from the different domains. The organisation of the pilot required intense coordination. Main points of criticism on the assessment were the lengthiness of the document and overlap of information throughout the assessment.

Conclusions: A reduction in the number of authoring organisations and individuals participating is necessary to avoid information overlap and increase efficiency in undertaking the assessment. Involving several organisations (e.g. five) in an in-depth review could still ensure the benefit of broad participation from various countries. The focus of a rapid REA should be on the first four domains of the Model.

Objectives: The aim of this study was to evaluate the comparative effectiveness of anti-vascular endothelial growth factor therapy in the treatment of diabetic macular edema (DME).

Methods: Searches focused on reports concerning treatment of DME with aflibercept, bevacizumab, pegaptanib, or ranibizumab published between January 2000 and June 30, 2012, with comparisons to laser photocoagulation, sham injection, or other control (e.g., triamcinolone). Effectiveness was based on best-corrected visual acuity (BCVA), in terms of letters gained. Direct meta-analyses were conducted on BCVA for each anti-vascular endothelial growth factor (VEGF) agent; indirect comparisons also were performed for each anti-VEGF pair.

Results: A total of fifteen randomized controlled trials (eleven fair- or good-quality) and eight observational studies were included. No direct comparative studies were identified. Improvement in visual acuity versus control was seen with all agents (range: 4–9 letters); outcomes were consistent across multiple timepoints. Meta-analyses showed no statistically significant and/or consistent differences between agents in BCVA changes or the percentage of patients gaining more than ten letters. No discernible differences in the potential harms of anti-VEGFs, including ocular events, MI, stroke, and other cardiovascular events, as well as death, were noted between aflibercept, pegaptanib, and ranibizumab. Data on harms for bevacizumab were underreported.

Conclusions: All anti-VEGF agents are effective in improving visual acuity in comparison to laser photocoagulation. Evidence is insufficient to distinguish the performance of any single anti-VEGF agent over others. The safety of bevacizumab remains the greatest element of uncertainty.

Background: The cost-effectiveness of triptans in the treatment of migraine has not been assessed since generic sumatriptan entered the Finnish market in 2008.

Methods: Using systematic review and mixed treatment comparison, the effectiveness of triptans was estimated with regard to 2-hour response, 2-hour pain-free, recurrence, and any adverse event, using published clinical data. Direct and indirect costs (2010 EUR, societal perspective) and quality-adjusted life-years (QALYs) were evaluated over one acute migraine attack using a decision-tree model.

Results: The meta-analysis combined data from fifty-six publications. The highest probability of achieving the primary outcome, “sustained pain-free, no adverse event” (SNAE), was estimated for eletriptan 40 mg (20.9 percent). Sumatriptan 100 mg was the treatment with lowest estimated costs (€20.86), and the incremental cost-effectiveness ratio of eletriptan 40 mg compared with sumatriptan 100 mg was €43.65 per SNAE gained (€19,659 per QALY gained).

Conclusion: Depending on the decision-maker's willingness-to-pay threshold, either sumatriptan 100 mg or eletriptan 40 mg is likely to be cost-effective.