From its inception in 1999, the National Institute for Health and Care Excellence (NICE) committed to including the expertise, experiences, and perspectives of lay people, patients and carers, and patient organizations in its health technology assessments (HTAs). This is our story of patient involvement in HTA: from early methods designed for use when assessing medicines, widening to address the different requirements of HTAs for interventional procedures, medical technologies, and diagnostic technologies. We also chart the evolution and development of all our patient involvement methods over the past 20 years through regular evaluation and by responding to external challenge. However, we know that processes and methods alone are not enough. Through case studies we demonstrate the value of patient involvement in HTA and highlight the unique perspectives and experiences that patients bring to HTA committees. Finally, we discuss the underpinning principles and commitments that have made NICE a world leader in delivering meaningful and legitimate patient involvement.

Objectives: The objective of this study was to conduct a cost-effectiveness analysis of Herceptin® from the hospital's point of view. This new biotechnological pharmaceutical is a humanized monoclonal antibody that targets the HER2 receptor, an important anti-cancer target.

Methods: A cost model with standard diagnostic and treatment options for breast cancer was set up for a Belgian university hospital in close collaboration with its specialists. Direct and indirect costs were calculated for each diagnostic and treatment option using the micro-costing method. Effectiveness was estimated through a literature study. The model allowed us to take cost consequences in other stages of the model into account and to calculate changes in monthly treatment costs from different “starting points.” With an incremental cost-effectiveness analysis, differences in costs and effectiveness with and without Herceptin® were compared.

Results: Over the complete treatment period from diagnosis until the metastatic phase, monthly costs for the hospital rose from €85.07 to €90.35 for stage I diagnosed breast cancer when adding Herceptin® to the model. In the metastatic phase alone, these costs rose from €1,132.33 to €1,256.23. With Herceptin®, we found an extra cost of €3,981.44 per extra life-month.

Conclusions: This cost-effectiveness ratio was rather high, because Herceptin® was quite expensive and the product was additive in its current use and did not replace existing treatments. Future research will concentrate on alternative applications of Herceptin® based on ongoing Herceptin® trials and expert opinions.

The differences in survival rates of colon cancer patients between hospital districts were used as an indicator of equity in cancer treatment in Finland. The merits of two different assessment methods were evaluated. The large variation was mainly due to confounding by age and the extent of the disease and to random variation.

Objectives: Lifestyle changes to prevent type 2 diabetes among high-risk persons have been shown to be cost-effective. This study investigates the cost-effectiveness of a community-based program promoting general population lifestyle changes to prevent diabetes.

Methods: The 10-year program was implemented in three municipalities in Sweden. Effectiveness was measured with a quasiexperimental cohort design, that is, risk factor levels in a population group aged 36–56 years at baseline and 8–10 years later (2,149 men; 3,092 women) in the program municipalities and a control area were compared. The incremental cost-utility analysis included future diabetes and cardiovascular disease-related health effects and societal costs (discounted 3 percent), estimated by a Markov model.

Results: In all areas, risk factor levels increased during follow-up, leading to increased societal costs of between SEK40,000 and 90,000 (1 Euro 2004 = SEK9.13; 1 US$ = SEK 7.35) and quality-adjusted life-year (QALY) losses between 0.12 and 0.48 per individual. Compared with the control area, the cost increases and QALY losses for women were more favorable in two program areas but less favorable in one, and less favorable for men in both areas (data unavailable for one municipality). The findings indicate that the program was cost-effective in only two female study groups.

Conclusions: Conflicting results on the cost-effectiveness of the program were obtained. As several potentially valuable aspects of the program are not included in the cost-effectiveness analysis, the societal value of the program might not be adequately reflected.

Objectives: The objective of this exploratory study was to survey international health technology assessment (HTA) professionals to determine attitudes toward ethics in HTA.

Methods: An exploratory, quantitative, cross-sectional study design was developed. The sample population (n = 636) was selected from authors of the 206 articles published in the International Journal of Technology Assessment in Health Care between 2005 and 2007. A survey instrument was piloted and e-mailed.

Results: The response rate was low (n = 104; 16.4 percent). Respondents were primarily middle-aged (46 ± 11 years) male (62 percent) health professionals from Western countries (n = 92; 88.5 percent), with a mean of 10 (± 6 years) years of HTA experience. Although at least 90 percent of respondents agreed that healthcare decisions involved value judgments and that ethical analysis was important to HTA, respondents were divided as to whether normative (n = 45; 44.6 percent) or descriptive (n = 38; 37.6 percent) ethical recommendations were necessary. Most respondents (n = 83; 81.4 percent) believed that HTA should include citizen participation, but two thirds (n = 67; 67.0 percent) agreed that the final decision should be restricted to decision makers. A majority of respondents thought that ethical analysis could be discussed anywhere within the HTA process, either by an expert trained in ethics (n = 62; 60.8 percent) or by an external consultant (n = 80; 78.4 percent).

Conclusions: This study showed that ethical discourse in HTA is constrained by practical considerations, which serves to limit moral inquiry. To increase ethical analysis, a positive attitude toward ethics needs to be fostered within the HTA community.

Objectives: An overview was produced of approaches currently used to evaluate chronic disease management in selected European countries. The study aims to describe the methods and metrics used in Europe as a first to help advance the methodological basis for their assessment.

Methods: A common template for collection of evaluation methods and performance measures was sent to key informants in twelve European countries; responses were summarized in tables based on template evaluation categories. Extracted data were descriptively analyzed.

Results: Approaches to the evaluation of chronic disease management vary widely in objectives, designs, metrics, observation period, and data collection methods. Half of the reported studies used noncontrolled designs. The majority measure clinical process measures, patient behavior and satisfaction, cost and utilization; several also used a range of structural indicators. Effects are usually observed over 1 or 3 years on patient populations with a single, commonly prevalent, chronic disease.

Conclusions: There is wide variation within and between European countries on approaches to evaluating chronic disease management in their objectives, designs, indicators, target audiences, and actors involved. This study is the first extensive, international overview of the area reported in the literature.

Objectives: The aim of this study was to summarize the current evidence for the cost-effectiveness of primarily human papillomavirus (HPV) -based cervical cancer screening in settings with already established Papanicolaou test (Pap) programs. Emphasis was placed on the German situation with annual Pap screening.

Methods: Medical, economic, and health technology assessment (HTA) databases were systematically searched for cost-effectiveness studies comparing HPV to Pap screening. Study data were extracted, standardized, and summarized in cost-effectiveness plots contrasting HPV strategies to Pap screening with 1-, 2-, 3-, and 5-years interval. For each Pap setting, the likelihood of cost-effective HPV screening was assessed depending on willingness-to-pay.

Results: We reviewed twelve decision-analytic cost-effectiveness models. Study results showed wide variation due to methodical heterogeneity. Data synthesis revealed that the cost-effectiveness of HPV screening depends on the interval of the established Pap screening strategy. In comparison with Pap screening every 2 years, only 25 percent of the HPV-based screening strategies were cost-effective. However, in comparison with Pap screening every 1, 3, or 5 years, 83 percent, 55 percent, and 92 percent of HPV screening strategies were cost-effective, respectively. Results for settings with annual Pap screening are based on models assuming 100 percent screening coverage.

Conclusions: The introduction of HPV-based screening programs is cost-effective if the screening interval of the established Pap program exceeds 2 years. In settings with biennial Pap screening, introduction of HPV-based screening is unlikely to be cost-effective. Results also suggest cost-effectiveness of HPV-based screening in settings with annual Pap screening; however, this finding should be confirmed under realistic screening adherence assumptions.

The dream of artificial blood has existed since the 1960s. Disease-free hemoglobin solutions will be commercially available in 1991. A willingness-to-pay (WTP) survey was undertaken to assess the tangible and intangible benefits to the public from this new product. The positive results suggest that third-party payers should cover this benefit and assist the diffusion of this new technology. From the perspective of industrial marketing, results suggest that there will be little consumer price resistance for hemoglobin solutions within the suggested price range of $225–300 per unit. Management implications of this new product are discussed.

Objectives: Europe has many health technology assessment (HTA) agencies, each producing their own HTA reports. Adapting HTA reports for different contexts could reduce the need for multiple reports on the same health technology with resultant saving of time and resources. This study aims to examine and understand the process of adaptation, and to develop a toolkit that would help the adaptation of reports produced by other countries.

Methods: The methods used were a review of the literature; a survey of twenty-nine European HTA organizations, two rounds of a Delphi survey, a face-to-face meeting of twenty-one European network for Health Technology Assessment (EUnetHTA) representatives, iterative rounds of review, and two rounds of quality assurance testing (termed applicability testing).

Results: Descriptions of previous examples of adaptation in the literature are sparse. Most respondents had previous experience in adapting reports, and all believed that adaptation was useful, and there was the ability to benefit from the use of a toolkit to aid in the process. EUnetHTA Partners developed and tested an adaptation toolkit. The toolkit is composed of a series of checklists and resources that identify or clarify the relevance, reliability, and transferability of data and information from existing reports.

Conclusions: Consensus of opinion from twenty-nine European organizations/networks has indicated that the adaptation of HTA reports would be desirable and beneficial. A toolkit was developed to help with the adaptation of HTA reports produced in other settings. This collection of resources is available for use by all HTA agencies and can be accessed at: http://www.eunethta.net/upload/WP5/EUnetHTA_HTA_Adaptation_Toolkit_October08.pdf.

Objectives: Our aim was to review the recommendations given by health technology assessment (HTA) institutions in their methodological guidelines concerning the use of surrogate outcomes in their assessments. In a second step, we aimed at quantifying the role surrogate parameters take in assessment reports.

Methods: We analyzed methodological papers and guidelines from HTA agencies with International Network of Agencies for Health Technology Assessment membership as well as from institutions related to pharmaceutical regulation (i.e., reimbursement, pricing). We analyzed the use of surrogate outcomes in a sample of HTA reports randomly drawn from the HTA database. We checked methods, results (including evidence tables), and conclusions sections and extracted the outcomes reported. We report descriptive statistics on the presence of surrogate outcomes in the reports.

Results: We identified thirty-four methodological guidelines, twenty of them addressing the issue of outcome parameter choice and the problematic of surrogate outcomes. Overall HTA agencies call on caution regarding the reliance on surrogate outcomes. None of the agencies has provided a list or catalog of acceptable and validated surrogate outcomes. We extracted the outcome parameter of 140 HTA reports. Only around half of the reports determined the outcomes for the assessment prospectively. Surrogate outcomes had been used in 62 percent of the reports. However, only 3.6 percent were based upon surrogate outcomes exclusively. All of them assessed diagnostic or screening technologies and the surrogate outcomes were predominantly test characteristics.

Conclusions: HTA institutions seem to agree on a cautious approach to the use of surrogate outcomes in technology assessment. Thorough assessment of health technologies should not rely exclusively on surrogate outcomes.

Objectives: The role and impact of health technology assessment (HTA) in health policy has been widely discussed. Researchers have started to analyze how decisions on coverage of new technologies are made. Although the involvement of HTA may be an indicator of a well established decision process, this hypothesis requires validation. Also, it is not known whether HTA involvement is associated with other characteristics of decision making like participation or transparency. The primary objective of this study was to develop and test statements on the association between the publication of an HTA and coverage decision making for newborn screening tests in European Union countries.

Methods: Five statements were defined on the relative role of HTA during the steps of decision processes: trigger, participation, publication, assessment, and appraisal. For this purpose, data on twenty-two decision processes in the area of newborn screening across Europe were analyzed, defined as a coverage decision for a given disorder in a specific country. Decision processes were compared by whether the decision was accompanied by the publication of an HTA report. To test differences, nonparametric statistical tests were used.

Results: The decision steps of trigger, participation and publication differed between the HTA and the non-HTA groups. No clear association between HTA and assessment methods in coverage decision making was identified.

Conclusions: It appeared that there is an association between HTA and coverage decision processes that are more explicit, inclusive, and transparent. It is unclear whether HTA is associated with formal evidence reviews and economic evaluations.

Objectives: Our objective was to gather perspectives from payers on how comparative effectiveness research (CER) in the United States and relative effectiveness (RE) research in Europe will impact evidentiary standards for access decisions of new drugs by 2020.

Methods: We conducted semi-structured interviews with fourteen senior officials representing public and private payers, health technology assessment groups, and pricing and reimbursement bodies in the United States and Europe. An online survey assessed current use of CER/RE evidence and potential trends that might influence its use for decision making by 2020. A semi-structured interview elicited payers' definitions of CER/RE and was structured around four hypothetical cases resembling drugs expected to be more common or poised to create policy challenges by 2020. Topics included acceptance of study designs and analytic methods associated with CER/RE. A systematic content review was done to extract relevant information.

Results: According to key informants, randomization will remain an essential component for assessing comparative or relative effectiveness. They anticipate greater use of policy levers such as conditional reimbursement or prior authorization to manage diffusion of new drugs. Case studies provided important insights into situations when certain types of CER evidence may be acceptable (e.g., observational data when differences between drugs are largely convenience).

Conclusions: Industry perceptions that CER/RE will change payers' evidentiary requirements in the future are consistent with our findings. Growing investment in payers' own data and increased reliance on policy tools to control diffusion of new drugs may also influence the type of evidence industry will be required to produce by 2020.

Objectives: The aim of this study was to conduct a review of health technology assessments (HTAs) in cervical cancer screening to highlight the most common metrics HTA agencies use to evaluate and recommend cervical cancer screening technologies.

Methods: The Center for Reviews and Dissemination (CRD), MedLine, and national HTA agency databases were searched using keywords (“cervical cancer screening” OR “cervical cancer” OR “cervical screening”) and “HTA” from January 2000 to October 2014. Non-English language reports without English summaries, non-HTA reports, HTAs unrelated to a screening intervention and HTAs without sufficient summaries available online were excluded. We used various National Institute for Health and Care Excellence (NICE) methods to extract key assessment criteria and to determine whether a change in screening practice was recommended.

Results: One hundred and ten unique HTA reports were identified; forty-four HTAs from seventeen countries met inclusion criteria. All reports evaluated technologies for use among women. Ten cervical screening technologies were identified either as an intervention or a comparator. The most common outcome metric evaluated was diagnostic accuracy, followed by economic effectiveness. Additional outcome metrics such as the use of adjunct testing, screening intervals, and age-specific testing were commonly evaluated. Nearly one-third (fifteen of forty-four) of HTAs recommended a change in practice.

Conclusions: This review highlights popular metrics used in HTAs for cervical cancer screening. Clinical and economic effectiveness metrics have been consistently assessed in HTAs, while the use of adjunct testing, screening intervals, and age-specific screening became increasingly prevalent from after 2007. Moreover, we observed an increase in optimized recommendations after 2007.

Objectives: The aim of this study was to determine the prevalence and methods of expert knowledge elicitation (EKE) for specifying input parameters in health economic decision models (HEDM).

Methods: We created two samples using the National Health System Economic Evaluations Database: (1) 100 randomly selected HEDM studies to determine prevalence of EKE and (2) sixty studies using a formal EKE process to determine methods used.

Results: Fifty-seven (57 percent) of the random sample included at least one EKE-derived parameter. Of these, six (10 percent) used a formal expert process. Thirty-four studies from our second sample of sixty studies (57 percent) described at least one aspect of the process (e.g., elicitation method) with reasonable clarity. In approximately two-thirds of studies the external experts estimated parameters de novo; the remainder confirmed or modified initial estimates provided by authors, or the method was unclear. The majority of elicitations obtained point estimates only, although a few studies asked experts to estimate ranges of parameter values.

Conclusions: The use of EKE for parameter estimation is common in HEDMs, although there is room for improvement in the methods used.

Objectives: To determine if published figures on the proportion of articles included in systematic reviews and identified in electronic databases are applicable to an example from medical imaging.

Methods: A systematic review was performed. Additionally, sensitivity and precision of a MEDLINE search were compared with values from three published searches, each customized for a specific field.

Results: All articles included in the systematic review were in electronic databases. The MEDLINE search had low precision compared with searches in other fields.

Conclusions: In a specific area of medical imaging, electronic databases, including MEDLINE, are reliable sources of articles.

In economic evaluation, the healthcare perspective has gradually given way to use of the societal perspective, as this perspective is often advocated for support in making optimal societal decisions. In practice, economic evaluations conducted from the societal perspective ignore, fail to measure and/or fail to monetize many of the costs that fall outside of the healthcare sector. To limit bias and increase decision-supportive power, researchers could strengthen their evaluations by adhering to a few basic principles. Five “pillars for the societal perspective” are proposed. First, who bears the cost and who does not is irrelevant. Second, it is imperative to consider including costs for sectors outside the healthcare sector. Third, both high frequent costs and costs with high unit prices should be considered. Fourth, double counting should be avoided. And fifth, researchers should reflect on choices related to costs, i.e. cost omission and problems with identifying, measuring, and valuing costs.

Objectives: Health Technology Assessment (HTA) as a method for producing evidence in the health-care sector has been used for more than 25 years but has grown in extent during the past years. The objective of this study is to explore a possible evolution in these HTAs, in type of assessed technologies, in type of assessors, and in its methods.

Methods: A structured literature review was conducted of 433 HTA reports from the period 1989 to 2002 by eleven leading HTA institutions worldwide. The review focused on the methodology used, the assessors, and the assessed technologies and was designed to elucidate general time-trends in the practical application of HTA.

Results: The study shows that literature reviews are still the most often used method of assessment and accounts for a relatively stable fraction of assessments. The fraction of economic evaluations shows a slightly decreasing trend in contrast to randomized controlled trials and modeling/evaluations, which are applied more frequently. The data also demonstrate a more frequent use of external partners as assessors and a shift between devices and pharmaceuticals as assessed technologies.

Conclusions: The study shows an increase in the number of HTAs but no major developments in assessment methods used and, therefore, no widespread spillover from the development in research methods in general to the field of HTA methodology.

Abstract: The purpose of this study is to explain the rationale for the value of information approach to priority setting for research and to describe the methods intuitively for those familiar with basic decision analytical modeling. A policy-relevant case study is used to show the feasibility of the method and to illustrate the type of output that is generated and how these might be used to frame research recommendations. The case study relates to the use of glycoprotein IIb/IIIa antagonists for the treatment of patients with non-ST elevation acute coronary syndrome. This is an area that recently has been appraised by the National Institute for Health and Clinical Excellence.