Objectives: To identify predictors of high-quality clinical practice guidelines.

Methods: A total of 86 guidelines from 11 countries were assessed by four independent appraisers per guideline using the AGREE instrument (23 items). Six aspects of guideline development were considered to explain the variation in quality scores: care level (primary/secondary care), scope (diagnosis/treatment), type of guideline (new/update), year of publication, type of agency (governmental/professional), and whether the guideline was produced within a structured and coordinated program.

Results: Guidelines produced within a guideline program and by governmental agencies had higher scores than their counterparts. Differences in the applicability of the guidelines could not be explained by the variables studied.

Conclusions: To ensure high quality, clinical guidelines should be produced within a structured and coordinated program. Professional organizations or specialist societies that aim to develop guidelines may adopt quality criteria from leading guideline agencies.

Background: Patient involvement is widely acknowledged to be a valuable component in health technology assessment (HTA) and healthcare decision making. However, quantitative approaches to ascertain patients' preferences for treatment endpoints are not yet established. The objective of this study is to introduce the analytic hierarchy process (AHP) as a preference elicitation method in HTA. Based on a systematic literature review on the use of AHP in health care in 2009, the German Institute for Quality and Efficiency in Health Care (IQWiG) initiated an AHP study related to its HTA work in 2010.

Methods: The AHP study included two AHP workshops, one with twelve patients and one with seven healthcare professionals. In these workshops, both patients and professionals rated their preferences with respect to the importance of different endpoints of antidepressant treatment by a pairwise comparison of individual endpoints. These comparisons were performed and evaluated by the AHP method and relative weights were generated for each endpoint.

Results: The AHP study indicates that AHP is a well-structured technique whose cognitive demands were well handled by patients and professionals. The two groups rated some of the included endpoints of antidepressant treatment differently. For both groups, however, the same six of the eleven endpoints analyzed accounted for more than 80 percent of the total weight.

Conclusions: AHP can be used in HTA to give a quantitative dimension to patients' preferences for treatment endpoints. Preference elicitation could provide important information at various stages of HTA and challenge opinions on the importance of endpoints.

How to obtain confidence intervals for cost-effectiveness ratios is complicated by the statistical problems of obtaining a confidence interval for a ratio of random variables. Different approaches have been suggested in the literature, but no consensus has been reached. We propose an alternative simple solution to this problem. By multiplying the effectiveness units by the price per effectiveness unit, both costs and benefits can be expressed in monetary terms and standard statistical techniques can be used to estimate a confidence interval for net benefits. This approach avoids the ratio estimation problem and explicitly recognizes that the price per effectiveness unit has to be known to provide cost-effectiveness analysis with a useful decision rule.

Background: In conjunction with other important movements in contemporary medicine, including evidence-based medicine (EBM), health technology assessment (HTA) has promoted a culture of critical evaluation. Despite this impact, institutional and methodological challenges are associated with HTA. For example, only in recent years has HTA attempted an open dialogue with patients; however, this is normally done by giving them a “seat” at the HTA decision-making table, rather than by more scientific means.

Objectives: The aim of this study was to develop a working definition of patient-based HTA, to identify the current barriers to adopting a patient-based model, and to formulate a vision of how a patient-based HTA could be used to promote patient empowerment and patient-centered care.

Results: In the ideal setting, a patient-based HTA would promote patient knowledge by providing access to information and promoting an informed dialogue between patients and their healthcare professionals. To implement a patient-based HTA, the focus must turn to the patient's issues and incorporate each patient's unique perspective and preferences. Processes must change to increase patient participation in all levels of HTA and aim to promote empowered patients who can make informed decisions.

Conclusions: Present-day HTA is broad and has numerous stakeholders, with none so important as the patient. By asking patient-oriented questions in HTA and better involving patients throughout the entire process, we can easily promote patient empowerment, and as such make patients more capable to play a more active role in healthcare decision making.

Objectives: The aim of this study was to review international experiences of patient or public involvement in the field of health technology assessment (HTA).

Methods: A systematic review of the scientific literature was conducted. A literature search was performed across nine databases. Other literature was identified through citation tracking, government websites (HTA agencies), and Internet search engines. Characteristics of the studies, description of the activities related to patient or public involvement, impact of these activities on the HTA process, and factors facilitating or limiting involvement were abstracted independently by two reviewers.

Results: A total of 1,441 potentially relevant papers were identified by the main search strategy. Among these, seventeen papers met the inclusion criteria; other search strategies identified seven additional documents. The findings reveal that patient or public involvement in HTA activities was reported in two domains, research and HTA process. In the research domain, patients are consulted to gather evidence about their perspectives, experiences, or preferences about a health technology. These perspectives could add key dimensions to the evaluation of health technologies that might otherwise be overlooked. In the domain of the HTA process, patients or public representatives participate in different stages of this process: prioritization, evidence assessment, or dissemination of findings.

Conclusions: There are few published examples of experiences involving patients and the public in HTA. These examples show that patients' or the public's perspectives could add important dimensions to the evaluation of health technologies. However, there is a need to develop more systematic approaches to considering patients' and the public's perspectives in HTA.

Diagnostic information within administrative data bases is generally in the form of lnternational Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes. The purpose of this article is to introduce ICD-9-CM, to review its strengths and limitations, and to suggest ways that it can be used through administrative files for assessing the quality of hospital care.

Background: The hazard ratio (HR) is the most appropriate measure for time to event outcomes such as survival. In systematic reviews, HRs can be calculated either from the raw trial data obtained as part of an individual patient data (IPD) meta-analysis or from the appropriate trial-level summary statistics. However, the information required for the latter are seldom reported in sufficient detail to allow reviewers to calculate HRs. In contrast, the median survival and survival rates at specific time points are frequently presented. We aimed to evaluate retrospectively the performance of meta-analyses using median survival times and survival rates by comparing them with meta-analyses using IPD to calculate HRs.

Methods: IPD from thirteen published meta-analyses (MAs) in cancers with high mortality rates were used. Median survival and survival rates were calculated from the IPD rather than taken from publications so that the same trials, patients, and extended follow-up are used in each analysis.

Results and Conclusions: We show that using median survival times or survival rates at a particular point in time are not reasonable surrogate measures for meta-analyses of survival outcomes and that, wherever possible, HRs should be calculated. Individual trial publications reporting on time to event outcomes, therefore, should provide more detailed statistical information, preferably logHRs and their variances, or their estimators.

Background: To ensure rapid access to new potentially beneficial health technologies, obtain best value for money, and ensure affordability, healthcare payers are adopting a range of innovative reimbursement approaches that may be called Managed Entry Agreements (MEAs).

Methods: The Health Technology Assessment International (HTAi) Policy Forum sought to identify why MEAs might be used, issues associated with implementation and develop principles for their use. A 2-day deliberative workshop discussed key papers, members' experiences, and collectively addressed four policy questions that resulted in this study.

Results: MEAs are used to give access to new technologies where traditional reimbursement is deemed inappropriate. Three different forms of MEAs have been identified: management of budget impact, management of uncertainty relating to clinical and/or cost-effectiveness, and management of utilization to optimize performance. The rationale for using these approaches and their advantages and disadvantages differ. However, all forms of MEA should take the form of a formal written agreement among stakeholders, clearly identifying the rationale for the agreement, aspects to be assessed, methods of data collection and review, and the criteria for ending the agreement.

Conclusions: MEAs should only be used when HTA identifies issues or concerns about key outcomes and/or costs and/or organizational/budget impacts that are material to a reimbursement decision. They provide patient access and can be useful to manage technology diffusion and optimize use. However, they are administratively complex and may be difficult to negotiate and their effectiveness has yet to be evaluated.

Objective: Individuals' recollections of the number and type of health service encounters are frequently required for health economic studies. We sought to establish whether the accuracy of self-reported healthcare resource utilization is a function of the duration of the recall period and the saliency of the health service encounter.

Methods: Patient recollections of a range of community services (general practitioner visits, community midwifery visits) and hospital services (accident and emergency attendances, hospital outpatient attendances, inpatient admissions) over 4-month and 8-month time periods were obtained from women participating in a randomized controlled trial. Comparisons were made with healthcare resource utilization data extracted from medical records. Where significant differences were identified between the self-reported and medically recorded data, a multivariate linear regression model was constructed to identify the factors associated with underreporting and overreporting of healthcare resource utilization.

Results: The study revealed a tendency to underreport community service utilization, which appears to be exacerbated when the recall period is extended. A number of sociodemographic and clinical factors significantly associated with this tendency to underreport community service utilization were identified. The self-reporting of hospital service utilization over varying periods of recall was found to be more accurate.

Conclusion: It is important that economic analysts establish optimal methods for estimating resource utilization quantities within health economic analytical designs. In particular, greater emphasis should be placed on extracting information on community service utilization from medical records or routine health service information systems.

Many people believe that cost-effectiveness (CE) and cost-benefit (CB) analyses require different assumptions. However, when CE analysis supports decisions to use medical resources, it makes the same assumptions that CB analysis requires. They are mathematically equivalent. Differences between CE and CB hinge more on reporting style than on fundamental assumptions.

Background: Increasingly, healthcare decision makers demand quality evidence in a short timeframe to support urgent and emergent decisions related to procurement, clinical practice, and policy. Health technology assessment (HTA) producers are responding by developing innovative approaches to evidence synthesis that can be executed more quickly than traditional systematic review. These approaches, and the broader implications they bring to bear on health decision making and policy development, however, are generally neither well-understood nor well-described. This study intends to contribute to an emerging literature around methodological approaches to rapid review in HTA by outlining those developed and implemented by the Canadian Agency for Drugs and Technologies in Health (CADTH).

Methods: Since 2005, CADTH has developed and implemented a rapid review approach that synthesizes evidence to support informed healthcare decisions and policy. Rapid Response reports are tailored to the identified needs of Canadian health decision makers, representing a range of options with regard to depth, breadth, and time-to-delivery.

Results: Preliminary observations indicate that CADTH's approach to rapid evidence review is generally well-received by Canadian health decision makers; real-world case studies provide pragmatic examples of how health decision makers have used Rapid Response reports to support evidence-informed health decisions across Canada.

Conclusions: Rapid review is becoming an increasingly important approach to evidence synthesis, both within and external to the field of HTA. Transparent reporting of the methods used to develop rapid review products will be critical to the assessment of their relevance, utility and effects in a range of contexts.

Objectives: The aim of this study was to develop and test a generic framework to enable international collaboration for producing and sharing results of health technology assessments (HTAs).

Methods: Ten international teams constructed the HTA Core Model, dividing information contained in a comprehensive HTA into standardized pieces, the assessment elements. Each element contains a generic issue that is translated into practical research questions while performing an assessment. Elements were described in detail in element cards. Two pilot assessments, designated as Core HTAs were also produced. The Model and Core HTAs were both validated. Guidance on the use of the HTA Core Model was compiled into a Handbook.

Results: The HTA Core Model considers health technologies through nine domains. Two applications of the Model were developed, one for medical and surgical interventions and another for diagnostic technologies. Two Core HTAs were produced in parallel with developing the model, providing the first real-life testing of the Model and input for further development. The results of formal validation and public feedback were primarily positive. Development needs were also identified and considered. An online Handbook is available.

Conclusions: The HTA Core Model is a novel approach to HTA. It enables effective international production and sharing of HTA results in a structured format. The face validity of the Model was confirmed during the project, but further testing and refining are needed to ensure optimal usefulness and user-friendliness. Core HTAs are intended to serve as a basis for local HTA reports. Core HTAs do not contain recommendations on technology use.

Objectives: Discounting of costs in health-related economic evaluation is generally regarded as uncontroversial, but there is disagreement about discounting health benefits. We sought to explore the current recommendations and practice in health economic evaluations with regard to discounting of costs and benefits.

Methods: Recommendations for best practice on discounting for health effects as set out by government agencies, regulatory bodies, learned journals, and leading health economics texts were surveyed. A review of a sample of primary literature on health economic evaluations was undertaken to ascertain the actual current practice on discounting health effects and costs.

Results: All of the official sources recommended a positive discount rate for both health effects and costs, and most recommended a specific rate (range, 1% to 8%). The most frequently specified rates were 3% and 5%.

A total of 147 studies were reviewed; most of these used a discount rate for health of either 0% (n = 50) or 5% (n = 67). Over 90% of studies used the same discount rate for both health and cost. While 28% used a zero rate for both health and cost, in 64% a nonzero rate was used for both. Studies where the health measure was in natural clinical units (direct) were significantly more likely to have a zero discount rate.

Conclusion: The finding that 28% of studies did not discount costs or benefits is surprising and concerning. A lower likelihood of discounting for benefits when they are in natural units may indicate confusion regarding the rationale for discounting health effects.

The application of conditionality to coverage decisions for healthcare technologies is increasing. Coverage with Evidence Development (CED) represents a specific approach to coverage for promising technologies for which the evidence remains uncertain. CED demands that additional evidence is generated to address the sources of uncertainty and secure ongoing coverage. This study explores the conceptual and policy issues relating to CED and discusses issues involved in operationalizing CED in practice, including presenting criteria for which technologies may be most suitable for CED. This study is intended to further the debate on the use of CED as well as highlight areas that warrant further research.

Objectives: This study sought to identify and compare various practical and current approaches of health technology assessment (HTA) priority setting.

Methods: A literature search was performed across PubMed, MEDLINE, EMBASE, BIOSIS, and Cochrane. Given an earlier review conducted by European agencies (EUR-ASSESS project), the search was limited to literature indexed from 1996 onward. We also searched Web sites of HTA agencies as well as HTAi and ISTAHC conference abstracts. Agency representatives were contacted for information about their priority-setting processes. Reports on practical approaches selected through these sources were identified independently by two reviewers.

Results: A total of twelve current priority-setting frameworks from eleven agencies were identified. Ten countries were represented: Canada, Denmark, England, Hungary, Israel, Scotland, Spain, Sweden, The Netherlands, and United States. Fifty-nine unique HTA priority-setting criteria were divided into eleven categories (alternatives; budget impact; clinical impact; controversial nature of proposed technology; disease burden; economic impact; ethical, legal, or psychosocial implications; evidence; interest; timeliness of review; variation in rates of use). Differences across HTA agencies were found regarding procedures for categorizing, scoring, and weighing of policy criteria.

Conclusions: Variability exists in the methods for priority setting of health technology assessment across HTA agencies. Quantitative rating methods and consideration of cost benefit for priority setting were seldom used. These study results will assist HTA agencies that are re-visiting or developing their prioritization methods.

The complexities and nuances of evaluating the costs associated with providing medical technologies are often underestimated by analysts engaged in economic evaluations. This article describes the theoretical underpinnings of cost estimation, emphasizing the importance of accounting for opportunity costs and marginal costs. The various types of costs that should be considered in an analysis are described; a listing of specific cost elements may provide a helpful guide to analysis. The process of identifying and estimating costs is detailed, and practical recommendations for handling the challenges of cost estimation are provided. The roles of sensitivity analysis and discounting are characterized, as are determinants of the types of costs to include in an analysis. Finally, common problems facing the analyst are enumerated with suggestions for managing these problems.

Objectives: During the past decade, telehealth has enjoyed a plethora of public funding and publication outlets around the world. Yet, rhetoric appears to be outpacing the actual diffusion and utilization of telehealth technologies for patient care. Several barriers, such as reimbursement and legal/regulatory issues, are commonly cited as impeding the successful deployment of this innovation. However, two separate studies carried out in Michigan that controlled for these barriers point out a more significant initial gatekeeper to the deployment of telehealth, namely providers.

Methods: Multiple data collection strategies were used in both the telehospice and telepsychiatry projects, including utilization logs, surveys, telehospice nursing notes, cost frame data collection, patient interviews, caregiver interviews and focus groups, and videotaped visits.

Results: This study summarizes data from the two studies to support the hypothesis that the provider is the most important initial gatekeeper for telemedicine.

Conclusions: The implications from this conclusion have important consequences for health system deployment strategies. Specifically, telemedicine project managers must keep providers' needs (ease of use and incentives) in mind when designing a telemedicine system.

Objectives: A mechanism to increase the influence of Health Technology Assessments (HTAs) on hospital policy decisions was developed.

Methods: We describe the process and results of an experiment in which a local in-hospital HTA unit was created to provide sound evidence on technology acquisition issues, and to formulate locally appropriate policy recommendations. The Unit consists of a small technical staff that accesses and synthesizes the evidence incorporating local health and economic data, and a Policy Committee that develops policy recommendations based on this evidence. It represents administration, health-care professionals, patients, and representatives of the clinical disciplines affected by each issue. The level of success of the Unit was independently evaluated.

Results: To date, 16 reports have been completed, each within 2–4 months. Five recommended unrestricted use, seven recommended rejection, and four recommended very limited use of the technology in question. All have been incorporated into hospital policy. Budget impact is estimated at approximately $3 million of savings per year.

Conclusions: This local in-house HTA agency has had a major impact on the adoption of new technology. Probable reasons for success are (i) relevance (selection of topics by administration with on-site production of HTAs allowing them to incorporate local data and reflect local needs), (ii) timeliness, and (iii) formulation of policy reflecting community values by a local representative committee. Because over one third of all health-care costs are incurred in the hospital, diffusion of this model could have a significant effect on the quantity and quality of health-care spending.

Technology is generally defined as “science or knowledge applied to a definite purpose.” Technology assessment has been defined as a form of policy research that examines short- and long-term consequences (for example, societal, economic, ethical, legal) of the application of technology. The goal of technology assessment was said to be to provide policy makers with information on policy alternatives. Health technology assessment (HTA) grew out of this field and was developed in the U.S. Office of Technology Assessment (OTA). However, the OTA was closed in 1995. The links between technology assessment and health technology assessment were more-or-less lost after the dissolution of OTA, and few workers in the field of HTA seem familiar with the roots of the field in the more general and social-oriented technology assessment.

Objectives: This study aimed to investigate whether evidence-based clinical practice in health care improves outcomes for patients, personnel, and/or organizations.

Methods: A systematic review of studies was conducted with various quantitative and qualitative methods up to the Spring of 2002. Protocols were used in quality assessment. Data synthesis is descriptive in a narrative form.

Results: Of 305 assessed articles, eight studies were included. The outcomes in the included studies were related to the experiences of patients and personnel and to organization concerning changed patient care and resource utilization. Because the included studies are heterogeneous in design, focus of research area, and scientific quality, the scientific foundation for the findings is weak. There is some support that evidence-based clinical practice guidelines, when put to use, improve outcomes (i) for patients—less likelihood of showing worsening of skin condition and disruption of skin condition improves more rapidly for infants; (ii) for personnel—support in daily work situation; and (iii) for organizations—decreased admission rates and length of stay, less resource utilization and reduced costs.

Conclusions: There is a need for further research as the findings are based on a rather limited number of studies. There is a tendency toward support for the idea that outcomes improve for patients, personnel, or organizations if clinical practice in health care is evidence-based, that is, if evidence-based clinical practice guidelines are used, although these findings could be specific to the settings and context of the studies reported in this systematic review.