Health technology assessment (HTA) currently focuses on efficiency, rather than equity, on the basis that its primary objective is to maximize population health. Yet a strict cost-effectiveness approach sometimes conflicts with important equity concerns, such as the reduction of socioeconomic health inequalities. Managing such equity–efficiency trade-offs on the basis of intuition is unsatisfactory in a democracy, as it arouses suspicions of special pleading and favoritism toward vested interests. Over the next few decades, therefore, decision making may progress through up to three further stages of development observed historically in other areas of resource allocation. Stage two involves case law, limited to principles distilled from precedent. Stage three involves codification, seeking to generalize these principles without specifying their relative weights. Finally, at stage four, quantitative trade-offs are incorporated into a formula. At stage four, deliberation centers on adjustments to the formula, which would then be applied impartially, transparently, and fair-mindedly to all future decisions. Methods already exist for valuing equity–efficiency trade-offs, based on established methodological principles for valuing trade-offs between different dimensions of health. Early findings indicate that the general public thinks that social class inequalities are more inequitable than those by smoking status, with inequalities between the sexes somewhere in between. Relative weights can be calculated from these data, although the data are not yet comprehensive enough to do this credibly for current policy purposes. In the mean time, the equity–efficiency trade-offs suggested by current decisions can be estimated using standard cost-effectiveness analysis. This is because every departure from a strict cost-effectiveness approach has an opportunity cost. The size of that opportunity cost is a test of how much weight a particular equity concern is deemed to merit.

Objectives: The complexity of health technology assessment (HTA) has increased, in part because of its evolution through three distinct phases: the machine, the clinical outcomes, and the delivery models. However, the theoretical foundation for the field remains underdeveloped.

Methods: It is high time for HTA to bring together aspects of conceptual and theoretical works from other fields to strengthen the foundation of HTA.

Results: Many challenges await the further development of HTA. They can be captured around three research themes: adapting HTA to an evolving analysis object; translating HTA results into policy, management, and practice decisions; and evaluating organizational models of HTA.

Conclusions: Consolidating the scientific basis of HTA is essential if we are to succeed in increasing the relevance of HTA in some of the most challenging health-related decisions that we will make as individuals and societies.

Leukoreduction, filtering white blood cells from transfusion blood, effectively avoids leukocyte-related complications of blood transfusion. The technology has proven its relative cost-effectiveness for specific patient populations. With the advent of variant Creutzfeldt–Jakob disease, a transmittable spongiform encephalopathy caused by mad cow disease (bovine spongiform encephalopathy), the hard hit United Kingdom introduced universal leukoreduction for all patients as a precaution for transmission of prions in 1999. This costly policy was followed by many other countries, in the absence of much evidence of an actual health problem or of a more than presumed effectiveness of leukoreduction in preventing prion transmission. The core problem proved to be legal. The blood banks are legally accountable for blood safety. This accountability is absolute, based on avoidance of all possible risks, regardless of costs. This strategy leads to inefficiencies in health care: (i) blood safety management is guided by available rather than cost-effective technology, and (ii) private insurance premiums for civil liability are sharply increasing, while they are in no way related to the expected returns and the high and increasing blood safety. A rational safety policy is to be optimal, taking into account costs and effects of the safety procedures. This issue will need an open discussion with the general public of the real risks and a clear and unambiguous definition of proportionality in the precautionary principle, based on the European law.

Objectives: The International Network of Agencies for Health Technology Assessment (INAHTA) has been tracking activities associated with the clinical use of positron emission tomography (PET) in its members' healthcare systems since 1997 and published its first Joint Project report on PET in 1999. Part 1 of this Joint Project report presents survey results on diffusion, assessment activities, and policy for clinical use related to PET among INAHTA members since 1999.

Methods: INAHTA members were surveyed in 2003–2004.

Results: Twenty-seven INAHTA agencies (69 percent response rate) from nineteen countries responded to the survey. Dedicated PET systems are the most universally installed systems to date. Mobile scanners and modified gamma cameras are used occasionally as lower cost alternatives, and interest in PET–computed tomography hybrid models is rising despite limited assessment of impact on service planning. PET was used and assessed most commonly for managing patients with cancer. All respondents reported having some form of public funding for clinical PET frequently linked to data collection for the purpose of gathering evidence to refine clinical use and guide resource allocation toward indications that maximize clinical and cost-effectiveness.

Conclusions: The use of HTA within a continuous quality improvement framework can help optimize scarce resources for evaluation and use of high cost diagnostic technologies such as PET, particularly where potential clinical or cost-effectiveness is considerable but conclusive evidence is lacking.

Battista has written a thoughtful and timely challenge to the field of health technology assessment (HTA). As the author points out, HTA is changing and evolving. One of the greatest challenges is that HTA now seems firmly established in a number of countries, and is being looked to more and more for the “answers.” We accept and endorse Battista's central thesis, and essentially all of his critical questions, including his ideas about future research topics. We hope to evoke further thoughts in the reader and perhaps stimulate other responses, either private and public.

Objectives: Australia, Canada, and many European countries now use various forms of health technology assessment (HTA) in decision making regarding the reimbursement of drugs and other health technologies. To achieve a better understanding of the potential for use of HTA in this context, an analytical framework was developed to describe and classify existing fourth hurdle systems.

Methods: Based on a review of published literature, and official documentation, the key aspects of a fourth hurdle system were identified at two levels: policy implementation and individual technology decision. Characteristics of the systems were grouped under four main headings: constitution and governance, objectives, use of evidence and decision processes, and accountability. The comprehensiveness and relevance of this framework was assessed by an independent group of experts in HTA. A pilot study was undertaken, using only published sources, to test the feasibility of obtaining the information needed to complete the framework.

Results: The framework was found to be sufficiently broad to encompass all the issues of interest regarding the systems, but the proportion of information available from published sources was variable between sections of the framework and between countries, with average availability of 45 percent.

Conclusions: The analytical framework will help researchers and policy-makers in individual countries to understand their own systems and will allow some preliminary sharing of experience between countries. More experience of its application is needed to judge whether it will provide the basis for more formal comparison of systems and whether it will determine the appropriateness for particular decision contexts.

Objectives: The aim of this study was to conduct a systematic review of the evidence for treatments for retinoblastoma in children.

Methods: Seventeen electronic databases were searched. Two reviewers independently selected studies. Studies of participants diagnosed with childhood retinoblastoma, any interventions, and all clinical outcomes were eligible. Randomized and nonrandomized controlled trials and cohort studies with clear comparisons between treatment groups were included. Methodological quality was assessed.

Results: Thirty-one observational comparative studies were included, of which twenty-seven were retrospective. The methodological quality was generally poor, with a high risk of selection bias in all studies. Although there were high levels of treatment success in many of the studies, due to the limitations of the evidence identified, it was not possible to make meaningful and robust conclusions about the relative effectiveness of different treatment approaches for retinoblastoma in children.

Conclusions: Good quality randomized controlled trials are required. Where controlled trials are not feasible, only high quality prospective, nonrandomized studies should be given consideration, due to the generally higher risk of bias in retrospective studies.

Objectives: Since 1997, members of the International Network of Agencies for Health Technology Assessment (INAHTA) have collaborated on a Joint Project to track the diffusion, evaluation, and clinical policy of positron emission tomography (PET). Part 2 of this updated Joint Project report summarizes HTA-based strategies for directing the clinical use of PET and a discussion on the value of HTA in managing the diffusion of high cost diagnostic technologies, which were presented at an INAHTA-sponsored workshop at the Health Technology Assessment International Annual Meeting in 2004 on strategies for managing high cost diagnostic technologies.

Methods: A summary of the workshop proceedings is presented.

Conclusions: Sharing assessment work, universal agreement in assessment conclusions, stakeholder input, and modeling techniques help manage the uncertainty in the evidence base while targeting clinical use of PET toward the most promising indications. Emphasis on HTA findings, linkage between financing of clinical PET and outcome evaluation, and targeted dissemination of scientific findings empower providers to reduce unnecessary utilization and contain costs within a quality improvement framework. Above all, a trustworthy source of HTA information and a process that is conducive to using scientific evidence as the basis for decision making are essential for managing the diffusion of complex and costly diagnostic technologies in patient care.

Objectives: The objective of this study is to analyze statistically the possible determinants and implications of including or not including recommendations in health technology assessments (HTAs).

Methods: A sample of 433 HTAs published by eleven leading institutions or agencies in nine countries was reviewed and analyzed statistically by multiple logistic regression.

Results: The extent of policy and research recommendations in HTAs varies greatly from country to country. The content and scope of HTAs have some impact on recommendations. Extensive assessment of economic and organizational aspects increases the likelihood of including policy recommendations. Extensive assessment of technological and patient aspects increases the likelihood of including research recommendations, whereas extensive focus on economic aspects is negatively related to research recommendations. The most striking result is that the use of external partners for assessment increases the likelihood of including research recommendations in HTAs but not policy recommendations.

Conclusions: HTA commissioners, agencies, institutions, and funding authorities need to be aware of the consequences of the choices they make in advance of assessing health technologies. Outsourcing HTA to external partners suggests a greater likelihood of being told that “more research is needed.” The scope and content of HTAs has an impact on the type of recommendations, and country-specific preferences are strong predictors of recommendations in HTAs.

Objectives: Health technology assessment (HTA) has been used as input in decision making worldwide for more than 25 years. However, no uniform definition of HTA or agreement on assessment methods exists, leaving open the question of what influences the choice of assessment methods in HTAs. The objective of this study is to analyze statistically a possible relationship between methods of assessment used in practical HTAs, type of assessed technology, type of assessors, and year of publication.

Methods: A sample of 433 HTAs published by eleven leading institutions or agencies in nine countries was reviewed and analyzed by multiple logistic regression.

Results: The study shows that outsourcing of HTA reports to external partners is associated with a higher likelihood of using assessment methods, such as meta-analysis, surveys, economic evaluations, and randomized controlled trials; and with a lower likelihood of using assessment methods, such as literature reviews and “other methods.” The year of publication was statistically related to the inclusion of economic evaluations and shows a decreasing likelihood during the year span. The type of assessed technology was related to economic evaluations with a decreasing likelihood, to surveys, and to “other methods” with a decreasing likelihood when pharmaceuticals were the assessed type of technology.

Conclusions: During the period from 1989 to 2002, no major developments in assessment methods used in practical HTAs were shown statistically in a sample of 433 HTAs worldwide. Outsourcing to external assessors has a statistically significant influence on choice of assessment methods.

Objectives: The objective of this study was to examine and explain the differential international diffusion of six health innovations.

Methods: A retrospective diffusion study was undertaken of sildenafil, cyclooxygenase-II (COX II) inhibitors, beta interferon, verteporfin, deep brain stimulators, and drug-eluting coronary stents in ten countries—Australia, Canada, Denmark, France, The Netherlands, Norway, Spain, Sweden, Switzerland, and the United Kingdom. We plotted diffusion curves of daily defined doses per quarter, vials or implants per million population, and examined the association between diffusion and five key variables.

Results: Canada, Switzerland, and Sweden are generally high users of new technologies; Spain, Denmark, and particularly the United Kingdom are low users. Almost all countries experienced rapid adoption of sildenafil with diffusion to a similar level; there was variable adoption and diffusion of COX II inhibitors, verteporfin, and interferon beta; drug-eluting stents penetrated the market in a similar way in all but one country; and two countries had very different adoption patterns for deep brain stimulators. Above average health spending and the presence of health technology assessment (HTA) or other guidance reports are consistently associated with increased diffusion. Early warning activity and a national coverage decision being taken are more likely to be associated with a reduced diffusion.

Conclusions: The significant differences in diffusion between different countries are not consistent with a neat evidence-based world. The tools available to policy makers to control diffusion (early warning systems, HTA, and a fourth hurdle) play some part in influencing diffusion but need close scrutiny of how successfully they operate.

Objectives: Current policy and practice regarding identification of and extent of use of data from conference abstracts in health technology assessment reviews (TARs) are examined.

Methods: The methods used were (i) survey of TAR groups to identify general policy and experience related to use of abstract data, and (ii) audit of TARs commissioned by the National Institute for Health and Clinical Excellence (NICE) and published between January 2000 and October 2004.

Results: Five of seven TAR groups reported a general policy that included searching for and including studies available as conference abstracts and presentations. A total of sixty-three published HTA reports for NICE were identified. Of these reports, thirty-eight identified at least one randomized controlled trial available as an abstract/presentation. Twenty-six (68 percent) of these thirty-eight TARs included studies available as abstracts.

Conclusions: There are variations in policy and practice across TAR groups regarding the searching for and inclusion of studies available as conference abstracts. There is a need for clarity and transparency for review teams regarding how abstract data are managed. If conference abstracts are to be included, reviewers need to allocate additional time for searching and managing data from these sources. Review teams should also be encouraged to state explicitly their search strategies for identifying conference abstracts, their methods for assessing these abstracts for inclusion and, where appropriate, how the data were used and their effect on the results.

Objectives: To apply the Theory of Constraints (TOC) to the Evidence-based Practice Center (EPC) process.

Methods: Participants in the EPC process were interviewed, and the technical contracting literature was reviewed. Undesirable effects (UDEs) that might occur in the EPC process were identified and, in a TOC Current Reality Tree (CRT), a fundamental UDE was selected and tracked back to potential root causes, focusing on the core “constraint.” The resulting draft CRT was presented at an EPC directors' meeting and finalized. From possible solutions obtained from interviews and literature, a Future Reality Tree (FRT) was constructed.

Results: Three UDEs were identified and that a report is not used was deemed fundamentally undesirable. Nine root causes were identified, and using the CRT, a core constraint emerged: EPC partner does not know how to conceptualize and articulate needs, objectives, and specifications. Two change strategies (“injections”) appeared to address the core constraint: establish an ongoing relationship between partners, EPC, and Agency for Healthcare Research and Quality early in the process, and use a changes clause; in a FRT the injections lead to the positive outcome—the production of a successful EPC report that meets partner needs and, therefore, will be used.

Conclusions: As with any complex enterprise, the EPC needs to engage in self-reflection and active improvement to maximize its value. The development of technical reports intended to inform decision making represents a complicated area, involving a network of interrelated processes. Using the TOC permitted us to understand process failures; results serve as a guide to improvement.

Objectives: The aim of this study was to develop a guide to identifying and managing risks for health technology assessment (HTA) programs and to obtain opinions on this topic from HTA agencies.

Methods: The risks and approaches to their management were compiled, drawing on experiences from HTA programs and the risk assessment literature. Opinion on this classification was obtained from members of the International Network of Agencies for Health Technology Assessment (INAHTA).

Results: Twenty-one risks for HTA programs were identified under the categories Formulation of HTA Questions, Preparation of the HTA Product, Dissemination, and Contracting. For each risk area, potential consequences and suggested management approaches were outlined. Responses from ten HTA programs indicated substantial agreement regarding the risks that had been identified and on the importance of risk management for their own operations.

Conclusions: Prudent management of HTA programs should take into account the risks related to external factors.

Objectives: The objective of this study is to assess the impact of CEDIT (French Committee for the Assessment and Dissemination of Technological Innovations) recommendations on the introduction of technological innovations within the AP-HP (Assistance Publique–Hôpitaux de Paris), the French hospital network to which this body is attached.

Methods: In 2002, a study based on semidirective interviews of fourteen people affected by these recommendations and a case study relating to thirteen recommendations issued between 1995 and 1998 were conducted.

Results: The CEDIT is very scientifically reputable among interviewees. There is generally widespread interest for the recommendations. They are used as decision-making tools by administrative staff and as negotiating instruments by doctors in their dealings with management. Based on the case study, ten of thirteen recommendations had an impact on the introduction of the technology in health establishments. One recommendation appears not to have had an impact. Furthermore, the impact of two technologies was impossible to assess.

Conclusions: This study highlights the significant impact of recommendations arising from a structure that is attached to a hospital network and the good match between CEDIT's objectives and its assignments.

Objectives: The aim of this study was to examine (i) the consistency of reporting research findings presented in conference abstracts and presentations and subsequent full publications, (ii) the ability to judge methodological quality of trials from conference abstracts and presentations, and (iii) the effect of inclusion or exclusion of data from these sources on the pooled effect estimates in a meta-analysis.

Methods: This report is a case study of a selected health technology assessment review (TAR) of a rapidly evolving technology that had identified and included a meta-analysis of trial data from conference abstracts and presentations.

Results: The overall quality of reporting in abstracts and presentations was poor, especially in abstracts. There was incomplete or inconsistent reporting of data in the abstract/presentations. Most often inconsistencies were between conference slide presentations and data reported in published full-text articles. Sensitivity analyses indicated that using data only from published papers would not have altered the direction of any of the results when compared with those using published and abstract data. However, the statistical significance of three of ten results would have changed. If conference abstracts and presentations were excluded from the early analysis, the direction of effect and statistical significance would have changed in one result. The overall conclusions of the original analysis would not have been altered.

Conclusions: There are inconsistencies in data presented as conference abstracts/presentations and those reported in subsequent published reports. These inconsistencies could impact the final assessment results. Data discrepancies identified across sources included in TARs should be highlighted and their impact assessed and discussed. Sensitivity analyses should be carried out with and without abstract/presentation data included in the analysis. Incomplete reporting in conference abstracts and presentations limits the ability of reviewers to assess confidently the methodological quality of trials.

Objectives: Evaluating the impact of recommendations based upon health technology assessment (HTA) represents a challenge for both HTA agencies and healthcare policy makers. This research sought to understand factors affecting the uptake of HTA recommendations to support decision making with respect to the introduction of three health technologies.

Methods: Using a multidimensional framework, based upon a combination of theoretical models, a case study was conducted. A total of twenty-eight semistructured interviews were done with physicians from fifteen hospitals and other stakeholders in Catalonia. Interview content was analyzed iteratively and classified according to theoretical dimensions and contextual factors.

Results: At the sociopolitical level, factors related to the organization and financing of the health system were found to affect the utilization of HTA recommendations. At the healthcare organization level, existing collaborations between the hospital and the HTA agency favored the integration of recommendations into practices. Formalism in the organization also influenced the utilization of HTA recommendations. At the professional level, the high degree of autonomy of specialists, the importance of peers and collegial control, and the definition of professional roles and responsibilities influenced physicians' willingness to integrate HTA recommendations into their practice.

Conclusions: This study offers a comprehensive framework to understand the complex dynamics that affect adoption of health technologies in organizational and professional practices. The findings suggest some avenues to promote the integration of HTA recommendations into practices and, thus, increase the utilization of scientific evidence to support decision making in health care.

Objectives: It is generally accepted that home peak flow monitoring increases patients' self-management and could lead to cost savings. The aim of this review was to analyze costs and the cost-effectiveness of self-management based on peak flow monitoring interventions in asthma.

Methods: Twenty-one studies were included in this review. Data were extracted, and methodological and economic quality were assessed. These studies presented economic information regarding self-management interventions based on peak flow monitoring in asthmatics. The mean methodological quality was 4.6 (maximum 8), and the mean economic quality was 12.0 (maximum 15).

Results: In eighteen studies, the interventions led to net savings compared with usual care or less intensive intervention. Only three studies found the total costs to be higher in the intervention group. In thirteen of the seventeen studies that analyzed health outcomes, at least one of the reported health outcomes improved statistically significantly after the intervention. However, the methods of economic evaluation differed among the studies and were not always in line with the standard methodology.

Conclusions: The interventions, costs, and outcomes were very diverse. The results emphasize the need for guidelines to increase the comparability of cost-effectiveness evaluations relating to asthma. Only then will it be possible to conclude whether interventions for asthmatics, such as self-management based on peak flow monitoring interventions, are cost-effective.

Objectives: The purpose of this project was to evaluate local decision support tools used in the Danish hospital sector from a theoretical and an empirical point of view.

Methods: The use of local decision support was evaluated through questionnaires sent to all county health directors, all hospital managers, and all heads of clinical departments in cardiology, orthopedic surgery, and intensive care. In addition, respondents were asked to submit whatever decision support tools they were using (including mini-HTAs, other forms or checklists, and special procedures for decision making concerning new health technologies). A theoretical analysis of the decision support tools (decision theory) was performed as well as a comparison with the business case method used in private companies. Finally, the Danish mini-HTA was compared with foreign production and use of HTA and HTA-like assessments as local decision support.

Results: The response rate was high (87 percent, 94 percent, 85 percent, respectively). We collected sixty different forms (of which forty-nine were mini-HTAs) and twenty variants of written procedures. We found theoretical and empirical evidence that local involvement in the process of making the HTA could be important for the use of the results from the HTA and for the process of implementing the new technology.

Conclusions: Doing mini-HTA in hospitals seems to balance the need for quality and depth with the limited time and resources for assessment.

Objectives: The purpose of this study was to identify the Evidence-Based Practice Center (EPC) network participants' perceptions of the characteristics of the EPC process and the relationship of the process to the success of EPC reports.

Methods: Semistructured interviews were conducted with the three groups involved in the EPC: EPC staff, Agency for Healthcare Research and Quality (AHRQ) staff, and representatives of partner organizations.

Results: The analysis of the coded transcripts revealed three related major themes, which form the conceptual basis for the interpretation presented here: the definition of a successful report, the determinants of a successful report, and the role of AHRQ in the process.

Conclusions: A successful report is a report that is used. The ultimate success of the core health technology assessment objective, moving from research to policy, depends on balancing two values: excellence and relevance. Our findings are consistent with the “two communities thesis,” which postulates the existence of two camps that confer different values to excellence and relevance, with resulting tension. A promising model for approaching this tension is integration or collaboration, which requires linking researchers and policy makers, promoting productive dialogues about the formulation and timing of analysis, and early consideration of how the resulting analysis will be used. This effort suggests that actively blurring the frontiers between these two groups will enhance their interaction. Furthermore, enhancing the role of the AHRQ as scientific broker will maximize the potential of the EPC network.