In 1982 we began discussions about the need for a new journal devoted to exploring the impact of technology on health care, and in 1984 we signed on with Cambridge University Press to launch the International Journal of Technology Assessment in Health Care. The first issue appeared in 1985 in connection to the inaugural meeting that year of the International Society for Technology Assessment in Health Care (ISTAHC), which has held annual meetings for 18 years starting in Washington DC in 1986. The Society was later reformed as Health Technology Assessment International (HTAi) in 2003. The Journal has served as the official scientific publication of both ISTAHC and HTAi for 24 years, and its mission remains as it was stated in the introduction to the first issue: to serve as an international forum to explore the effects of health care technology, and to provide “a channel of communication among the diverse and world-wide audience of scholars, practitioners, policy makers, and administrators concerned with this issue. . .to nurture the discipline of technology assessment” (1).

Health technology assessment (HTA) is a dynamic, rapidly evolving process, embracing different types of assessments that inform real-world decisions about the value (i.e., benefits, risks, and costs) of new and existing technologies. Historically, most HTA agencies have focused on producing high quality assessment reports that can be used by a range of decision makers. However, increasingly organizations are undertaking or commissioning HTAs to inform a particular resource allocation decision, such as listing a drug on a national or local formulary, defining the range of coverage under insurance plans, or issuing mandatory guidance on the use of health technologies in a particular healthcare system. A set of fifteen principles that can be used in assessing existing or establishing new HTA activities is proposed, providing examples from existing HTA programs. The principal focus is on those HTA activities that are linked to, or include, a particular resource allocation decision. In these HTAs, the consideration of both costs and benefits, in an economic evaluation, is critical. It is also important to consider the link between the HTA and the decision that will follow. The principles are organized into four sections: (i) “Structure” of HTA programs; (ii) “Methods” of HTA; (iii) “Processes for Conduct” of HTA; and (iv) “Use of HTAs in Decision Making.”

Objectives: Investments in health research should lead to improvements in health and health care. This is also the remit of the main HTA program in the Netherlands. The aims of this study were to assess whether the results of this program have led to such improvements and to analyze how best to assess the impact from health research.

Methods: We assessed the impact of individual HTA projects by adapting the “payback framework” developed in the United Kingdom. We conducted dossier reviews and sent a survey to principal investigators of forty-three projects awarded between 2000 and 2003. We then provided an overview of documented output and outcome that was assessed by ten HTA experts using a scoring method. Finally, we conducted five case studies using information from additional dossier review and semistructured key informant interviews.

Results: The findings confirm that the payback framework is a useful approach to assess the impact of HTA projects. We identified over 101 peer reviewed papers, more than twenty-five PhDs, citations of research in guidelines (six projects), and implementation of new treatment strategies (eleven projects). The case studies provided greater depth and understanding about the levels of impact that arise and why and how they have been achieved.

Conclusions: It is generally too early to determine whether the HTA program led to actual changes in healthcare policy and practice. However, the results can be used as a baseline measurement for future evaluation and can help funding organizations or HTA agencies consider how to assess impact, possibly routinely. This, in turn, could help inform research strategies and justify expenditure for health research.

Objectives: The aim of this study was to analyze the current status of population screening for colon/rectum cancer in Europe to compare the different strategies, the coverage, the existence of pilot experiences, regional coverages, and the risk factors considered in each strategy.

Methods: A comprehensive, systematic search was performed in the literature for documents addressing population screening for colon/rectum cancer in Europe. An ad hoc questionnaire was prepared including questions considered relevant. The questionnaire was reviewed by experts in the area. To identify key informants, colleague members of the International Network of Agencies for Health Technology Assessment (INAHTA), participants in the EUnetHTA project, or representatives of the ministries of health of the different European countries were contacted. The information provided by key informants was checked with information directly obtained from the ministries of health, gray literature, and research documents.

Results: An 88 percent response rate was obtained. In countries for which no questionnaire data were collected, information was directly retrieved from the Web sites of the corresponding ministries. Four countries were found to perform population screenings: Austria, France, Germany, and the United Kingdom. However, they used different strategies. Five countries had begun regional or local strategies: Denmark, Finland, Italy, Spain, and Switzerland, and two additional countries (the Netherlands and Norway) reported ongoing research studies intended to determine the best strategy to implement a population-based screening program. Differences were found in age range, procedure chosen, and follow-up period.

Conclusions: Even though the European Council recommended a wider implementation of population screening for colon/rectum cancer, our results suggest that this recommendation continues to be valid. The differences found in screening strategies (in terms of age range, procedures, risk factors considered, and follow-up periods) are not warranted by the results obtained in research studies or regional-national cancer registries.

Objectives: The semiprivate health system in Iran has created an opportunity for unnecessary uses of advanced medical equipments including magnetic resonance imaging (MRI). This study aimed to evaluate the evidence for MRI overuse in private diagnostic imaging centers in Tehran, Iran. The objectives of this study were to determine the frequency of use of MRI scans for different complaints and to explore frequency of normal MRI findings as a function of unnecessary MRI use.

Methods: We conducted a survey among private MRI centers in Tehran, Iran, to study the proportion of MRI scans that may result in significant clinical finding. All MRI reports at a specific point in time at selected MRI centers were reviewed by a physician and the findings were recorded as normal, abnormal, or substantial changes.

Results: Of all the MRI reports, 17.2 percent had resulted in normal findings; 9.8 percent ordered for examination of headache, and 4.8 percent for lower back pain.

Conclusion: Unnecessary MRIs are most likely to result in normal finding; although not all the MRI with normal results could be identified as unnecessary. Negative findings from MRI scans may be reassuring to both clinicians and patients. The proportion of normal findings in MRI scans did not provide evidence of MRI overuse in Iran. The results of this study warrant formation of guidelines for the use of MRIs for headache and low back pain disorders.

Objectives: The aim of this study was to examine the impact of the multi-hundred million dollar investment by the federal government in the developing Patient Outcomes Research Teams (PORTs) in over a dozen major academic medical centers in the United States throughout the 1990s. The objective of the PORTs was to reduce unnecessary clinical variation in medical treatment.

Methods: Using an economic derivation of welfare loss attributable to medical practice variation and hospital admission claims data for 2 million elderly patients generalizable to the nation, we estimate the change in welfare between 1991 and 2000, the period within which the PORTs were designed and executed and their results disseminated.

Results: Our results show inpatient admission types targeted by the PORTs did have less welfare loss relative to their total expenditure by 2000, but that there was not a net decrease in the welfare loss for all hospital admissions affected by the PORT.

Conclusions: We conclude that PORTs may have had favorable effects on welfare, most likely by reducing variation in clinical care, but that causality cannot be proved, and the effects were not equal across all conditions targeted by PORTs. This research provides a methodological template that may be used to evaluate the impact of patient safety research on welfare loss and on variation in medical treatment in both hospital and ambulatory settings.

Objectives: The study aims at evaluating the economic evidence related to testing for genetic variants of the drug-metabolizing enzyme, TPMT. Detecting TPMT genetic variants before the administration of azathioprine (AZA) has the potential to prevent serious and costly adverse drug reactions (ADRs), such as neutropenia. In particular, our analysis concentrated on assessing the reliability of data on costs of neutropenia and performing the tests, the two main cost categories that could inform an economic evaluation of TPMT pharmacogenetic testing.

Methods: A systematic literature review was performed to gather evidence on the costs of testing and neutropenia. Articles were critically appraised for their comprehensiveness and quality. To better estimate costs of TPMT tests, a small-scale survey of European diagnostic laboratories was conducted.

Results: Only seven articles were retrieved specifying the costs associated with the management and treatment of AZA-induced neutropenia. Most of these studies are based on theoretical modeling reconstructed with key-informants or on very few cases of ADRs, and either the methodology for cost calculation is not specified or costs are based on national cost databases and tariffs. After critical appraisal of these studies, we considered €2,116 as the most reliable estimate for the cost of a case of neutropenia. Literature review accompanied by the survey of several diagnostic laboratories also provided an estimate (€68) for TPMT testing. Based on these values, the net cost per prevented case of neutropenia equals to €5,300.

Conclusions: Solid economic considerations related to TPMT pharmacogenetic testing are still limited by underreporting of ADRs and high level of approximation related to cost data. Ad hoc observational studies and the ADR recording process embedded in pharmacovigilance systems, established across Europe, should represent more reliable sources of cost data in the future.

Background: Home-fortification is a new strategy of adding micronutrients including zinc and iron to home-made foods. Zinc supplementation may prevent morbidity and mortality related to diarrheal illnesses, and iron supplementation may improve cognitive development, in children.

Objectives: To project clinical and economic effects of home-fortification in children in an urban slum of Karachi, Pakistan.

Methods: This is a cost benefit analysis of 5,000 simulated male and female infants (6–12 months) assigned to micronutrients or placebo for 4 months and followed for 55 years. We linked the effect of zinc on longitudinal prevalence of diarrhea to mortality, and the effect of iron on hemoglobin to IQ scores and lifetime earnings. Cost estimates were based on volumes of resource utilization from the Pakistan Sprinkles Diarrhea study. Main outcome was incremental benefit defined as the gain in lifetime earnings after accounting for the incremental costs of micronutrients over placebo (societal perspective).

Results: Our model projected that the reduction in diarrhea and improvement in hemoglobin concentrations through home-fortification was associated with reduced child mortality, higher IQ scores, and higher earnings. The present value of incremental benefit was $106 (95 percent probability interval = $17 to $193) U.S. dollars, which corresponds to $464.79 ($74.54 to $846.27) international dollars using a purchasing power parity exchange rate.

Conclusions: Home-fortification appears to improve clinical outcomes at a reasonable cost, and may actually be cost beneficial when lifetime earnings are considered.

Objectives: The aims of this study were to assess the 1-year cost-effectiveness of a new combined immunosuppressive and anti-infectious regimen in kidney transplantation to prevent both rejection and infectious complications.

Methods: Patients (pts) transplanted from January 2000 to March 2003 (Group A) and treated with a conventional protocol were compared with pts submitted to a combined regimen including universal cytomegalovirus (CMV) prophylaxis between April 2003 and July 2005 (Group B). Costs were computed from the hospital accounting system for hospital stays, and official tariffs for outpatient visits. Patients with incomplete costs data were excluded from analysis.

Results: Fifty-three patients were analyzed in Group A, and 60 in Group B. Baseline characteristics including CMV serostatus were not significantly different between the two groups. Over 12 months after transplantation, acute rejections decreased from 41.5 percent in Group A to 6.7 percent in Group B (p < .001), and CMV infections from 47 percent to 15 percent (p < .001). Overall, readmissions decreased from 68 percent to 55 percent (p = .160), and average hospital days from 28 ± 19 to 20 ± 11 days (p < .007). The average number of outpatient visits decreased from 49 ± 10 to 39 ± 8 (p < .001). Average 1-year immunosuppressive and CMV prophylaxis costs (per patient) increased from CHF20,402 ± 7,273 to 27,375 ± 6,063 (p < .001), graft rejection costs decreased from CHF4,595 ± 10,182 to 650 ± 3,167 (p = .005), CMV treatment costs from CHF2,270 ± 6,161 to 101 ± 326 (p = .008), and outpatient visits costs from CHF8,466 ± 1’721 to 6,749 ± 1,159 (p < .001). Altogether, 1-year treatment costs decreased from CHF39’957 ± 16,573 to 36,204 ± 6,901 (p = .115).

Conclusions: The new combined regimen administered in Group B was significantly more effective, and its additional costs were more than offset by savings associated with complications avoidance.

Objectives: The increasing use of full economic evaluations has led to the development of various instruments to assess their quality. The purpose of this study was to compare the frequently used British Medical Journal (BMJ) check-list and two new instruments: the Consensus Health Economic Criteria (CHEC) list and the Quality of Health Economic Studies (QHES) instrument. The analysis was based on a practical exercise on economic evaluations of the surgical treatment of obesity.

Methods: The quality of nine selected studies was assessed independently by two health economists. To compare instruments, the Spearman rank correlation coefficient was calculated for each assessor. Moreover, the test–retest reliability for each instrument was assessed with the intraclass correlation coefficient (ICC) (3,1). Finally, the inter-rater agreement for each instrument was estimated at two levels: comparison of the total score of each article by the ICC(2,1) and comparison of results per item by kappa values.

Results: The Spearman's rank correlation coefficient between instruments was usually high (rho > 0.70). Furthermore, test–retest reliability was good for every instruments, that is, 0.98 (95 percent CI, 0.86–0.99) for the BMJ check-list, 0.97 (95 percent CI, 0.73–0.98) for the CHEC list, and 0.95 (95 percent CI, 0.75–0.99) for the QHES instrument. However, inter-rater agreement was poor (kappa < 0.40 for most items and ICC(2,1) ≤ 0.5).

Conclusions: The study shows that the results of the quality assessment of economic evaluations are not so much influenced by the instrument used but more by the assessor. Therefore, quality assessments should be performed by at least two independent experts and final scoring based on consensus.

Objectives: Of the fourteen counties and two municipalities that until recently were responsible for healthcare provision in Denmark, five introduced mammography screening (MS) programs. The objective of this research is to explain this decision-making variation and to gain insight into priority setting processes in health-care provision at the county level in Denmark.

Methods: Literature on priority setting in health care was used to derive seven explanatory factors for comparing decision making on MS between four selected counties, of which two had implemented MS. The relative importance of each explanatory factor in each county was determined by analyzing policy documents, supplemented with interviews of selected stakeholders. The results were combined and compared at the county level.

Results: Evidence of effectiveness of MS was considered satisfactory and ethical issues related to MS were perceived relatively unproblematic only in those counties that introduced MS. Lack of resources, that is, radiologists, was an additional important factor for counties not implementing MS. Local opinion leaders have played a stimulating role, whereas advisory policy documents at the central government level and even legislation have had a minor impact.

Conclusions: The four counties have based their decision making on the introduction of MS on different combinations of a limited number of factors that have been differentially weighted. The pattern of relevant factors in both counties not introducing MS is rather similar. The study elucidates the role of complementary factors to evidence in decision making. Of interest, recent public sector reforms have resulted in the decision to have MS implemented nationwide.

Objectives: To catalogue and comparatively assess the quality of Clinical Practice Guidelines (CPG) for ischemic stroke taking into account format and development methodology.

Methods: We performed a comprehensive, systematic bibliographic search of CPGs addressing the management of ischemic stroke. We designed a sensitive strategy, using methodological filters in the following databases: Medline, IME and Lilacs, National Guidelines Clearinghouse, National electronic Library for Health, NICE, Guidelines International Network (GIN), Canadian Medical Association Infobase, development groups such as Scottish Intercollegiate Guidelines Network (SIGN), New Zealand Guidelines Group (NZGG), Agency for Healthcare Research and Quality (AHRQ), Ministry of Health Singapore, Institute for Clinical Systems Improvement (ICSI); and scientific societies: American Heart Association, American Medical Association, Royal College of Physicians London. We included all CPGs published in English, French, Italian, Portuguese, or Spanish from 1999 to 2005 and excluded those CPGs whose scope was primary prevention and rehabilitation from ischemic stroke. Four researchers independently assessed the structure and methodologies followed in drafting the CPGs using the Changing Professional Practice (CPP) and Appraisal of Guidelines Research & Evaluation (AGREE) instruments.

Results: We retrieved 117 documents; following application of exclusion criteria, twenty-seven CPGs were appraised. With regard to methodological quality (using the AGREE instrument), the domains that scored highest were “Scope and purpose” and “Clarity and presentation.” The lowest scoring domains were “Stakeholder involvement,” “Rigor of development,” and “Applicability.” Most guidelines received an overall score of “would not recommend” (77.8 percent). Finally, based on the CPP instrument, most of the CPGs evaluated were aimed at secondary care and did not provide updating procedures.

Conclusions: The overall quality of the CPGs published for ischemic stroke management did not have minimum methodological quality. Quality improvement has been observed in more recent CPGs and may be due to the publication of new tools such as the AGREE or CPP instruments, as well as international initiatives for CPG improvement.

Objectives: This report is a scoping review of the literature with the objective of identifying definitions, conceptual models and frameworks, as well as the methods and range of perspectives, for determining appropriateness in the context of healthcare delivery.

Methods: To lay groundwork for future, intervention-specific research on appropriateness, this work was carried out as a scoping review of published literature since 1966. Two reviewers, with two screens using inclusion/exclusion criteria based on the objective, focused the research and articles chosen for review.

Results: The first screen examined 2,829 abstracts/titles, with the second screen examining 124 full articles, leaving 37 articles deemed highly relevant for data extraction and interpretation. Appropriateness is defined largely in terms of net clinical benefit to the average patient and varies by service and setting. The most widely used method to assess appropriateness of healthcare services is the RAND/UCLA Model. There are many related concepts such as medical necessity and small-areas variation.

Conclusions: A broader approach to determining appropriateness for healthcare interventions is possible and would involve clinical, patient and societal perspectives.

Objectives: A treatment pathway model was developed to examine the costs and benefits of the current bowel cancer service in England and to evaluate potential alternatives in service provision. To use the pathway model, various parameters and probability distributions had to be specified. They could not all be determined from empirical evidence and, instead, expert opinion was elicited in the form of statistical quantities that gave the required information. The purpose of this study is to describe the procedures used to quantify expert opinion and note examples of good practice contained in the case study.

Methods: The required information was identified and preparatory discussion with four experts refined the questions they would be asked. In individual elicitation sessions they quantified their opinions, mainly in the form of point and interval estimates for specified variables. New methods have been developed for quantifying expert opinion and these were implemented in specialized software that uses interactive graphics. This software was used to elicit opinion about quantities related to measurable covariates.

Results: Assessments for thirty-four quantities were elicited and available checks supported their validity. Eight points of good practice in eliciting and using expert judgment were evident. Parameters and probability distributions needed for the pathway model were determined from the elicited assessments. Simulation results from the pathway model were used to inform policy on bowel cancer service provision.

Conclusions: The study illustrates that quantifying and using expert judgment can be acceptable in real problems of practical importance. For full benefit to be gained from expert knowledge, elicitation must be conducted carefully and should be reported in detail.

Objectives: Individual patient data (IPD) meta-analyses have been proposed as a major improvement in meta-analytic methods to study subgroup effects. Subgroup effects of conventional and IPD meta-analyses using identical data have not been compared. Our objective is to compare such subgroup effects using the data of six trials (n = 1,643) on the effectiveness of antibiotics in children with acute otitis media (AOM).

Methods: Effects (relative risks, risk differences [RD], and their confidence intervals [CI]) of antibiotics in subgroups of children with AOM resulting from (i) conventional meta-analysis using summary statistics derived from published data (CMA), (ii) two-stage approach to IPD meta-analysis using summary statistics derived from IPD (IPDMA-2), and (iii) one-stage approach to IPD meta-analysis where IPD is pooled into a single data set (IPDMA-1) were compared.

Results: In the conventional meta-analysis, only two of the six studies were included, because only these reported on relevant subgroup effects. The conventional meta-analysis showed larger (age < 2 years) or smaller (age ≥ 2 years) subgroup effects and wider CIs than both IPD meta-analyses (age < 2 years: RDCMA -21 percent, RDIPDMA-1 -16 percent, RDIPDMA-2 -15 percent; age ≥2 years: RDCMA -5 percent, RDIPDMA-1 -11 percent, RDIPDMA-2 -11 percent). The most important reason for these discrepant results is that the two studies included in the conventional meta-analysis reported outcomes that were different both from each other and from the IPD meta-analyses.

Conclusions: This empirical example shows that conventional meta-analyses do not allow proper subgroup analyses, whereas IPD meta-analyses produce more accurate subgroup effects. We also found no differences between the one- and two-stage meta-analytic approaches.

The article “Key principles for the improved conduct of health technology assessments for resource allocation decisions” presents fifteen principles for health technology assessment. Many of these cannot be disputed, and application of the principles as stated would undoubtedly improve HTA as it is developing in the world at large. My question is, are these the most important principles? The document does not really try to answer this question.

The paper presents a set of principles that cover the broad scope of HTA from formulation of the question to effects on the decision-making process. It also provides goals for HTA programs to aim for in producing assessments. The principles are supported by material that includes illustrations from the experience of several programs. The authors have had long contact with the field and are well able to authoritatively discuss a range of methodological and other topics. The paper includes a range of views and details that are relevant to those involved with HTA.

“Key principles for the improved conduct of health technology assessment for resource allocation decisions.” The title says it all. I am amazed that such a summary is even possible and that it defines this field so well. Perhaps this report will be cited for years to come as the best and central definition of HTA. A field like this can be defined in other ways.

Our goal in formulating the proposed principles was to stimulate discussion of HTA program goals and procedures to enhance the rigor, validity, and usefulness of HTA. The thoughtful and constructive comments by David Banta, David Hailey, and Duncan Neuhauser are an excellent beginning to what we hope will be an extended and interactive process. We appreciate the opportunity to respond to some of their comments, suggestions, and critiques.

In the article entitled “Rapid reviews versus full systematic reviews: An inventory of current methods and practice in health technology assessment,” by Watt et al. in volume 24 number 2 (Spring 2008) of International Journal of Technology Assessment in Health Care, the affiliation of Stephen Blamey is incorrectly listed as Department of Health & Ageing. Dr. Blamey is the current Chair of the Medical Services Advisory Committee (MSAC). MSAC is an independent scientific committee comprising individuals with expertise in clinical medicine, health economics, and consumer matters. The Department of Health & Ageing administers funding and operations for MSAC. However, members of MSAC act independently of the Department. As Chair of MSAC, Dr. Blamey can be contacted through the Department. Dr. Blamey is not affiliated with the Department of Health and Ageing and his contribution to the above-mentioned article does not reflect its policy. Dr. Blamey wishes to apologize for this misunderstanding.