Objectives: As an initiative of the International Network of Agencies for Health Technology Assessment (INAHTA), a checklist for assessment reports was developed as a means of improving transparency and consistency in HTA.

Methods: Preparation of a summary of key elements in HTA reports, drawing on experience in preparation of such documents, information from guidelines for HTA, and individual assessments. Review by INAHTA agencies and modification of the summary to reflect the consensus.

Results: The resulting checklist includes 17 questions, with supporting detail. General areas covered include preliminary information, why and how the assessment has been prepared, the results of the assessment, implications of the results, and conclusions.

Conclusions: The checklist is intended to be considered by those preparing or using an HTA report. It reflects the views of INAHTA members and is seen as a mechanism to improve the standard of HTA reports, being complementary to the more detailed guidelines on how to conduct assessments.

Objectives: To explore one aspect of the external validity of the randomized controlled trial (RCT), specifically how being selected for inclusion in a trial and having participated has influenced the use and cost of asthma-related health services.

Methods: Services used by asthmatic users of inhaled corticosteroids (iCSTs) having previously participated in an RCT (TS, n = 46) were compared with individuals who had never participated (NS, n = 51).

Results: TS were more likely to use higher (≥400 μg) daily doses of iCSTs than NS (OR, 3.3; 95% CI, 1.1–8.3) but less likely to visit emergency departments (OR, 0.3; 95% CI, 0.1–0.7). Total asthma-related costs did not differ significantly.

Conclusions: Subject differences may impede generalizing from RCTs to real life.

Objective: To determine if a pre-assessment can be used to establish whether cost-effectiveness results would meet the actual information needs of Dutch healthcare decision makers.

Methods: Two recent studies in rehabilitation medicine served as study material. Based on Wholey, a limited pre-assessment was performed in which the potential impact of cost-effectiveness analysis (CEA) results on intended users' decision making was assessed. Desk research and semi-structured interviews with several intended users of CEA results were performed. These included general practitioners, representatives of health insurance companies, the Health Care Insurance Board (CvZ), and medical guidelines committees.

Results: In day-to-day decision making of the interviewed decision makers, a cost-effectiveness criterion seemed to be of limited importance. Instead, results from clinical effectiveness studies and budget impact studies appeared to be sufficient. CvZ, however, preferred relative cost-effectiveness to be a criterion for inclusion in future reimbursement guidelines. In both cases the limited pre-assessments changed the expectations of the investigators regarding decision-making impact of an economic evaluation.

Conclusion: This study revealed that the use of CEA results for Dutch micro- and meso-level healthcare decision making is not self-evident. The main purpose of CEA results is to support health policy making and planning at a macroeconomic level. Pre-assessment can be a valuable tool in designing a CEA to support the actual information needs of the decision makers.

This study examines the parameters crucial to cost-effectiveness of universal hepatitis B immunization in India. An incremental cost-effectiveness analysis was done using a decision tree (Markov model) to follow up a hypothetical cohort of 100,000 newborns for the effects of hepatitis B acquired vertically at birth. The measure of effectiveness was disability-adjusted life-years gained. Uncertainty analysis and Scenario analysis were done using Latin hypercube sampling. Hepatitis B endemicity is the most important factor, followed by the cost of vaccine. Other factors of some influence are vaccination coverage, vaccine efficacy, HBeAg positivity, and vaccine wastage.

Objectives: To review the scientific evidence on the efficacy, effectiveness, and safety of intra-articular injections of Hylan G-F 20 for the treatment of knee osteoarthritis.

Methods: Systematic review of experimental and observational studies performed in humans up to December 1999. Qualitative and quantitative (meta-analytic) techniques were used for data synthesis.

Results: A single course of intra-articular Hylan G{-}F 20 provides a statistically significant and clinically relevant short-term decrease of the painful symptomatology of knee osteoarthritis and improves joint function. It also seems to delay the need for knee replacement, if results observed in noncontrolled studies are confirmed. Hylan G-F 20 has a comparable efficacy to that of oral NSAID, and a smaller risk of gastrointestinal adverse effects. It seems to be well tolerated and safe, but the short follow-up in most studies limits any extrapolation of the effectiveness and safety over the longer term. There is also scarce evidence on the effect of multiple courses of Hylan G-F 20, and the scientific rigor of both experimental and nonexperimental studies reviewed is somewhat limited.

Conclusions: Whereas there is good quality scientific evidence showing that Hylan G-F 20 is a safe and well-tolerated therapy providing a short-term decrease of the pain symptoms while improving joint function, the delay of the need for knee replacement as well as the durability of the effect over the longer term have only been demonstrated in noncontrolled clinical series. The available evidence is not sufficient to reach firm conclusions on the effect of multiple courses of intra-articular injections of Hylan G-F 20 on health outcomes.

Objectives: To develop and test an appropriateness of indications tool for total knee replacement (TKR) in patients with osteoarthritis.

Methods: Criteria were developed using a modified Delphi panel judgment. Another panel rated the same indications, and the results were compared with the main panel. Test-retest of the main panel was performed. Regression models were used to assess the contribution of each algorithm variable. A classification tree was developed.

Results: The procedure was considered appropriate in 167 (26.8%) scenarios, and there was agreement on 112 (67.1%) of them. When the rates of the main panel were compared with those of a second panel, the result was a kappa statistic of 0.75. The test-retest kappa for the main panel was 0.78. Neither in the first case nor in the second was there an instance in which a scenario classified as appropriate shifted to inappropriate or vice versa. The regression models showed that symptomatology and radiology were the variables that explained most of the variability of appropriateness as determined by panelists. In the classification tree performed, the probability of misclassification was 3.8% with 150 scenarios, of the 156 analyzed and classified correctly.

Conclusions: The previous parameters tested showed acceptable results for an evaluation tool. These results support the use of this algorithm as an aid in formulating clinical practice guidelines and to promote the appropriateness of TKR.

Objectives: This study compared the relative cost-effectiveness of daily recombinant human deoxyribonuclease (rhDNase), with alternate day rhDNase and hypertonic saline (HS) for treating children with cystic fibrosis (CF).

Methods: A randomized controlled trial with a crossover design allocated 40 CF children consecutively to 12 weeks of daily rhDNase, alternate day rhDNase, or HS. The primary outcome measure was forced expiratory volume in 1 second (FEV1), a measure of lung function. All health resource use was prospectively documented for each patient and multiplied by unit costs to give a total health service cost for each 12-week treatment period. The nonparametric bootstrap method was used to present cost-effectiveness acceptability curves and net benefit statistics for each treatment comparison, for various hypothetical levels of the decision maker's ceiling ratio.

Results: Compared with HS, there was a 14% improvement in FEV1 for daily rhDNase (95% CI, 5% to 23%), and a 12% improvement (95% CI, 2% to 22%) for alternate day rhDNase. For a ceiling ratio of £200 per 1% gain in FEV1, the mean net benefits of daily and alternate day rhDNase compared with HS were £1,158 (95% CI, −£621 to 2,842) and £1,188 (95% CI, −847 to 3,343), respectively; the mean net benefit of daily compared with alternate day rhDNase was *minus;£30 (95% CI, −£2,091 to 1,576).

Conclusions: If decision makers are prepared to pay £200 for a 1% gain in FEV1 over a 12-week period, then on average either rhDNase strategy is cost-effective.

Objectives: To compare the cost-effectiveness of generic psychological therapy (counseling) with routinely prescribed antidepressant drugs in a naturalistic general practice setting for a follow-up period of 12 months.

Methods: Economic analysis alongside a randomized clinical trial with patient preference arm. Comparison of depression-related health service costs at 12 months. Cost-effectiveness analysis of bootstrapped trial data using net monetary benefits and acceptability curves.

Results: No significant difference between the mean observed costs of patients randomized to antidepressants or to counseling (£342 vs £302, p = .56 [t test]). If decision makers are not willing to pay more for additional benefits (value placed on extra patient with good outcome, denoted by K, is zero), then we find little difference between the treatment modalities in terms of cost-effectiveness. If decision makers do place value on additional benefit (K > £0), then the antidepressant group becomes more likely to be cost-effective. This likelihood is in excess of 90% where decision makers are prepared to pay an additional £2,000 or more per additional patient with a good global outcome. The mean values for incremental net monetary benefits (INMB) from antidepressants are substantial for higher values of K (INMB = £406 when K = £2,500).

Conclusions: For a small proportion of patients, the counseling intervention (as specified in this trial) is a dominant cost-effective strategy. For a larger proportion of patients, the antidepressant intervention (as specified in this trial) is the dominant cost-effective strategy. For the remaining group of patients, cost-effectiveness depends on the value of K. Since we cannot observe K, acceptability curves are a useful way to inform decision makers.

Objectives: Many hospitals in New Zealand have been using clinical priority assessment criteria (CPAC) to select and prioritize patients for access to publicly funded elective surgery. CPAC usually consist of clinical, patient-experienced, and social measures. The objective of this study was to determine how robust patient rankings were and the extent to which the patients selected were those who benefited the most from surgery.

Methods: Patients prioritized for cataract (n = 101), prostate (n = 103), and hip or knee joint replacement (n = 137) surgery according to CPAC were assessed using the EQ-5D, SF-12, and condition-related patient-experienced health status measures before and after treatment. Correlations between the rankings of patients on the CPACs and the alternative instruments were explored.

Results: For each surgery group, the CPAC ranking of patients was not strongly correlated with rankings obtained using their before-treatment EQ-5D (valued) profiles or the SF-12, although there was some correlation with rankings according to the condition-related measures. Improvements in the health status of patients who were operated on, as measured by the change in their EQ-5D values, were poorly correlated with equivalent changes on the SF-12 and condition-related measures. Patients' baseline health status according to the CPAC, the EQ-5D, and the SF-12 patient-experienced measures was only slightly related to the magnitude of benefit following surgery. The strongest predictors of improvement in health status were the baseline condition-related measures.

Conclusions: The current method of prioritizing patients in New Zealand requires reconsideration, although a gold standard method for prioritization is not immediately apparent from these results.

Objective: To evaluate the long-term impact of an intervention designed to reduce the ordering of three tumor markers frequently prescribed for gastroenterologic diseases (carcinoembryonic antigen, alpha-fetoprotein, carbohydrate antigen 19-9).

Methods: A prospective study with time series analysis in a teaching hospital. Local clinical guidelines were developed and implemented through a new order form, designed as a reminder to the physician, restricting the ordering of laboratory tests. Ratios between the number of markers ordered and number of admissions were recorded during a 3-month period before and after intervention in the whole hospital and monthly on a 4-year period in two wards of the hospital (Department of Gastroenterology and Department of Internal Medicine). To evaluate the appropriateness of tumor marker orders, audits were performed on a sample of order forms, before and after (1 month and 2 years after) the implementation of the new order form.

Results: The analysis of covariance showed a significant effect of the intervention in the hospital (p < .01), and in the Departments of Gastroenterology (p < .01) and Internal Medicine (p < .007). The decrease of tumor marker orders ranged from 25% (Internal Medicine Department) to 55% (whole hospital). A similar decrease was observed for the three studied markers. The appropriateness of prescriptions increased from 54.6% before to 73.6% after the implementation of the new order form, but decreased to 52.9% 2 years after intervention.

Conclusions: Providing a reminder to clinicians through a specific order form represents an inexpensive and easy way to implement guidelines on use of laboratory tests.

Objective: A United Kingdom Department of Health directive (EL[94]72) asked Health Authorities to manage the entry of new drugs into practice. There seem to be costs associated with the decision-making process of managed entry, but no clear evidence of benefit to patient populations. The objective of this study was to assess the potential costs and outcomes of different models of managed entry, using the example of donepezil in the North West Health Region of the U.K. National Health Service. This is a preliminary study designed to identify the key pieces of information required to evaluate the value of managed entry.

Methods: Decision analytic models of three Health Authorities' approaches to manage the entry of donepezil were used to estimate the expected costs and effectiveness of the process. Resource use data were obtained from published sources and the relevant Health Authority. Probabilistic sensitivity analysis was used to determine the robustness of the results.

Results: The process of managed entry of donepezil was associated with higher expected costs and higher expected outcome than no managed entry. The 95% confidence intervals for the net expected costs and net expected outcomes were relatively narrow and did not cross zero, which suggests a statistical difference between managed entry and no managed entry for donepezil. The incremental cost-effectiveness ratios for managed entry of donepezil indicate that, compared with no managed entry, there were substantial differences between the different models used in the three study sites. The expected cost per unit of cognitive function gained was between £18,000 in study site 001 to £28,000 in study site 010. The expected cost per person with a clinically significant improvement was between £140,000 and £230,000. The expected cost per QALY ranged from £470,000 to £19.3 million.

Conclusions: Managed entry does not appear to be a worthwhile mechanism to introduce drugs into practice. However, poor accessibility and availability of data means that the results are highly uncertain. The lack of data presents serious obstacles for both researchers and policy makers wishing to develop evidence-based policy and practice.

Objectives: Publication bias is widely appreciated, but considerable time and effort are needed to locate and obtain data from unpublished randomized controlled trials (RCTs), those published in non-English language journals or those reported in the gray literature; for this publication, we will call this collection of trials the “gray+ literature.” However, excluding such trials from systematic reviews could introduce bias and give rise to misleading conclusions.

Methods: We aimed to explore and quantify the impact of inclusion of gray+ literature on the results of all completed individual patient data (IPD) reviews coordinated by our group (13 meta-analyses). For each IPD review, results were calculated for RCTs fully published in English language journals and RCTs fully published in English language journals and the gray+ literature.

Results: The IPD meta-analyses based only on RCTs that were fully published in English language journals tended to give more favorable results than those that included RCTs from the gray+ literature. Although in most cases the addition of gray+ data gave less encouraging results, moving the estimated treatment effect toward a null result, the direction of effect was not always predictable.

Conclusions: We recommend that all systematic reviews should at least attempt to identify trials reported in the gray+thinsp;literature and, where possible, obtain data from them.

Objectives: To develop and validate a paper-based instrument that is simple to administer and produces a reliable estimate of patient standard gamble (SG) utilities for current health status.

Methods: A 1-page paper questionnaire instrument, paper standard gamble (PSG), was designed to estimate SG utilities. We performed two studies to assess the validity of PSG. First we compared PSG and SG utilities for current health in patients with prostate cancer. They randomly received either PSG followed by SG or vice versa, always with an intervening SF-12. In the second validity study, we assessed the test-retest reliability of PSG by administering it to prostate cancer patients twice, at least 2 weeks apart.

Results: In the first study, utilities were assessed in 64 men (32 per SG/PSG order group). A paired-comparison t test suggested no difference between SG and PSG (mean difference = −0.007; 95% confidence interval (CI), $-$0.022 to 0.008). The concordance correlation coefficient was 0.92 (95% CI, 0.79 to 0.99). In the second study, test and retest PSGs were available for 184 patients. The concordance correlation coefficient was 0.88 (95% CI, 0.73 to 0.94).

Conclusions: These data suggest that PSG may serve as a reliable substitute for SG when current health utility is of interest. PSG may have particular advantages for acquisition of health-related quality-of-life data in longitudinal studies.

Objectives: To identify predictors of high-quality clinical practice guidelines.

Methods: A total of 86 guidelines from 11 countries were assessed by four independent appraisers per guideline using the AGREE instrument (23 items). Six aspects of guideline development were considered to explain the variation in quality scores: care level (primary/secondary care), scope (diagnosis/treatment), type of guideline (new/update), year of publication, type of agency (governmental/professional), and whether the guideline was produced within a structured and coordinated program.

Results: Guidelines produced within a guideline program and by governmental agencies had higher scores than their counterparts. Differences in the applicability of the guidelines could not be explained by the variables studied.

Conclusions: To ensure high quality, clinical guidelines should be produced within a structured and coordinated program. Professional organizations or specialist societies that aim to develop guidelines may adopt quality criteria from leading guideline agencies.

Objective: We have used multidisciplinary expert panels to assess the health benefits from two different emergency medical service programs in Norway. This gave the opportunity to study the reliability of the expert panel method.

Methods: Two panels assessed case reports for 18 children, and two other panels assessed case reports for 64 adult patients. The assessments of each case report were compared. These assessments were also compared with assessments of the same case reports, done by the same panels 1 and 9 years earlier.

Results: Two different panels agreed on the benefit/no benefit conclusion in at least 75% of the patients, both for children and adult patients (kappa 0.88–0.50). For groups of patients assessed to have some health benefit, the magnitude of the benefit estimates differed by 25% between the panels. When the same panels assessed the same patient groups twice, 1 and 9 years apart, their estimates of total benefit differed up to 30%. However, estimates for single patients, as well as estimates from single panel members, varied considerably more.

Conclusions: Use of multidisciplinary expert panels is a useful method for estimating health benefits on program level or for groups of patients. But assessments from single panelists, and for single patients may be seriously biased.

Objective: To evaluate the sensitivity and precision of various extended search methods in identifying randomized controlled trials (RCTs) for systematic reviews.

Method: Prospective analysis of extended search methods (specialized databases or trial registries, reference lists, hand-searching, personal communication, and Internet) used in two systematic reviews of RCTs. The gold standard was the total number of RCTs identified by major databases (MEDLINE, EMBASE, etc.) and extended search strategies combined. Sensitivity was the proportion of all known RCTs identified by any extended search method. Precision reflected the proportion of all items uncovered by any extended search method that actually were RCTs.

Results: The extended search identified 94 additional RCTs for the systematic reviews beyond those identified with the major databases. Specialized databases and trial registries had the highest sensitivity and precision for the lipid-lowering project (13.6% and 52.7%, respectively; p < .05) followed by scanning of reference lists (7.2% sensitivity and 41.9% precision; p < .05). Hand-searching was more effective than personal communication and Internet searching (1.7% sensitivity and 12.2% precision; p < .05). The acupuncture project had slightly different results, with the specialized databases and trial registries tied with the review of reference lists for highest sensitivity (14.2%). The precision followed the same trend as the lipid-lowering project (17.6% specialized databases; 8.3% reference lists; p < .05). A post-hoc analysis showed that 75 of the 94 RCTs were indexed in the major databases but missed by the major database search.

Conclusions: Extended searching identified additional RCTs for the systematic reviews beyond those found in major databases. Specialized databases and trial registries were most effective. An important number of RCTs were missed by the major database search. Timing and accuracy of indexing may explain this finding. The definitive measure, whether there is an association between the method used to uncover RCTs, the quality of the items uncovered and their impact on systematic review results, is yet to be determined.

Objective: Health technology assessment (HTA) can be used both to promote access to safe, efficacious, and cost-effective technologies, and to discourage access to undesirable ones. Yet HTA has had less success than might be hoped in pursuing the latter goal. This paper examines the scope of HTA as currently practiced to contribute to regulation of access to undesirable technologies.

Design: The study design is a critical analysis of HTA's methods, based on an exposition of the normative issues involved in restriction of access to health technologies. The paper classifies technologies that might figure as potential candidates for exclusion into five categories and underscores the key social and ethical dilemmas associated with limiting their use.

Results: For four of the five categories of technology outlined, limitation of access necessarily involves denial of benefit. Limitation of access thus inevitably raises difficult normative issues. We show that these are ill-addressed by the range of “evidence” typically considered in technology assessments, which centers predominantly on clinical and technical features such as efficacy, safety, and costs.

Conclusions: If HTA is to enhance our ability to make reasonable decisions concerning the use and diffusion of health technologies, it must better integrate consideration of the social, political, and ethical dimensions of health technologies into the process of technology assessment. We suggest a framework within which to approach this goal.

Objectives: This preliminary study had two objectives: a) charting the considerations relevant to decisions about acquisition of new medical technology at the hospital level; and b) creating a basis for the development of a research tool that will examine the function of the Israeli health system in assessment of new medical technologies.

Methods: A comprehensive literature review and in-depth interviews with decision makers at different levels allowed formulation of criteria considered by decision makers when they decide to purchase and use (or disallow the use) of new medical technology. The resulting questionnaire was sent to medical center directors, along with a letter explaining the goals of the study. The questionnaire included 31 possible considerations for decision making concerning the acquisition of new medical technology by medical centers. The interviewees were asked to indicate the relevance of each consideration in the decision-making process.

Results: The most relevant criteria for the adoption of new technologies related to the need for a large capital investment, clinical efficacy of the technology as well as its influence on side effects and complication rates, and a formal approval by the Ministry of Health. Most interviewees stated that pressures exerted by the industry, by patients, or by senior physicians in the hospital are less relevant to decision making. Very small and usually not statistically significant differences in the ranking of hospital directors were found according to the hospitals' ownership, size, or location.

Conclusions: The present study is a basis for a future study that will map and describe the function of hospital decision makers within the area of new technology assessment and the decision-making process in the adoption of new healthcare technologies.

Objectives: To identify core issues that contribute to the gap between pre-marketing clinical research and practice as seen from the perspective of medical practice, as well as possible changes and potential barriers for closing this gap.

Methods: Interviews with 47 physicians and pharmacists who were liaised to drug regulation through their role in the pre- and post-marketing shaping of new cardiovascular drugs. Data were analyzed using methods of grounded theory and analytical evaluations.

Results: Six core issues were identified that referred to the standards in drug regulation, the organization of the regulatory system, and conflicting interests. Pre-marketing trials should focus more on populations and research questions relevant to medical practice. In particular, variability in drug responses between subgroups of patients and demonstration of effectiveness should become major principles in drug regulation. An interactive post-marketing process in which public interests are represented was considered necessary to further guide research and development according to the needs in daily practice. Strategies for change could be applied within the present system of drug regulation, or affect its basic principles. Regulatory authorities were primarily identified to initiate changes, but many other parties should be involved. Barriers for change were identified regarding differences in interests between parties, organizational matters, and with respect to broader healthcare policies.

Conclusions: Based on the respondents' opinions, there is a need to focus regulatory standards more on the needs in medical practice. Therefore, regulatory authorities should further develop their influence in the pre- and post-marketing drug development process, together with other parties involved, in order to bridge the gap between clinical research and medical practice.

Objective: The objective of this paper is to determine prevalence and characteristics of acute care for elders (ACE) units and hospital characteristics associated with the presence of an ACE unit.

Methods: Data on characteristics and prevalence of ACE units were obtained by surveying all established geriatric medical divisions across U.S. medical schools. Data on hospital characteristics such as number of beds, revenue, number of Medicare inpatients, and average length of stay were obtained from the 1999 American Hospital Association Annual Survey Data. Descriptive statistics and t test were used to analyze the characteristics of ACE units. Stepwise logistic regression was used to analyze the hospital characteristics associated with the presence of an ACE unit.

Results: The survey identified 16 geriatric divisions and programs with ACE units. Hospitals that have ACE units differ significantly with respect to number of beds and total revenue, compared with institutions that do not have an ACE unit. Stepwise logistic regression indicated total hospital revenue was the only factor significantly associated with the presence of an ACE unit.

Conclusions: ACE units are attractive interdisciplinary models to address the particular needs of the elderly during their hospital stay. Low presence of ACE units warrants further research as to reasons more hospitals have not included them, given the available evidence for clinical, functional, and economic benefits.