This issue of the International Journal of Technology Assessment in Health Care (IJTAHC) brings to the readers a lively discussion on the possibility of benchmarking health technology assessment (HTA) organizations. Michael Drummond and colleagues present a paper discussing “the conceptual and methodological challenges associated with benchmarking” (1). They propose a set of criteria for comparing the quality of organizations that produce assessments on health technologies. Aiming at explicit and transparent comparisons, Drummond et al. also highlight the possible problems in their approach, and present their paper as a starting point for discussion.

Pharmaceutical sales totaled an estimated USD 700 billion in OECD countries in 2011, approximately one-fifth of total healthcare spending (5). In the past, pharmaceutical expenditure has increased faster than total healthcare expenditure in many countries. The increase in pharmaceutical expenditure has slowed down in recent years due to the economic recession, patent expirations of important blockbusters, and cost-containment measures already taken. However, pharmaceutical expenditure still continues to rise, albeit at a reduced rate. Alongside an increased consumption of medicines, an important reason for growing costs is the adoption of new—and typically more expensive—products. To ensure that pharmaceutical expenditure does not become excessive and that patients have access to medicines they need, most countries now weigh up what products are to be included in the reimbursement system and at what price. Both the criteria to assess products and the processes to make reimbursement decisions vary from country to country and so far there have been no analytically oriented comparison studies.

In this theme section on early awareness and alert (EAA) systems and activities, we are pleased to present a series of articles covering the structure of EAA systems, their development and integration into decision making for the adoption and use of new and emerging health technologies, and the methods used by current networks, systems, and organizations.

The 27th year of the International Journal of Technology Assessment in Health Care (IJTAHC) introduced major changes in editorial work. Submission processes were modernized by introducing Web-based submission at http://mc.manuscriptcentral.com/thc in September 2010. The change has been a learning process to staff, authors, and reviewers alike. During the transition period, responses were often slower than usual, with occasional technical problems. Now that the change is completed, we expect to get back to normal reviewing times. From this issue onward, all manuscripts, including letters, are accepted through the Web site only.

Objectives: The aim of this study was to summarize and assess economic evaluations of poison centers (PCs) from the perspectives of society, the payer, and the healthcare system.

Methods: A systematic review was performed to identify complete economic evaluations regardless of the language or publication status. Two reviewers evaluated the abstracts for eligibility, extracted the data, and assessed the study quality using a standardized tool.

Results: In total, 422 non-duplicated studies were retrieved, but only nine met the eligibility criteria. Five of the eligible studies were published in the 1990s, and four were published in the 2000s. Six studies met at least seven of ten quality criteria. In all studies, the presence of PCs was compared with a scenario of their absence. Eight studies used cost–benefit analyses and one used a cost-effectiveness approach. The cost–benefit ratios ranged from 0.76 to 7.67, which indicates that each United States dollar (USD) spent on poison centers can save almost 8 USD on medical spending. A cost-effectiveness analysis showed that each successful outcome achieved by a PC avoids a minimum of 12,000 USD to 56,000 USD in other healthcare spending.

Conclusions: The data in our review show that PCs are economically viable. PCs improve the efficiency of healthcare expenditure and contribute to the sustainability of the healthcare system. An investment in PCs is a rational public health policy approach that contrasts the current trend of reducing spending on PCs.

Objectives: The aim of this study is to examine the cost-effectiveness of a group-based peer support intervention in general practice for patients with type 2 diabetes.

Methods: Incremental cost utility analysis combining within trial and beyond trial components to compare the lifetime costs and benefits of alternative strategies: Control: standardized diabetes care; Intervention: group-based peer support in addition to standardized diabetes care. Within trial analysis was based on a cluster randomized controlled trial of 395 patients with type 2 diabetes in the east of Ireland. Beyond trial analysis was conducted using the United Kingdom Prospective Diabetes Study (UKPDS) Outcomes Model. Uncertainty was explored using a range of sensitivity analyses and cost-effectiveness acceptability curves were generated.

Results: Compared with the control strategy, the intervention was associated with an increase of 0.09 (95 percent confidence interval [CI], −0.05 to 0.25) in mean quality-adjusted life-years per patient and savings of €637.43 (95 percent CI, −2455.19 to 1125.45) in mean healthcare cost per patient and €623.39 (95 percent CI, −2507.98 to 1298.49) in mean total cost per patient respectively. The likelihood of the intervention being cost-effective was appreciably higher than 80 percent for a range of potential willingness-to-pay cost-effectiveness thresholds.

Conclusions: Our results suggest that while a group-based peer support intervention shows a trend toward improved risk factor management, we found no significant differences in final cost or effectiveness endpoints between intervention and control. The probabilistic results suggest that the intervention was more cost-effective, with probability values of higher than 80 percent across a range of potential cost-effectiveness threshold values.

Objectives: The aim of our study is to compare five European drug reimbursement systems, describe similarities and differences, and obtain insight into their strengths and weaknesses and formulate policy recommendations.

Methods: We used the analytical Hutton Framework to assess in detail drug reimbursement systems in Austria, Belgium, France, the Netherlands, and Sweden. We investigated policy documents, explored literature, and conducted fifty-seven interviews with relevant stakeholders.

Results: All systems aim to balance three main objectives: system sustainability, equity and quality of care. System impact, however, is mainly assessed by drug expenditure. A national reimbursement agency evaluates reimbursement requests on a case-by-case basis. The minister has discretionary power to alter the reimbursement advice in Belgium, France, and the Netherlands. All systems make efforts to increase transparency in the decision-making process but none uses formal hierarchical reimbursement criteria nor applies a cost-effectiveness threshold value. Policies to deal with uncertainty vary: financial risk-sharing by price/volume contracts (France, Belgium) versus coverage with evidence development (Sweden, the Netherlands). Although case-by-case revisions are embedded in some systems for specific groups of drugs, systematic (group) revisions are limited.

Conclusions: As shared strengths, all systems have clear objectives reflected in reimbursement criteria and all are prepared to pay for drugs with sufficient added value. However, all systems could improve the transparency of the decision-making process; especially appraisal lacks transparency. Systems could increase the use of (systematic) revisions and could make better use of HTA (among others cost-effectiveness) to obtain value for money and ensure system sustainability.

Health systems are challenged continuously to provide the highest quality universal health care within their means. While for 30 years, health technology assessment (HTA) has contributed to the process of evidence-informed decision making and the managed entry of new technologies, its remit has not expanded to include assessment of technologies currently in use, as a means of managing their use and potentially their exit. We propose that health technology reassessment (HTR) become standard practice, an integral part of all health technology assessment agencies, and that we develop standardized models and methodologies for reassessment drawing from what we have learned from HTA.

Background: Health systems face rising patient expectations and economic pressures; decision makers seek to enhance efficiency to improve access to appropriate care. There is international interest in the role of HTA to support decisions to optimize use of established technologies, particularly in “disinvesting” from low-benefit uses.

Methods: This study summarizes main points from an HTAi Policy Forum meeting on this topic, drawing on presentations, discussions among attendees, and an advance background paper.

Results and Conclusions: Optimization involves assessment or re-assessment of a technology, a decision on optimal use, and decision implementation. This may occur within a routine process to improve safety and quality and create “headroom” for new technologies, or ad hoc in response to financial constraints. The term “disinvestment” is not always helpful in describing these processes. HTA contributes to optimization, but there is scope to increase its role in many systems. Stakeholders may have strong views on access to technology, and stakeholder involvement is essential. Optimization faces challenges including loss aversion and entitlement, stakeholder inertia and entrenchment, heterogeneity in patient outcomes, and the need to demonstrate convincingly absence of benefit. While basic HTA principles remain applicable, methodological developments are needed better to support optimization. These include mechanisms for candidate technology identification and prioritization, enhanced collection and analysis of routine data, and clinician engagement. To maximize value to decision makers, HTA should consider implementation strategies and barriers. Improving optimization processes calls for a coordinated approach, and actions are identified for system leaders, HTA and other health organizations, and industry.

Objectives: This study aims to estimate the annual U.S. societal costs associated with treatment of metastatic breast cancer (MBC) patients using an incidence-based cost-of-illness (COI) framework.

Methods: An incidence-based COI model was constructed in which MBC patients were simulated from diagnosis through active treatment, palliative care, and death over 5 years. Key model parameters included: annual incidence of breast cancer in the metastatic stage, utilization of cancer therapies and other medical care resources, treatment-related adverse events, unit costs, work days missed by patient and caregiver, and wage rates. Overall survival was based on SEER data and costs were assigned to living patients monthly, according to their disease management phase. The outcomes measures were total discounted societal costs, cost/year, and cost/patient-year.

Results: The annual incidence of MBC in the United States in 2007 was estimated to be 49,674 patients (de novo and progressed from earlier stages). The total discounted cost to society attributable to MBC was $12.2 billion for the incident cohort, or $98,571 per patient-year. The 5-year direct medical cost of this incident cohort was $9.3 billion, or $75,415 per patient-year. Treatment-related costs (active treatment, toxicity management, and medical follow-up) contributed 44 percent of MBC expenditure, followed by palliative/best supportive care costs (31 percent). Lost productivity accounted for approximately 21 percent of the total cost ($2.6 billion over 5 years or $21,153 per patient-year).

Conclusions: The societal burden of MBC in the United States is substantial. Earlier detection and effective treatment could lead to a significant decrease in costs while improving overall disease prognosis.

Objectives. In a democratic system, decision makers are accountable for the reasonableness of their decisions. This presumes (i) transparency, (ii) relevance of the decision criteria, (iii) revisability of decisions, and (iv) enforcement/regulation. We aim to (i) evaluate the extent to which drug reimbursement decision-making processes in different contexts meet these conditions and (ii) develop, starting from these findings, a framework for improving the transparency and the relevance of used decision criteria.

Methods. We evaluated the Austrian, Belgian, French, Dutch, and Swedish drug reimbursement systems. Based on this evaluation, we developed a framework for improving the transparency of drug reimbursement decision-making processes. It makes explicit the questions often addressed implicitly during decision-making processes as well as criteria for answering each question.

Results. Transparency of appraisal processes varies across systems. Justification with explicit criteria is generally limited. Although relevant criteria are similar across systems, their operationalization varies and their role in the appraisal process is not always clear. All systems seem to implicitly address five key questions, relating to (i) the medical, therapeutic, and societal need for treatment; (ii) preparedness to pay for treating the condition as a principle and (iii) for using the treatment under consideration; (iv) preparedness to pay more compared with alternatives; and (v) actual willingness to pay from public resources.

Conclusions. Transparency of the appraisal process can be improved by using an explicit decision framework. Systematic use of such a framework enhances consistency across decisions, allows justification of value judgments, and thus enhances legitimacy of societal decision making.

A decision analytic model often comprises a significant part of a health technology assessment. As health technology assessment in the hospital setting evolves, there is an increased need for modeling methods that account for patient care pathways and interactions between patients and their environment. For example, an evaluation of a computed tomography (CT) scanner for a new indication would need to consider the current and increased demand of the machine and how that may affect service in other areas of the hospital. This problem solving approach views “problems” through a systems perspective.

Objectives: The aim of this study was to determine how evidence from systematic review (SR) is perceived and negotiated by expert stakeholders in considering a technology for potential disinvestment.

Methods: An evidence-informed stakeholder engagement examined results from a diagnostic accuracy SR of vitamin B12 and folate tests. Pathologists deliberated around the SR findings to generate an informed contribution to future policy for the funding of B12 and folate tests. Deliberations were transcribed and subject to qualitative analysis.

Results: Pathologists did not engage with findings from the SR in depth; rather they sought to contest the terms of the problem driving the review and attempted to reframe it. Pathologists questioned the usefulness of SR outcomes given the variable definitions of B12 deficiency and deferred addressing disinvestment options specifically pertaining to B12 testing. However, folate testing was proffered as a potential disinvestment candidate, based upon pathologists' definition of “appropriate” evidence beyond the bounds of the SR.

Conclusions: The value of SR to informing disinvestment deliberations by expert stakeholders may be a function of timing as well as content. Engagement of stakeholders in co-produced evidence may be required at two levels: (i) Early in the synthesis phase to help shape the SR and harmonize expert views with the available evidence (including gaps); (ii) Collaboration in primary research to fill evidence-gaps thus supporting evidence-based disinvestment. Without this, information asymmetry between clinically engaged experts and decision makers may preclude the collaborative, informed, and technical discussions required to generate productive policy change.

Introduction: The supplementation of alanyl-glutamine dipeptide in critically ill patients necessitating total parenteral nutrition (TPN) improves clinical outcomes, reducing mortality, infection rate, and shortening intensive care unit (ICU) hospital lengths of stay (LOSs), as compared to standard TPN regimens.

Methods: A Discrete Event Simulation model that incorporates outcomes rates from 200 Italian ICUs for over 60,000 patients, alanyl-glutamine dipeptide efficacy data synthesized by means of a Bayesian random effects meta-analysis, and national cost data has been developed to evaluate the alternatives from the cost perspective of the hospital. Simulated clinical outcomes are death and infection rates in ICU, death rate in general ward, and hospital LOSs. Sensitivity analyses are performed by varying all uncertain parameter values in a plausible range.

Results: The internal validation process confirmed the accuracy of the model in replicating observed clinical data. Alanyl-glutamine dipeptide on average results more effective and less costly than standard TPN: reduced mortality rate (24.6% ± 1.6% vs. 34.5% ± 2.1%), infection rate (13.8% ± 2.9% vs. 18.8% ± 3.9%), and hospital LOS (24.9 ± 0.3 vs. 26.0 ± 0.3 days) come at a lower total cost per patient (23,409 ± 3,345 vs. 24,161 ± 3,523 Euro).Treatment cost is completely offset by savings on ICU and antibiotic costs. Sensitivity analyses confirmed the robustness of these results.

Conclusions: Alanyl-glutamine dipeptide is expected to improve clinical outcomes and to do so with a concurrent saving for the Italian hospital.

Objectives: The aim of this study was to evaluate the cost-utility of Docetaxel with doxorubicin and cyclophosphamide (TAC) and 5-fluorouracil, doxorubicin, cyclophosphamide (FAC) in node-positive breast cancer patients in the south of Iran.

Methods: A double blind study was done on a cohort of 100 patients suffering from breast cancer with node-positive over 8 months in the radiotherapy center of Namazi hospital, Shiraz-Iran. Health-related quality of life was assessed using questionnaire (QLQ-C30) from European Organization for Research and Treatment of Cancer (EORTC). QLQ-C30 scale scores were mapped to 15D and EuroQol 5D utilities to measure the quality-adjusted life-years (QALYs).Third party payer point of view was applied to measure and value the cost of treatments. Cost data were extracted from hospital and health insurance organizations. Robustness of the results was checked through a two way sensitivity analysis.

Results: TAC was associated with higher deterioration in HRQoL during treatment and higher improvements over 4 months follow-up. On average, the cost of treatment per patient in TAC was 15 times higher than FAC (p < .001). In overall, TAC was resulted in lower QALYs and higher cost over study period.

Conclusions: FAC was a dominant option versus TAC in short-term. The higher improvement in HRQoL over follow-up in TAC may not compensate the more intensive deterioration caused during treatment in short-term. The short time horizon of study may limit the generalizability of our findings and, hence, there is a need to conduct long-term economic evaluation studies whenever data is available to inform decision making.

Objectives: The aim of this study is to examine key informants’ perceptions of how the New Zealand Pharmaceutical Management Agency (PHARMAC) operates in New Zealand.

Methods: We carried out qualitative analysis of semi-structured interviews with key informants. We obtained ethics approval from the University of Otago School of Pharmacy, and all participants gave informed consent. We digitally recorded the interviews, which were then transcribed, and coded in NVivo. The data were analyzed by theme using constant comparison methods. Twenty informants who had previously published research or commentary on New Zealand's access to medicines, acted as spokespersons for interest groups, or held positions that gave them key insights into New Zealand's medicines system agreed to participate. Informants were purposefully selected to ensure a wide range of views, including five people working in medicine, four in pharmacy, three Members of Parliament from different parties, and two each from PHARMAC and the pharmaceutical industry.

Results: Respondents saw PHARMAC as an organization that contained medicine costs effectively, was politically neutral, and resistant to lobbying. It enjoyed broad political support and, with extremely rare exceptions, had been allowed to carry out its functions independently regardless of who was in government. As a result of this political stability, the relationship between PHARMAC and the pharmaceutical industry has been improving.

Conclusion: PHARMAC's longevity and increasing influence are largely due to political choices made to prioritize containing pharmaceutical expenditure and to respecting PHARMAC's independence. This may be difficult to replicate in other countries.

Background: Keeping clinical practice recommendations up-to-date with a continually evolving evidence base presents challenges. Resources required to update recommendations compete with those needed to evaluate newer treatments.

Methods: We describe an approach developed by the UK National Institute for Health and Clinical Excellence (NICE) for updating clinical practice recommendations for new interventional procedures and we evaluate relevant initial experience of using this system. Depending on whether evidence for a procedure is judged adequate or inadequate for safety and efficacy, use in clinical practice is usually recommended with either “normal” or “special” arrangements for patient consent, data collection and institutional oversight, respectively. We examined whether differences in the state of the evidence at the initial and the updated appraisal of procedures were associated with changed recommendations.

Results: Since 2008, updating of recommendations focuses on procedures with initially inadequate evidence. “Special arrangements” recommendations about eleven procedures were updated after 3.3–6.5 years (median, 5.3 years), and recommendations for six were changed to “normal arrangements.” Overall, procedures with changed (“special-to-normal”) recommendations had a greater increase in the number of patients included in observational studies published since the initial guidance.

Conclusions: Procedures with changed (“special-to-normal”) recommendations generally had greater increases in their evidence base. Although uncertainties about optimal methods for keeping evidence-based recommendations up-to-date remain, this experience should be useful to policy makers in developing processes for prioritizing scarce resources for updating clinical practice recommendations. Further studies are needed about the value placed on “updated” recommendations by clinicians, policy-makers, and patients.

There has been an increased focus on the relationship between health technology assessment (HTA) and regulatory assessments and how regulatory, HTA and coverage bodies, and industry can work better together to improve efficiency and alignment of processes. There is increasingly agreement across sectors that improved communication and coordination could contribute to facilitating timely patient access to effective, affordable treatments that offer value to the health system. Discussions on aspects of this relationship are being held in different forums and various forms of coordination and collaboration are being developed or piloted within several jurisdictions. It is therefore both timely and of value to stakeholders to describe and reflect on current initiatives intended to improve interactions between regulatory, HTA and coverage bodies, and industry. Drawing on 2011 meetings of the HTAi Policy Forum and the Center for Innovation in Regulatory Science (CIRS), this study aims to describe and compare initiatives, and point to success factors and challenges that are likely to inform future work and collaboration.

Objectives: The aim of this study is to describe and illustrate a method to obtain early estimates of the effectiveness of a new version of a medical device.

Methods: In the absence of empirical data, expert opinion may be elicited on the expected difference between the conventional and modified devices. Bayesian Mixed Treatment Comparison (MTC) meta-analysis can then be used to combine this expert opinion with existing trial data on earlier versions of the device. We illustrate this approach for a new four-pole implantable cardioverter defibrillator (ICD) compared with conventional ICDs, Class III anti-arrhythmic drugs, and conventional drug therapy for the prevention of sudden cardiac death in high risk patients. Existing RCTs were identified from a published systematic review, and we elicited opinion on the difference between four-pole and conventional ICDs from experts recruited at a cardiology conference.

Results: Twelve randomized controlled trials were identified. Seven experts provided valid probability distributions for the new ICDs compared with current devices. The MTC model resulted in estimated relative risks of mortality of 0.74 (0.60–0.89) (predictive relative risk [RR] = 0.77 [0.41–1.26]) and 0.83 (0.70–0.97) (predictive RR = 0.84 [0.55–1.22]) with the new ICD therapy compared to Class III anti-arrhythmic drug therapy and conventional drug therapy, respectively. These results showed negligible differences from the preliminary results for the existing ICDs.

Conclusions: The proposed method incorporating expert opinion to adjust for a modification made to an existing device may play a useful role in assisting decision makers to make early informed judgments on the effectiveness of frequently modified healthcare technologies.

Objectives: Obsolescence is a natural phase of the lifecycle of health technologies. Given increasing cost of health expenditures worldwide, health organizations have little choice but to engage in health technology reassessment (HTR); a structured, evidence-based assessment of the medical, social, ethical, and economic effects of a technology, currently used within the healthcare system, to inform optimal use of that technology in comparison to its alternatives. This research was completed to identify and summarize international HTR initiatives for non-drug technologies.

Methods: A systematic review was performed using the terms disinvestment, obsolescence, obsolete technology, ineffective, reassessment, reinvestment, reallocation, program budgeting, and marginal analysis to search PubMED, MEDLINE, EMBASE, and CINAHL until November 2011. Websites of organizations listed as members of INAHTA and HTAi were hand-searched for gray literature. Documents were excluded if they were unavailable in English, if the title/abstract was irrelevant to HTR, and/or if the document made no mention of current practices. All citations were screened in duplicate with disagreements resolved by consensus.

Results: Sixty full-text documents were reviewed and forty were included. One model for reassessment was identified; however, it has never been put into practice. Eight countries have some evidence of past or current work related to reassessment; seven have shown evidence of continued work in HTR. There is negligible focus on monitoring and implementation.

Conclusions: HTR is in its infancy. Although health technology reassessments are being conducted, there is no standardized approach. Future work should focus on developing and piloting a comprehensive methodology for completing HTR.