Objectives: The aim of this study was to assess if the use of Markov modeling (MM) or discrete event simulation (DES) for cost-effectiveness analysis (CEA) may alter healthcare resource allocation decisions.

Methods: A systematic literature search and review of empirical and non-empirical studies comparing MM and DES techniques used in the CEA of healthcare technologies was conducted.

Results: Twenty-two pertinent publications were identified. Two publications compared MM and DES models empirically, one presented a conceptual DES and MM, two described a DES consensus guideline, and seventeen drew comparisons between MM and DES through the authors’ experience. The primary advantages described for DES over MM were the ability to model queuing for limited resources, capture individual patient histories, accommodate complexity and uncertainty, represent time flexibly, model competing risks, and accommodate multiple events simultaneously. The disadvantages of DES over MM were the potential for model overspecification, increased data requirements, specialized expensive software, and increased model development, validation, and computational time.

Conclusions: Where individual patient history is an important driver of future events an individual patient simulation technique like DES may be preferred over MM. Where supply shortages, subsequent queuing, and diversion of patients through other pathways in the healthcare system are likely to be drivers of cost-effectiveness, DES modeling methods may provide decision makers with more accurate information on which to base resource allocation decisions. Where these are not major features of the cost-effectiveness question, MM remains an efficient, easily validated, parsimonious, and accurate method of determining the cost-effectiveness of new healthcare interventions.

Objectives: The aim of this study was to assist in the development of a health technology assessment (HTA) program for the Ministry of Health (MOH) of the Republic of Kazakhstan

Methods: Mentoring of an initial HTA program in Kazakhstan was provided by the Canadian Society for International Health (CSIH) by means of a partnership with the Kazakhstan MOH. HTA materials, courses, and one-on-one support for the preparation of a series of initial HTA reports by MOH HTA staff were provided by a seven-member CSIH team over a 2.5-year project.

Results: Guidance documents on HTA and institutional strengthening were prepared in response to an extensive set of deliverables developed by the MOH and the World Bank. Introductory and train-the-trainer workshops in HTA and economic evaluation were provided for MOH staff members, experts from Kazakhstan research institutes and physicians. Five short HTA reports were successfully developed by staff in the Ministry's HTA Unit with assistance from the CSIH team. Challenges that may be relevant to other emerging HTA programs included lack of familiarity with some essential underlying concepts, organization culture, and limited time for MOH staff to do HTA work.

Conclusions: The project helped to define the need for HTA and mentored MOH staff in taking the first steps to establish a program to support health policy decision making in Kazakhstan. This experience offers practical lessons for other emerging HTA programs, although these should be tailored to the specific context.

Objectives: The objective of this study was to explore the degree to which databases other than MEDLINE contribute studies relevant for inclusion in rapid health technology assessments (HTA).

Methods: We determined the extent to which the clinical, economic, and social studies included in twenty-one full and four rapid HTAs published by three Canadian HTA agencies from 2007 to 2012 were indexed in MEDLINE. Other electronic databases, including EMBASE, were then searched, in sequence, to assess whether or not they indexed studies not found in MEDLINE. Assessment topics ranged from purely clinical (e.g., drug-eluting stents) to those with broader social implications (e.g., spousal violence).

Results: MEDLINE contributed the majority of studies in all but two HTA reports, indexing a mean of 89.6 percent of clinical studies across all HTAs, and 88.3 percent of all clinical, economic, and social studies in twenty-four of twenty-five HTAs. While EMBASE contributed unique studies to twenty-two of twenty-five HTAs, three rapid HTAs did not include any EMBASE studies. In some instances, PsycINFO and CINAHL contributed as many, if not more, non-MEDLINE studies than EMBASE.

Conclusions: Our findings highlight the importance of assessing the topic-specific relative value of including EMBASE, or more specialized databases, in HTA search protocols. Although MEDLINE continues to be a key resource for HTAs, the time and resource limitations inherent in the production of rapid HTAs require that researchers carefully consider the value and limitations of other information sources to identify relevant studies.

Objectives: Reporting bias potentially threatens the validity of results in health technology assessment (HTA) reports. Our study aimed to explore policies and practices of HTA agencies regarding strategies to include previously unpublished data in their assessments, focusing on requests to industry for unpublished data.

Methods: We included international HTA agencies with publicly available methods papers as well as HTA reports. From the methods papers and recent reports we extracted information on requests to industry and on searches in trial registries, regulatory authority Web sites and for conference abstracts.

Results: Eighteen HTA agencies and seventy-three reports were included. Agencies’ methods papers showed variability regarding requests to industry (requests are routinely carried out in seven cases, not mentioned in six, at the discretion of HTA authors in three, and based on manufacturer applications in two), which were reflected in the reports investigated. As reporting of requests was limited, it often remained unclear whether unpublished data had been received. Searches in trial registries, at regulatory authorities or for conference abstracts are described as a routine or optional part of the search strategy in the methods papers of 9, 11, and 8 included agencies, respectively. A total of 52 percent, 39 percent, and 16 percent of reports described searches in trial registries, at regulatory agencies, and hand searching of conference proceedings.

Conclusion: International HTA agencies currently differ considerably in their efforts to address the issue of unpublished data. Requests to industry may constitute one strategy to access and include unpublished data, while agencies can learn from each other concerning successful practice.

Objectives: Procedures and new medical devices are typically introduced into healthcare systems with limited evidence, when they might be ineffective or unsafe. Systematic data collection (“registers”) can provide valuable “real world” evidence, but difficulties in funding registers are a major obstacle. A good economic case for the value of registers would therefore be useful.

Methods: (i) Literature search on specific purposes of registers. (ii) Surveys (a) of senior clinicians involved with registers, seeking examples of beneficial outcomes, and (b) of administrators, regarding costs of running registers. (iii) A scoping exercise for possible methods to value (financially) the outputs of registers.

Results: Four main categories of beneficial outcomes from registers were identified. These were—safety and quality assurance; training and quality improvement; complementing trial evidence and reducing uncertainty; and supporting trial research. Explicit examples of all these are presented, together with information about the costs of registers. Combining these with the scoping exercise we present suggestions for a methodology of assessing the value of registers across each of the categories.

Conclusions: This study is unique in addressing methods for determining the financial value of registers, based on the amount they cost versus the financial benefits which may result from the evidence generated. Developing the suggested methods could support the case for funding new registers, by showing that their use can benefit healthcare systems through more efficient use of resources, so justifying their costs.

Background: Ethics has been part of health technology assessment (HTA) from its beginning in the 1970s, and is currently part of HTA definitions. Several methods in ethics have been used in HTA. Some approaches have been developed especially for HTA, such as the Socratic approach, which has been used for a wide range of health technologies. The Socratic approach is used in several ways, and there is a need for harmonization to promote its usability and the transferability of its results. Accordingly, the objective of this study was to stimulate experts in ethics and HTA to revise the Socratic approach.

Methods: Based on the current literature and experiences in applying methods in ethics, a panel of ethics experts involved in HTA critically analyzed the limitations of the Socratic approach during a face-to-face workshop. On the basis of this analysis a revision of the Socratic approach was agreed on after deliberation in several rounds through e-mail correspondence.

Results: Several limitations with the Socratic approach are identified and addressed in the revised version which consists of a procedure of six steps, 7 main questions and thirty-three explanatory and guiding questions. The revised approach has a broader scope and provides more guidance than its predecessor. Methods for information retrieval have been elaborated.

Conclusion: The presented revision of the Socratic approach is the result of a joint effort of experts in the field of ethics and HTA. Consensus is reached in the expert panel on an approach that is considered to be more clear, comprehensive, and applicable for addressing ethical issues in HTA.

Objectives: In many economic evaluations and reimbursement decisions, quality-adjusted life-years (QALYs) are used as a measure of benefit to assess effectiveness of novel therapies, often based on the EQ-5D 3-level questionnaire. As only five dimensions of physical and mental well-being are reflected in this tool, significant aspects of the patient experience may be missed. We evaluate the use of the EQ-5D as a measurement of clinical change across a wide range of disorders from dermatological (acne) to life-threatening (metastatic cancers).

Methods: We analyze published studies on the psychometric properties of the EQ-5D 3-level questionnaire, extracting information on the Visual Analogue Scale versus Index score, Standardized Response Mean, and Effect Size. These are compared with ranges generally accepted to represent good responsiveness in the psychometric literature.

Results: We find that only approximately one in five study populations report subjective health state valuation of patients within 5 percent of the score attributed by the EQ-5D index, and more than 40 percent of studies report unacceptable ceiling effects. In the majority of studies, responsiveness of the EQ-5D index was found to be poor to moderate, based on Effect Size (63 percent poor–moderate) and Standardized Response Mean (72 percent poor–moderate).

Conclusions: We conclude that the EQ-5D index does not adequately reflect patient health status across a range of conditions, and it is likely that a significant proportion of the subjective patient experience is not accounted for by the index. This has implications for economic evaluations of novel drugs based on evidence generated with the EQ-5D.

Objectives: When incorporating treatment effect estimates derived from a random-effect meta-analysis it is tempting to use the confidence bounds to determine the potential range of treatment effect. However, prediction intervals reflect the potential effect of a technology rather than the more narrowly defined average treatment effect. Using a case study of robot-assisted radical prostatectomy, this study investigates the impact on a cost-utility analysis of using clinical effectiveness derived from random-effects meta-analyses presented as confidence bounds and prediction intervals, respectively.

Methods: To determine the cost-utility of robot-assisted prostatectomy, an economic model was developed. The clinical effectiveness of robot-assisted surgery compared with open and conventional laparoscopic surgery was estimated using meta-analysis of peer-reviewed publications. Assuming treatment effect would vary across studies due to both sampling variability and differences between surgical teams, random-effects meta-analysis was used to pool effect estimates.

Results: Using the confidence bounds approach the mean and median ICER was €24,193 and €26,731/QALY (95%CI: €13,752 to €68,861/QALY), respectively. The prediction interval approach produced an equivalent mean and median ICER of €26,920 and €26,643/QALY (95%CI: -€135,244 to €239,166/QALY), respectively. Using prediction intervals, there is a probability of 0.042 that robot-assisted surgery will result in a net reduction in QALYs.

Conclusions: Using prediction intervals rather than confidence bounds does not affect the point estimate of the treatment effect. In meta-analyses with significant heterogeneity, the use of prediction intervals will produce wider ranges of treatment effect, and hence result in greater uncertainty, but a better reflection of the effect of the technology.

Background: Increasingly, healthcare decision makers demand quality evidence in a short timeframe to support urgent and emergent decisions related to procurement, clinical practice, and policy. Health technology assessment (HTA) producers are responding by developing innovative approaches to evidence synthesis that can be executed more quickly than traditional systematic review. These approaches, and the broader implications they bring to bear on health decision making and policy development, however, are generally neither well-understood nor well-described. This study intends to contribute to an emerging literature around methodological approaches to rapid review in HTA by outlining those developed and implemented by the Canadian Agency for Drugs and Technologies in Health (CADTH).

Methods: Since 2005, CADTH has developed and implemented a rapid review approach that synthesizes evidence to support informed healthcare decisions and policy. Rapid Response reports are tailored to the identified needs of Canadian health decision makers, representing a range of options with regard to depth, breadth, and time-to-delivery.

Results: Preliminary observations indicate that CADTH's approach to rapid evidence review is generally well-received by Canadian health decision makers; real-world case studies provide pragmatic examples of how health decision makers have used Rapid Response reports to support evidence-informed health decisions across Canada.

Conclusions: Rapid review is becoming an increasingly important approach to evidence synthesis, both within and external to the field of HTA. Transparent reporting of the methods used to develop rapid review products will be critical to the assessment of their relevance, utility and effects in a range of contexts.

Objectives: The use of value of information methods to inform trial design has been widely advocated but there have been few empirical applications of these methods and there is little evidence they are widely used in decision making. This study considers the usefulness of value of information models in the context of a real clinical decision problem relating to alternative diagnostic strategies for patients with a recent non-ST elevated myocardial infarction.

Methods: A pretrial economic model is constructed to consider the cost-effectiveness of two competing strategies: coronary angiography alone or in conjunction with fractional flow reserve measurement. A closed-form solution to the expected benefits of information is used with optimal sample size estimated for a range of models reflecting increasingly realistic assumptions and alternative decision contexts.

Results: Fractional flow reserve measurement is expected to be cost-effective with an incremental cost-effectiveness ratio of GBP 1,621, however, there is considerable uncertainty in this estimate and consequently a large expected value to reducing this uncertainty via a trial. The recommended sample size is strongly affected by the reality of the assumptions of the expected value of information (EVI) model and the decision context.

Conclusions: Value of information models can provide a simple and flexible approach to clinical trial design and are more consistent with the constraints and objectives of the healthcare system than traditional frequentist approaches. However, the variation in sample size estimates demonstrates that it is essential that appropriate model parameters and decision contexts are used in their application.

Background: Health technology reassessment (HTR) is “a structured, evidence-based assessment of the clinical, social, ethical, and economic effects of a technology currently used in the healthcare system, to inform optimal use of that technology in comparison to its alternatives.” The purpose of this study is to describe the key themes in the context of current HTR activities and propose a way forward for this newly emerging field.

Methods: Data were gathered from a workshop held as part of the 2012 Canadian Agency for Drugs and Technology in Health (CADTH) symposium. The workshop consisted of two panel presentations followed by discussion; data gathered, including presentations and rich audience discussion transcripts, were analyzed for key themes emerging in the field of HTR using constant comparative analysis.

Results: The language chosen to describe HTR will set the tone for engagement. The identification of champions at multiple levels and political will are essential. Key lessons from international experience are: disinvestment is difficult, focus on clinical areas not specific technologies, identify clear goals of the HTR agenda. Six key themes were identified to move the HTR agenda forward: emphasize integration over segregation, focus on development of HTR methods and processes, processes are context-specific but lessons must be shared, build capacity in synergistic interdisciplinary fields, develop meaningful stakeholder engagement, strengthen postimplementation monitoring and evaluation.

Conclusions: To move this field forward, we must continue to build on international experiences with a focus on developing novel methodological approaches to generating, incorporating, and implementing evidence into policy and practice.

Objectives: The aim of this study was to develop and apply an instrument to map the level of health technology assessment (HTA) development at country level in selected countries. We examined middle-income countries (Argentina, Brazil, India, Indonesia, Malaysia, Mexico, and Russia) and countries well-known for their comprehensive HTA programs (Australia, Canada, and United Kingdom).

Methods: A review of relevant key documents regarding the HTA process was performed to develop the instrument which was then reviewed by selected HTAi members and revised. We identified and collected relevant information to map the level of HTA in the selected countries. This was supplemented by information from a structured survey among HTA experts in the selected countries (response rate: 65/385).

Results: Mapping of HTA in a country can be done by focusing on the level of institutionalization and the HTA process (identification, priority setting, assessment, appraisal, reporting, dissemination, and implementation in policy and practice). Although HTA is most advanced in industrialized countries, there is a growing community in middle-income countries that uses HTA. For example, Brazil is rapidly developing effective HTA programs. India and Russia are at the very beginning of introducing HTA. The other middle-income countries show intermediate levels of HTA development compared with the reference countries.

Conclusions: This study presents a set of indicators for documenting the current level and trends in HTA at country level. The findings can be used as a baseline measurement for future monitoring and evaluation. This will allow a variety of stakeholders to assess the development of HTA in their country, help inform strategies, and justify expenditure for HTA.