Objectives: The aim of this paper is to develop a methodological framework to facilitate the application of Multi-Criteria Decision Analysis (MCDA) for a comprehensive economic evaluation of disease management programs (DMPs).

Methods: We studied previously developed frameworks for the evaluation of DMPs and different methods of MCDA and we used practical field experience in the economic evaluation of DMPs and personal discussions with stakeholders in chronic care.

Results: The framework includes different objectives and criteria that are relevant for the evaluation of DMPs, indicators that can be used to measure how DMPs perform on these criteria, and distinguishes between the development and implementation phase of DMPs. The objectives of DMPs are categorised into a) changes in the process of care delivery, b) changes in patient lifestyle and self-management behaviour, c) changes in biomedical, physiological and clinical health outcomes, d) changes in health-related quality of life, and e) changes in final health outcomes. All relevant costs of DMPs are also included in the framework. Based on this framework we conducted a MCDA of a hypothetical DMP versus usual care.

Conclusions: We call for a comprehensive economic evaluation of DMPs that is not just based on a single criterion but takes into account multiple relevant criteria simultaneously. The framework we presented here is a step towards standardising such an evaluation.

Background: In recent years, there has been a surge in the development of frameworks to assess the value of different types of health technologies to inform healthcare resource allocation. The reasons for, and the potential of, these value frameworks were discussed during the 2017 Health Technology Assessment International (HTAi) Policy Forum Meeting.

Methods: This study reflects the discussion, drawing on presentations from invited experts and Policy Forum members, as well as a background paper.

Results: The reasons given for a proliferation of value frameworks included: rising healthcare costs; more complex health technology; perceived disconnect between price and value in some cases; changes in societal values; the need for inclusion of additional considerations, such as ethical issues; and greater empowerment of clinicians and patients in defining and using value frameworks. Many Policy Forum participants recommended learning from existing frameworks. Furthermore, there was a desire to agree on the core components of value frameworks, defining the additional value elements as necessary and considering how they might be measured and used in practice. Furthermore, adherence to the principles of transparency, predictability, broad stakeholder involvement, and accountability were widely supported, along with being forward looking, explicit, and consistent across decisions.

Conclusions: Value frameworks continue to evolve with significant implications for global incentives for innovation and access to health technologies. There is a role for the HTA community to address some of the key areas discussed during the meeting, such as defining the core components for assessing the value of a health technology.

Objectives: The aim of this article is to describe the diffusion of magnetic resonance imaging (MRI) in Iran, including regional variations during the period of 1990 to 2005 and international comparisons.

Methods: Data on the diffusion of MRI were obtained from the Medical Equipment Office of the Ministry of Health (MOH) and, using self-administered questionnaires, from forty-one universities specializing in medical sciences. Data were gathered from the year of first purchase up to mid-2005. Information for international comparisons was obtained from the Organization for Economic Cooperation and Development health data of 2006.

Results: Iran purchased its first MRI unit in 1990. Since then, the number of MRI units has increased remarkably. The diffusion curve of MRI in Iran follows an S-shaped curve with a very slow speed in the period of 1991–95. Accelerated adoption occurred later coinciding with a significant influence from the private sector, especially from 1999. Iran had ninety-three MRI units in 2005, and the number of MRI units per million in the population was 1.36.

Conclusions: The number of MRI units in provinces is not in direct proportion to the number of their inhabitants. Rational adoption and equitable diffusion of MRI may require the MOH and regulatory bodies to improve their ability in health technology assessment and integrate it into the policy making regarding adoption, diffusion, and utilization of health technologies.

Objectives: This study assesses the impact of the English National Health Service (NHS) Health Technology Assessment (HTA) program using the “payback” framework.

Methods: A survey of lead investigators of all research projects funded by the HTA program 1993–2003 supplemented by more detailed case studies of sixteen projects.

Results: Of 204 eligible projects, replies were received from 133 or 65 percent. The mean number of peer-reviewed publications per project was 2.9. Seventy-three percent of projects claimed to have had had an impact on policy and 42 percent on behavior. Technology Assessment Reports for the National Institute for Health and Clinical Excellence (NICE) had fewer than average publications but greater impact on policy. Half of all projects went on to secure further funding. The case studies confirmed the survey findings and indicated factors associated with impact.

Conclusions: The HTA program performed relatively well in terms of “payback.” Facilitating factors included the program's emphasis on topics that matter to the NHS, rigorous methods and the existence of “policy customers” such as NICE.

Objectives: To assess the relative cost-effectiveness of laparoscopic methods of inguinal hernia repair compared with open flat mesh and open non-mesh repair.

Methods: Data on the effectiveness of these alternatives came from three systematic reviews comparing: (i) laparoscopic methods with open flat mesh or non-mesh methods; (ii) open flat mesh with open non-mesh repair; and (iii) methods that used synthetic mesh to repair the hernia defect with those that did not. Data on costs were obtained from the authors of economic evaluations previously conducted alongside trials included in the reviews. A Markov model was used to model cost-effectiveness for a five-year period after the initial operation. The outcomes of the model were presented using a balance sheet approach and as cost per hernia recurrence avoided and cost per extra day at usual activities.

Results:Open flat mesh was the most cost-effective method of preventing recurrences. Laparoscopic repair provided a shorter period of convalescence and less long-term pain compared with open flat mesh but was more costly. The mean incremental cost per additional day back at usual activities compared with open flat mesh was €38 and €80 for totally extraperitoneal and transabdominal preperitoneal repair, respectively.

Conclusions:Laparoscopic repair is not cost-effective compared with open flat mesh repair in terms of cost per recurrence avoided. Decisions about the use of laparoscopic repair depend on whether the benefits (reduced pain and earlier return to usual activities) outweigh the extra costs and intraoperative risks. On the evidence presented here, these extra costs are unlikely to be offset by the short-term benefits of laparoscopic repair.

Objectives: To compare the cost-effectiveness of generic psychological therapy (counseling) with routinely prescribed antidepressant drugs in a naturalistic general practice setting for a follow-up period of 12 months.

Methods: Economic analysis alongside a randomized clinical trial with patient preference arm. Comparison of depression-related health service costs at 12 months. Cost-effectiveness analysis of bootstrapped trial data using net monetary benefits and acceptability curves.

Results: No significant difference between the mean observed costs of patients randomized to antidepressants or to counseling (£342 vs £302, p = .56 [t test]). If decision makers are not willing to pay more for additional benefits (value placed on extra patient with good outcome, denoted by K, is zero), then we find little difference between the treatment modalities in terms of cost-effectiveness. If decision makers do place value on additional benefit (K > £0), then the antidepressant group becomes more likely to be cost-effective. This likelihood is in excess of 90% where decision makers are prepared to pay an additional £2,000 or more per additional patient with a good global outcome. The mean values for incremental net monetary benefits (INMB) from antidepressants are substantial for higher values of K (INMB = £406 when K = £2,500).

Conclusions: For a small proportion of patients, the counseling intervention (as specified in this trial) is a dominant cost-effective strategy. For a larger proportion of patients, the antidepressant intervention (as specified in this trial) is the dominant cost-effective strategy. For the remaining group of patients, cost-effectiveness depends on the value of K. Since we cannot observe K, acceptability curves are a useful way to inform decision makers.

The objective of this study was to test whether the relatively new, noninvasive technique of extracorporeal shock-wave lithotripsy (ESWL) for renal stones resulted in a measurably better outcome from the patients' point of view than percutaneous surgery. The claimed superiority of ESWL was not demonstrated with the data available.

The availability and efficacy of recombinant deoxyribonucleic acid yeast-derived hepatitis-B vaccine, at a price much lower than the previously available plasma-derived hepatitis-B vaccines against hepatitis-B virus infections, motivate a new cost-benefit analysis of hepatitis-B vaccination. Spanish data were used to calculate direct and indirect costs of hepatitis-B infection and the costs and benefits of different vaccination strategies in defined risk groups of the Spanish population. A vaccination program will reduce direct expenditures for hepatitis B if the attack rate in the target population is higher than 4.9%. If indirect costs are included, the threshold for cost saving is reduced to 0.9%. The results are sensitive to the price of the vaccine, the duration of protection, assumptions about consequences for quality of life, and to indirect costs.

Objectives: Colloquial evidence (CE) has been described as the informal evidence that helps provide context to other forms of evidence in guidance development. Despite challenges around quality, and the potential biases, the use of CE is becoming increasingly important in assessments where scientific literature is sparse and to also capture the experience of all stakeholders in discussions, including that of experts and patients. We aimed to ascertain how CE was being used at the National Institute for Health and Care Excellence (NICE).

Methods: Relevant data corresponding to the use of CE was extracted from all NICE technical and process manuals by two reviewers and quality assured and analyzed by a third reviewer. This was considered in light of the results of a focused literature review and a combined checklist for quality assessment was developed.

Results: At NICE, CE is utilised across all guidance producing programmes and at all stages of development. CE could range from information from experts and patient/carers, grey literature (including evidence from websites and policy reports) and testimony from stakeholders through consultation. Six tools for critical appraisal of CE were available from the literature and a combined best practice checklist has been proposed.

Conclusions: As decisions often need to be made in areas where there is a lack of published scientific evidence, CE is employed. Therefore to ensure its appropriateness the development of a validated CE data quality check-list to assist decision makers is essential and further research in this area is a priority.

Objectives: Since 1988, four government-funded health technology assessment (HTA) agencies have been established in Canada. This paper is a descriptive review of reports issued by these organizations during the period from 1988 to 1998.

Methods: Publications from the national and three provincial HTA agencies in Canada were obtained and reviewed. Only the 117 assessment reports that were reported to have undergone external review were included in this analysis. Each report was classified on a standard abstraction form according to criteria such as technology type(s), assessment focus, whether a specific policy question was clearly stated and relevant decision maker(s) identified, description of search strategy and selection criteria, sources of data and assessment methods used, whether recommendations or conclusions were made, and duplication or overlap of reports. The trends in these qualities over the 10-year period were also examined.

Results: Therapeutic technologies have received the most attention from all four agencies, although the focus on devices, drugs, and procedures varied between agencies. The policy question under investigation was clearly identified in 82% of reports, and 71% clearly identified the decision maker toward whom the assessment was targeted. Efficacy or effectiveness was examined in 81% of reports, and costs were assessed in 65% of studies. These were the two most frequently examined aspects. Almost all assessments were descriptive literature reviews; 9% included meta-analyses and 32% had cost analyses or economic evaluations. Forty-four percent of reports had a clear description of the literature search strategy, and selection criteria were clearly specified in 38% of studies, but there was considerable variation among agencies in the level of description of these methods. Conclusions were clearly stated in 83% of the assessments' conclusions, and 13% had recommendations. When analyzed longitudinally, it is apparent that the quality of reports has improved markedly during the past decade. This was determined by examining the clarity of specifying the policy question(s) under investigation, the identification of the target audience of decision makers for the information, and by evaluating the thoroughness of the description of the methods used in the assessment.

Conclusions: Canadian government agencies have contributed a considerable quantity of health technology assessments. There has been very little duplication of technologies evaluated, and the quality of the assessment reports has markedly improved during the past decade.

Background: Endoscopic retrograde cholangiopancreatography (ERCP) is considered the gold standard for imaging of the biliary tract but is associated with complications. Less invasive imaging techniques, such as magnetic resonance cholangiopancreatography (MRCP), have a much lower complication rate. The accuracy of MRCP is comparable to that of ERCP, and MRCP may be more effective and cost-effective, particularly in cases for which the suspected prevalence of disease is low and further intervention can be avoided. A model was constructed to compare the effectiveness and cost-effectiveness of MRCP and ERCP in patients with a previous history of cholecystectomy, presenting with abdominal pain and/or abnormal liver function tests.

Methods: Diagnostic accuracy estimates came from a systematic review of MRCP. A decision analytic model was constructed to represent the diagnostic and treatment pathway of this patient group. The model compared the following two diagnostic strategies: (i) MRCP followed with ERCP if positive, and then management based on ERCP; and (ii) ERCP only. Deterministic and probabilistic analyses were used to assess the likelihood of MRCP being cost-effective. Sensitivity analyses examined the impact of prior probabilities of common bile duct stones (CBDS) and test performance characteristics. The outcomes considered were costs, quality-adjusted life years (QALYs), and cost per additional QALY.

Results: The deterministic analysis indicated that MRCP was dominant over ERCP. At prior probabilities of CBDS, less than 60 percent MRCP was the less costly initial diagnostic test; above this threshold, ERCP was less costly. Similarly, at probabilities of CBDS less than 68 percent, MRCP was also the more effective strategy (generated more QALYs). Above this threshold, ERCP became the more effective strategy. Probabilistic sensitivity analyses indicated that, in this patient group for which there is a low to moderate probability of CBDS, there was a 59 percent likelihood that MRCP was cost-saving, an 83 percent chance that MRCP was more effective with a higher quality adjusted survival, and an 83 percent chance that MRCP had a cost-effectiveness ratio more favorable than $50,000 per QALY gained.

Conclusions: Costs and cost-effectiveness are dependent upon the prior probability of CBDS. However, probabilistic analysis indicated that, with a high degree of certainty, MRCP was the more effective and cost-effective initial test in postcholecystectomy patients with a low to moderate probability of CBDS.

Background: Most industrialized countries, including France, have undertaken to generalize colorectal cancer screening using guaiac fecal occult blood tests (G-FOBT). However, recent researches demonstrate that immunochemical fecal occult blood tests (I-FOBT) are more effective than G-FOBT. Moreover, new generation I-FOBT benefits from a quantitative reading technique allowing the positivity threshold to be chosen, hence offering the best balance between effectiveness and cost. We aimed at comparing the cost and the clinical performance of one round of screening using I-FOBT at different positivity thresholds to those obtained with G-FOBT to determine the optimal cut-off for I-FOBT.

Methods: Data were derived from an experiment conducted from June 2004 to December 2005 in Calvados (France) where 20,322 inhabitants aged 50–74 years performed both I-FOBT and G-FOBT. Clinical performance was assessed by the number of advanced tumors screened, including large adenomas and cancers. Costs were assessed by the French Social Security Board and included only direct costs.

Results: Screening using I-FOBT resulted in better health outcomes and lower costs than screening using G-FOBT for thresholds comprised between 75 and 93 ng/ml. I-FOBT at 55 ng/ml also offers a satisfactory alternative to G-FOBT, because it is 1.8-fold more effective than G-FOBT, without increasing the number of unnecessary colonoscopies, and at an extra cost of €2,519 per advanced tumor screened.

Conclusions: The use of an automated I-FOBT at 75 ng/ml would guarantee more efficient screening than currently used G-FOBT. Health authorities in industrialized countries should consider the replacement of G-FOBT by an automated I-FOBT test in the near future.

Objectives: Immediate post-mastectomy breast reconstruction (IBR) is a procedure that has proven advantages, but it also entails risks. The aim of this study was to identify risk factors for reconstruction failure.

Methods: A review was made of all the IBR carried out at a general hospital from 2002 to 2009. Retrospective information was obtained about postoperative complications and the characteristics of patients and treatments applied. The minimum follow-up period was 9 months. Cox's regression analysis was performed on the variables related to the reconstruction failure requiring the removal of the prosthesis, with an explanatory model in which all the study variables were introduced and a predictive model that contained only the variables known before the intervention.

Results: A total of 115 IRB interventions carried out on 112 women with breast cancer were analyzed. The mean follow-up period was 25.5 months. In sixty cases (52.2 percent), there were no complications; in sixteen cases (13.9 percent) minor complications appeared, and in 39 (33.9 percent) the complications were moderate. In twenty-six cases (22.6 percent), a reconstruction failure occurred. Cox's regression model revealed that the reconstruction failures were related to the patient's age (Hazard Ratio 1.08), to neoadjuvant chemotherapy (HR 6.24) and to postoperative tamoxifen (HR 3.10). The predictive model included the age of the patient (HR 1.05) and the use of neoadjuvant chemotherapy (HR 5.11).

Conclusions: A significant proportion of the patients receiving IBR developed reconstruction failure. Multivariate analysis identified three variables related to this complication, two of which were known before the intervention.

Objectives: The Turkish healthcare system is currently undergoing reform, and efficient use of resources has become a key factor in determining the allocation of resources. The objective of this study was to analyze strengths, weaknesses, opportunities, and threats (SWOT) in the development of a health technology assessment (HTA) program in Turkey.

Methods: A SWOT analysis was performed using a literature review and interviews with key people in the Turkish Ministry of Health and Ministry of Labor and Social Security.

Results: Regarding recent reforms in health care, investments for information network and databank are the strengths, but the traditional “expert-based” decision making, poor availability of data, and poor quality of data could be seen as some of the weaknesses. Another major weakness is lack of general awareness of HTA. Increasing demand for transparency in decision making, demand for evidence, and demand for credibility by decision makers are some of the opportunities, and current healthcare reforms, i.e., restructuring of healthcare and general health insurance, could also be seen as major opportunities. These opportunities unfortunately could be threatened by lack of funding, and resources are challenged by large, recent national investments.

Conclusions: There is a good opportunity for Turkey to use the skills in HTA currently being developed through activities in Europe and the Americas to assist in the development of a much more cost-effective and transparent healthcare system in Turkey.

Objectives: Canada has witnessed expansion of the health technology assessment (HTA) infrastructure in the last 25 years. Local HTA entities at the hospital or regional level are emerging to assist decision makers in the acquisition, implementation, maintenance, and disinvestment of healthcare technologies. There is a need to facilitate collaboration and exchange of expertise and knowledge between these entities regarding the role of local HTA in Canada.

Methods: In November 2013, the pan-Canadian Collaborative hosted a symposium, Hospital/Regional HTA: Local Evidence-based Decisions for Health Care Sustainability, bringing together over 60 HTA producers, researchers, stakeholders, and manufacturers involved in local HTA across Canada. The objective was to showcase the diversity of local HTA in Canada, while highlighting common gaps to be addressed.

Results: The Symposium focused on current practices in local HTA in Canada to support informed decision making, and opportunities for information sharing and provide equal access to timely evidence-based information to decision makers. The main themes included assessment of evidence for local HTA, contextualization, stakeholder engagement in local HTA, knowledge translation and impact of recommendations, and challenges and opportunities for local HTA.

Conclusions: Local HTA in Canada complements HTAs conducted at the provincial and federal levels to improve the efficient and effective health service delivery in institutions or regions faced with limited resources. Some challenges faced by local HTA producers to influence hospital policies and clinical practice involve the engagement of healthcare professionals and potential lack of training and support necessary for the introduction of a new technology.

Objectives: There is relatively little evidence on the test–retest reliability of utility scores derived from multiattribute measures. The objective was to estimate test–retest reliability for Health Utilities Index Mark 2 (HUI2) and Mark 3 (HUI3) utility scores in patients recovering from hip fracture.

Methods: We enrolled an inception cohort of hip fracture patients within 3 to 5 days of surgery. Baseline assessments included the Functional Independence Measure (FIM™), Folstein Mini-Mental State Examinations, and the HUI2 and HUI3 questionnaire. Follow-up assessments at 1, 3, and 6 months also included a global change question. Test–retest reliability was assessed as agreement between 3- and 6-month scores using the intraclass correlation coefficient (ICC). Two approaches were used to classify patients as stable; a third approach based on the generalizability theory was also used. Patients were classified as stable if their FIM™ overall scores changed by 10 points or fewer and if they classified themselves as having experienced no or only a little change according to their global change question.

Results: Complete data at both the 3- and 6-month assessments based on self-report were available for 196 patients; 141 patients with complete data were classified as stable. The ICCs for HUI2 and HUI3 for stable patients were 0.71 and 0.72; the ICCs derived from the generalizability theory were 0.76 and 0.77.

Conclusions: Test–retest reliability for HUI in this cohort was similar to reliability estimates for other preference-based multiattribute and generic health-profile measures—in the acceptable range for making valid group-level comparisons.

Objectives: Even when policy makers show interest and evidence-informed and convincing HTA studies are available, use of assessment products is not guaranteed. In this article, we report our experience with knowledge brokering to foster evidence-informed policy making on cost-effective treatment and reimbursement of assisted reproduction in The Netherlands.

Methods: From earlier work in the field of knowledge brokering, we foresaw the need for a deliberative strategy to manage the inherent tension between scientific rigor demanded by researchers and responsiveness to real-time needs demanded by policy makers. Therefore, we structured the process in three distinct steps: (i) agreement about the main messages from the research, (ii) analysis of the policy context and of the meaning of the main messages for the actors involved, and (iii) an invitational meeting to make recommendations for action.

Results: One of the recommendations that would require changes in ministerial policy was followed up instantly, whereas the other recommendation is still under debate. The Dutch Society of Obstetrics and Gynecology activated the revision of two guidelines. The patient organization uses the new scientific insights in informing members and the public. Closing the loop, The Netherlands Organisation for Health Research and Development (ZonMw) funded research to close knowledge gaps that became apparent in the process.

Conclusions: Knowledge brokering is a promising approach to bring HTA into practice. We conclude that the methodologies to feed research results into the policy process are still in an incipient stage and need further development.

Objectives: Promoting the adoption of clinical practice guidelines is a challenging task. Research shows that adopters' beliefs about guidelines and the process of their development are important predictors of guideline use. Our objective is to report on the development and refinement of an instrument designed to measure oncologists' assessments of guidelines in oncology.

Method: A theoretically derived instrument was drafted. Oncologists were asked to complete a questionnaire for each draft guideline they were asked to review. Factor analyses and multilevel modeling techniques were undertaken to explore the properties of the instrument and refine it.

Results: A total of 488 Ontario clinicians were sent 1,494 new questionnaires regarding 34 clinical practice guidelines produced between 1999 and 2002. A refined eighteen-item questionnaire with four stable factors that predicted 60 percent of the variance emerged. The factors are interpreted as guideline quality, applicability, acceptability, and comparative value. The four factors predicted oncologists' endorsements of draft guidelines and, with the exception of quality, predicted their intentions to use the guidelines. As expected, variation in the factor scores could be attributed more to the differences among the oncologist who completed the survey than to the differences among the guidelines themselves.

Conclusions:An instrument composed of four stable and theoretically relevant factors emerged. The findings support the hypothesis that beliefs about guideline attributes and development attributes relate to oncologists' endorsement of and intentions to use guidelines. Our next step is to link the responses of the survey with actual clinical behavior.

Objectives: This study describes the diffusion of advanced diagnostic devices in India and assess implications for efficiency in resource use and equity.

Methods: Commodity-level import statistics, household survey data, and interviews with medical device sellers are used to assess the spread of diagnostic devices. Published qualitative evidence, case studies of diagnostic service providers, and cross-country analyses are used to identify the reasons underlying the spread of medical devices in India. Case studies of public and private providers and data from 150 hospitals in one Indian state are used to assess efficiency in resource use and the distributive impacts of diagnostic devices.

Results: High-end medical device inflows rose during the 1990s, with both supply- and demand-side factors influencing this trend. Although our results suggest that the overall quantity of advanced diagnostics in India is not excessive, there is some evidence of inefficiency in public facilities and possibly unethical practices in private diagnostic facilities. The unequal geographical distribution of magnetic resonance imaging facilities, coupled with inefficient use of medical devices in public facilities suggests inequality in access.

Conclusions: The study points to major regulatory gaps and health system inefficiencies and suggests ways in which these gaps can be addressed.