Health Technology Wales (HTW) is a relatively new Health Technology Assessment (HTA) agency which focuses on non-medicines. In common with other HTA organizations, it identifies and appraises a range of technologies. However, HTW is also looking beyond the publication of guidance, to assess the adoption of advice and its eventual impact.

HTW commissioned development of an Evaluation Plan from independent experts (Matter of Focus). A literature review was carried out to inform an options appraisal of methods for assessing impact. The selected approach was Contribution Analysis, which estimates the counterfactual through engagement of stakeholders.

Whilst it is too early to report the full impact of HTW's guidance, a number of activities have taken place to prepare for evaluation. The core HTW team developed a series of logic models to describe the anticipated impact, the mechanisms by which it would be achieved, and key assumptions. Stakeholders were consulted for insight from a range of perspectives, and to manage expectations. This was achieved through individual interviews, presentation and discussion at committee meetings, and the sharing of written materials for feedback. This information was collated to populate bespoke software (OutNav). The collection of data relating to processes, outputs and outcomes is already an ongoing routine task of researchers and support staff.

HTW has an opportunity to build impact evaluation into its culture from the beginning. This will facilitate the future reporting of HTW's influence using a well-designed, evidence-based approach. Furthermore, this pioneering work will clearly demonstrate the value of HTA to funders, commissioners, governments, and other decision-making bodies.

The re-organization of the supply chain (SC) of medicines and medical devices may improve the efficacy and efficiency of the National Health Service (NHS). The aims of this study were to (i) identify the offers provided by private operators to NHS, and (ii) analyze the organizational model of the public healthcare SC system and its criticalities.

Two online surveys have been designed. Regarding the first survey, managers of private providers associated with the National Association of Commercial and Logistic Operators (ASSORAM) have been interviewed to identify the offers provided to the NHS. The second has been submitted to managers of local health authorities and university hospitals associated to the Italian Association of Hospitals (FIASO) to gather both organizational/managerial information (warehouse capacity, purchasing, registry, security) and qualitative aspects of the SC. Data was collected in 2015.

On the supply side, 41 providers have been interviewed. More than 70 percent of associates managed mainly hospital products; 67 percent of interviewees delivered less than 30 percent of products to hospitals, and only eight percent delivered about 70 percent of the products to hospitals. The providers’ infrastructure (warehouses, transport, information technology, cold chain, gross domestic product) were adequately regulated and they adopted a wide list of indicators for monitoring performance. Private providers showed high interest in investing in the hospital sector. On the demand side (56 hospitals from 28 regions) the main weaknesses of SC are related to infrastructure, information technology, human resources, a lack of financial resources and inadequate process control.

The study highlighted extremely limited outsourcing in the hospital field to date, weaknesses in the public system and a high interest of private providers in investing in public hospital SC.

Since 2005, new treatment and diagnostic methods (NUBs) were reimbursed by individual supplementary fees. The assessment procedure for NUBs is induced by hospitals submitting a request for additional compensation of healthcare treatment to the Institute for the Fee System in the Hospital (InEK). In 2016, the legal norm §137h SGB V was introduced to evaluate medical devices (MD) of high risk classes by the Federal Joint Committee (G-BA). InEK grants a status that is valid for twelve months and impacts additional compensation as well as assessment required by G-BA. The effects of this rating seem to differ between hospitals and Statutory Health Insurance (SHI).

The published InEK decisions on NUBs were analyzed according the decision criteria and possible impact on price negotiations with SHI.

In 2018, 705 NUB requests were assessed by InEK. NUB Status 1, granting negotiation of additional coverage, was assigned to 171 procedures. Status 2 – no additional reimbursement possible - was given in 472 cases, the remaining had not sufficient information. Most NUBs (n = 368) requests did not fall under §137h; however, those with sub-Status “B” (allocated to 12) led to controversies; no participant had requested an evaluation according to §137h for the NUB. Two consultation requests receiving Status 1 B were regarded as not eligible according to § 137h by the G-BA. To avoid price negotiation delays, early consultations according to § 137h are recommended by G-BA during the NUB application.

The NUB process enables hospitals to receive a supplemental payment when using innovative technologies not listed in the existing German healthcare system. The question of which requirements must be fulfilled to guarantee the reimbursement should be asked at an early stage. Consultation requests to the G-BA in due time are strongly recommended. Contact between manufacturers and hospitals are advisable to support the NUB application.

Due to an aging population, shortage of healthcare staff, and escalating healthcare costs, there has been a recent shift in the professional roles and responsibilities in acute care settings to help bridge the care gap. Paramedics, whose primary responsibilities have been in emergency/transportation services, are increasingly involved in the management of chronic diseases in the community setting. However, even with additional training, there are concerns about the safety and effectiveness of this expanded role. The objective of this presentation is to highlight some of the key findings from a health technology assessment report on the safety and effectiveness of community paramedicine in assessing and managing conditions/diseases with low acuity.

A systematic review was conducted to identify studies that evaluated the safety and effectiveness of different community paramedicine programs.

Four systematic reviews and 20 primary studies (one randomized controlled trial (RCT) and 19 observational studies) were identified. Of these, two systematic reviews and 14 primary studies focused on the safety and effectiveness of Emergency Care Practitioner (ECP) programs ̶ widely implemented programs whereby a paramedic or nurse undertakes activities traditionally performed by physicians, such as the initial assessment of patients, provision of simple treatment, or referral of patients to other clinical care. Limited evidence showed that ECP programs are promising in reducing repeated emergency calls, emergency department visits, hospital admissions/readmissions, and emergency transport charges. While the majority of included studies did not report any safety outcomes, no significant safety issues were identified from the cluster RCT. Evidence for other types of community paramedicine is limited.

Evaluation of the impact of community paramedicine programs remains methodologically challenging. Additional cluster RCTs may help determine the effectiveness of community paramedicine programs; safety outcomes should be a key element of future observational studies.

Current clinical practice is based on guidelines and local protocols that are informed by clinical evidence. This means that clinical variability is reduced, but can lead to inefficient clinical decision-making, and can increase medical errors, decreasing patient's safety. The aim of the EXCON project is to investigate the innovative concept of Intelligent Clinical History (ICH), and to develop functional prototypes of high added-value in healthcare services.

The innovative EXCON project will take advantage of recent advances in technologies for coding, structuring and semantizing medical information. Thanks to this new structuring, the EXCON platform will be developed. Final users will be health professionals and other decision-makers. Doctors, nurses, epidemiologists and information specialists will be involved in the development and subsequent validation of the platforms.

To develop the ICH platform clinical data on a highly prevalent symptom with high variability in clinical practice, such as non-traumatic chest pain in emergency services, has been collected from different electronic medical record databases. The extraction of clinical data to implement new techniques of artificial intelligence requires tasks that must be automated, which today is difficult and tedious (data is often not computerized). Through techniques applied in EXCON, such as natural language processing, relevant clinical data have been extracted and a Decision Support System has been developed and validated. This tool optimizes resources and improves clinical management, reducing errors and increasing patient's safety.

In coming decades, patient management will be impacted by the application of new advanced data analytics tools. This will allow for safer and more efficient clinical management, decrease variability in clinical practice, and improve equity. That is why the development and assessment of these technologies is necessary.

In Hungary, the procedure for health technology assessment of innovative pharmaceutical products allows 13 assessors 43 calendar days to evaluate reimbursement submissions. These short timelines have created a need for smart capacity building, namely, streamlining the scientific evaluation process while making sure that the quality of the critical appraisals remain high. The objective of this study was to present and evaluate the implementation of an online knowledge base to distill community knowledge, and also for management purposes.

The scope and the content-, functional-, and technical specification was developed, and information technology security requirements were identified during the pre-implementation phase. An existing platform was chosen for adaptation, ensuring that descriptive follow-up data is available on uptake for monitoring purposes. Both the adaptation and maintenance were carried out internally by the Department of Health Technology Assessment at the National Institute of Pharmacy and Nutrition.

The key requirements identified when developing the specification were searchability, low maintenance need, low operating costs and attractivity for users. An already existing open-source, flat file content management system was chosen for adaptation. In terms of content, a health technology assessment handbook, process documentation, a news bulletin section was created, and corporate identity elements were added. Since the start of the service in September 2018, the number of total daily page downloads to the knowledge base varied between four and 1,193 (average 205 per day), with the assessment handbook topping the overall page visit statistics.

The implementation of this knowledge base enables the Department of Technology Assessment to rely more on the formalized community knowledge when carrying out critical appraisal, while enabling better knowledge and quality management. Uptake remains an issue on the long run, indicating a need for continuous content development.

eHealth is a new approach for managing several health conditions, but up to now not so many interventions have shown their efficacy/effectiveness. The AUTAPP Project tries to add knowledge in eHealth interventions targeted to Mental Health disorders, specifically Autism Spectrum Disorder (ASD) management that requires complex interventions that integrate different psychosocial interventions. AUTAPP aims to develop an evidence based Clinical Decision Support System (CDSS) using mobile technology for improving the decision process on psychosocial therapies in ASD. This study aimed to identify recommendations on which the algorithm of the CDSS will be developed.

A systematic review (November 2009-November 2018) was carried out to identify the existing scientific evidence published in relation to the effectiveness of: (i) early detection protocols; (ii) assessment tools; (iii) existing non-pharmacological therapies. Main databases were consulted (PubMed, Cochrane Library, PsychoInfo). Articles were reviewed by two independent reviewers. The quality of included publications and recommendations were assessed according to SIGN criteria.

A total number of 147 publications were included (477 identified): 96 for non-pharmacological therapies, 33 for assessment tools and eighteen for early detection. Regarding early detection and assessment, 12 recommendations were identified and six obtained the highest level (A), such as the convenience of multidisciplinary diagnosis teams and the usefulness of the Modified Checklist for Autism in Toddlers (M-CHAT) for ASD confirmation. For non-pharmacological therapies, 16 recommendations were collected. Those with higher levels of recommendations were family, environmental and educational (three As and one B). Interventions with lower levels of recommendation (C) were interventions which exercise, computers and neurological approaches.

This systematic review allows both to identify gaps and opportunities in psychosocial interventions research and be the base for the CDSS algorithm. In the future professionals, careers and people diagnosed with ASD will validate the mobile CDSS.

We estimate the life-cycle value of risperidone – Second-Generation Antipsychotics (SGA) – to balance the view that cost per Quality-Adjusted Life Year (QALY) estimates at launch are enough to guide access decisions. Study results will also drive discussion on access and price to recognize the dynamic nature of pharmaceutical pricing over the long-run.

We estimated number of patients treated for schizophrenia with risperidone in Sweden and the United Kingdom (UK) between 1994-2017 based on usage data form national statistics and volume sales data from IQVIA. We collected data from literature on the effectiveness (QALYs) and costs (EUR 2017) of risperidone (SGA) and haloperidol – First-Generation antipsychotic (FGA). We estimate the life-cycle value added by risperidone versus haloperidol, and the life-cycle distribution of the social surplus between the payer (consumer surplus) and the innovator (producer surplus).

We estimated the consumer surplus, the producer surplus, the Net Monetary Benefit (NMB) and Incremental Cost-Effectiveness Ratio (ICER) at each year and in aggregate terms (1993-2017). For the UK the producer surplus was ~28 percent out of the total surplus before patent expiration and five percent after patent expiration. In Sweden, producer surplus was around 6 percent out of the total surplus before patent expiration and one percent thereafter. In both countries, during the life-cycle of risperidone, the NMB per patient increased and the ICER decreased as a response to: (i) the launch of Risperidone Long-Acting Injectable (RLAI); and (ii) the generic entry.

The value added by risperidone increased during the life-cycle due to the launch of RLAI and the generic competition. This suggests that, considering the entire life-cycle, the value added by SGAs to the system is higher than the expected value estimated using cost-effectiveness analysis at launch. Pricing and reimbursement decisions should take into account the dynamic nature of pharmaceutical markets and the value added by innovative medicines over the long-run.

With the discovery of new direct-acting antivirals, the cure for hepatitis C appears to be a reality, but its high price and the availability of new antivirals are a major obstacle. In Brazil, treatments for hepatitis C have been available in the public health system since the 1990s, and in 2015 were made available the antivirals sofosbuvir, daclatasvir and simeprevir. The calculation of the budgetary impact of this merger estimated expenditures between 467 and 666 million Reais (USD 121 and 172 million) per year. This study aims to present and discuss the cost and effectiveness of hepatitis C treatment with direct-acting antivirals with or without alfapeginterferon and ribavirin, based on real-life data, and compare it with the world scenario.

We analyzed the treatment data and outcomes of 253 patients from a retrospective cohort performed in a Specialized Care Service, in the city of Porto Alegre. In relation to costs, the direct costs of antiviral drugs, per unit (tablet), were considered according to financial receipts from public purchases. The total cost of the medications used by each individual in each treatment and the cost per cure obtained, expressed in Sustained Viral Response (SVR), were calculated.

Most patients (66.8 percent) had genotype 1 of the hepatitis virus and 92.9 percent achieved SVR. The mean total cost of treatment of patients with genotype 1 was USD 5,862.31 and USD 6,310.34 per cure; while in patients with genotype 3 the cost was USD 5,144.27 and USD 5,974.76 per cure. The cost with the most commonly used treatment regimen, sofosbuvir, daclatasvir and ribavirin was USD 5,961.25 and USD 6,536.46 per cure. These values were 30 percent lower than the values estimated at the time of drug incorporation.

Cost and effectiveness data contextualize a real-life scenario in Brazil. The evaluated treatments presented good effectiveness, but high costs.

Drug development is a risky business. Manufacturers are faced with the dilemma of whether or not to invest at any stage in the development process. Even once marketing authorization has been attained, payers are becoming increasingly demanding of evidence to justify price premiums in the face of increasing budgetary pressures. Cost-effectiveness is a critical decision-making criterion for many payers, and restrictions to sub-populations is common. Early economic modelling at very early phases of the development pathway can inform optimal investment decision-making, including go/no-go decisions and clinical trial design, particularly in population selection. To test the hypothesis of changing payer requirements, we carried out a study on the trends in reimbursement submissions where payers approved but ultimately restricted the population compared to the marketing license or the company's target population.

A systematic literature review of all single technology appraisals (STAs) by the National Institute for Health and Clinical Excellence (NICE) was carried (01/01/2006- to 16/11/2018). We used a linear regression model to examine the relationship between frequency of optimizations and time.

In total, 357 STAs outcomes were identified, 55 percent were recommended and 26 percent were optimized. The proportion of optimized recommendations increased over time vs all other outcomes (p = 0.01), with more technologies being optimized over time (p < 0.01).

The results indicate an increasing trend by NICE towards maximization of value through approval of drugs in select groups of patients. From a manufacturer's perspective, prediction of such outcomes at an early stage is fundamental for investment purposes and to maximize financial returns. An early stage model provides a framework to examine these issues as well as identifying data gaps, where real world evidence can be planned to support the value argument for products, and to inform clinical trial design through value of information analysis.

Health Technology Assessment (HTA) and regulatory decisions involve value judgements. As patient groups, industry, and regulatory agencies conduct more preference studies to quantify these judgements, a better understanding of the methods and practices is needed. Currently, there is no systematic mapping of the use of preference data in Europe. This study aimed to identify (i) the use of quantitative preference data by all relevant HTA bodies and regulatory authorities of the European Union (EU) member states, and (ii) key standards and guidelines.

This study used a mixed method approach based on a systematic literature review, survey and subsequent interviews with decision makers and experts.

A total of 62 survey responses were received. Many respondents reported that their agencies were responsible for supporting more than one type of decision, with 69.0 percent supporting approval decisions, 64.3 percent supporting reimbursement decisions, 61.9 percent supporting pricing decisions, and 64.2 percent supporting guideline development. Respondents reported that their agencies supported these decisions in multiple ways: 78.6 percent by assessing health technologies; 54.8 percent by appraising health technologies; 45.2 percent by compiling an HTA report; 7.1 percent by conducting primary research; 9.5 percent by conducting secondary research. More than 40 percent (42.9 percent) of agencies had the final say on one of the decisions of interest – approval, reimbursement, or pricing. Of the 31 countries studied, 71 percent (n = 22) used quantitative preference data in their reimbursement and pricing decisions. Of those, 86 percent (n = 19) used general population preferences to inform the estimation of quality-adjusted life years (QALY) as part of cost utility analysis.

Much of this use of preference data can be understood within the standard framework of economic analysis adopted by many HTA agencies; either in in the form of: standard ways to estimate QALYs; ways to broaden the impacts of technologies captured in the QALY; or ways to weigh health gain with other decision-making criteria, such as disease severity or innovativeness.

Recent studies have investigated the re-organization and automation of laboratory medicine as a challenge for the hospital in terms of reduction of costs, turnaround time, workload, optimization of human and technological resources and improvement of safety. The purpose of this study was to conduct a health technology assessment (HTA) evaluation process about the possibility to re-organize and automate laboratory medicine at Bambino Gesù Children's Hospital.

The decision-oriented HTA (Do-HTA) method, involving the integration of the European Network for HTA (EUnetHTA) CoreModel and the Analytic Hierarchy Process, was applied to assess the best technology solution. Twenty-one professionals were involved to define tender specifications related to the adaptation works of the new dedicated rooms, and the automatic technologies and organizational solutions for the new laboratory department. Finally, two manufacturer companies were consulted.

Using Do-HTA, the study was focused on laboratory technologies while the infrastructure evaluation was conducted by the Engineering and Logistic Units of the hospital. Results showed that the total performance score of the first proposal was slightly higher (2.5 percent) than the second one, proving the comparable high qualitative level of both manufactures technologies. After an accurate analysis, evaluating all aspects (safety, clinical efficacy, cost, organization & technical criteria) and integrating the infrastructure evaluation, the decision has fallen upon the first company offer.

This HTA project provided an in-depth examination of two proposed technological and organizational solutions. Thanks to the Do-HTA method, which produced and developed data and all needed information, it was possible to guide and assist the decision makers on the choice between the two technical solutions.

A health technology assessment (HTA) process to evaluate the best intensive care ventilator manufacturers has been carried out in different pediatric intensive care units (ICUs) of Bambino Gesù Children's Hospital (OPBG). The purpose of this study is to determine: (i) the most relevant features of a ventilator to be considered between different manufacturers, and (ii) the methodology to conduct the assessment to support the decision-making process about the choice to adopt the suitable technology for OPBG.

The decision-oriented HTA method (Do-HTA), developed by the HTA unit of OPBG, was applied to conduct the assessment. Do-HTA involves the integration of the European Network for HTA (EUnetHTA) CoreModel and the Analytic Hierarchy Process with the support of an informatics tool. It provides the definition and numerical evaluation of assessment parameters to evaluate the performance of technologies. A literature review involving ICU professionals was used to define and weight the assessment elements on clinical, technical, organizational, economic, and safety domains. In particular, a subgroup of these domains has been included in a checklist for the comparative evaluation of different ventilator models, each of which was tested in three independent runs performed in three different ICUs.

Results show that safety and clinical effectiveness had highest the impact within the evaluation, followed by organizational, technical and economic aspects. A percentage value per each ventilator has been assigned, representing the global performances regarding the assessment elements.

This study presents and discusses the benefits and drawbacks of innovative features of ventilators, all characteristics to be taken into account during the evaluation process and a methodology to conduct it. The project identified the best performing ventilator model through a collective decision, giving a reliable recommendation to the Hospital Decision Makers.

Multiple Sclerosis is a progressive, degenerating disease of the central nervous system (CNS), which affects more than 2.5 million people worldwide. The monoclonal antibody natalizumab (TysabriTM) has been approved by the European Medicines Agency in 2006. Yet, the treatment is associated with an increased risk of developing progressive multifocal encephalopathy (PML). The aim of the systematic review was to investigate whether natalizumab is more effective and safer than alternative pharmacological therapies or placebo over a prolonged period (≥ 36 months) with respect to annualised relapse rate (ARR), disability progression, quality of life and number of serious adverse events (SAEs).

A systematic literature search was conducted considering randomized controlled trials (no restriction in length) and prospective, non-randomized controlled trials. In terms of safety, prospective single arm studies were additionally included. The risk of bias (RoB) was assessed using the Cochrane RoB tool (RCT), the ROBINS-I tool (NRCT) and the Institute of Health Economics quality appraisal checklist (IHE-20) for case series (single-arm studies). The quality of evidence was determined using the GRADE-method (Grading of Recommendations, Assessment, Development and Evaluation).

For the assessment of clinical effectiveness, three studies (one RCT and two NRCTs) met the inclusion criteria. No significant differences regarding the ARR and disability progression were detected, if natalizumab was compared to an alternative treatment with fingolimod. Yet, if compared to placebo or a group of natalizumab interrupters, a 70 percent reduction in the ARR was observed. For the assessment of safety, seven studies met the inclusion criteria. The proportion of patients suffering from SAEs ranged from 2.4 percent to 16.0 percent. In total, 35 cases of PML occurred. The results were supported by a very low quality of evidence.

Future research should provide more head-to-head RCTs comparing natalizumab with other disease modulating drugs along with a comprehensive documentation of adverse events.

Mobile health systems (MHS) are one of the more spreading technologies in the field of medicine. However, identification of useful MHS is rather challenging. Few of them are, or could be, connected medical devices (cMD). Like other medical devices, cMD must be assessed to validate claimed benefits for reimbursement purposes. Clinical added value demonstration is a major criterion used to satisfy administrative requirements. With the increase of clinical studies that are including MHS, study registries can be used for insight into the type of evidence expected to become available in the near future.

In 2018, the French National Authority for Health (HAS) performed a review of registered MHS clinical study designs. The Clinicaltrials.gov database was consulted for all studies indexed with the terms “mHealth” and “mobile health” for the search fields “study title”, “conditions” and “interventions”.

Four hundred and fifteen clinical studies were registered. Three hundred and eighty studies were interventional with most comprised of a randomized study design (75 percent). Fifteen had a crossover design. Only few observational studies (n = 35) were registered. These mainly concerned (59 percent) patient use of an app on a smartphone without any other device.

Patterns of clinical studies were not found to significantly differ between MHS and other medical devices. Most of the clinical studies were randomized and specific criteria to assess MHS could easily be identified. However, specific methodologies for clinical development are not used in practice for cMD health technology assessment. In the absence of validated and specific methodology for clinical development, current methods that are being used in these ongoing studies will nonetheless be generating evidence for the upcoming years.

Evidence supporting the use of pacemakers is well established. However, evidence about the optimal use of pacemaker telemonitoring for disease management in heart failure is not. Health Technology Wales (HTW) held a national adoption event to encourage implementation and best practice in use of pacemaker telemonitoring in the National Health Service (NHS) Wales to improve patient outcomes in heart failure.

Multi-stakeholder national adoption workshop using a mixture of expert presentations, case studies and interdisciplinary group and panel discussions to agree key actions to understand the value and promote optimal use of pacemakers for remote disease monitoring in patients with heart failure in Wales.

The workshop was attended by forty-five senior professionals with an interest in improving care of patients with heart failure. Actions to progress included: providing a centralized Welsh system to support technical issues that arise with telemonitoring; considering interoperability with other NHS Wales systems; encouraging value-based procurement with collection of a core outcome set; agreeing implementation issues with both professionals and patients; audit to understand experience, resource use and outcomes; and sharing manufacturer evidence on the accuracy of telemanagement algorithms. It was suggested that these actions be progressed via an All-Wales multi-stakeholder approach, led by the Welsh Cardiac Network.

Developing a more agile, lifecycle approach to technology appraisal is currently advocated; recalibrating the focus from technology assessment to technology management across the complete technology lifecycle. HTW will endeavour through regular adoption events to facilitate such a paradigm shift that aims to understand value and optimise use of evidence-based technologies.

This work aims to update the previous robotic surgery health technology assessment (HTA) study conducted in 2013 in Bambino Gesù Children's Hospital. The study, focused on the evaluation of the newest evidence that have emerged over the last three years, aims to identify if there are new perspectives and advantages of introducing this technology in the hospital.

Decision-oriented HTA (DoHTA) method was applied to conduct the assessment. It involved the integration of the European Network for HTA Core Model® (version 3.0) and the analytic hierarchy process providing the definition and the numerical evaluation of assessment parameters through which it is possible to evaluate the performance of the technologies compared. Three years after the first technology's evaluation, an updated literature review was conducted, using the same 2013 key words, to identify changes in the indicators’ performance score. The performance values have been updated through a quantitative and qualitative evaluation of data gathered from the literature review, expert opinion and context analysis. The global weights’ system, developed in 2013, has not been updated because the relative importance of each domain remained unchanged. The performance values of safety, efficacy, costs, and social aspects have been estimated, identifying the differences in terms of percentage values in comparison with the previous study.

Results showed a slight improvement on safety and organizational aspects in robotic surgery; however, clinical effectiveness and economic, social and legal aspects remained unvaried. More specifically, it has been registered a 3 percent reduction of the difference of the distance between robotic and laparoscopic performance values (2013: 14, 15 percent; 2017: 11, 29 percent).

Results highlighted a slight improvement in robotic surgery performances even if it confirmed the previous results for which the laparoscopic system outperformed the others and currently is keeping the best performance techniques. Finally, sensitivity analysis and a Monte Carlo simulation were carried out proving the stability and reliability of the solution.

The National Institute for Health and Care Excellence (NICE) may recommend temporary funding through managed access agreements (MAAs) for oncology drugs (via the Cancer Drugs Fund [CDF]) and highly specialized therapies for rare diseases. MAAs allow for the collection of evidence to address key areas of clinical uncertainty, while providing access of medicines to patients, prior to re-appraisal by NICE. Observational data and other real-world evidence (RWE) are crucial requirements for all MAAs and herein we examine the extent these data are being used to inform HTA decisions at re-appraisal.

Existing MAAs entered into between the National Health Service (NHS) England and manufacturers as of 30 October 2018 were identified; for drug:indication pairings with NICE re-appraisals, all information was reviewed and the key data extracted.

Of the twenty-two MAAs identified, only two drug:indication pairings have been subsequently re-appraised by NICE: BV(brentuximab vedotin):non-Hodgkin lymphoma (’recommended’) and pembrolizumab:relapsed or refractory classical Hodgkin lymphoma (’recommended’). Data from a retrospective questionnaire regarding the proportion of patients that received curative stem cell transplant (SCT) post-BV (from patients who received BV in the old CDF) were accepted to provide sufficient evidence on the post-BV SCT rate by NICE. Meanwhile, for pembrolizumab, long-term survival benefit was the key clinical uncertainty; the primary data collection source was updated phase III randomized controlled trial data. At re-appraisal no reference was made to the observational data component; more mature survival data reduced uncertainty over survival benefits and were sufficient to support a positive NICE recommendation.

Of the twenty-two MAAs to date, only two drugs have been re-appraised thus far, with both receiving positive NICE recommendations. Observational data were successfully used to address key clinical uncertainties regarding subsequent real-world treatment patterns for BV, but observational data were not referred to in the NICE recommendation for pembrolizumab. The re-appraisal of more drugs in the future will clarify the importance being placed on observational data collection requested by NICE for existing MAAs.

In a context of rapidly evolving technologies and growing evaluative challenges, the National Institute of Excellence in Health and Social Services (INESSS) is developing an institutional ethical framework making explicit and transparent the guiding principles and new modalities of process for health technology assessment for public coverage.

This framework is co-built by the INESSS experts - drugs, social services, technology and health services and cross-cutting methodologies - through literature and practice reviews as well as a consultative process on key topics with external collaborators.

The development process aims to: (i) identify the principles applicable to all the objects evaluated, (ii) define the evaluation strategies used to appropriately address evaluation challenges in the clinical, organizational, economic and societal dimensions, (iii) equip the scientific teams to successfully integrate diversified knowledge from the literature, stakeholders participation and medico-administrative data banks, and (iv) facilitate deliberation leading to evidence-informed recommendations. It is envisioned as a fully integrated process rooted in a reflexive multi-criteria approach supporting fair and reasonable decision. The presentation will focus on one of the key aspects of this framework, i.e., the development of principles for stakeholder participation based on a recent INESSS methodological forum on the topic, and the agile deployment of innovative processes and tools in various projects, including the patient partnership developed with a pioneering academic centre.

This framework provides explicit, transparent and cross-cutting processes and a framework for continuous improvement. The goal is to promote stakeholder engagement and enable increasingly complex arbitration aimed at equity and social justice, in a context of rising costs and uncertainty, and focused on the creation of value for our fellow citizens.

After surveying its members on ethical issues (2003), the International Network of Agencies for Health Technology Assessment (INAHTA) mandated its Ethics Working Group (2005) to reflect on the role of health technology assessment (HTA) organizations in meeting social expectations. Some aspects of these have since been clarified by two studies addressing either the official position of INAHTA's members or the publication authors. An international survey was carried out on the perception of HTA professionals’ expectations when producing HTA reports: how to fulfil HTA's social role, which value judgments should be made explicit and what should be the status of ethical analysis.

A twenty-two question, web-based, anonymous survey was devised from our recent systematic review on the integration of ethics into HTA and carried from April to July 2018. The information on 328 HTA agencies/contact persons from seventy-five countries was collected from the website of INAHTA, Health Technology Assessment International (HTAi), the European Network for Health Technology Assessment (EUnetHTA), EuroScan International Network, the HTA Network of the Americas (RedETSA) and the HTA Network of Asia (HTAsiaLink), a 2015 World Health Organization survey, HTAi members, and our local HTA network (Québec, Canada).

Eighty-nine participants completed and submitted a finalized survey for a 27 percent participation rate representing thirty-three countries. Regarding how the HTA reports should fulfil their social role, our results showed that over 84 percent of the respondents agreed upon the necessity to address it to decision makers, patients and citizens. At a lower and more variable level, the same result was found about the necessity to make value judgements explicit in different sections of the report, including ethical analysis. This contrasts with the variability of responses obtained on the status of ethical analysis although an agreement on the expertise required was observed. Variability in the usefulness of patient, public or stakeholder participation was observed.

At the dawn of this decade, this study reveals high expectations on context-dependent decisions in HTA: the necessity to integrate the ‘explicitation’ of value judgements and systematic ethical analysis to fulfil HTA's social role.