The Patient Experience, Expectations and Knowledge (PEEK) protocol was developed so that a holistic, comprehensive, independent, proactive and systematic approach could be taken to inform decisions made in the context of health technology assessment and other parts of the health sector. Each PEEK study is made publicly available which over time will result in a global repository of patient experience data.

The PEEK protocol is a single protocol that can be implemented across disease areas and includes a quantitative and qualitative component. The quantitative component is based on a series of validated tools that provide baseline health and demographic data for the study population. The qualitative component is the result of two years of protocol testing to develop a structured interview that solicits comprehensive and holistic patient experience data, and provides participants with the opportunity to provide advice on their future expectations.

PEEK studies in breast cancer, bladder cancer, lung cancer, spinal muscular atrophy, atopic dermatitis, chronic kidney disease, chronic heart failure and mitochondrial disease have been completed in the Australian context (www.cc-dr.org/peek). Holistic patient experience themes are presented commencing with symptoms and diagnosis experience, through to communication, information, treatments experienced and quality of life. Information is also available in relation to participant's expectations of future treatment, care, information and communication. The result is a freely available repository of patient experience data that anyone in the sector can access to complement clinical and economic evidence.

The process of providing patient feedback and real-world evidence in the context of health technology assessment is often ad-hoc. The lack of consistency means that it has been difficult to assess the impact of patient engagement and feedback in the context of health technology assessment. The PEEK protocol and program is an example of a systematic, independent and holistic approach to patient experience and real-world evidence data collection that provides the sector with an opportunity to proactively engage the community in decisions that are made about treatment, care and support.

Patient-reported outcomes (PRO) data are important in understanding patients’ experience of disease and treatment; however, PRO data are not universally collected or consistently included as part of a Health Technology Assessment (HTA) submission. Additionally, the HTA bodies’ response to PRO data vary, making the impact unclear. To understand the impact of PRO data on reimbursement decisions for Acute Myeloid Leukemia (AML) indications, an in-depth analysis of HTA bodies’ appraisals of AML and analogous indications was conducted.

This analysis was conducted using IQVIA's HTA Accelerator, which contains HTA appraisals from ≥100 HTA agencies in thirty-nine countries. Included in the analysis were single-technology assessments (original submissions, resubmissions, extensions of original indications, and renewals); relevant regulatory approvals and pivotal trials were also analyzed.

Of the 185 AML appraisals from sixteen HTA bodies, 66 (36%) included PRO data. Within these, thirteen different PRO instruments were identified, none of which have been validated in patients with AML. For seven of twenty in-scope products, PRO evidence positively impacted ≥1 of the HTA decisions. Although the same HTA bodies (i.e., Scottish Medicines Consortium, pan-Canadian Oncology Drug Review, and the National Institute of Health and Care Excellence) generally accepted the PRO evidence, others were critical of the evidence (i.e., Haute Autorité de Santé and the Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen). The most common concerns raised by the HTA bodies regarding the PRO evidence included trial design and low patient response rate.

Of the products that included PRO evidence in their HTA submissions, 35% received positive feedback from ≥1 HTA body on their submitted PRO evidence. Attention to PRO data collection is key to demonstrate the value of AML products to HTA bodies. Without these data, a clear gap in the understanding of patients’ experience is evident.

Hospital-based health technology assessment (HB-HTA) needs to consider all relevant data to help decision-making, including patients’ preferences. In this study, we comprehensively describe the process of identification, refinement and selection of attributes and levels for a discrete choice experiment (DCE).

A mixed-methods design was used to identify attributes and levels explaining low back pain (LBP) patients’ choice for a non-surgical treatment. This design combined a systematic literature review with a patients’ focus group, one-on-one interactions with experts and patients, and discussions with stakeholder committee members. Following the patient's focus group, ranking exercises were conducted. A consensus about the attributes and levels was researched during discussions with committee members.

The literature review yielded 40 attributes to consider in patients’ treatment choice. During the focus group, one additional attribute emerged. The ranking exercises allowed selecting eight attributes for the DCE. These eight attributes and their levels were discussed and validated by the committee members who helped reframe two levels in one of the attributes and delete one attribute. The final seven attributes were: treatment modality, pain reduction, onset of treatment efficacy, duration of efficacy, difficulty in daily living activities, sleep problem, and knowledge about their body and pain.

This study is one of the few to comprehensively describe the selection process of attributes and levels for a DCE. This may help ensure transparency and judge the quality of the decision-making process. In the context of a HB-HTA unit, this strengthens the legitimacy to perform a DCE to better inform decision-makers in a patient-centered care approach.

Controversy regarding the efficacy of transcatheter mitral valve repair with a clip (TMVRc) in reducing the mitral regurgitation is related to the lack of solid scientific evidence. Worldwide, refusal or conditional acceptance for implementation of TMVRc, reflect ongoing uncertainty. We sought to apply a systematic multicriteria framework to ensure a fair and reasonable decision regarding the use of TMVRc in Quebec.

The framework included the following domains: context, quality of evidence concerning safety, efficacy and effectiveness, unmet patient needs, expected volume of patients, and impact on the health system including costs. Each domain within the framework was examined by a review of the literature and through consultations with a scientific advisory committee, a TMVRc clinical expert committee, TMVRc clinical teams, industry representatives and the Institut national d'excellence en santé et en services sociaux (INESSS) clinical excellence committee.

The literature review indicated that uncertainty about the efficacy and effectiveness of TMVRc persists, particularly in the real world context, and this view was supported by scientific experts. The TMVRc clinical teams provided insight into the burden of mitral insufficiency on patients and the health system and their belief in the promise of TMVRc. They also highlighted the challenges of patient selection and organizational issues related to the introduction of TMVRc within their institutions. The advisory committee stressed the need for further evaluation prior to wide diffusion.

Using a multicriteria framework facilitated a more standardized and transparent approach to our literature review and consultations as well as to the development of the proposed recommendations. This was especially important in the context of an evaluation of a promising new approach to treat mitral valve disease with many important uncertainties. This multi-criteria approach will facilitate a more standardized process for deliberation on how new health technologies should be implemented into the Quebec health system.

Although stochastic analysis has become the accepted standard for decision analytic cost effectiveness models, deterministic one-way sensitivity analysis continues to be used to meet the needs of decision makers to understand the impact that changing the value taken by one specific parameter has on the results of the analysis. However, there are a number of problems with this approach.

We review the reasons why deterministic one-way sensitivity analysis will provide decision makers with biased and incomplete information. We then describe a new method - stochastic one-way sensitivity analysis (SOWSA), and apply this to a previously published cost effectiveness analysis, to produce a stochastic tornado diagram and conditional incremental net benefit curve. We then discuss how these outputs should be interpreted and the potential barriers to the implementation of SOWSA.

The results illustrate the shortcomings of the current approaches to deterministic one-way sensitivity analysis. For SOWSA, the expected costs and outcomes are captured, along with the sampled value of the parameter and these are linked to the probability that the parameter takes that value – which can be read off the probability distribution for the parameter used in the stochastic analysis. From these results it is possible to gain insights into probability that a parameter will take a value that will change a decision.

Although a well-used technique, one-way deterministic sensitivity analysis has a number of shortcomings that may contribute to incorrect conclusions being drawn about the importance of certain parameter values on model results. By providing fuller information on uncertainty in model results, it is hoped that the methods here will lead to more informed decision making. Although, as with all developments in the presentation of analytic results to decision makers, care will be required to ensure that the decision makers understand the information provided to them.

Economic models contain several parameters ordinarily subject to uncertainty. Unlike most other model parameters, costs can constitute numerous distinct components. For example, a surgical intervention can require a variety of disposables and reusable equipment. A micro-costing output may be disaggregated or presented as a total cost. Uncertainty could be applied to individual cost components or to total cost. We aimed to explore how disaggregation of cost data may impact on uncertainty using a case study.

A set of simulations using hypothetical scenarios were developed with uncertainty set at ± 20 percent. The simulations investigated the impact of number of cost components, balance between components, and correlation between them. A cost-utility model from an assessment of robot-assisted radical prostatectomy was analyzed; procedure cost was divided into 32 individual cost components or treated as a total cost.

Based on five equal cost components, uncertainty reduces from ± 20 percent for correlated variables to ± 9 percent for uncorrelated variables. With increasing numbers of uncorrelated cost components, the uncertainty in the total cost decreases markedly. The uncertainty around total robot-assisted surgery procedure equipment costs was ± 19.7 percent when components were correlated and ± 9.4 percent when uncorrelated. The impact on uncertainty in the incremental cost effectiveness ratio (ICER) was negligible but the ranking of parameters in the univariate sensitivity analysis changed.

Analyzing uncertainty by aggregated or disaggregated costs can have implications for presenting uncertainty in costs to decision makers. Applying uncertainty to aggregated costs essentially implies that variation in the cost of individual components is perfectly correlated. By disaggregating cost components they are being treated as uncorrelated, which can substantially reduce uncertainty in the total cost. In this case study we found that although the reduction in uncertainty could be clearly seen in the cost parameter, it had a negligible impact on uncertainty in the ICER.

For the last decades, health technology assessment has been strongly promoted in order to provide evidence-based rather than eminence-based healthcare. However, when it comes to implementing interventions that are based on processes and behavior rather than products, importing evidence-based interventions is likely to fail because it ignores the strong influence of contextual factors. In a recently started research project, an alternative approach is tested.

The project aims at improving identification and support of children who have parents with a mental illness in an Austrian region (Tyrol). A theory- and stakeholder-driven approach has been designed in order to co-develop, implement and evaluate practice approaches for improving the situation for children. The former addresses the questions whether, how and why suggested practices may work and the later brings together evidence and practice to develop interventions that are feasible and take the regional context and service settings into account.

Based on evidence from various sources (literature, international and local stakeholder interviews), theories that describe the key mechanisms of action to influence outcomes for children have been developed. A regional and interdisciplinary group of practitioners and people with lived experience has been established to facilitate the development of the theory of change and practice approaches for the region and local service settings. The final practice model is then monitored in the respective organizations with implementation support from the research team.

Theory-driven and co-designed-based approaches are a feasible alternative to ‘off-the-shelf’ evidence-based practices for supporting decision makers in implementing complex interventions. However, they require a broad variety of skills within the research team as well as willingness to accept uncertainties of the final outcomes produced, which can also be of risk to funders. The ongoing project will demonstrate whether the developed practices will be implemented successfully and result in benefits for the affected children.

Australia has a well-developed HTA system for approving, funding and disinvesting in medical services, but how does it cope when it assesses new or existing services when health system frameworks, policy and legislation do not keep pace? This presentation will present a number of case studies where HTA methods have adapted to suit emerging health needs of the Australian community.

Australia's HTA system has been adapted over recent years to allow it to perform HTA on novel services that do not fit into its standard HTA pathway to public funding. There has also been an increasing number of assessments where HTA has been unusually sponsored by the funder to assess priority health care needs.

More Medicare funding for addiction and sexual health consultation services are an example of a novel HTA which led to more public funding for these services. Limited evidence for these services was available and there was difficulty in demonstrating that increased public funding would lead to better outcomes for patients. A range of techniques, such as examining real world data and stakeholder views were partnered with HTA to assess these services. Currently, HTA is being utilised to assess a novel medical treatment known as the anti-CD19 chimeric antigen receptor T cell (CAR-T) therapeutic process. This is a complex, non-standard HTA which encompasses aspects of the Australian hospital funding system, Medicare and the Pharmaceutical Benefits Scheme and requires an adapted HTA process to assess evidence across a range of funder systems.

Australia's well established HTA system has adapted to become more agile to suit emerging health care needs for a range of interest groups, the government, sponsors and consumers. Consultation with stakeholders and the community have assisted in developing and refining these new processes.

Mental health disorders and their treatments produce costs and benefits in both healthcare and non-healthcare sectors. The latter one is often referred to as inter-sectoral costs and benefits (ICBs). Limited research is available on the inclusion of these inter-sectoral costs and benefits (ICBs) in economic evaluations. In this study, we focus on the identification and classification of ICBs of mental health-related interventions within the criminal justice sector in a broader European context. This study was conducted as part of the PECUNIA-project, which aims to develop new standardized, harmonized and validated methods and tools for the assessment of costs and outcomes in European healthcare systems. The aim of the study is to further conceptualize an internationally applicable list of ICBs of mental health-related interventions in the criminal justice sector. Additionally, we aim to facilitate the inclusion of ICBs in economic evaluations across EU by prioritizing important ICBs.

Data was collected via a systematic literature search on PubMed and PsychINFO. Additionally, a grey literature search was carried out in six European countries. In order to validate the international applicability of the list and prioritize the ICBs, a survey was conducted with an international group of experts from the criminal justice sector.

The literature search identified ICBs and resulted in a comprehensive list of items. A multi-dimensional list was constructed, distinguishing between costs as consequence of crime, and costs in response to crime. Based on the expert survey, the international applicability of the list was validated and the most important ICBs from the economic perspective were identified.

This study laid further foundations for the inclusion of important societal costs of mental health-related interventions within the criminal justice sector. More research is needed to facilitate the greater use of ICBs in economic evaluations.

This study aimed to reach patients using different languages while providing an opportunity to enter symptoms in their everyday language text besides medical expressions of symptoms.

Named entity recognition (NER) techniques, based on natural language processing (NLP), were applied to develop a language independent predictive model. The research was based on extracting symptoms entered to the system by patient using NER method of NLP. In order to implement the system, python was used while pre-processing the data and string similarity function was used to estimate similarity with disease symptoms. Two sets were used for classification, one including only symptoms, and the other the matching diseases. Four thousand two hundred and eighty different symptoms were processed for the corresponding 880 diseases.

Each user symptom had a similarity score for each symptom in all diseases. Top N results with highest similarities were chosen from this list. The final N results are matched with diseases. According to these results, matched diseases were ordered in terms of the percentage of matched symptoms in the disease's symptoms. Extracted terms were implied as an input of the model and analysed for a matching diagnosis where an accuracy of 83 percent was accomplished when it is tested and compared using Mayo Clinic data for specific foreign languages other than English.

This language independent online diagnostic tool is a solution for both personal and clinical use and provides maintainable, updatable and more reliable diagnostics. This tool is particularly relevant today, with global mobility growing at a rate faster than the world`s population. We aim to upgrade the system by adding speech recognition and engaging it with the background (if available, electronic health records) of the patient.

Robot-Assisted Surgery (RAS) has been available in Australia since 2003. There are 50 da Vinci RAS systems in Australia (18 in NSW and 12 in Victoria) with most in the private hospital sector. In Australia the capital cost of a da Vinci RAS system is up to AUD 4 million (USD 2.8 million), which excludes annual maintenance fees of AUD 250,000 (USD 175,000) and consumable costs of AUD 3,500 (USD 2,450) for each procedure.

The NSW Ministry of Health and Victorian Department of Health and Human Services commissioned a health technology assessment (HTA) to explore the benefits, risks and economic implications of surgical robotics, which involved a review of the peer reviewed literature, a cost benefit analysis of public sector patients who received RAS and broad stakeholder consultation to document current perspectives on RAS applications.

RAS is as safe and effective as other surgical modalities when performed by sufficiently skilled surgeons, although evidence generally comes from small studies with limited follow-up time and few studies report long term mortality, morbidity or patient-reported outcomes. Comparative benefits of RAS are uncertain as most studies conclude little or no difference in procedure related or functional outcomes. While RAS reduces length of stay, which offers patient and health system benefits, this is insufficient to fully offset high capital and consumable costs currently charged to Australian providers. Government and clinical stakeholders identified that establishing an RAS service requires consideration of important factors, including: i) Governance is critical; ii) Higher case volumes may improve financial viability; and iii) a need for state-wide/national standards for surgeon training and credentialing.

RAS is as safe and effective as other modalities when performed by skilled surgeons. However, uncertainty remains around long-term outcomes and clinical and cost effectiveness. An accredited training program, monitoring and evaluation will be critical to ensure outcomes data inform ongoing evidence assessment and government policy and investment.

The Brazilian Unified Health System (SUS) is known worldwide for ensuring universal assistance to citizens, which includes the supply of medicines free of charge. Biological drugs consume about 40 percent of the public budget for pharmaceutical services in the SUS and Rheumatoid Arthritis (RA) is the largest consumer of these resources, serving about 110,000 patients. Since 2002 there has been a clinical guideline for the care of patients with rheumatoid arthritis in Brazil, currently providing 10 biological drugs for treatment of RA. The objective of this study is to present data about the provision, expenditure and profile of users of biological drugs for rheumatoid arthritis in SUS.

Retrospective and exploratory study, using administrative data regarding the purchase and consumption of biological drugs Infliximab, Etanercept, Adalimumab, Rituximab, Abatacept (intravenous and subcutaneous), Tocilizumab, Golimumab and Certolizumab pegol for the treatment of RA between 2012 and 2017 in SUS.

There was an expenditure of approximately USD 421.7 million from the Brazilian Ministry of Health with the supply of about 2 million pharmaceutical units of biological drugs for treatment of rheumatoid arthritis, 79 percent of them destined for female users and 89.2 percent for the 40-69 age group. The M05.8 and M.06.0 codes of the International Classification of Diseases (ICD-10) were the most prevalent among the arthritic population served. Adalimumab and Etanercept accounted for 68.3 percent of total expenditure. A reduction in the use of these medicines were observed after the availability of new drugs for the treatment of the disease between 2014 and 2017.

Brazil is one of the largest consumers of biological medicines in the world. The use of real-life data allows monitoring trends and costs of the use of these drugs as well as changes with the entry of new therapies and biosimilar medicines.

Stroke is a major contributor to mortality, disability and long-term use of healthcare services. As for all chrono-dependant conditions, clinical results are associated with timely access to appropriate care. Thrombectomy (EVT) is an effective treatment for large vessel occlusions, but can only be provided in highly-specialized centers by experienced personnel. We sought to develop a framework to aid decision-making on the appropriateness of opening new EVT centers in Québec, Canada.

Data sources included provincial administrative healthcare databases, population density statistics, field evaluation of Québec's four existing EVT care networks, and literature review concerning structural and performance criteria for EVT centers. We consulted EVT clinical teams, interdisciplinary stroke experts, patients, professional association representatives, healthcare managers and decision-makers.

Access to EVT is suboptimal in all 17 regions of Québec, with virtually no access in remote areas. Results of key performance indicators indicated favorable treatment delays after arrival at the EVT center. However, door-to-needle and door-in-door-out times were long for patients transferred from non-EVT centers. High use of ambulances indicated the potential to transport patients to the most appropriate center. In light of ‘real world’ results and other sources of information, the need for a new EVT center should consider the following criteria: sub-optimal EVT access within the region; transport time to an existing EVT center >1 hour; expected patient volume within 2 hours of transport; impact on volume of existing programs; availability of long-term financial support; availability of a critical mass of neurointerventionists, vascular neurologists, and neurosurgeons; demonstrated quality of stroke care; and, presence of a stroke unit.

The triangulation of literature, clinician experience and the Québec context enriched the evaluation process. Furthermore, this facilitated the development of a framework that was broadly applicable across regions to the real-world setting of decision-making in a complex system of care.

The objectives of this study were to systematically review published research on the relationship between nursing staff coverage, care hours, and quality of care (QoC) in long-term care (LTC) facilities; and to conduct a real world evidence (RWE) analysis using Alberta real world data (RWD) to inform policy makers on whether any amendments could be made to current regulations.

A systematic review (SR) of research evidence published between January 2000 and May 2018 on the relationship between nursing staff coverage, care hours, and QoC in LTC facilities was conducted. Panel data regressions using available RWD from Alberta, Canada, were performed to assess associations between nursing care hours and LTC outcomes. Outcomes of interest included quality indicators related to resident outcomes, hospital admissions, emergency room visits and family satisfaction. Nursing care hours considered in SR and RWE analysis included those provided by registered nurses (RNs) and licensed practical nurses (LPNs).

The SR found inconsistent and poor quality evidence relevant to the questions of interest, indicating a great uncertainty about the association between nursing staff time and type of coverage and QoC. Although some positive indications were suggested, major weaknesses of reviewed studies limited interpretation of SR results. RWE analysis found that impact of care hours on LTC outcomes was heterogeneous, dependent on outcome measurements. There was evidence that total staff, RN, and LPN hours had positive effects on some resident outcomes and magnitude of effect differed for different nursing staff.

No definitive conclusion could be drawn on whether changing nursing staff time or nursing staff coverage models would affect residents’ outcomes based on the research evidence gathered in the SR. RWE analysis helped to fill a gap in the available published literature and allowed policy makers to better understand the impact of revising current regulations based on actual outcomes.

When conducting a Network Meta-Analysis (NMA) for a Health Technology Assessment (HTA), the submitting company typically will have access to Individual Patient Data (IPD) from their own trials, but only aggregate data (AgD) for the comparator. In this case, they can re-weight the IPD so that the covariate characteristics in the IPD trials match that of the AgD trials, using the increasingly popular method of Matching-Adjusted Indirect Comparison (MAIC).

We carried out a simulation study to investigate this method in a Bayesian setting. We simulated three IPD trials comparing treatments A and B (AB-IPD trials), and one aggregate data trial comparing treatments B and C (BC-AgD trial). We investigated two options of weighting covariates: 1. all three studies are weighted separately to match the BC-AgD trial (MAIC Separate Trials). 2. patients are weighted across all three IPD studies to match the BC-AgD trial, but the NMA still considers each trial separately (MAIC Pooled Trials). We compared the results of the MAIC to a standard NMA and a mixed IPD/AgD NMA. We applied these methods to a network of treatments for multiple myeloma.

MAIC can provide more accurate estimates of the relative treatment effects than a standard NMA in the BC-AgD trial population. However, MAIC may decrease the accuracy of the relative treatment effects in the overall population. Treatment rankings were unchanged when applying MAIC to the multiple myeloma network.

MAIC is beneficial as a sensitivity analysis to demonstrate that results hold across patient populations. If there is a difference in relative treatment effects attributable to population imbalances, then it is useful to be able to quantify this difference. However, we recommend using either a standard NMA or a mixed IPD/AgD NMA for the base case analysis, given the potential bias that can arise in an MAIC.

Efforts to harmonize health technology assessment (HTA) processes and methods across Europe are currently intensified. In this context, the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach has been proposed as a “common ground” in joint HTAs. However, GRADE has been primarily developed to support authors of clinical guidelines. Therefore, it is unclear whether HTA reports based on GRADE are compatible with the methods currently applied by European HTA organizations.

We contrasted IQWiG's methods paper and publications by the GRADE Working Group with regard to the following domains: 1) risk of bias (RoB) assessment 2) prerequisites for “greater benefit” (assuming that IQWiG's “greater benefit” corresponds to a GRADE assessment of at least low certainty and a small important effect) and 3) consideration of non-randomized studies (NRS). We present illustrative differences and highlight similarities.

Overall, RoB assessments are very similar under both approaches. However, we identified several important differences. In case of very severe publication bias, IQWiG methods preclude drawing a conclusion, whereas GRADE requires only downgrading the certainty of evidence while still allowing for a conclusion on effect sizes. Secondly, IQWiG generally requires a statistically significant effect for a “greater benefit”, while GRADE does not (statistically non-significant effects would only necessitate downgrading the certainty of results for imprecision). Another difference is that in general, NRS are not included in IQWiG assessments when randomized studies (RS) are available and thus possible. In contrast, preliminary GRADE guidance recommends considering NRS in addition to RS when the RS evidence is of low or very low certainty.

While GRADE and IQWiG's method share some similarities, our exemplary analysis shows that there are some notable differences. Therefore, GRADE should not be used “out of the box” for European HTAs. To foster further discussion, more research (including a comprehensive comparison of methods and an analysis of resources for adaptation) is needed.

Intention to treat (ITT) is a gold standard strategy to analyze the results of randomized controlled trials (RCTs). ITT analysis has been considered a methodological indicator of the quality of clinical trials. The extent to which the use of ITT is related to the treatment effects observed in RCTs has not been rigorously explored. Therefore, the main objective of this study was to determine the association between biases related to attrition and missing data and the use of intention to treat principle, and changes in effect size estimates in RCTs.

This was a meta-epidemiological study. A random sample of RCTs included in meta-analyses was identified. Data extraction including assessments of the use of intention to treat principle, missing data and drop-outs was conducted independently by two reviewers. To determine the association between biases related to attrition, missing data, and the use of intention to treat and effect sizes, a two-level analysis was conducted using a meta-meta-analytic approach.

Three-hundred and ninety-three trials included in 43 meta-analyses, analyzing 44,622 patients contributed to this study. From these, 134 trials (34.1%) used ITT and 218 (55.5%) did not use ITT. Trials which did not use the ITT principle, or which were assessed as having an inappropriate control of incomplete outcome data (based on the Cochrane risk of bias tool) tended to underestimate the treatment effect when compared with trials with adequate use of ITT (ES= -0.13; 95%CI -0.26, -0.01) or trials which were assessed as having an appropriate control of incomplete outcome (ES= -0.18; 95%CI -0.29, -0.08).

Our results suggest that when evaluating risk of bias of primary RCTs, systematic reviewers should pay attention to these biases since they could underestimate treatment effects. Systematic reviewers should perform sensitivity analysis including trials with low risk of bias in these domains.

Most new oncology therapies are studied in the advanced/metastatic setting. However, there is an increasing focus on earlier stage disease. Nevertheless, measuring Overall Survival (OS) in neo-/adjuvant therapy trials can be very challenging due to the increased life expectancy and the confounding effects of subsequent treatments. Thus, their primary endpoints tend to be surrogate survival metrics (e.g. metastases-free survival). This research aims evaluates the health technology assessment (HTA) acceptability of such endpoints through recent neo-/adjuvant HTA assessments.

The European Medicines Agency (EMA) website was screened for any neo-/adjuvant oncology therapies approved (1 January 2013-22 October 2018) and any corresponding publicly-available assessments by HTA bodies (NICE, SMC, IQWiG, G-BA, CADTH, PBAC, HAS) were identified and key data extracted.

Six neo-/adjuvant therapies have received marketing authorization by the European Commission (EC). These six have been on the market for an average of 8.9 months (range: 0.9-39.3 months, median: 3.3 months). In four of the six, the pivotal trial primary endpoints were measures of relapse-/disease-free survival, (others: pathological complete response and PFS/OS co-primary). Only one had mature OS data available at EC-approval. Four of the six therapies had received at least draft guidance by an HTA body, encompassing 11 HTA assessments in total (4: NICE, 2: IQWiG, HAS; 1: SMC, CADTH, G-BA). Only two of 11 (18%) were positive outcomes (both NICE), the remaining nine were negative.

Oncology therapies are increasingly receiving regulatory approval in the neo-/adjuvant setting. However, their pivotal trials are frequently powered to show benefits in disease-/metastases-free survival. Whilst sufficient for regulatory approval, translating this to favorable HTA decisions has been more challenging. Clearly establishing linkages between surrogate survival metrics and OS alongside measuring metrics that clearly portray patient benefits (e.g. time to symptomatic progression) could improve HTA-acceptability. Further, some payers allow for temporary reimbursement whilst additional evidence is generated (e.g. Cancer Drugs Fund in England).

The introduction of fast-track licensing strategies increases the approval of anti-cancer drugs with ambiguous benefit-risk profiles. Thus, in many instances there is lacking evidence about overall survival (OS) at the time of marketing authorisation. Our objective was to monitor and characterise therapies with ambiguous benefit-risk profiles and identify any post-approval updates on median OS after at least three years of approval by the European Medicines Agency (EMA).

We included all originator anti-cancer drugs with initially ambiguous benefit-risk profiles that received marketing authorization from the EMA between 1 Jan 2009 and 31 May 2015. Our monitoring timeframe was at least three years after EMA-approval. To identify study updates, the following three sources were included: clinicaltrials.gov, European Public Assessment Reports (EPARs), and PubMed.

In total, we identified 102 eligible approval studies. Out of these, a negative difference in median OS or no information was available in forty-three (42.2%) instances. During monitoring, eleven updates with accessible information on median OS could be identified. Including monitoring results, there are still thirty-two remaining therapies (31.4%) where no or negative information (n = 27 [26.5%] and n = 5 [4.9%], respectively) regarding median OS was present at least three years after EMA approval.

One-third of oncology drugs with ambiguous benefit-risk profiles failed to demonstrate a survival benefit even several years following marketing authorization. Systematic and transparent post-approval monitoring mechanisms will be of high relevance to assure a clinically relevant patient benefit, since the trend towards faster access to medicines with uncertain benefit is increasing rather than declining.