The expansion of electronic health record (EHR) data networks over the last two decades has significantly improved the accessibility and processes around data sharing. However, there lies a gap in meeting the needs of Clinical and Translational Science Award (CTSA) hubs, particularly related to real-world data (RWD) and real-world evidence (RWE).

We adopted a mixed-methods approach to construct a comprehensive needs assessment that included: (1) A Landscape Context analysis to understand the competitive environment; and (2) Customer Discovery to identify stakeholders and the value proposition related to EHR data networks. Methods included surveys, interviews, and a focus group.

Thirty-two CTSA institutions contributed data for analysis. Fifty-four interviews and one focus group were conducted. The synthesis of our findings pivots around five emergent themes: (1) CTSA segmentation needs vary according to resources; (2) Team science is key for success; (3) Quality of data generates trust in the network; (4) Capacity building is defined differently by researcher career stage and CTSA existing resources; and (5) Researchers’ unmet needs.

Based on the results, EHR data networks like ENACT that would like to meet the expectations of academic research centers within the CTSA consortium need to consider filling the gaps identified by our study: foster team science, improve workforce capacity, achieve data governance trust and efficiency of operation, and aid Learning Health Systems with validating, applying, and scaling the evidence to support quality improvement and high-value care. These findings align with the NIH NCATS Strategic Plan for Data Science.

In France, decisions for pricing and reimbursement for medicinal products are based on appraisals performed by the National authority for health (Haute Autorité de Santé (HAS)). During the appraisal process, additional real-world evidence can be requested as “Post-Registration Studies” (PRS) when there are uncertainties in evidence that could be resolved by additional data collection. To facilitate PRS planning, a retrospective exploratory analysis was conducted to identify the characteristics of medicinal products associated with a PRS request.

This analysis encompassed all appraisals finalized between January 1, 2016 and December 31, 2021 and compared products for which the appraisal led to a PRS request with those that did not.

Six hundred positive opinions for reimbursement were identified, with a PRS request present in 17 percent (n = 103) of cases. The independent characteristics associated with a PRS request were a mild or moderate clinical benefit score, a major to moderate or minor clinical added value score, previous availability under an early access program, and certain therapeutic areas (neurology, pulmonology, and endocrinology). These findings suggest two different profiles of PRS requests: (i) products for which there is uncertainty in the size of the clinical benefit and (ii) innovative products for which a substantial benefit is expected but uncertainties persist.

These results will assist health technology developers to better anticipate data generation to promptly address uncertainties identified by HAS. It may also help HAS and other assessment agencies to work together to improve postlaunch evidence generation according to the characteristics of the medicinal products.

The Health Technology Assessment (HTA) process aims to optimize health system funding of technologies. In recent years there has been an increase in what is known as Real-World Evidence (RWE) as a complement to clinical trials. The objective of Health Technology Assessment International’s Latin American Policy Forum 2022 was to explore the utility of incorporating RWE into HTA and decision-making processes in the region.

This article is based on a background document, survey, and the deliberative work of the country representatives who participated in the Forum.

There is a growing interest in the use of Real-World Data / Real-World Evidence in HTA processes in Latin America, although currently there are no specific local guidelines for RWE use by HTA agencies. At present, its use is limited to certain areas such as adding context to HTA reports, the evaluation of adverse events, or cost estimation.

Potential future uses of RWE were identified, including the creation of risk-sharing agreements, the assessment of technology performance in routine practice, providing information on outcomes that are not so easily evaluated in clinical trials (e.g., the identification of specific subpopulations or quality of life), and the estimation of input parameters for economic evaluations.

The participants agreed that there are several areas presenting significant potential to expand the application of RWD/RWE and that the development of normative frameworks for its use could be helpful.

Increasing interest in real-world evidence has fueled the development of study designs incorporating real-world data (RWD). Using the Causal Roadmap, we specify three designs to evaluate the difference in risk of major adverse cardiovascular events (MACE) with oral semaglutide versus standard-of-care: (1) the actual sequence of non-inferiority and superiority randomized controlled trials (RCTs), (2) a single RCT, and (3) a hybrid randomized-external data study.

The hybrid design considers integration of the PIONEER 6 RCT with RWD controls using the experiment-selector cross-validated targeted maximum likelihood estimator. We evaluate 95% confidence interval coverage, power, and average patient time during which participants would be precluded from receiving a glucagon-like peptide-1 receptor agonist (GLP1-RA) for each design using simulations. Finally, we estimate the effect of oral semaglutide on MACE for the hybrid PIONEER 6-RWD analysis.

In simulations, Designs 1 and 2 performed similarly. The tradeoff between decreased coverage and patient time without the possibility of a GLP1-RA for Designs 1 and 3 depended on the simulated bias. In real data analysis using Design 3, external controls were integrated in 84% of cross-validation folds, resulting in an estimated risk difference of –1.53%-points (95% CI –2.75%-points to –0.30%-points).

The Causal Roadmap helps investigators to minimize potential bias in studies using RWD and to quantify tradeoffs between study designs. The simulation results help to interpret the level of evidence provided by the real data analysis in support of the superiority of oral semaglutide versus standard-of-care for cardiovascular risk reduction.

Real-world data, such as administrative claims and electronic health records, are increasingly used for safety monitoring and to help guide regulatory decision-making. In these settings, it is important to document analytic decisions transparently and objectively to assess and ensure that analyses meet their intended goals.

The Causal Roadmap is an established framework that can guide and document analytic decisions through each step of the analytic pipeline, which will help investigators generate high-quality real-world evidence.

In this paper, we illustrate the utility of the Causal Roadmap using two case studies previously led by workgroups sponsored by the Sentinel Initiative – a program for actively monitoring the safety of regulated medical products. Each case example focuses on different aspects of the analytic pipeline for drug safety monitoring. The first case study shows how the Causal Roadmap encourages transparency, reproducibility, and objective decision-making for causal analyses. The second case study highlights how this framework can guide analytic decisions beyond inference on causal parameters, improving outcome ascertainment in clinical phenotyping.

These examples provide a structured framework for implementing the Causal Roadmap in safety surveillance and guide transparent, reproducible, and objective analysis.

Ideally, real-world data (RWD) collected to generate real-world evidence (RWE) should lead to impact on the care and health of real-world patients. Deriving from care in which clinicians and patients try various treatments to inform therapeutic decisions, N-of-1 trials bring scientific methods to real-world practice.

These single-patient crossover trials generate RWD and RWE by giving individual patients various treatments in a double-blinded way in sequential periods to determine the most effective treatment for a given patient.

This approach is most often used for patients with chronic, relatively stable conditions that provide the opportunity to make comparisons over multiple treatment periods, termed Type 1 N-of-1 trials. These are most helpful when there is heterogeneity of treatment effects among patients and no a priori best option. N-of-1 trials also can be done for patients with rare diseases, potentially testing only one treatment, to generate evidence for personalized treatment decisions, designated as Type 2 N-of-1 trials. With both types, in addition to informing individual’s treatments, when uniform protocols are used for multiple patients with the same condition, the data collected in the individual N-of-1 trials can be aggregated to provide RWD/RWE to inform more general use of the treatments. Thereby, N-of-1 trials can provide RWE for the care of individuals and for populations.

To fulfill this potential, we believe N-of-1 trials should be built into our current healthcare ecosystem. To this end, we are building the needed infrastructure and engaging the stakeholders who should receive value from this approach.

Clinical trials provide the “gold standard” evidence for advancing the practice of medicine, even as they evolve to integrate real-world data sources. Modern clinical trials are increasingly incorporating real-world data sources – data not intended for research and often collected in free-living contexts. We refer to trials that incorporate real-world data sources as real-world trials. Such trials may have the potential to enhance the generalizability of findings, facilitate pragmatic study designs, and evaluate real-world effectiveness. However, key differences in the design, conduct, and implementation of real-world vs traditional trials have ramifications in data management that can threaten their desired rigor.

Three examples of real-world trials that leverage different types of data sources – wearables, medical devices, and electronic health records are described. Key insights applicable to all three trials in their relationship to Data and Safety Monitoring Boards (DSMBs) are derived.

Insight and recommendations are given on four topic areas: A. Charge of the DSMB; B. Composition of the DSMB; C. Pre-launch Activities; and D. Post-launch Activities. We recommend stronger and additional focus on data integrity.

Clinical trials can benefit from incorporating real-world data sources, potentially increasing the generalizability of findings and overall trial scale and efficiency. The data, however, present a level of informatic complexity that relies heavily on a robust data science infrastructure. The nature of monitoring the data and safety must evolve to adapt to new trial scenarios to protect the rigor of clinical trials.

Real-world data privacy is a complex yet underexplored topic. To date, few studies have reported adult perspectives around real-world data privacy and willingness to share real-world data with researchers.

Relevant survey items were identified in the literature, adapted and pilot tested among a small convenience sample, and finalized for distribution. The survey was distributed electronically in April 2021 among adults (≥18 years of age) registered in ResearchMatch (www.researchmatch.org). Microsoft Excel was used to assess descriptive statistics across demographical items and four privacy-related items.

Of 402 completed responses received, half of respondents (∼50%) expressed willingness to share their prescription history data and music streaming data with researchers and unwillingness to share real-world data from several other sources. Most (53–93%) of participants expressed concern with five statements reflecting the sharing and use of their digital data online. Most participants (71–75%) agreed with four statements focused on individual measures taken to protect their personal privacy and disagreed (77–85%) with two statements centered on not being concerned about sharing or 3rd party access to their personal data online.

Our observations indicate an important yet unmet need to further explore and address real-world data privacy concerns among US adults engaging as prospective research participants.

Noninterventional naturalistic studies are an important complement to randomized controlled trials. Aripiprazole once-monthly (AOM) is an atypical antipsychotic in a long-acting injectable formulation.

A pooled analysis of two noninterventional studies was undertaken to validate previous results on AOM effectiveness and safety in a larger population and improve statistical power for preplanned subgroup analyses. We analyzed data from 409 patients with schizophrenia who were treated with AOM and were enrolled in noninterventional studies in Germany (via noninterventional studies registry 15,960 N) and Canada (NCT02131415). Data collected at baseline, 3 and 6 months were analyzed. Among the endpoints were psychopathology (brief psychiatric rating scale [BPRS]) and disease severity (clinical global impression [CGI]).

Mean patient age was 38.9 (SD 14.8) years, and 59.9% were male. BPRS decreased from 48.1 (SD 15.6) at baseline to 36.5 (SD 13.7) at month 6 (p < 0.001). CGI decreased from 4.47 (SD 0.90) at baseline to 3.64 (SD 1.16) at month 6 (p < 0.001). A total of 54.4% were responders (at least 20% reduction) on the BPRS, and 56.5% had a CGI-S-score that was at least 1 level better than baseline. A total of 43.4% were considered responders on both the BPRS and CGI scales. A total of 45.2% were considered in remission. Adverse events were rare and corresponded to the previously known safety profile of AOM.

Treatment with AOM for patients with schizophrenia appeared effective and safe under real-life conditions.

The Royal Australian and New Zealand College of Psychiatrists clinical practice guidelines for mood disorders (MDcpg2015 and MDcpg2020) provide evidence-based and consensus-based recommendations for managing mood disorders.

We examined Australian real-world prescribing habits to determine whether management in clinical practice aligned with MDcpg2015 recommendations.

A retrospective analysis of a cohort of patients ≥16 years old who had been dispensed a Pharmaceutical Benefits Scheme (PBS)-listed antidepressant between July 2013 and June 2019 was conducted using Australian Commonwealth Department of Human Services PBS 10% sample data.

Between July 2013 and June 2019, 239 944 patients in Australia commenced antidepressant treatment. Of these, 22% (52 694 patients) received a second treatment (a new class of treatment after a period of discontinuation or additional antipsychotic therapy) and 6% (15 741 patients) received a third treatment. Patients were initially prescribed primarily selective serotonin reuptake inhibitors (SSRIs; 52% of prescriptions) or tricyclic antidepressants (TCAs; 25%), even though TCAs are not recommended for first-line treatment. Fewer than one-quarter of patients were prescribed serotonin–noradrenaline reuptake inhibitors (13%) or other agents (10%). General practitioners (GPs) were more likely to initiate TCAs than psychiatrists (22% v. 7%).

Once initiated, the overall median time patients remained on treatment was 4.5 months; this was highest with SSRIs (5.8 months) and lowest with TCAs (0.9 months).

First-line prescribing broadly follows guidelines. GP and psychiatrist prescribing patterns differ, perhaps reflecting different patient groups and the need to tailor treatment to individuals. Future guidelines should aim to capture the different presentations and complexity of depression.

Technological and computational advancements offer new tools for the collection and analysis of real-world data (RWD). Considering the substantial effort and resources devoted to collecting RWD, a greater return would be achieved if real-world evidence (RWE) was effectively used to support Health Technology Assessment (HTA) and decision making on medical technologies. A useful question is: To what extent are RWD suitable for generating RWE?

We mapped existing RWD sources in Europe for three case studies: hip and knee arthroplasty, transcatheter aortic valve implantation (TAVI) and mitral valve repair (TMVR), and robotic surgery procedures. We provided a comprehensive assessment of their content and appropriateness for conducting the HTA of medical devices. The identification of RWD sources was performed combining a systematic search on PubMed with gray literature scoping, covering fifteen European countries.

We identified seventy-one RWD sources on arthroplasties; ninety-five on TAVI and TMVR; and seventy-seven on robotic procedures. The number, content, and integrity of the sources varied dramatically across countries. Most sources included at least one health outcome (97.5%), with mortality and rehospitalization/reoperation the most common; 80% of sources included resource outcomes, with length of stay the most common, and comparators were available in almost 70% of sources.

RWD sources bear the potential for the HTA of medical devices. The main challenges are data accessibility, a lack of standardization of health and economic outcomes, and inadequate comparators. These findings are crucial to enabling the incorporation of RWD into decision making and represent a readily available tool for getting acquainted with existing information sources.

This study aims to assess stakeholder perceptions on the challenges and value of real-world evidence (RWE) post approval, the differences in regulatory and health technology assessment (HTA) real-world data (RWD) collection requirements under the German regulation for more safety in drug supply (GSAV), and future alignment opportunities to create a complementary framework for postapproval RWE requirements.

Eleven semistructured interviews were conducted purposively with pharmaceutical industry experts, regulatory authorities, health technology assessment bodies (HTAbs), and academia. The interview questions focused on the role of RWE post approval, the added value and challenges of RWE, the most important requirements for RWD collection, experience with registries as a source of RWD, perceptions on the GSAV law, RWE requirements in other countries, and the differences between regulatory and HTA requirements and alignment opportunities. The interviews were recorded, transcribed, and translated for coding in Nvivo to summarize the findings.

All experts agree that RWE could close evidence gaps by showing the actual value of medicines in patients under real-world conditions. However, experts acknowledged certain challenges such as: (i) heterogeneous perspectives and differences in outcome measures for RWE generation and (ii) missing practical experience with RWD collected through mandatory registries within the German benefit assessment due to an unclear implementation of the GSAV.

This study revealed that all stakeholder groups recognize the added value of RWE but experience conflicting demands for RWD collection. Harmonizing requirements can be achieved through common postlicensing evidence generation (PLEG) plans and joint scientific advice to address uncertainties regarding evidence needs and to optimize drug development.

Health Canada is committed to the modernization of the use of real-world data (RWD) and evidence (RWE) to support regulatory decisions. As such, telephone interviews with stakeholders, including government decision makers, health technology assessment (HTA) producers, industry, and patients, to understand their experiences with and perspectives on how to enhance RWE use for medical devices were performed.

Thirty-four semi-structured telephone interviews with forty key informants were conducted. Transcripts were reviewed independently by one individual to identify, define, and categorize key concepts and were verified by a second reviewer.

There are expectations for Health Canada to provide a framework and guidance on RWE use, identify relevant outcomes for data collection and criteria for data quality, conduct post-market surveillance more systematically, and partner with HTA organizations to develop methods for RWE generation. Stakeholders interviewed support the RWE use for regulatory decisions and HTA recommendations. Moreover, robust scientific methods for RWE generation will be critical to ensure that relevant questions are asked and rigorous statistical analyses are done to answer them. Patients are likely to consent to share their anonymized or de-identified medical information for nonprofit purposes.

Key concepts from the interviews centered on the current and future RWE use for medical devices, considerations for the organizational, medical, scientific, and legal aspects and privacy issues of RWD collection or RWE generation, and options to implement the use of RWD and RWE. Our study findings will help inform the development of an RWE framework for regulatory decisions and HTA recommendations.