Although classical randomized clinical trials (RCTs) are the gold standard for proof of drug efficacy, randomized discontinuation trials (RDTs), sometimes called “enriched” trials, are used increasingly, especially in psychiatric maintenance studies.

A narrative review of two decades of experience with RDTs.

RDTs in psychiatric maintenance trials tend to use a dependent variable as a predictor: treatment response. Treatment responders are assessed for treatment response. This tautology in the logic of RDTs renders them invalid, since the predictor and the outcome are the same variable. Although RDTs can be designed to avoid this tautologous state of affairs, like using independent predictors of outcomes, such is not the case with psychiatric maintenance studies

Further, purported benefits of RDTs regarding feasibility were found to be questionable. Specifically, RDTs do not enhance statistical power in many settings, and, because of high dropout rates, produce results of questionable validity. Any claimed benefits come with notably reduced generalizability.

RDTs appear to be scientifically invalid as used in psychiatric maintenance designs. Their purported feasibility benefits are not seen in actual trials for psychotropic drugs. There is warrant for changes in federal policy regarding marketing indications for maintenance efficacy using the RDT design.

Personalized medicine has exposed wearable sensors as new sources of biomedical data which are expected to accrue annual data storage costs of approximately $7.2 trillion by 2020 (>2000 exabytes). To improve the usability of wearable devices in healthcare, it is necessary to determine the minimum amount of data needed for accurate health assessment.

Here, we present a generalizable optimization framework for determining the minimum necessary sampling rate for wearable sensors and apply our method to determine optimal optical blood volume pulse sampling rate. We implement t-tests, Bland–Altman analysis, and regression-based visualizations to identify optimal sampling rates of wrist-worn optical sensors.

We determine the optimal sampling rate of wrist-worn optical sensors for heart rate and heart rate variability monitoring to be 21–64 Hz, depending on the metric.

Determining the optimal sampling rate allows us to compress biomedical data and reduce storage needs and financial costs. We have used optical heart rate sensors as a case study for the connection between data volumes and resource requirements to develop methodology for determining the optimal sampling rate for clinical relevance that minimizes resource utilization. This methodology is extensible to other wearable sensors.

Human-centered design (HCD) training offers the potential to improve both team processes and products. However, the use of HCD to improve the quality of team science is a relatively recent application, and its benefits and challenges have not been rigorously evaluated. We conducted a qualitative study with health sciences researchers trained in HCD methods. We aimed to determine how researchers applied HCD methods and perceived the benefits and barriers to using HCD on research teams.

We conducted 1-hour, semi-structured interviews with trainees from three training cohorts. Interviews focused on perceptions of the training, subsequent uses of HCD, barriers and facilitators, and perceptions of the utility of HCD to science teams. Data analysis was conducted using Braun and Clarke’s process for thematic analysis.

We interviewed nine faculty and nine staff trained in HCD methods and identified four themes encompassing HCD use, benefits, challenges, and tensions between HCD approaches and academic culture.

Trainees found HCD relevant to research teams for stakeholder engagement, research design, project planning, meeting facilitation, and team management. They also described benefits of HCD in five distinct areas: creativity, egalitarianism, structure, efficiency, and visibility. Our data suggest that HCD has the potential to help researchers work more inclusively and collaboratively on interdisciplinary teams and generate more innovative and impactful science. The application of HCD methods is not without challenges; however, we believe these challenges can be overcome with institutional investment.

Identification of evidence-based factors related to status of the clinical research professional (CRP) workforce at academic medical centers (AMCs) will provide context for National Center for Advancing Translational Science (NCATS) policy considerations and guidance. The objective of this study is to explore barriers and opportunities related to the recruitment and retention of the CRP workforce.

Qualitative data from a series of Un-Meeting breakout sessions and open-text survey questions were analyzed to explore barriers and recommendations for improving AMC CRP recruitment, retention and diversity.

While certain institutions have established competency-based frameworks for job descriptions, standardization remains generally lacking across CTSAs. AMCs report substantial increases in unfilled CRP positions leading to operational instability. Data confirmed an urgent need for closing gaps in CRP workforce at AMCs, especially for attracting, training, retaining, and diversifying qualified personnel. Improved collaboration with human resource departments, engagement with principal investigators, and overcoming both organizational and resource challenges were suggested strategies, as well as development of outreach to universities, community colleges, and high schools raising awareness of CRP career pathways.

Based on input from 130 CRP leaders at 35 CTSAs, four National Institute of General Medical Sciences’ Institutional Development Award (IDeA) program sites, along with industry and government representatives, we identified several barriers to successful recruitment and retention of a highly trained and diverse CRP workforce. Results, including securing institutional support, champions, standardizing and adopting proven national models, improving local institutional policies to facilitate CRP hiring and job progression point to potential solutions.

Many clinical trials leverage real-world data. Typically, these data are manually abstracted from electronic health records (EHRs) and entered into electronic case report forms (CRFs), a time and labor-intensive process that is also error-prone and may miss information. Automated transfer of data from EHRs to eCRFs has the potential to reduce data abstraction and entry burden as well as improve data quality and safety.

We conducted a test of automated EHR-to-CRF data transfer for 40 participants in a clinical trial of hospitalized COVID-19 patients. We determined which coordinator-entered data could be automated from the EHR (coverage), and the frequency with which the values from the automated EHR feed and values entered by study personnel for the actual study matched exactly (concordance).

The automated EHR feed populated 10,081/11,952 (84%) coordinator-completed values. For fields where both the automation and study personnel provided data, the values matched exactly 89% of the time. Highest concordance was for daily lab results (94%), which also required the most personnel resources (30 minutes per participant). In a detailed analysis of 196 instances where personnel and automation entered values differed, both a study coordinator and a data analyst agreed that 152 (78%) instances were a result of data entry error.

An automated EHR feed has the potential to significantly decrease study personnel effort while improving the accuracy of CRF data.

Advancing understanding of human health promotion and disease prevention and treatment often requires teamwork. To evaluate academic medical institutions’ support for team science in the context of researchers’ career development, we measured the value placed on team science and specificity of guidance provided for documenting team science contributions in the promotion and tenure (P&T) documents of Colleges/Schools of Medicine (CoMs) in the National Center for Advancing Translational Sciences’ Clinical and Translational Science Award (CTSA) program.

We reviewed complete P&T documents from 57 of 63 CTSA CoMs to identify career paths defined by three dimensions: academic rank (associate versus full professor), tenure eligibility (tenure track versus not), and role (research, clinical, education, and administrative), and we rated team science value and documentation guidance for each path. Multilevel models were estimated to compare team science value and documentation guidance as a function of the three career path dimensions while accounting for the clustered data (N = 357 career paths within 57 CoMs).

Team science value was greater for associate than full professors, non-tenure-eligible versus tenure-eligible positions, and roles prioritizing clinical, education, and administrative responsibilities versus those prioritizing research. Guidance for documenting team science achievements was more explicit for roles that prioritized research.

Although P&T policies at most CTSA institutions express value for team science, inconsistent within-institutional patterns of recognition and reward across career paths may have implications for researchers’ involvement in team science. We discuss the implications of our findings for research and for P&T policies that promote team science.

In response to a call issued by the National Research Council to investigate the knowledge, skills, and attitudes of effective science teams, we designed a team training program for conducting science in collaborative contexts.

We reviewed the literature to develop an evidence-based competency model for effective science teams along with exemplary behaviors that can be used for founding team training and evaluation. We discuss the progress of teamwork and team development research that serves as a foundation for this work, as well as previous research involving team-based competencies.

Three overarching competencies emerged from the literature as key for science team effectiveness: psychological safety, awareness and exchange, and self-correction and adaptation. These competencies are fully described, including their evidence base.

We developed a competency model and implementation plan for a team training program specific to science teams – TeamMAPPS (Team Methods to Advance Processes and Performance in Science). This paper details steps in the implementation process, including plans for consortia dissemination, evaluation, and future development.

Implementation scientists increasingly recognize that the process of implementation is dynamic, leading to ad hoc modifications that may challenge fidelity in protocol-driven interventions. However, limited attention to ad hoc modifications impairs investigators’ ability to develop evidence-based hypotheses about how such modifications may impact intervention effectiveness and cost. We propose a multi-method process map methodology to facilitate the systematic data collection necessary to characterize ad hoc modifications that may impact primary intervention outcomes.

We employ process maps (drawn from systems science), as well as focus groups and semi-structured interviews (drawn from social sciences) to investigate ad hoc modifications. Focus groups are conducted with the protocol’s developers and/or planners (the implementation team) to characterize the protocol “as envisioned,” while interviews conducted with frontline administrators characterize the process “as realized in practice.” Process maps with both samples are used to identify when modifications occurred across a protocol-driven intervention. A case study investigating a multistage screening protocol for autism spectrum disorders (ASD) is presented to illustrate application and utility of the multi-method process maps.

In this case study, frontline administrators reported ad hoc modifications that potentially influenced the primary study outcome (e.g., time to ASD diagnosis). Ad hoc modifications occurred to accommodate (1) whether providers and/or parents were concerned about ASD, (2) perceptions of parental readiness to discuss ASD, and (3) perceptions of family service delivery needs and priorities.

Investigation of ad hoc modifications on primary outcomes offers new opportunities to develop empirically based adaptive interventions. Routine reporting standards are critical to provide full transparency when studying ad hoc modifications.

Trainees and investigators from underrepresented minority (URM) backgrounds face unique challenges to establishing successful careers in clinical and translational research. Structured training for mentors is an important mechanism to increase the diversity of the research workforce. This article presents data from an evaluation of the University of California, San Francisco (UCSF) Center for AIDS Research (CFAR) Mentoring the Mentors program aimed at improving mentors’ competency in working with diverse mentees in HIV research.

Mentors from around the USA who had in one of seven separate 2-day training workshops conducted from 2013 to 2020 were invited to participate in an online evaluation survey of their experiences with the training and their subsequent mentoring activities.

There was a high response rate (80%) among the 226 mentors invited to complete the survey. The 180 respondents were diverse in demographics, professional disciplines, and geographic distribution. Quantitative and qualitative data indicate a lasting positive impact of the training, with sustained improvements documented on a validated measure of self-appraised mentoring competency. Respondents also endorsed high interest in future, follow-up training with continued focus on topics related to mentoring in the context of diversity.

The evaluation of the UCSF CFAR Mentoring the Mentors program showed lasting impact in improving mentoring practices, coupled with high interest in continued in-depth training in areas focused on diversity, equity, and inclusion.

Rigor and reproducibility are two important cornerstones of medical and scientific advancement. Clinical and translational research (CTR) contains four phases (T1–T4), involving the translation of basic research to humans, then to clinical settings, practice, and the population, with the ultimate goal of improving public health. Here we provide a framework for rigorous and reproducible CTR.

In this paper we define CTR, provide general and phase-specific recommendations for improving quality and reproducibility of CTR with emphases on study design, data collection and management, analyses and reporting. We present and discuss aspects of rigor and reproducibility following published examples of CTR from the literature, including one example that shows the development path of different treatments that address anaplastic lymphoma kinase-positive (ALK+) non-small cell lung cancer (NSCLC).

It is particularly important to consider robust and unbiased experimental design and methodology for analysis and interpretation for clinical translation studies to ensure reproducibility before taking the next translational step. There are both commonality and differences along the clinical translation research phases in terms of research focuses and considerations regarding study design, implementation, and data analysis approaches.

Sound scientific practices, starting with rigorous study design, transparency, and team efforts can greatly enhance CTR. Investigators from multidisciplinary teams should work along the spectrum of CTR phases, and identify optimal practices for study design, data collection, data analysis, and results reporting to allow timely advances in the relevant field of research.