Hostname: page-component-cd9895bd7-gbm5v Total loading time: 0 Render date: 2024-12-25T20:25:52.316Z Has data issue: false hasContentIssue false

The Recruitment Innovation Center: Developing novel, person-centered strategies for clinical trial recruitment and retention

Published online by Cambridge University Press:  19 August 2021

Consuelo H. Wilkins
Affiliation:
Vanderbilt Institute for Clinical and Translational Research, Nashville, TN, USA Department of Medicine, Vanderbilt University Medical Center, Nashville, TN, USA Department of Internal Medicine, Meharry Medical College, Nashville, TN, USA Office of Health Equity, Vanderbilt University Medical Center, Nashville, TN, USA
Terri L. Edwards
Affiliation:
Vanderbilt Institute for Clinical and Translational Research, Nashville, TN, USA
Mary Stroud
Affiliation:
Vanderbilt Institute for Clinical and Translational Research, Nashville, TN, USA
Nan Kennedy
Affiliation:
Vanderbilt Institute for Clinical and Translational Research, Nashville, TN, USA
Rebecca N. Jerome
Affiliation:
Vanderbilt Institute for Clinical and Translational Research, Nashville, TN, USA
Colleen E. Lawrence
Affiliation:
Vanderbilt Institute for Clinical and Translational Research, Nashville, TN, USA
Sheila V. Kusnoor
Affiliation:
Center for Knowledge Management, Strategy and Innovation, Vanderbilt University Medical Center, Nashville, TN 37203 USA
Sarah Nelson
Affiliation:
Vanderbilt Institute for Clinical and Translational Research, Nashville, TN, USA
Loretta M. Byrne
Affiliation:
Vanderbilt Institute for Clinical and Translational Research, Nashville, TN, USA
Leslie R. Boone
Affiliation:
Vanderbilt Institute for Clinical and Translational Research, Nashville, TN, USA
Julia Dunagan
Affiliation:
Vanderbilt Institute for Clinical and Translational Research, Nashville, TN, USA
Tiffany Israel
Affiliation:
Vanderbilt Institute for Clinical and Translational Research, Nashville, TN, USA
Casey Rodweller
Affiliation:
Vanderbilt Institute for Clinical and Translational Research, Nashville, TN, USA
Bethany Drury
Affiliation:
Vanderbilt Institute for Clinical and Translational Research, Nashville, TN, USA
Rhonda G. Kost
Affiliation:
Center for Clinical and Translational Science, The Rockefeller University, New York, NY, USA
Jill M. Pulley
Affiliation:
Vanderbilt Institute for Clinical and Translational Research, Nashville, TN, USA
Gordon R. Bernard
Affiliation:
Vanderbilt Institute for Clinical and Translational Research, Nashville, TN, USA Department of Medicine, Vanderbilt University Medical Center, Nashville, TN, USA
Paul A. Harris*
Affiliation:
Vanderbilt Institute for Clinical and Translational Research, Nashville, TN, USA Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, USA
*
Address for correspondence: P. Harris, PhD, Vanderbilt Institute for Clinical and Translational Research, Vanderbilt University Medical Center, 2525 West End Avenue, Nashville, TN 37203, USA. Email: [email protected]
Rights & Permissions [Opens in a new window]

Abstract

Clinical trials continue to face significant challenges in participant recruitment and retention. The Recruitment Innovation Center (RIC), part of the Trial Innovation Network (TIN), has been funded by the National Center for Advancing Translational Sciences of the National Institutes of Health to develop innovative strategies and technologies to enhance participant engagement in all stages of multicenter clinical trials. In collaboration with investigator teams and liaisons at Clinical and Translational Science Award institutions, the RIC is charged with the mission to design, field-test, and refine novel resources in the context of individual clinical trials. These innovations are disseminated via newsletters, publications, a virtual toolbox on the TIN website, and RIC-hosted collaboration webinars. The RIC has designed, implemented, and promised customized recruitment support for 173 studies across many diverse disease areas. This support has incorporated site feasibility assessments, community input sessions, recruitment materials recommendations, social media campaigns, and an array of study-specific suggestions. The RIC’s goal is to evaluate the efficacy of these resources and provide access to all investigating teams, so that more trials can be completed on time, within budget, with diverse participation, and with enough accrual to power statistical analyses and make substantive contributions to the advancement of healthcare.

Type
Special Communications
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
© The Author(s), 2021. Published by Cambridge University Press on behalf of The Association for Clinical and Translational Science

Introduction

Difficulty in recruiting and retaining participants, long understood as a key research challenge, continues to plague the clinical trial landscape. Estimates suggest that nearly one in five studies either terminate for failed accrual or finish with underpowered numbers [Reference Carlisle, Kimmelman, Ramsay and MacKinnon1]. These issues not only add expense to research, but can also compromise generalizability and representativeness, raise ethical concerns, and defer the discovery of answers to significant health issues [Reference Huang, Bull and McKee2].

In recent years, increased attention to the adverse impact of recruitment shortfalls has led to identification of numerous contributing issues [Reference Boden-Albala, Carman and Southwick3Reference Lovato, Hill, Hertert, Hunninghake and Probstfield7]. NIH’s National Center for Advancing Translational Sciences (NCATS) launched the Trial Innovation Network (TIN) in 2016 to create efficiencies, streamline processes, and discover and share solutions to common problems facing multicenter trial investigators. The TIN collaborative initiative includes three Trial Innovation Centers (TICs), one Recruitment Innovation Center (RIC), and the Clinical and Translational Science Awards (CTSA) Program Hubs [8,Reference Shah, Culp and Gersing9]. Specific aims of the RIC are to:

  1. 1. Provide a national home and collaborative ‘storefront’ for the creation, storing, and sharing of recruitment education, programs, and best practices.

  2. 2. Catalyze enrollment by developing and disseminating novel technical and procedural approaches to support researchers in recruiting participants.

  3. 3. Enhance national awareness of research through patient education, and facilitate participant identification of studies with novel online patient facing tools.

  4. 4. Conduct rigorous studies on methods to enhance recruitment efficacy/efficiency and make modifications based on these data.

The RIC Transdisciplinary Team

Participant recruitment has evolved into a specialized field of knowledge requiring expertise from multiple stakeholders, including trialists, patients, front-line recruitment staff, advocacy groups, and community members. To achieve its aims, the RIC has assembled a transdisciplinary team of experts in diverse fields (Fig. 1). By tapping into the multiple perspectives and insights of this team, the RIC is empowered to design informatics-driven and community-engaged recruitment approaches across the continuum of research. With this holistic approach, the RIC develops custom strategies to optimize recruitment and retention for each study. The RIC team refines these potential solutions, disseminates them to the broader research community, then reconvenes in an iterative process to improve methods and explore new ground for innovation. This report describes the RIC’s unique team approach to resource design and delivery; details RIC resources; reports on progress and research team satisfaction metrics; and delineates plans to evaluate the effectiveness of its resources in improving clinical trial recruitment and retention.

Fig. 1. RIC transdisciplinary team of experts generates innovations in clinical trial recruitment and retention.

RIC, Recruitment Innovation Center; IRB, Institutional Review Board; EHR, electronic health records.

CTSA Engagement and the RIC Model for Design of Resources

The RIC’s resources are conceptualized and developed in parallel with trial teams to provide generalizable solutions to common recruitment challenges [Reference Huang, Bull and McKee2]. Typically, innovations are driven by and designed for a specific trial, and the resulting resources are tailored to the policies, guidelines, and cultures at participating CTSA Hubs. This accommodation of variability is a core principle of the RIC—every resource is designed to support generalizability while respecting local context. The local CTSA Points of Contact, who report to CTSA Principal Investigators (PIs) and Directors, are invited to participate in all consultations, enabling RIC consultants to determine what the local CTSA team can already provide to the PI.

Input from colleagues across the CTSA Network informed the RIC’s structure and resource offerings. In 2016 (Year 1 of the grant funding cycle), the RIC conducted a survey of CTSA PIs, Directors, and Recruitment Specialists at 53 responding institutions. Their ratings of the usefulness of forecasted RIC resources supported our choices for resource development (Supplemental Table 1). Sites also expressed a desire for the RIC to work directly with trialists to provide guidance on improving recruitment. Areas of recruitment cited as requiring extra support are listed in Supplemental Table 2.

Ongoing input from CTSAs is continuously collected through additional channels. Regularly scheduled TIN Collaboration Webinars enable CTSA Hub leaders and other team members to share their expertise, methods, and strategies in recruitment and retention. Monthly Open Forums and CTSA Liaison Team Meetings invite Network Project Leads and PIs to present on topics of importance to CTSA Hubs and provide an opportunity to share information and ask questions. The RIC also created and actively manages a Recruitment & Retention Working Group communications platform to support rapid online exchange of ideas, messaging, and discussions. This forum is used regularly by CTSA-wide experts to discuss current recruitment and retention topics.

The RIC’s Community Advisory Board (CAB), established in September 2017, also contributes to the creation, refinement, and evaluation of RIC-generated recruitment and engagement methods, as well as to the development of innovative practices. The CAB’s 12 members represent various geographic regions and diverse communities across the USA. They provide regular feedback that fosters a stronger, more productive relationship between the community and the research enterprise.

Additional rationale for developing RIC resources, as well as underlying hypotheses, resource allocation details, resource usage, and planned evaluation are documented in the RIC metrics and evaluation section.

The Continuum of Research Recruitment Efforts

A guiding goal of the RIC is to shift the perception of recruitment as a one-time activity to that of an ongoing, collaborative effort across the continuum of research, from promoting awareness of research opportunities through dissemination of results and continued relationship-building (Fig. 2). The RIC focuses on building a spectrum of resources to support and integrate each component of this continuum.

Fig. 2. Recruitment Innovation Center continuum of participant recruitment efforts.

General Awareness

Promotion of clinical trials increases awareness of opportunities for research participation related to an individual’s personal needs. The RIC has worked to increase clinical trial awareness nationally with Trials Today [Reference Pulley, Jerome and Bernard10], a novel public-facing interface that helps patients and families search for relevant clinical trials. Launched in 2015 and partially supported by the RIC, the site resides within the ResearchMatch volunteer platform (described later) and uses ClinicalTrials.gov data. To promote this tool, the RIC launched a national digital and print campaign in 2017 [Reference Jerome, Dunkel and Kennedy11]. Integration between Trials Today and ResearchMatch allows volunteers to view research opportunities on a personalized ResearchMatch volunteer dashboard.

In addition, the Trials Today ‘Local’ platform allows partner institutions to display to the public the studies affiliated with their institution on a custom-branded, centrally hosted website (projecttrialstoday.org/). Trials Today Local also powers the National Covid Cohort Collaborative (N3C) clinical trial listing service [12].

Feasibility

A major reason studies fail to meet recruitment goals is an inaccurate estimate of the number of eligible participants. To improve feasibility exploration prior to undertaking a study, the RIC developed a set of informatics resources, including cross-institutional Electronic Health Record (EHR) Cohort Assessments to determine counts of individuals with a specific diagnosis or comorbidity. The RIC has operationalized a low-tech, low-burden approach to cohort identification that has proven both feasible and adaptable across diverse types of studies and institutions, depicted and described in Supplemental Figure 1.

Because prospective sites may already be conducting trials with similar participant profiles and inclusion/exclusion criteria, the RIC has developed a Competing Trial Tool, again based on data retrieved daily from ClinicalTrials.gov. The user interface for this tool allows RIC consultants to rapidly search conditions, eligibility criteria, and expected enrollment periods for a specific trial to generate a list of known trials that might be co-enrolling at CTSA Hubs. This information is then factored into site feasibility assessment and selection. The tool exports data into a structured report that summarizes the trial, enrollment period, recruitment goals, and investigator contact information. The Competing Trial Tool is currently accessible by RIC personnel and is deployed in the context of informing specific trials during RIC consultations. Planning is underway to test and refine a public-facing tool based on experiential feedback obtained from consultations.

Another integral aspect of study feasibility is a team-based assessment of the risks, cost, time, human resources, and communications required to successfully recruit and retain participants. Often a PI does not give sufficient attention to the ways in which a study’s recruitment and retention could be impacted by these variables. Recognizing an unmet need articulated by CTSAs in our initial needs survey and substantiated in initial consultations with trialists, the RIC developed a Recruitment and Retention Plan (RRP) template to help investigators fully evaluate recruitment factors prior to grant submission. The template is comprehensive, bringing to light logistical, motivational, and behavioral barriers that could affect clinical trial recruitment and retention, and breaking them down to simplify an otherwise daunting task. In practice, completing this template can broaden the perspective of study teams, while enabling RIC consultants to generate more robust recommendations. Once complete, the RRP serves as a blueprint in determining which RIC resources could help support the study. The RIC has promulgated wide usage of this template [13], and offers assistance to PIs in completing it. The RRP can be used to meet NIH protocol guidelines, which now require an RRP.

In addition, as part of our overall comprehensive RRP, the RIC has developed a recruitment Feasibility Assessment model that may reduce barriers to clinical trial participation among diverse and rare populations. Additional components of a recruitment feasibility assessment include participant eligibility criteria such as age, gender, race, ethnicity, and health status; and potential study site environmental strengths and weaknesses such as location, competition, prior success in recruiting, and potential participant pool.

Opportunity

The concept of opportunity refers to the participant’s ability to enroll in a specific trial. In a clinical trial consultation, the RIC guides the PI through an assessment of potential barriers that may reduce a participant’s opportunity to enroll, such as lack of clinician knowledge of the trial or willingness to refer patients to the study. The RIC suggests methods, tailored to the trial, to help minimize these obstacles.

Elements in the design of a trial, often not apparent to investigators, may unintentionally impede an individual’s opportunity to participate. Through the use of Community Engagement (CE) Studios, developed in 2009 by the Meharry-Vanderbilt Community-Engaged Research Core [Reference Joosten, Israel and Williams14], the RIC facilitates evaluation of design elements for each trial by bringing together the study PI and multiple stakeholders (including community members) to consider the participant’s perspective when deciding such aspects as the number, duration, and frequency of study visits; study procedures; and appropriate compensation.

ResearchMatch, a disease-neutral, online service that matches study investigators with volunteers looking for trials in which to participate, is another method for increasing research participation opportunities. The platform was launched in 2009 and is funded by NIH through CTSA and RIC grants. The RIC recently supported a language translation of the ResearchMatch platform to broaden its ability to engage Spanish speakers [Reference Byrne, Cook and Kennedy15].

Screening

Participant screening is the process by which an individual is deemed eligible to join a particular study. The RIC has created a Clinical Systems Optimization (CSO) resource line to work directly with local study teams and their recruitment sites to better understand site-specific recruitment workflows, information technology (IT) capabilities, and study protocol, enabling them to provide recommendations to enhance screening efforts through EHR platforms. The CSO resource can provide investigator teams with pre-award grant language detailing how the EHR can identify eligible patients for screening and align EHR-based queries with recruitment workflows. The resource also includes documentation and guidance for leveraging bulk messaging capabilities within EHR systems to streamline the distribution of recruitment materials to potentially eligible patients. One example includes using patient portals (e.g., Epic’s MyChart) to deploy electronic bulk messaging to reach individuals potentially meeting study eligibility criteria. Retrieving patient data for research purposes requires significant policy and procedure development and implementation related to a range of issues, including regulatory requirements, ethics, participant burden, and privacy. The RIC offers guidance on using EHR workflow tools for recruitment of a targeted sample of an institution’s available population. Also provided are instructions for use, templates for patient-portal based recruitment messaging, and feedback as needed.

Invitation to Participate

The invitation to participate in a clinical trial can be delivered via clinicians, study staff, word of mouth, mass media, social media, or printed recruitment materials such as posters, flyers, and brochures. Clinicians and study staff participating in in-person recruitment efforts need a clear understanding of the study, including eligibility criteria, study purpose, participation requirements, and potential risks and benefits. Print and electronic materials can extend a specific invitation from clinical staff to potential participants and, when appropriate, family members. The RIC provides templates for designing information sheets that detail a study for clinicians, and for designing brochures and flyers that explain requirements to potential participants. All messages and printed materials are evaluated for plain language usage, literacy level, and readability. Our continuing target is to lower the reading level of participant recruitment materials to a 6th-to-7th grade level average to accommodate potential participants with lower literacy.

Because clinician referrals are so critical to a study’s enrollment success, the RIC has developed a Referring Providers Outreach Guide, available for download in the TIN Toolbox, that provides extensive guidance to study teams in addressing provider questions and concerns about referring patients to clinical trials. The RIC has also created a Clinician Study App (CSA) template, a customizable digital resource for clinicians and coordinators to access study information (primarily on cellphones and tablets) such as inclusion/exclusion criteria, study videos, guides, and graphics (Supplemental Figure 2). The CSA allows clinicians, with a single click, to connect via email or phone call with a local study coordinator. Similarly, the RIC developed a customizable Participant Study App (PSA) template to enable rapid study information exchange with descriptive study content tailored to potential research participants. Quick Response (QR) codes can be automatically generated and printed on flyers or other print or electronic media to support smartphone utilization of PSAs. This feature allows potential participants to discreetly access study information on their phones, and also provides ‘touch-free’ access to this information during a pandemic such as COVID-19.

Traditional recruitment materials can sometimes be less effective with underrepresented groups, including minorities, who often encounter a greater array of obstacles to clinical trial participation than the non-minority population [Reference Denicoff, McCaskill-Stevens and Grubbs16]. Lack of diverse participation in research can result in limited generalizability of findings and, ultimately, to health disparities [Reference Dirks, Huth, Yates and Meibohm17Reference Schaefer, Caracciolo, Frishman and Charney19]. The RIC provides support to investigators for developing specially designed recruitment strategies and materials for underrepresented groups to reduce barriers to participation. The RIC created Faster Together, Enhancing the Recruitment of Minorities in Clinical Trials [20], a free online training program available worldwide that teaches methods to foster recruitment and retention of racially and ethnically diverse participants in clinical research studies. The course has been shown to improve attendees’ knowledge of the material, and learners have indicated that they intend to make changes to their recruitment and retention practices as a result of their course participation [Reference Kusnoor, Villalta-Gil and Michaels21].

Consent

Almost one in ten clinical trial participants finds the informed consent form difficult to understand [22]. Providing researchers with strategies to maximize communication and understanding during the consent process is another key RIC resource. STRIDE (Strengthening Translational Research in Diverse Enrollment) is a CTSA partnership with Vanderbilt University Medical Center, the University of Massachusetts Medical School, the University of Alabama at Birmingham, and the Community Campus Partnerships for Health, initiated to assist researchers in making the consent process more appropriate for participants from culturally diverse or low-literacy backgrounds [23]. STRIDE’s development of a 21-CFR Part 11 compliant eConsent platform within REDCap (Research Electronic Data Capture) [Reference Harris, Taylor, Thielke, Payne, Gonzalez and Conde24,Reference Harris, Taylor and Minor25] was a milestone in the consenting process [Reference Lawrence, Chiriboga and Goins26]. The RIC has played a pivotal role in refining and disseminating this work by creating an information sheet and best practices document for the consenting process, currently available online in the TIN Toolbox [13]. Additional information on eConsent is found in the Supplemental Materials.

Engagement and Retention

Systematic efforts toward retention of enrolled participants are critical to achieving timely completion of a study and collecting sufficient data to power the appropriate statistical analyses. Participants can and do withdraw from a study for myriad reasons, some of which are unavoidable, such as an inconvenient location, lack of perceived improvement in their medical condition, or side effects. Some participants, however, leave a study because of difficulty with scheduling, feeling unappreciated, or through simple forgetfulness [27].

The RIC’s development of retention resources is assisted by feedback from patients and community representatives who are willing to be embedded in study teams, take surveys, act as user-testers, participate in Community Engagement Studios, or serve on advisory panels. One emerging informatics-based engagement tool is the MyCap platform, which was developed initially by our Vanderbilt REDCap team to support patient reported outcome (PRO) data from research participants in support of longitudinal studies [28] In addition to data capture, researchers can now connect to participants through the native mobile application (iOS and Android) to schedule tasks such as surveys or assessments, share announcements, and send secure messages and reminders. MyCap thus fosters enhanced communication and a general sense of connectedness and ownership among participants. Recent RIC-sponsored development work for MyCap includes language abstraction for non-English research participants (Spanish, French, and Nepali) based on local COVID-19 related use cases.

Return of Value

A key motivator for participating in a research study is the opportunity to receive individual results, whether or not these results are clinically relevant [Reference Shalowitz and Miller29,Reference Kaufman, Baker, Milner, Devaney and Hudson30]. Sharing results helps researchers to demonstrate the ethical principle of respect for persons [31]. The RIC has developed and disseminated extensive guidelines, templates, and examples to assist researchers in sharing study results with research participants [13].

The RIC also conducted a national survey among likely research participants [Reference Wilkins, Mapes, Jerome, Villalta-Gill, Pulley and Harris32] and found that respondents highly value results showing the impact of their personal genetic profile on likely medication response and prediction of disease risk. In addition, they often desire information about nearby clinical trials and updates on how their data are being used. The RIC is using findings from subpopulations in this study to aid researchers in identifying opportunities to deepen participants’ experience of the value yielded by their study participation.

Building and Maintaining Trust

A major issue affecting the entire recruitment continuum is a participant’s level of trust or mistrust in researchers and institutions. Woven through many of the RIC resources is a focus on building and maintaining trust [Reference Wilkins, Mapes, Jerome, Villalta-Gill, Pulley and Harris32,Reference Wilkins33], which can contribute to a positive relationship with research that could and should last a lifetime.

While the completion of data analysis and dissemination of results signal the technical end of a study, participants who perceive value in their contribution and who trust the researchers they have encountered will likely want to participate in future studies. Former participants can continue to interact with ResearchMatch and Trials Today for future research needs. Researchers too can continue to build trust and maintain connections with former participants by sending out newsletters or staying in contact with local cultural or disease communities.

RIC Results

Consultations

For each study approved for support, the RIC provides a consultation with our recruitment experts. The RIC has developed multiple resources and templates that it continues to refine, while developing new ones to meet the needs of clinical trial investigators nationwide. Any given study might receive value from the RIC across a spectrum of resources: at one end, the RIC may provide an investigator with a brief consultation that includes study-specific recommendations, templates, and guidelines that have been previously built and field-tested. Mid-spectrum, the RIC may offer a mature resource line offering or innovation in need of development or refinement. At the far end, the RIC may become deeply involved in developing a needs-based recruitment or retention solution, built around a new idea, that supports the specific trial while potentially generating a solution that is generalizable to future studies. In addition, resources such as ResearchMatch and eConsent may be used by researchers on an ad hoc basis, outside of the formal consultation structure.

The RIC has completed 173 initial recruitment consultations, 164 of which were conducted for CTSA institutions. RIC resource support was offered for 107 of the consultations, while the remainder, 66, were provided with custom recruitment and retention recommendations only. Supplemental Table 3 shows the diversity of disease areas under study. The length of a RIC consultation may vary considerably (from 2 weeks to several years), based on individual study needs and joint interests in innovation or resource service line testing.

Milestones and Achievements

The RIC, currently in Year 5 of the grant cycle, has achieved or made significant progress on all milestones proposed in our grant submission, as defined by NCATS’ strategy to develop, demonstrate, and disseminate translational research innovations [34] (Table 1).

Table 1. RIC milestone achievements

RIC, Recruitment Innovation Center; TIN, Trial Innovation Network; EHR, electronic health records; CTSA, Clinical and Translational Science Awards; PSA, Participant Study App; CDSHooks, Clinical Decision Support Hooks.

* Data from October 2016 through February 2021, except where noted.

** Data from February 5, 2021 to June 28, 2021.

*** Data from March 1, 2019 to June 28, 2021.

Dissemination of Recruitment Resources

A primary aim of the RIC is to disseminate not only those recruitment innovations the RIC team has designed, field-tested, and formalized, but also those developed by the broader clinical trial community. To this end, the RIC has designed and implemented these resources: the TIN website [41], which houses the TIN Toolbox, a showcase of recruitment and retention tools and resources gathered from across the CTSA community; a Collaboration Webinar series to share recruitment expertise, methods, and best practices in study recruitment; and the ‘RIC Download,’ a bi-weekly bulletin of study recruitment news and publications that highlights the latest approaches to recruitment and best practices from across the TIN.

RIC Vignettes: Increased Study Enrollment

The RIC, in collaboration with the Project LUNA team, helped accelerate recruitment for a randomized controlled trial evaluating the efficacy of three smoking cessation strategies in long-term smokers eligible for low-dose CT lung cancer screening. The study was awarded the Recruitment Materials resource line, which consisted of a short pilot social media campaign aimed at increasing study recruitment. During the campaign, 23 participants enrolled as a direct result of the social media messaging, compared to 11 through the study’s standard recruitment channels. The initial consultation for this study spanned about 3 months, from the introductory call through submission of the recommendations report. The social media design and implementation required two months, and the manuscript describing the project was written and accepted for publication in a 20-month timeframe [Reference Cook, Jerome and Dunagan40].

Another example of accelerated study recruitment facilitated by the RIC was the TARGET-RA (Rheumatoid Arthritis) study. This study was challenged with complex eligibility criteria as well as participant burden and at the time of the consult had enrolled 23 participants. The RIC provided an in-depth consultation and a myriad of resources, some of which were piloted with this study, including the CSA, Competing Trial Tool, revamped recruitment brochures, and a social media campaign. The study was eventually able to meet their revised enrollment goal of 160 in just over 24 months, and the study team has since requested RIC involvement for a new grant to employ these same types of strategies.

Additional examples of RIC consultation recommendations found helpful by investigators are contained in the Supplemental Materials.

RIC Metrics and Evaluation

The RIC has identified metrics for evaluating research team satisfaction as well as outcomes associated with consultation and service line resource sharing efforts. The RIC regularly measures PI satisfaction with the study-specific recruitment and retention recommendations provided in our initial consultations, and has obtained consistently high ratings (Table 2). Likewise, all researchers participating in RIC Community Engagement Studios have found them worth their time and agree that the studios improved their projects. In addition, attendees of the Collaboration Webinars report high levels of satisfaction with them.

Table 2. Preliminary data on satisfaction with RIC consultations and resources

RIC, Recruitment Innovation Center; TIN, Trial Innovation Network.

a Data available from 2019 through 2020.

b Data from October 2016 through July 23, 2021.

The RIC has recently begun conducting in-depth, semi-structured quality assurance interviews with PIs receiving RIC resources for their studies, to learn about their use of the resources and any barriers or facilitators to usage they have encountered. Preliminary results suggest that lack of institutional capacity and leadership buy-in may be barriers to RIC resource usage, while perceived utility, ease of use, and potential for speedy dissemination can facilitate usage. The RIC is currently conducting interviews with PIs and study coordinators of RIC-led consultations that received resources and will be obtaining study enrollment tables to gain a larger picture of enrollment trends after implementation of RIC resources.

In addition to these qualitative interviews, Table 3 includes evaluation methods for each of the RIC’s five resource service lines (see also Fig. 1) using metrics specific to that resource. Collection of some metrics has begun, while others are in the planning stages. An overarching Evaluation Plan for the RIC program was developed using a Causal Pathway approach [Reference Grembowski42] and employing logic models to describe the anticipated relationship between RIC activities and their ultimate impacts. In a causal pathway, the models specifically articulate the inputs (resources) and outputs (deliverables), and the early, intermediate, and later anticipated impacts of the initiatives, each with associated performance measures. The RIC developed a Logic Model for each of the Specific Aims of the grant for internal use as a roadmap to determine what aspects of the team’s work to evaluate. The Causal Pathway and Logic Model are described more fully in the Supplemental Materials, and an example is provided. While the models sometimes overlap—innovation developed in one aim may feed activities of another aim—they eventually converge on the common goal of the RIC and TIN: to accelerate the completion of clinical trials with representative enrollment. As the Evaluation Plan is further refined, the RIC will continue to gather a range of outcomes data to establish best practices and will report on results.

Table 3. Development of standard RIC resources, usage, and plans for evaluation

CTSA, Clinical and Translational Science Awards; EHR, Electronic Health Records; NIH, National Institutes of Health; VUMC, Vanderbilt University Medical Center; CE, Community Engagement; CAB, Community Advisory Board; FTE, Full-Time Effort; CSA, Clinician Study App; PSA, Participant Study App; PI, Principal Investigator; QR, Quick Response.

* Data from October 2016 through February 2021. Note that while all resource lines were operational from the start, individual elements within those lines may have been developed over time.

Limitations

A limitation of the RIC model is that studies may encounter recruitment and retention challenges unique to localities or specific populations, especially those underrepresented in research. The RIC attempts to address these challenges by applying generalizable principles while also obtaining local feedback for culturally appropriate messaging, engaging local community groups for referrals, and customizing recruitment and retention planning and recommendations for each study and targeted population.

Next Steps and Future Plans

The RIC’s suite of resources is continually evolving as more information is gained about recruitment efficacy and best practices and as new opportunities are identified. The RIC is currently developing and testing a novel expert advice feedback mechanism that can solicit opinions of ResearchMatch volunteers on a study’s proposed protocol, recruitment/engagement plan, and desired return of value. The RIC team is also developing and piloting measures of trust and person-centeredness. Future plans for the RIC include the activation of participant compensation guidelines and preparations to evaluate their effectiveness.

The RIC is in the process of adding social media engagement and recruitment website generation to our suite of recruitment resources to increase clinical trial awareness. The RIC is also developing educational materials, including videos, to train clinicians and study staff in best practices and techniques for obtaining informed consent.

The RIC will continue to innovate in the area of informatics platforms and applied methods in support of participant recruitment and engagement. Specifically, our informatics team will refine and improve methods of creating and socializing electronic phenotyping algorithms for use across diverse health systems, and has plans to advance methods for developing and deploying workflows at the intersection of research and EHR systems. New mandates from the Office of the National Coordinator for Health Information Technology (ONC) for EHR adoption of HL7 Fast Healthcare Interoperability Resources (FHIR) standard adherence are enabling systems to receive, process, and utilize clinical data in real-time [43]. The RIC is developing new models and methods for use of these data in both REDCap and ResearchMatch platforms for use as recruitment and retention tools. Work will continue to evolve in building accessible eConsent modules and participant-facing applications (e.g., Study Applications, MyCap).

The RIC intends to continue its work in developing sustainable models for grant-supported resources, services lines, and informatics tools to ensure maximal dissemination and continuity in serving the clinical trial community.

Conclusion

The RIC is our nation’s only federally funded center charged with catalyzing enrollment in clinical trials. To address the complex recruitment and retention challenges affecting our national trial infrastructure, the RIC has created a patient-centered suite of recruitment and engagement resources to accelerate trial completion. The RIC’s overall innovation and implementation strategy scales to support the greatest number of studies with the smallest investment and allows local sites to leverage their own capabilities to take advantage of these resources. Innovations move through a life cycle of novelty, testing, and evaluation until each one becomes the new standard or best practice. The RIC’s unique team approach continues to draw on diverse expertise to build, test, and disseminate new resources that can support recruitment on all NIH-funded studies, including those outside of the TIN. Our ongoing challenge is to keep raising the bar.

Supplementary material

To view supplementary material for this article, please visit https://doi.org/10.1017/cts.2021.841.

Acknowledgements

The authors extend their sincere thanks to Stephanie Mayers, Sarah Cook, Leah Dunkel, Meredith O’Brien, Michelle Jones, Bridgett Swindell, and the entire Recruitment Innovation Center team, as well as the members of the RIC Community Advisory Board. We thank the Trial Innovation Centers (Utah University; Johns Hopkins University with Tufts University; and Duke University with Vanderbilt University Medical Center) for their efforts in supporting the Recruitment Innovation Center’s activities. We also thank Meghan Vance for her assistance in designing figures for the paper. This project was supported by awards no. UL1 TR002243 from the National Center for Advancing Translational Sciences (NCATS), the Vanderbilt Institute for Clinical and Translational Research grant no. UL1 TR000445 from NCATS/NIH, The Rockefeller University Center for Clinical and Translational Science award UL1TR001866 from NCATS/NIH, and The Recruitment Innovation Center – RIC (U24TR001579) from NCATS. Its contents are solely the responsibility of the authors and do not necessarily represent official views of NCATS or NIH.

Disclosures

The authors have no conflicts of interest to declare.

References

Carlisle, B, Kimmelman, J, Ramsay, T, MacKinnon, N. Unsuccessful trial accrual and human subjects protections: an empirical analysis of recently closed trials. Clinical Trials: Journal of the Society for Clinical Trials 2015; 12: 7783. doi: 10.1177/1740774514558307. CrossRefGoogle ScholarPubMed
Huang, GD, Bull, J, McKee, KJ, et al. Clinical trials recruitment planning: a proposed framework from the clinical trials transformation initiative. Contemporary Clinical Trials 2018; 66: 7479. doi: 10.1016/j.cct.2018.01.003. CrossRefGoogle ScholarPubMed
Boden-Albala, B, Carman, H, Southwick, L, et al. Examining barriers and practices to recruitment and retention in stroke clinical trials. Stroke 2015; 46: 22322237. doi: 10.1161/STROKEAHA.114.008564. CrossRefGoogle ScholarPubMed
Epstein, S. The rise of ‘recruitmentology’: clinical research, racial knowledge, and the politics of inclusion and difference. Social Studies of Science 2008; 38: 801832. doi: 10.1177/0306312708091930. CrossRefGoogle Scholar
Gren, L, Broski, K, Childs, J, et al. Recruitment methods employed in the prostate, lung, colorectal, and ovarian cancer screening trial. Clinical Trials 2009; 6: 5259. doi: 10.1177/1740774508100974. CrossRefGoogle ScholarPubMed
Harris, KJ, Ahluwalia, JS, Catley, D, Okuyemi, KS, Mayo, MS, Resnicow, K. Successful recruitment of minorities into clinical trials: The Kick It at Swope project. Nicotine & Tobacco Research 2003; 5: 575584. doi: 10.1080/1462220031000118540. CrossRefGoogle ScholarPubMed
Lovato, LC, Hill, K, Hertert, S, Hunninghake, DB, Probstfield, JL. Recruitment for controlled clinical trials: literature summary and annotated bibliography. Controlled Clinical Trials 1997; 18: 328352.CrossRefGoogle ScholarPubMed
National Center for Advancing Translational Sciences. New funding opportunities for trial and recruitment innovation centers . (https://ncats.nih.gov/news/releases/2015/ctsa-ric-tic-funding)Google Scholar
Shah, M, Culp, MA, Gersing, KR, et al. Early vision for the CTSA program trial innovation network: a perspective from the National Center for Advancing Translational Sciences. Clinical and Translational Science 2017; 10: 311313. doi: 10.1111/cts.12463. CrossRefGoogle ScholarPubMed
Pulley, JM, Jerome, RN, Bernard, GR, et al. Connecting the public with clinical trial options: The ResearchMatch Trials Today tool. Journal of Clinical and Translational Science 2018; 2: 253257. doi: 10.1017/cts.2018.327.CrossRefGoogle ScholarPubMed
Jerome, RN, Dunkel, L, Kennedy, N, et al. To end disease tomorrow, begin with trials today: digital strategies for increased awareness of a clinical trials finder. Journal of Clinical and Translational Science 2019; 3(4): 190198. doi: 10.1017/cts.2019.404. CrossRefGoogle ScholarPubMed
Joosten, YA, Israel, TL, Williams, NA, et al. Community engagement studios: a structured approach to obtaining meaningful input from stakeholders to inform research. Academic Medicine 2015; 90: 16461650. doi: 10.1097/ACM.0000000000000794. CrossRefGoogle ScholarPubMed
Byrne, LM, Cook, SK, Kennedy, N, et al. Opening doors to clinical trial participation among Hispanics: lessons learned from the Spanish translation of ResearchMatch. Journal of Clinical and Translational Science 2021; 5(1): e46. doi: 10.1017/cts.2020.539. Google Scholar
Denicoff, AM, McCaskill-Stevens, W, Grubbs, SS, et al. The National Cancer Institute-American Society of Clinical Oncology Cancer Trial Accrual Symposium: summary and recommendations. Journal of Oncology Practice 2013; 9: 267276. doi: 10.1200/JOP.2013.001119. CrossRefGoogle ScholarPubMed
Dirks, NL, Huth, B, Yates, CR, Meibohm, B. Pharmacokinetics of immunosuppressants: a perspective on ethnic differences. International Journal of Clinical Pharmacology and Therapeutics 2004; 42: 701718. doi: 10.5414/cpp42701. CrossRefGoogle ScholarPubMed
Phan, VH, Moore, MM, McLachlan, AJ, Piquette-Miller, M, Xu, H, Clarke, SJ. Ethnic differences in drug metabolism and toxicity from chemotherapy. Expert Opinion on Drug Metabolism & Toxicology 2009; 5: 243257. doi: 10.1517/17425250902800153. CrossRefGoogle ScholarPubMed
Schaefer, BM, Caracciolo, V, Frishman, WH, Charney, P. Gender, ethnicity and genetics in cardiovascular disease: part 1: basic principles. Heart Disease 2003; 5: 129143.CrossRefGoogle ScholarPubMed
Faster Together. Enhancing the recruitment of minorities in clinical trials. Coursera. (https://www.coursera.org/learn/recruitment-minorities-clinical-trials)Google Scholar
Kusnoor, SV, Villalta-Gil, V, Michaels, M, et al. Design and implementation of a massive open online course on enhancing the recruitment of minorities in clinical trials – Faster Together. BMC Medical Research Methodology 2021; 21: 44. doi: 10.1186/s12874-021-01240-x. CrossRefGoogle ScholarPubMed
Charts & Statistics. Center for Information & Study on Clinical Research Participation. (https://www.ciscrp.org/education-center/charts-statistics/)Google Scholar
STRIDE (Strengthening Translational Research in Diverse Enrollment). (https://strideproject.org/)Google Scholar
Harris, PA, Taylor, R, Thielke, R, Payne, J, Gonzalez, N, Conde, JG. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. Journal of Biomedical Informatics 2009; 42: 377381. doi: 10.1016/j.jbi.2008.08.010. CrossRefGoogle ScholarPubMed
Harris, PA, Taylor, R, Minor, BL, et al. The REDCap consortium: building an international community of software platform partners. Journal of Biomendical Informatics 2019; 95: 103208. doi: 10.1016/j.jbi.2019.103208. CrossRefGoogle ScholarPubMed
Lawrence, CE, Chiriboga, G, Goins, KV, et al. A REDCap-based model for electronic consent (eConsent): moving toward a more personalized consent. Journal of Clinical and Translational Science 2020; 4(4): 345353. doi: 10.1017/cts.2020.30. CrossRefGoogle Scholar
MyCap – Mobilizing the Participant Voice. (https://projectmycap.org/)Google Scholar
Shalowitz, DI, Miller, FG. Disclosing individual results of clinical research: implications of respect for participants. JAMA 2005; 294: 737740. doi: 10.1001/jama.294.6.737. CrossRefGoogle ScholarPubMed
Kaufman, DJ, Baker, R, Milner, LC, Devaney, S, Hudson, KL. A survey of U.S adults’ opinions about conduct of a nationwide Precision Medicine Initiative® cohort study of genes and environment. PLoS One 2016; 11: e0160461. doi: 10.1371/journal.pone.0160461. CrossRefGoogle ScholarPubMed
National Academies of Sciences, Engineering, and Medicine, et al. Principles for the Return of Individual Research Results: Ethical and Societal Considerations. Washington, DC: National Academies Press (US), 2018.Google Scholar
Wilkins, CH, Mapes, BM, Jerome, RN, Villalta-Gill, V, Pulley, JM, Harris, PA. Understanding what information is valued by research participants, and why. Health Affairs 2019; 38: 399407. doi: 10.1377/hlthaff.2018.05046. CrossRefGoogle Scholar
Wilkins, CH. Effective engagement requires trust and being trustworthy. Medical Care 2018; 56(Suppl 1): S6S8. doi: 10.1097/MLR.0000000000000953. CrossRefGoogle ScholarPubMed
National Center for Advancing Translational Sciences. Strategic plan introduction. (https://ncats.nih.gov/strategicplan/introduction)Google Scholar
Kusnoor, SV, Villalta-Gill, V, Michaels, M, et al. Design and implementation of a massive open online course on enhancing the recruitment of minorities in clinical trials – Faster Together. BMC Medical Research Methodology 2021; 21: 44. doi: 10.1186/s12874-021-01240-x. CrossRefGoogle ScholarPubMed
Helmer, TT, Lewis, AA, McEver, M, et al. Creating and implementing a COVID-19 recruitment Data Mart. Journal of Biomedical Informatics 2021; 117: 103765. doi: 10.1016/j.jbi.2021.103765. CrossRefGoogle ScholarPubMed
Cook, S, Mayers, S, Goggins, K, et al. Assessing research participant preferences for receiving study results. Journal of Clinical and Translational Science 2019; 4: 243249. doi: 10.1017/cts.2019.427. CrossRefGoogle ScholarPubMed
Wilkins, CH, Mapes, BM, Jerome, RN, Villalta-Gill, V, Pulley, JM, Harris, PA. Understanding what information is valued by research participants, and why. Health Affairs 2019; 38: 399407. doi: 10.1377/hlthaff.2018.05046. CrossRefGoogle Scholar
Marquis-Gravel, G, Robertson, H, Jones, WS, et al. Streamlining the institutional review board process in pragmatic randomized clinical trials: challenges and lessons learned from the aspirin dosing: a patient-centric trial assessing benefits and long-term effectiveness (ADAPTABLE) trial. Trials 2021; 22: 90. doi: 10.1186/s13063-021-05026-w. CrossRefGoogle ScholarPubMed
Cook, SK, Jerome, RN, Dunagan, J, et al. Engaging smokers in research: utility of Facebook in facilitating recruitment to a smoking cessation study. Contemporary Clinical Trials 2021; 107: 106461. doi: 10.1016/j.cct.2021.106461. CrossRefGoogle ScholarPubMed
Grembowski, D. The Practice of Health Program Evaluation. Thousand Oaks, CA: SAGE Publications, 2015.Google Scholar
H.R.34 - 114th Congress (2015–2016): 21st Century Cures Act | Congress.gov | Library of Congress. (https://www.congress.gov/bill/114th-congress/house-bill/34)Google Scholar
Figure 0

Fig. 1. RIC transdisciplinary team of experts generates innovations in clinical trial recruitment and retention.RIC, Recruitment Innovation Center; IRB, Institutional Review Board; EHR, electronic health records.

Figure 1

Fig. 2. Recruitment Innovation Center continuum of participant recruitment efforts.

Figure 2

Table 1. RIC milestone achievements

Figure 3

Table 2. Preliminary data on satisfaction with RIC consultations and resources

Figure 4

Table 3. Development of standard RIC resources, usage, and plans for evaluation

Supplementary material: File

Wilkins et al. supplementary material

Wilkins et al. supplementary material

Download Wilkins et al. supplementary material(File)
File 588.4 KB