Advancing diversity, equity, and inclusion within clinical and translational science training programs: A qualitative content analysis of the training breakout session at the national CTSA program meeting

Abstract

Background:

Diversity, equity, and inclusion (DEI) in clinical and translational science (CTS) are paramount to driving innovation and increasing health equity. One important area for improving diversity is among trainees in CTS programs. This paper reports on findings from a special session at the November 2020 Clinical and Translational Science Award (CTSA) national program meeting that focused on advancing diversity and inclusion within CTS training programs.

Methods:

Using qualitative content analysis, we identified approaches brought forth to increase DEI in KL2 career development and other training programs aimed at early-stage CTS investigators, beyond the six strategies put forth to guide the breakout session (prioritizing representation, building partnerships, making it personal, designing program structure, improving through feedback, and winning endorsement). We used an inductive qualitative content analysis approach to identify themes from a transcript of the panel of KL2 program leaders centered on DEI in training programs.

Results:

We identified four themes for advancing DEI within CTS training programs: 1) institutional buy-in; 2) proactive recruitment efforts; 3) an equitable application process; and 4) high-quality, diverse mentorship.

Conclusion:

Implementing these strategies in CTS and other training programs will be an important step for advancing DEI. However, processes need to be established to evaluate the implementation and effectiveness of these strategies through continuous quality improvement, a key component of the CTSA program. Training programs within the CTSA are well-positioned to be leaders in this critical effort to increase the diversity of the scientific workforce.

Introduction

A diverse medical and research workforce has been shown to drive innovation and improve access to high-quality, culturally competent patient-centered care for racial and ethnic minorities [1,2]. Further, diversity in academic medicine and clinical and translational science (CTS) may broaden the medical research agenda [3], allow for more diverse clinical trial recruitment [4], support institutional excellence [5,6], and enhance education [4]. As such, diversity, equity, and inclusion (DEI) in CTS are paramount to driving science forward and increasing health equity [7]. One important area for improving diversity is among trainees in CTS programs [8].

All National Institutes of Health (NIH) Clinical and Translational Science Award (CTSA)-funded hubs have KL2 career development programs (soon to be called K12) to train early career scientists in CTS [9]. Many hubs also have TL1 programs (soon to be called T32) aimed at providing CTS training for predoctoral and postdoctoral trainees [9]. While increasing diversity among KL2 scholars has been a goal across CTSA programs nationwide, a 2016 survey of KL2 programs found that only 12% of KL2 scholars came from underrepresented groups [10]. In recent years, all CTSA hubs participated in collecting common metrics, including the number and percent of KL2 and TL1 graduates who are engaged in CTS and who are from underrepresented groups. According to the multi-year common metrics report, in 2019, the median percent of KL2 scholars from underrepresented groups engaged in CTS was 15% across 57 hubs (range: 0%–100%) [9]. Similarly, the median percent of TL1 graduates from underrepresented groups engaged in CTS was 19% across 48 hubs (range: 0%–100%) [9].

As part of the common metrics initiative, the Center for Leading Innovation & Collaboration (CLIC) has recognized hubs with the year-over-year greatest increases in diversity among their group of KL2 scholars and TL1 trainees. CLIC conducted 19 interviews with leaders from those 18 hubs to better understand the strategies used to successfully increase diversity [9]. Through qualitative analysis of interviews and submitted common metrics narratives, CLIC identified in their Insights to Inspire series six main strategies used by hubs to advance DEI in CTS training programs: prioritizing representation, building partnerships, making it personal, designing program structure, improving through feedback, and winning endorsement [9]. In November 2020, at the national CTSA program meeting, one session focused on advancing diversity and inclusion within CTSA training programs, with speakers organized around these six strategies, followed by a question-and-answer session with attendees. During the question-and-answer session and throughout the panel discussion, participants shared many useful strategies beyond the original six identified by CLIC. Using qualitative methods, the aim of this analysis was to identify these additional approaches brought forth at the meeting to increase DEI in KL2 career development and other training programs aimed at early-stage CTS investigators.

Materials and Methods

Due to COVID-19 precautions, the November 2020 national CTSA program meeting was held virtually. Forty-four individuals participated in the breakout session focused on advancing diversity and inclusion in CTS training programs, including program leaders, investigators, administrators, and other stakeholders. The session consisted of a panel discussion from KL2 leaders from four CTSA hubs that were identified as being among those with the greatest increase in diversity among the KL2 scholars and was organized around the six strategies identified by CLIC, followed by a question-and-answer session and open discussion among the full group. The 90-minute session was held virtually via Zoom, and the question-and-answer section was a synchronous, live discussion among the attendees and panel presenters. The session was audio-recorded and transcribed via Zoom. Session attendees were made aware that the session was being recorded and that the data would be aggregated and used to inform DEI initiatives.

To analyze the transcript from the session, we used an inductive qualitative content analysis approach [11,12]. Inductive approaches derive categories from the data and are recommended when moving from specific to more general with the overarching goal of categorically assigning to a larger whole [12]. The goal of this analysis was to identify themes that were unique from the six organizing strategies listed above. For our analysis, three coders independently examined the transcript by using inductive coding (i.e., guided by the text without predetermined codes) following the recommended phases for inductive content analysis: 1) preparing; 2) organizing; and 3) reporting [12]. To organize the data, first, open coding was completed to generate initial categories. After open coding, categories were created by grouping, followed by abstraction, which allowed for main categories to be identified. The coders discussed the codes and categories to ensure consensus on main categorical themes identified. While the discussion was led by KL2 leaders, session participants discussed strategies for improving DEI across the spectrum of CTS training programs. We have thus left the language broader (CTS training programs) throughout unless it was clear that a specific quote or idea solely related to KL2 programs, in which case we indicated as such.

Results

In addition to the six strategies put forth to guide the breakout session, we identified four themes for advancing DEI: 1) institutional buy-in; 2) proactive recruitment efforts; 3) an equitable application process; and 4) high-quality, diverse mentorship (Fig. 1). For each of these four themes, we present illustrative verbatim quotes from the breakout session.

Fig. 1. Diversity, equity, and inclusion strategies.

Institutional Buy-In

First, session presenters and participants reported that institutional buy-in and support are key to the success of diversity efforts. Institution-wide support for diversity efforts provides the scaffolding for increasing diversity within training programs. As one session participant explained:

“The support from the institution is really key. We talk about diversity as a pragmatic priority and an institutionally supportive priority at the beginning of all of our recruitments […] for scholars […], but also with students and faculty who are part of those programs, we also host a wide variety of perspectives. This is important to encourage; it’s an important component of emphasizing diversity.”

KL2 and other training program leaders viewed themselves as having an important role as diversity advocates. Their role, therefore, should be to promote diversity both through recruitment efforts and through providing trainees with diverse role models. One presenter described this role as follows:

“I think role modeling is really important. And so, we're all successful investigators, ourselves. Two of us [program leaders] […] are [from] underrepresented groups […] within our program; because we have been successful with increasing the number of racial and ethnic minorities, they then serve as role models.”

Another program leader described another path to advancing program diversity through succession planning. By building up and institutionally supporting underrepresented trainees and scholars through career development and training opportunities, they can eventually take on leadership roles within their institution, including in training programs; in becoming role models for future scholars; and in further advancing institutional support for diversity over time.

Proactive Recruitment Efforts

Session participants also discussed the importance of proactive recruitment efforts in increasing diversity. A common refrain was that KL2 scholars, and other trainees can be only as diverse as the pool of applicants. In order to increase applicant diversity, program leaders must proactively recruit these potential scholars. This begins with broadening outreach to a wider variety of departments and specialties. Traditionally, recruitment for KL2 and other similar programs has focused on biomedical sciences and medical schools. However, junior faculty from less traditional scholar applicant pools, such as social work, public health, allied health sciences, and nursing may be more diverse, not only by gender and race/ethnicity but also by experience and discipline. For example, one participant said,

“[We needed to] extend our program beyond focus[ing] mainly on preclinical undergraduates. We went to the school of social work. We extended our relationships with the school of public health because they have […] undergraduates as well as large postdoc populations. We went to the nursing school. And so, we had to broaden our view of a training program.”

Participants and presenters described potential “on-ramps” or long-term pathways for increasing the diversity of the workforces via the applicant pool by targeting high school students, community college students, undergraduate and graduate students, and postdoctoral fellows. Additionally, partnering with or recruiting from minority-serving institutions, such as Historically Black Colleges and Universities (HBCUs) and Hispanic Serving Institutions (HSIs) can diversify the on-ramp of applicants. One participant remarked,

“It makes a huge difference in terms of recruitment retention and the ability to actually engage the community… for most of us who are in areas where there are historically black colleges and universities. And so even thinking more broadly about who are our partners outside of the institution, that has really been a great source of collaboration and also a potential opportunity to encourage folks to enter the kinds of careers that we know will ultimately be important as we try to increase diversity.”

Equitable Application Process

Presenters and participants spent much of the session discussing the importance of having an equitable application process in order to promote diversity among scholars and trainees. Some potential areas for hidden inequities in the application process include the following: 1) disparities in the amount of support and experience applicants have in preparing such applications; 2) application review criteria and reviewer instructions that may favor applicants from majority groups; and 3) possible disparities in the application review process itself.

First, some underrepresented persons may have less support and experience with preparing their application. One presenter described the problem as follows:

“There are sources of variation that intentionally or more often unintentionally disadvantage some groups. And so, thinking about the application process, really beginning to think about equity – application preparation even across our institutions – there may be a different access to application prep programs or other resources that allow for pre-submission review.”

Some potential solutions regarding unequal application preparation include providing preparation programs or helping to match applicants with mentors who can help support them through the application process. One participant suggested:

“… creating an application prep program for those unable to do that. There are often programs across institutions. So, are there real opportunities to partner with other career development programs or other entities to provide for scholars some assistance with preparing applications […]?”

Second, the traditional review criteria for KL2 and other training programs likely favor majority groups. Traditional measures such as publications [13] and prior grant history [14,15] can be biased against underrepresented groups and may not show the “full story” of the applicant, including where they have come from and what they have achieved. One participant suggested,

“… really thinking about opportunities for folks to highlight their strengths. That can be in the application process. Some programs actually also will include interviews. And what are the markers of the potential for success? We often think about things like previous grants or papers. But are there really other ways that people can show where they're going in ways that are important for consideration in the selection process?”

Further, session participants thought the application review process may lead to inequities. Some suggestions for a more equitable review process include making sure review criteria are clear and transparent, both for the applicants and for the reviewer panel.

“This may certainly be their first time preparing an application for submission, a formal application for submission to a grant and being clear upfront about what our review criteria are as they are preparing those applications.”

Also important is to ensure that there is a diverse reviewer panel, including by gender, race/ethnicity, and fields of research.

“There needs to be an effort to make sure that [the] KL2 review committee is very diverse […] in terms of gender, in terms of race, and also in terms of discipline. Our scores in science are biased – depending on my background, what I'm interested in, I'm discussing differently – but when you create […] a diverse, equitable group, you actually create opportunity to balance out the scores.”

Session participants also shared that they felt that many underrepresented applicants prefer health services and public health research (T3-T4 translational research) to preclinical and lab-based work (T0-T2 translational research) [16], so it is important to ensure that the reviewer panel has the expertise and ability to review in these areas to promote scholar and trainee diversity.

“So really thinking about diversity in our committee members, how are we valuing research across the translational spectrum, ensuring that all of our scholars, no matter what they're studying really have an opportunity to advance their science and many of these awards and many of the review committees will include at least some members who are internal to the institution.”

Another potential solution is to consider interviewing all underrepresented applicants, regardless of their initial application review score. Several program leaders reported that prioritizing interviews with this group has allowed them to see the full candidate and their strengths rather than relying only on reviewer scores.

Mentorship

Fourth, session presenters and participants discussed the importance of high-quality mentorship for the success of diverse scholars and trainees. Faculty members who belong to racial and ethnic minority groups are often underrepresented in academic medicine as compared with scholars and trainees [17]. This, together with the “minority tax”, that is postdoctoral, additional, financially uncompensated responsibilities such as mentoring and committee service that minorities are asked to perform in efforts to increase diversity at their institutions [18], means that 1) a focus on increasing diversity among senior faculty is critical, 2) mentoring activities, especially in regard to increasing diversity, should be incentivized, and 3) underrepresented scholars and trainees may need to be mentored by people who are not of their same race/ethnicity, provided that additional mentor training to better serve underrepresented scholars is available and utilized. One participant stated,

“Mentorship training may be something that our training programs can partner with or sponsor. There are a number of programs. One is through the National Research Mentoring [Program], which has some focus on mentorship for underrepresented groups.”

Additionally, creating peer mentors and mentorship teams/committees can help ensure that underrepresented groups receive support and that scholar needs are being met. KL2 program alumni can serve as ambassadors, role models, or informal mentors to future and current scholars. One participant stated,

“Building an inclusive community, having been a KL2 scholar myself, one of the things that I enjoy is being part of a community that extends beyond my time in the KL2 program. This is really a wonderful opportunity to really expand some of the things that we know are important, like peer mentorship and role modeling that can happen among scholars as they learn from each other.”

Another way to increase diversity in mentorship for all scholars and trainees is for programs to engage the broader community beyond academic medicine. Further, community partners can provide critical feedback to the program. One program leader stated,

“One idea is to have a diversity advocate [and] community partners […] to give us feedback on our program. Accessible mentors and role models are critically important [… for] recruitment across the translational perspective.”

Another stated,

“Mentorship is really important, I think [it] is certainly a focus of all of our training programs. And I really like to think of this as we consider our scholars […] moving across the trajectory of their career, that good mentorship really can help to identify those people that help to make them successful.”

Discussion

Our findings point to the importance of both a top-down and bottom-up approach to increase diversity among KL2 scholars and other CTS trainees. From the top down, session participants described the importance of meaningful institutional buy-in, with supportive leadership and policies both within the CTSA program and the broader academic institution. From the bottom up, guiding applicants through the application process, ensuring diverse application reviewers, and providing high-quality, diverse mentorship will help to build the workforce and future leadership, which, in turn, will foster institutional environments that continue to grow in their support of DEI efforts.

These four additional strategies complement the six identified by CLIC through qualitative analysis of interviews and common metrics narratives and the CTSA Program Meeting presenters (i.e., prioritizing representation, building partnerships, making it personal, designing program structure, improving through feedback, and winning endorsement). Together, these 10 strategies highlight the importance of a multipronged approach that that may begin further upstream and earlier in the process and require reaching beyond the hub institutions to develop meaningful partnerships with the community and other institutions. Relationships built on mutual trust with minority-serving institutions may be helpful to create the infrastructure and develop pathways appropriate for increasing DEI in career development and identifying and recruiting diverse KL2 and early-stage CTS applicants. Session participants and presenters suggested cross-institution collaboration on mentorship programs and trainings as well as creating application support models.

While relatively little has been published on DEI efforts among KL2 programs nationwide, similar strategies have been found to be helpful. In one publication, the University of Utah CTSA hub listed creating an inclusive environment through institutional support, faculty recruitment efforts, and salary equity for new and existing researchers, as well as developing a holistic mentoring program with formal mentor training [19]. Additionally, a publication from the Duke University hub described their success with identifying and recruiting underrepresented applicants, providing application preparation assistance, promoting structured mentoring, and setting recruitment goals that can be used as a specific “metric of success” [20]. The importance of mentoring is highlighted in both these previous publications as well as by the discussion during the CTSA program meeting. Similarly, a systematic review of mentorship of physicians and trainees who are underrepresented in medicine further concluded with recommendations focused on diversity efforts with strong institutional support, tailored approaches based on institutional needs, and mentor training [21]. While early-onset, effective, and culturally responsive mentoring may not be the only strategy, it is one that may be more easily accomplished, especially if hubs collaborate to develop mentorship programs and incorporate existing resources such as the National Center for Faculty Development and Diversity, a center focused on supporting faculty members, postdoctoral fellows, and graduate students by providing professional development in strategic planning, productivity, building healthy relationships, and maintaining a work-life balance [22].

While there is general agreement around the importance of these strategies to improve diversity, rigorous impact assessments are lacking. Urgently needed are studies focused on empirically investigating the impact of these 10 strategies for increasing DEI among CTS training programs (the original six identified by CLIC and the additional four described here) on program and scholar outcomes including increase in diversity over time, scholar career metrics, institutional cultural changes, and scholar experiences. As discussed in the Duke University CTSA hub publication on increasing diversity [20], setting specific “metrics of success” and then evaluating how specific DEI strategies help to meet these goals is critical to understanding the true impact of DEI efforts among CTS training programs. Future research should focus on examining the pathways in which added diversity in training programs will contribute to research, patient care, and improved outcomes.

We acknowledge several limitations of this analysis. First, the analysis is limited to the discussion that occurred during one virtual breakout session, and thus, the strategies presented here are likely not exhaustive. Additionally, we did not collect demographic or institutional data on the participants and thus cannot comment on the diversity of the participants or whether all CTSA hubs were represented in the discussion. Further, we cannot comment on the relationship between the strategies described by programs and the success of programs in achieving diversity. Additionally, the original six strategies discussed in the session were derived from hubs with the greatest increases in diversity among their KL2 scholars, and thus, their experiences may differ from hubs that already had high and consistent diversity among their KL2 scholars. Finally, many of these strategies may be difficult to implement without financial support from NCATS or CTSA hubs themselves. However, these results incorporated feedback from a geographically diverse group of CTSA hubs across the US noting consistent strategies that need strong institutional support to advance DEI efforts in KL2 and other training programs. Implementing all ten strategies may be cost-prohibitive and unnecessary; future studies should empirically test which strategies have the greatest impact on diversity efforts and on relevant program and scholar outcomes. Similarly, some strategies may work better in specific training settings (e.g., TL1/T32 vs. KL2/K12), and those should also be examined in future studies.

Implementing these strategies in CTS and other training programs will be an important step for advancing DEI, and it is important that hubs evaluate these efforts so that effective strategies can be identified and disseminated. Protocols need to be established to evaluate the implementation and effectiveness of these strategies through continuous quality improvement and program analysis, key components of the CTSA program. Diversity in the workforce can improve science, allow for more diverse clinical trial participation [4], and translate it into improved patient care [2]. Training programs within the CTSA are well-positioned to be leaders in this critical effort to diversify our scientific workforce.