If you build it, will they come? Linking researcher engagement and scientific productivity in large infrastructure grants

Abstract

Introduction:

The NIH Clinical and Translational Science Awards (CTSA) Program supports the creation of program infrastructure promoting scientific collaboration and improvement in translational research. While most evaluations of these and similar programs focus on scientific outcomes such as grants and publications, few studies investigate the underlying mechanisms through which large infrastructure grants produce scientific or translational benefits. This study investigated how engagement – researchers’ interactions with CTSA-funded resources – can help to increase scientific productivity.

Methods:

Authors 1) developed process indicators to define engagement in the CTSA infrastructure at Washington University in St. Louis in four general categories (core service use, internal funding, mentor-mentee opportunities, and leadership roles); 2) explored the relationship between CTSA engagement and scholarly productivity; and 3) compared the relationships between engagement and productivity across gender and race/ethnicity. Mixed effects Poisson regressions modeled productivity outcomes on engagement, controlling for demographic and academic characteristics.

Results:

CTSA members who were engaged were more likely to publish papers and submit grants when compared to others. They were more likely to receive external grant awards – 10% to 20% percent more – than those who were not engaged. Productivity disparities between men and women and to a lesser extent across categories of race and ethnicity persisted even in samples matched on previous productivity levels.

Conclusions:

CTSAs could see larger growth in scientific productivity by increasing researcher engagement and addressing demographic disparities – possibly through focused communications to raise awareness of opportunities – and dissemination of case studies and success stories of engagement to membership.

Introduction

The National Institutes of Health and its National Center for Advancing Translational Sciences (NCATS) launched the Clinical and Translational Science Awards (CTSA) Program in 2006 to support the creation of program infrastructure promoting scientific collaboration and improvement in translational research [1]. It is important to assess the impact of the financial investments, capital contributions, and human resources of such large infrastructure grants for accountability, evaluation of impact, and future funding. While several ways to measure the outputs of such programs exist, such as subsequent grants and publications, or downstream outcomes of disease prevention and increased health care access [2,3], it is more difficult to see exactly how these outputs are connected to initial investments. Here, we investigate the underlying mechanisms – processes and interactions occurring between inputs and outputs – that transform large infrastructure grants into scientific benefits. The overarching research question we address is: How does participation in large infrastructure grant projects increase scientific productivity for members?

NCATS funds approximately 60 CTSA hubs in the US to advance the translational research process and shorten the time it takes to implement clinical findings into practice or move research findings from “bench to bedside.” The investments in infrastructure address operational, organizational, and scientific challenges at a system level within hubs and in their collaborations with local and regional partner institutions [1]. The explicit goals of the CTSA program from NCATS are to:

1. 1. Train and cultivate the translational science workforce;

2. 2. Engage patients and communities in every phase of the translational process;

3. 3. Promote the integration of special and underserved populations in translational research across the human lifespan;

4. 4. Innovate processes to increase the quality and efficiency of translational research, particularly of multisite trials; and

5. 5. Advance the use of cutting-edge informatics [1].

Given these goals, CTSA evaluations often focus on scholarly outputs and outcomes, for example, publications, citations, faculty and team productivity, career success, community engagement, social return on investment, and skill development and training; many evaluations also consider personal characteristics like race and gender [4–9]. Other research evaluations focus on the interdisciplinarity of publications and collaborations [10–12], or the translational benefits from such grants [2]. A handful of other studies have focused on process indicators such as start-up barriers, protocol approval time, regulatory redundancy, and recruitment of research participants [13,14]. Studies that have applied the principles of return on investment in evaluating the impact of large infrastructure grants have considered direct and indirect costs and returns such as operational costs, use of shared core facilities, profits, publications, patents, and career development, but have focused less on process measures [15,16]. To bridge process and outcome indicators, Searles and colleagues constructed the Framework to Assess the Impact from Translational health research (FAIT), a logic model that includes both process metrics like feedback mechanisms of existing practices and translational impact such as new knowledge and community benefits [3].

The current study builds on this literature to focus on how funding specifically helps to increase scientific productivity, and why some researchers benefit more than others from such programs. We frame the nature of the pipeline from investment to scientific output as a process of engagement with the programs, services, and other resources created by large infrastructure grants. Engagement is the set of activities by which researchers benefit from their membership in a CTSA setting, including use of CTSA-funded resources, receipt of internal funding, being involved in mentorship activities, and serving in leadership roles. We measure how different types of engagement are associated with productivity by 1) developing process indicators to operationalize levels of engagement in the CTSA infrastructure at Washington University in St. Louis; 2) exploring relationships between CTSA engagement and scholarly productivity; and 3) comparing these relationships between engagement and productivity across sociodemographic and academic characteristics.

Our study aims to advance our understanding of how large infrastructure investments are transformed into outputs and investigates how researchers interact with newly formed resources to increase scientific productivity focusing on grant submissions, grant awards, and publications. As shown in the Engagement and Productivity Model (Fig. 1), we identify five main types of engagement for researchers that link inputs to outputs in the Institute for Clinical and Translational Science (ICTS), the CTSA at Washington University in St. Louis. These are: leadership roles, internal funding opportunities, mentorship roles, mentee opportunities, and use of ICTS core services. Large infrastructure grants support internal efforts toward recruitment of new members and capacity building. Members then take advantage of and utilize newly created resources and subsequently realize added value of the infrastructure through greater scientific productivity in the form of grant applications, grant awards, and publications. While we recognize that there are other, less formal ways to engage with CTSA infrastructure and leadership, here we focus on five measurable ways ICTS members formally engage, utilize, and take advantage of services and other resources.

Fig. 1. Engagement & Productivity Model: moving from inputs to outputs.

Focusing on these mechanisms of engagement, the current study has three goals. First, we describe patterns of engagement and of productivity over time. Next we model the relationship between engagement and productivity while controlling for several demographic and academic characteristics. Finally, we explore differences across gender and race in our models. While we focus on CTSAs, other, similar types of programs that provide infrastructure for increased knowledge generation, collaboration, and scientific and translational outcomes could also benefit from this framework, including Transdisciplinary Tobacco Use Research Centers, Centers for Population Health and Health Disparities, Cancer Centers, Physical Sciences in Oncology Centers, and Alzheimer’s Disease Research Centers [14,17,18].

Methods

Setting and Participants

Our study comes from the evaluation of a large-scale research institute, the Institute of Clinical and Translational Science (ICTS) at Washington University in St. Louis. The ICTS was formed in 2007 following receipt of a Clinical and Translational Science Award, and since then has focused on supporting innovative, rigorous, and impactful clinical and translational science, with a rich array of career development, pilot funding, and research core services. Membership in the ICTS is open to all faculty and trainees at Washington University and regional partner institutions who conduct clinical or translational research. While all CTSA hubs define membership more or less formally, the ICTS uses a highly inclusive membership model, with a simple online application. This paper utilizes 11 years of ICTS evaluation data collected between 2008 and 2018. This evaluation includes data from 1,746 ICTS members who were faculty, research staff, or students during this time. Years of ICTS membership range from 2 to 11, comprising 9,290 member–year observations. Our study focuses only on members at Washington University, who represent the vast majority of members, and for whom grant submission data were available.

Measures

Given the primary focus on unpacking how engagement with ICTS is associated with scientific activities and outcomes, the measures used here fall into three broad categories: engagement, demographic and academic characteristics, and scientific productivity. Data sources for all the variables include ICTS archives (e.g., executive and operations committee rosters, research core service logs, participation in mentoring programs), human resources data, Scopus [19], and university administrative data (e.g., grant submissions and awards).

Engagement

Core service use

ICTS provides 17 research cores (Appendix) that provide specialized services to members. We identified which members used these core services each year, ranging from 157 to 454 members.

Internal funding

ICTS has several internal pilot funding mechanisms competitively awarded to members across a broad range of subject areas. These 1 year grants range from $5,000 to $50,000. The number of funded members each year ranged from 15 to 66.

Mentors and mentees

We identified mentors from those who formally served as mentors in ICTS MTPCI and T- and K-type programs, or were paired with junior faculty to review early stage research at research forums. Mentees either presented work for review by experts at research forums or were trainees in one of the career development programs above. The number of mentees each year ranged from 23 to 67, and mentors numbered 34 to 117.

Leadership

We included as leadership roles members of the executive and operations committees, leaders or directors of ICTS research cores, and members who led and served on internal funding application review committees. The number of members in leadership roles each year ranged from 60 to 102.

Demographic & Academic Characteristics

The demographic indicators for race, ethnicity, and gender were obtained from university human resources records. For race and ethnicity, employees are asked to provide responses to two questions: whether they identify as Hispanic or Latino, and then to select all race categories that apply from 1) American Indian or Alaska Native, 2) Asian, 3) Black or African American, 4) Native Hawaiian or Pacific Islander, and 5) White. Academic characteristics included academic rank (i.e., Professor, Associate Professor, Assistant Professor, and non-faculty) and discipline. Membership application requests discipline from a list of NIH fields which we collapsed into four overarching categories: Allied Health (e.g., public health, dentistry), Basic Science (e.g., non-clinical, lab-based), Clinical Science (i.e., clinical disciplines), or Social Science/Statistics (e.g., non-clinical psychology, research methods), as described elsewhere [12]. As a proxy for years of experience, we also included the number of years since earning highest degree (also collected from human resources).

Scientific Productivity

Annual counts for the three indicators of productivity were obtained through two sources. Librarians from the Becker Medical Library at the Washington University School of Medicine disambiguated author names from Scopus to curate annual ICTS publication lists. We used institutional records of grant applications and awards, provided by the Office of the Vice Chancellor of Research. All members cited as key personnel in grant applications and awards were counted. More than half of the grant applications and awards were from federal funders, followed next by foundations.

Analyses

To investigate the role of member engagement in scientific productivity, we used these data to describe and analyze patterns of engagement and productivity over time. Furthermore, we explored the role of gender and race/ethnicity in productivity through inclusion of these characteristics in our models.

The main analytic models focused on understanding three outcomes – grant applications and awards and publications as a function of engagement, controlling for demographic and academic characteristics, and productivity from the previous year. For each engagement variable, we used binary indicators where 0 = “not engaged” and 1 = “engaged in this manner at least once.” We used raw counts for each outcome (e.g., five publications or two grant awards in year X). We lagged engagement metrics by 1 year to 1) allow time to realize the benefits of engagement (e.g., internal funding in 2010 related to an external grant submission in 2011); and 2) avoid measuring engagement that may have occurred as a result of new funding in the same year (e.g., a grant award in early 2012 that leads to core service use later in the year).

In this quasi-experimental longitudinal research design, we were unable to have a control group, a condition that introduces potential problems of endogeneity or bias. Possible endogeneity arises out of idiosyncratic qualities of members that influence them to be more productive. Put more simply: maybe some members are just more productive than others. Probable bias can be introduced into observational data analyses when pre-existing characteristics across different groups are imbalanced, for example in our data those who receive internal funding may have more experience than those who do not or those in clinical science may be less inclined or encouraged to publish than others in basic science or allied health.

To address these potentialities, we pre-processed the dataset through coarsened exact matching [20]. In general, matching methods work to select and match observations from an observational dataset so that distributions of covariates across groups are similar. This can guard against potential confounding effects toward causal inference. Compared to the commonly used method of propensity score matching, coarsened exact matching generally estimates effects with lower variance and bias and has been used on longitudinal datasets [20,21]. We employed matching methods since we suspect that endogenous characteristics of researchers cause some to be more productive than others and matching on pre-existing characteristics and recent past productivity can help to balance the groups of engaged and non-engaged members. For the current analyses, we used each engagement metric separately as the treatment variable for each of the three outcomes and produced 15 matched datasets to later pass on to longitudinal regression models (five engagement metrics by three productivity outcomes). For each analysis year, observations from members who were engaged in a specific way were matched with those who were not engaged in this way on the following variables: academic rank, discipline, years since earning highest degree, and the previous year’s productivity metric. For example, for the publications and core service usage model, there were two groups: those who used core services in the previous year and those who did not. Observations between the two groups were matched exactly on the categorical variables of rank and discipline and matched in coarsened categories of years since highest degree and number of publications in the previous year. The latter two continuous variables were coarsened into four categories each using natural breaks.

For the main analyses, we used separate mixed effects generalized Poisson regressions on the matched datasets to model the relationship between each engagement variable and each indicator of scientific productivity. Random effects for year nested within each individual with autoregressive errors (order 1) were fitted to address temporal autocorrelation and potential endogeneity within individuals from year to year [22]. We conducted analyses in R: A language and environment for statistical computing, version 4.0.1 [23] along with the following R packages: tidyverse [24], glmmTMB [25], cem [26], and emmeans [27].

Results

We first present descriptive statistics of the ICTS members included in the analysis for each variable by year, and then offer a comparison of members in the observed dataset to those in the matched samples. Following this are the results from the mixed effects models, along with overall marginal estimates and comparisons across groups.

Descriptive Statistics

Table 1 shows the characteristics of members included by year. Initial membership was small and consisted disproportionately of successful researchers. Concerted efforts to increase membership in subsequent years – especially for junior researchers – increased the diversity of members’ experience and research activity levels. Outcomes and demographic and academic characteristics are shown for 2009 through 2018, and engagement metrics are shown for 2008 through 2017 since they were lagged 1 year for the analyses. Individuals increased from 264 members in 2009 to 1297 in 2018. As membership grew over these years, the overall number engaged in each of the five ways tended to increase, while the percentage of engaged members decreased. For example, 59.5% of members used core services in 2009 and 31.7% did so in 2017. Over the same time, the percentage of members who were mentees ranged from 8.7% to 3.5%, 12.9% to 8.4% for mentors, and 22.7% to 7.5% for those in leadership positions. The proportion of members who were women increased overall from 31.8% (2009) to 38.1% in 2018. Racial and ethnic diversity also increased somewhat during the analysis period from being over 83.7% White alone in 2009 to 73.0% in 2018. The distribution of members’ disciplines stayed relatively constant, as did rank, except for increased percentages of non-faculty members by 2018 and slightly lower percentages of professors and assistant professors. Similarly, the average number of years since earning one’s highest degree hovered around 20 years, though the standard deviation was also consistently large, ranging from 10 to 11.5 years from 2009 to 2018. Average outcomes of productivity decreased gradually as membership grew from 5.6 grant applications per member in 2009 to 3.0 in 2018, 1.9 average annual grant awards in 2009 to 1.1 in 2018, and 5.6 publications in 2009 to 4.7 in 2018.

Table 1. Descriptive statistics of Institute for Clinical and Translational Science (ICTS) members included in original dataset

Coarsened Exact Matching

Before modeling the relationships between engagement and productivity in mixed effects regression approach, we used coarsened exact matching to obtain more balanced samples for analysis. While the core focus in this matching strategy is to balance covariate characteristics between the two groups (here, engaged and not engaged in each way), it is useful to examine the samples for representativeness overall, especially for those covariates not included in the matching set – gender and race/ethnicity. As an example of how the distributions of covariates in the samples resembled the original dataset, Fig. 2 shows the full set of observations for 2018 (top, larger panel) and the 2018 subsample from each of the 15 matched sample datasets. All the sample distributions follow the same patterns in the observed data. For discipline, those in clinical science comprised about half of the original dataset, and they are present in higher proportions for some of the samples for internally funded members and mentees and mentors. Professors and associate professors also make up about half of the original sample. They are present in higher numbers for the mentor and leader model samples, roles that tend to go to more senior scholars, and lower numbers for the mentee and internally funded member matched samples, roles generally filled by more junior scholars. The composition of gender and the three categories for race/ethnicity in the matched samples all closely resemble the observed dataset.

Fig. 2. Example of included Institute for Clinical and Translational Science (ICTS) members by demographic and academic characteristics for 2018. Top figure shows distributions from the original dataset and each smaller figure shows the distributions for a matched sample used in analytic models.

The supplemental appendix includes more details about the matched samples. The smallest sample matched contained 597 members and the largest contained 1,688 across the 15 matched datasets, from the original 1,746 unique members included. Comparing average rates of productivity between engaged and non-engaged members in the raw data, engaged members showed higher rates for four of the five types of engagement. However, those who were mentees had lower productivity rates than non-mentees before matching. Importantly, in all the matched samples, the gap in the average count outcome (grant application, award, publication) between engaged and non-engaged members was narrower than in the observed dataset, suggesting further improved balance between each set of paired groups (Fig. A1 in the appendix).

Mixed Effects Longitudinal Poisson Models

Using the matched samples, we estimated 15 longitudinal mixed effects regression models for research productivity count data. The results of these models are in Table 2. Dispersion tests for Poisson data indicated that each of the matched samples were underdispersed, so all models are generalized (Conway–Maxwell) Poisson that address this characteristic of count distributions [28]. Each column contains odds ratios and 95% confidence intervals from one specific model, along with model statistics. On average, the models performed well on the observed data, predicting between 82% and 95% of cases correctly. Bolded estimates are statistically significant (p < 0.05).

Table 2. Results of 15 mixed effects Poisson models, each productivity outcome (3) by each type of engagement (5); odds ratios and 95% intervals

Notes: Fixed effects estimates are odds ratios with 95% intervals immediately below. Bolded estimates have p < 0.05. Models are mixed effects generalized Poisson with random effects for year nested within individuals and order 1 autoregressive errors. Correct predictions show the estimates which contain the observed value in 95% interval.

All the engagement metrics resulted in statistically significant and positive effects on productivity (top row Table 2), except for the model focused on leadership roles and publications (last column). Members who used any ICTS core services in the previous year were estimated to submit 20% more, or 1.2 times the number of grant applications as those who did not use services. The largest estimate for engagement was for mentors and grant awards where those who were mentors during the previous year received almost one-and-a-half times the number of grant awards (1.4) as those who were not mentors in the matched sample.

Next, the models showed differences between men and women who were otherwise similar in rank, discipline, years since degree, and previous productivity. Except for the matched samples of men and women in mentorship and leadership roles in the grant applications models where no differences were found, men submitted 24%–33% more grant applications, received 29%–41% more grant awards, and published 18%–37% more articles than women. Additionally, while in 10 of the 15 models no differences were found between the race and ethnicity categories, Asian members were predicted to receive 75%–84% of the number of grant awards of their white counterparts in two models and members of other races or ethnicities were predicted to publish at rates that were 71%–80% of their white colleagues in three models. We note here that hypothesizing that engagement may have mitigated some differences in productivity across gender and race/ethnicity. However, interactions between engagement metrics and these demographics were tested but were not statistically significant in any models.

Academic characteristics serving as control variables were statistically significant in most of the models based on the matched samples. Full professors were expected to be more productive than their junior colleagues and non-faculty researchers. Basic and social scientists/statisticians were predicted to submit and receive more grants and publish more articles than clinicians and allied health researchers in most cases, and the years since highest degree, modeled quadratically to reflect an observed peak in productivity during career trajectories, followed that pattern in many cases. Figures A2–A4 in the appendix show the average predicted counts of each productivity outcome by each of the academic characteristic variables.

Marginal Estimates: Engagement

Here we present the marginal estimates for each engagement-productivity scenario and the differences between groups of interest. Figure 3 shows the count estimates of the outcomes for average members who were or were not engaged in each way in the previous year (top panel), along with the percentage differences for the two groups (bottom panel). On average, ICTS members submit between one and two grant applications, receive about 0.5 grant awards, and publish around three articles per year. In varying magnitudes, each model estimates that non-engaged members are less productive than their engaged counterparts. The bottom panel of Fig. 3 further illustrates the differences in estimates between members engaged and not engaged in each way. Models estimated that those who were engaged would submit on average 6% (internal funding) to 16% (mentees) more grant applications than others, depending on the type of engagement. Engaged members also typically receive 10% (internal funding) to 20% (mentors) more grant awards and publish 3% (core service use) to 10% (mentees) more articles than their non-engaged counterparts.

Fig. 3. Count estimates and percentage differences for grant applications and awards and publications comparing those engaged in each way and those who were not; continuous covariates held at mean and categorical ones averaged across all categories.

Marginal Estimates: Gender and Race/Ethnicity

The next two Figures 4 and 5 focus on differences across categories of gender and race/ethnicity for average ICTS members. Figure 4 shows the average number of grant applications, awards, and publications estimated for women and men, along with the percentage differences. While the count estimates for each productivity outcome (top panel) are similar to those comparing engagement, the differences are more or less pronounced between women and men, depending on the engagement-productivity scenario. Three of the five models indicated differences across gender ranging from 11% and 15% more grant applications submitted by males than by otherwise similar female researchers, while the models for mentors and leaders showed no statistically significant differences. Across the remaining models based on matched samples, the average number of grant awards and publications annually, men were estimated to receive 12%–19% more grant awards and publish 9%–17% more articles than their female counterparts. Figure 4 also shows that the largest differences between women and men across all productivity outcomes were predicted from the models using the matched samples for mentees.

Fig. 4. Count estimates and percentage differences for grant applications and awards and publications comparing women and men; continuous covariates held at mean and categorical ones averaged across all categories.

Fig. 5. Percentage differences in count estimates for grant applications and awards and publications between race/ethnicity categories; continuous covariates held at mean and categorical ones averaged across all categories.

Finally, Figure 5 displays the percentage differences predicted for ICTS members across the three categories of race/ethnicity. Few of the models indicated statistically significant differences, including none for the expected number of grant applications between Asians, Whites, and members of other races or ethnicities. For grant awards, no differences were seen between Asians and those of other races or ethnicities, though differences were found between Whites and Asians for three of the five grant award models, the largest of which was 14% more grants received by Whites than Asians (the mentor model). Models also estimated that Whites receive 8% or 9% more grants in the core service and internal funding models when compared to Asians and 12% more grant awards than members of other races or ethnicities (core service model). Two of the models (core service use and mentors) predicted that Asians would publish 10% or 11% more articles than those of other races or ethnicities. While no differences were seen between Whites and Asians for publications, three of the publications models predicted 11%–16% more articles for Whites than members of other races or ethnicities (core service use, mentor, and leader models).

Discussion

Our framework for analyzing return on investment investigates how engagement in CTSA activities relates to scientific productivity. We operationalize engagement as taking on leadership or mentor/mentee roles, receiving internal funding, and using core services, and productivity as grant submissions, grant awards, and publications. While research infrastructure grants offer the means to increase the production of scientific knowledge, our research suggests that providing the means is not enough – engagement is a key mechanism through which infrastructure is transformed into increased scientific productivity. Among the population of ICTS members at Washington University, those who engaged with services and programs were more productive than their non-engaged counterparts.

We found that engagement facilitates the relationship between investments in research infrastructure and increased scientific output by illuminating the black box of how new services, resources, and opportunities can increase productivity. While CTSA members are generally likely to submit and receive grants and to publish regardless of engagement, levels of doing so increase with engagement. This held true across all types of engagement measured. Interestingly, in the raw data engaged members were more productive than non-engaged ones, except for the case of mentees. Once the samples were matched and more balanced in terms of discipline, years of experience (proxied by years since highest degree), and past productivity, mentees had slightly higher rates of authorship and grant activity. This is likely because mentees are generally junior scholars and in the raw data the average years of experience was 20, and much lower in the matched samples for mentees (ranging from 13 to 15 years). Balancing the datasets through matching creates more comparable groups to help isolate the impact of the treatment variable – in this case engaging as a mentee – which resulted in the positive and statistically significant effects (from 21% to 35% higher productivity for mentees versus non-mentees). Perhaps one of the most compelling findings, since external funding is always scarce and competitive, is that engaged members are likely to receive more external grant awards – 10% to 20% percent more awards – than those who are not engaged.

While gender disparities in academia are well-documented [29,30], it is discouraging to see that disparities persist even after matching samples on previous productivity metrics before modeling the effects of engagement. In 13 of 15 matched samples, models predicted women to apply for and receive fewer grants and publish fewer articles than their male counterparts – regardless of previous track records or recent ICTS engagement. Another compelling finding is that the largest differences found through modeling engagement, gender, and productivity were for samples matched on those engaged as mentees. These are by nature junior scholars. Results showing gaps of at least 15% across all productivity metrics by gender suggest that women early in their careers face more challenges than more established female scholars.

Disparities across race and ethnicity categories were less common in this study yet were still found in one-third of the models. While the confidence intervals around the estimated differences neared zero in most cases they should not be ignored, as they also reached 20% or more at the upper bound. We acknowledge that it is not ideal to group all other races and ethnicities besides those who were Asian or white alone into one group. All other categories comprised less than 7% of the study population. We recognize this speaks to the need for continued and additional attention to equity in hiring and recruitment. Studies at other CTSA hubs that have more diverse memberships could shed further light on this finding.

Another possible limitation of our study is that, while employing the matching strategy increased balance in the samples and decreased bias, our data only allowed for matching samples on one engagement metric at a time. This prevented us from investigating any possible additive or interaction effects of different combinations of engagement such as junior scholars who were internally funded, served as mentees, and used core services. We also acknowledge that all CTSAs, large infrastructure grants, or institutions are not the same. While this could diminish the generalizability of our results to other programs and places, it is also feasible that the trends found here – that those actively engaged in large infrastructure grants are more productive than those who are not, regardless of previous productivity – are similar elsewhere.

In sum, providing scientific and operational infrastructure to academic investigators is related to positive productivity outcomes, but mostly for those members who engage with these programs and services. CTSAs and other large infrastructure grant programs may see more impact on institutional productivity by supporting activities that draw translational researchers into closer engagement with CTSA activities, resources, and collaborations. For example, one strategy might be greater allocation of resources to communications, to raise awareness of their services, programs and other opportunities. Part of this communication strategy may be dissemination of case studies and success stories highlighting how engagement translates to increased productivity for active members. Promoting the benefits of engagement and availability of services to investigators is necessary for members to understand how the investment of their time can translate to increased productivity. Only about one-third of ICTS members studied here used core services in the last 2 years – the most popular form of engagement – so there is still room to promote increased service utilization in the institute.

To address potential disparities in productivity for women and people of color, all these communications strategies and efforts to increase engagement could be tailored to specific populations, even more so than they currently are. Making programs and services more accessible by waving fees or reallocating funds from elsewhere for targeted populations could also help to narrow disparities in productivity. However, our results show that even when women or racial and ethnic minorities engage with available services and support, their productivity boosts are not as great as those for white males. This illuminates the need for more efforts focused on understanding and overcoming barriers to academic success among women and underrepresented groups. On a broader and further downstream scale, engagement with translational research infrastructure results in real-world benefits in clinical, community, economic, and policy spaces [2], which further demonstrate the value of these investments. Keeping decision-makers aware of the outcomes of these investments is crucial to secure future research funding, reduce disparities, and decrease the time-to-uptake of new translational science findings.