Hostname: page-component-cd9895bd7-p9bg8 Total loading time: 0 Render date: 2024-12-26T07:54:20.144Z Has data issue: false hasContentIssue false

The evolving collaborative relationship between Practice-Based Research Networks (PBRNs) and Clinical and Translational Science Awardees (CTSAs)

Published online by Cambridge University Press:  28 December 2017

Maureen Riley-Behringer
Affiliation:
Department of Social Work, Elizabethtown College, Elizabethtown, PA, USA
Melinda M. Davis*
Affiliation:
Oregon Rural Practice-Based Research Network, Family Medicine (School of Medicine) and Community Health (OHSU-PSU School of Public Health), Oregon Health & Science University, Portland, OR, USA
James J. Werner
Affiliation:
Department of Family Medicine, Case Western Reserve University, Cleveland, OH, USA
L. J. Fagnan
Affiliation:
Department of Family Medicine and Director of Oregon Rural Practice-Based Research Network, Oregon Health & Science University, Portland, OR, USA
Kurt C. Stange
Affiliation:
Center for Community Health Integration, Departments of Family Medicine & Community Health, Population & Quantitative Health Sciences, and Sociology, The Case Comprehensive Cancer Center, Cleveland, OH, USA
*
*Address for correspondence: M. M. Davis, Oregon Rural Practice-Based Research Network, Oregon Health & Science University, Mail Code L222, 3181 SW Sam Jackson Park Road, Portland, OR 97239-3098, USA. (Email: [email protected])
Rights & Permissions [Opens in a new window]

Abstract

Purpose

Clinical and Translational Science Awards (CTSAs) and Practice-Based Research Networks (PBRNs) have complementary missions. We replicated a 2008 survey of CTSA-PBRN leaders to understand how organizational relationships have evolved.

Methods

We surveyed 60 CTSA community engagement (CE) Directors and 135 PBRN Directors and analyzed data using between and within-group comparisons.

Results

In total, 43% of CTSA CE Directors (26/60) and 42% of PBRN Directors (57/135) responded. Quantitative responses revealed growing alignment between CTSA/PBRN perceptions, with a few areas of discordance. CE Directors noted declining financial support for PBRNs. PBRN Directors identified greater CTSA effectiveness in PBRN engagement, consultation, and collaborative grant submissions. Qualitative data revealed divergent experiences across CTSA/PBRN programs.

Conclusions

Relationships between CTSAs and PBRNs are maturing; for some that means strengthening and for others a growing vulnerability. Findings suggest a mutual opportunity for PBRNs and CTSAs around applied research. Studies to characterize exemplar CTSA-PBRN collaborations are needed.

Type
Implementation, Policy and Community Engagement
Creative Commons
Creative Common License - CCCreative Common License - BYCreative Common License - NCCreative Common License - ND
This is an Open Access article, distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives licence (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is unaltered and is properly cited. The written permission of Cambridge University Press must be obtained for commercial re-use or in order to create a derivative work.
Copyright
© The Association for Clinical and Translational Science 2017

Introduction

Essential elements in the research pipeline are the translational steps: moving evidence from bench to bedside and then out into routine use in practices and community-based settings [Reference Westfall, Mold and Fagnan1, Reference Fort2]. Woolf et al. summarized the 2 distinct “translational blocks” identified by the Institute of Medicine’s (IOM) Clinical Research Roundtable which are still relevant today: first, moving evidence from laboratory discoveries into their first testing in humans and second, from clinical studies into “everyday clinical practice and health decision making [Reference Woolf3].” A recent systematic review identified an emerging consensus 5-phase definition of translational research, which emphasizes research along a continuum rather than across “gaps” and proposed a circular rather than linear trajectory of the research phases from basic research to populations and back [Reference Fort2]. Clinician and Translational Science Awards (CTSAs) and Practice-Based Research Networks (PBRNs) play an important role in current and historical efforts to improve translation research. However, few studies explore the structure and changing relationships between CTSAs and PBRNs over time [Reference Fagnan4, Reference Haggerty5].

The CTSA program was established by the National Institutes of Health (NIH) in 2006 to improve the conduct of biomedical research, reduce the time required for translation of discoveries into practice, engage communities in research, and train the next generation of translational scientists [Reference Fagnan4]. The CTSA program was designed in part to help address a key focus of the 2004 NIH Roadmap, to speed-up the translational research timeline [Reference Westfall, Mold and Fagnan1, Reference Zerhouni6]. Yet over the past decade, the NIH Roadmap and CTSA program have experienced many changes [Reference Leshner7]. For example, in 2012 the CTSA program shifted leadership to the National Center for Advancing Translational Sciences (NCATS) with an espoused mission to “transform the translational science process so that new treatments and cures for disease can be delivered to patients faster” by supporting a national network of medical research institutions—called hubs—that work together to improve the translational research process [8]. Concurrently, NCATS stopped requiring that a community engagement core was proposed within CTSA renewal applications despite later conclusions by the IOM that training and education as well as community engagement were important strengths of the CTSA program [Reference Leshner7]. Despite growing awareness of the work done by CTSAs, in late 2015 the cornerstone journal for the CTSA program, Clinical and Translational Science, mandated a shifted ownership, editorial staff, and focus away from the full, rich spectrum of translational research back to spotlighting laboratory discoveries [Reference Feldman9]. There are currently CTSAs in ~60 academic medical institutions in the USA [10].

PBRNs were established in the USA in the 1970s, as collaborative groups of primary care clinicians committed to conducting research focused on the experience and delivery of care to the patients they serve [Reference Mold11, 12]. Early PBRN studies explored problems faced by practitioners on the front lines of primary care; findings demonstrated the gap between research conducted in controlled specialty settings and real-world practices [Reference Davis13]. Over the past 40 years PBRNs have evolved and expanded their focus to engage additional front-line providers and to address increasingly complex questions that emerge from daily practice [Reference Davis13]. In addition to engaging primary care clinicians, PBRNs now engage and/or are led by clinicians from multiple specialties, public health professionals, and key population groups (e.g., patients and providers for people with disabilities) [Reference Davis13Reference Gilbert18]. Moreover, PBRNs have become multifaceted health improvement networks [Reference Williams and Rhyne19] that conduct community-engaged research [Reference Davis13, Reference Spears20Reference Pham22], lead quality improvement and practice transformation initiatives [Reference Mold and Peterson23], support participatory implementation research [Reference Ramanadhan24], enable continuing education and maintenance of board recertification [Reference Bakken25, Reference Gibson26], and help train the future generation of translational researchers [27]. Concurrently, PBRNs have expanded beyond the walls of primary care to engage patient and community-based partners in addressing the social determinants of health [Reference Fagnan28, Reference DeVoe29] and in supporting dissemination and implementation research [Reference Heintzman30]. Field leaders estimate that PBRNs reach ~15% of the US population [Reference Werner31]; many PBRNs serve regions and populations experiencing health disparities. In 2012, prior to disbanding of funding for the PBRN resource center, there were 143 active PBRNs in the USA and nearly half reported a CTSA affiliation [Reference Peterson32].

CTSA and PBRNs have many complementary characteristics and goals. PBRNs can play an important role in helping questions across the translational research pipeline stay grounded in the needs and interests of practicing clinicians and community-based stakeholders. In 2008, we published findings from a web-based survey that examined early relationships between PBRNs and CTSAs as perceived by CTSA community engagement (CE) directors as well as PBRN directors [Reference Fagnan4]. Our findings revealed important opportunities and challenges for these partnerships in relation to program roles, relationships, and structures for collaboration [Reference Fagnan4]. Although our findings suggested that both CTSA CE Directors and PBRN Directors found the relationship important and that the PBRN environment created opportunities for bidirectional exchanges for participatory and translational research we concluded that with only 3 years of experience together, PBRN/CTSA relationship were in the early discovery phase, with the collaborators negotiating expectations [Reference Fagnan4]. Although a recent study found that PBRNs who reported an affiliation with a CTSA was more likely to conduct pragmatic clinical research trials, participate in research projects with more than one other PBRN, to perceive fewer barriers to securing funding to support PBRN infrastructure [Reference Haggerty5], we found no additional studies that have explored the continued evolution of CTSA and PBRN relationships over time.

Many years have passed since we first characterized PBRN and CTSA relationships. These changes in the CTSA program focus and the expansion and diversification of PBRNs may have important implications on the relationships between CTSAs and PBRNs. Therefore, we conducted this new study to ascertain how CTSA CE and PBRN Directors viewed current relationships and to assess the changes in these associations since the original survey [Reference Fagnan4].

Methods

Modeled after the 2008 survey content and methods, we conducted an anonymous cross-sectional web-based survey of CTSA CE and PBRN Directors [Reference Fagnan4]. The current study was approved by the Case Western Reserve University Institutional Review Board.

Selection of Participants and Survey Administration

The Collaboration/Engagement Domain Task Force (formerly Community Engagement Key Function Committee, KFC) and the Community Engagement Survey Facilitation Reviewers of the CTSA endorsed the survey in October 2013, and provided access to the email listserv of 60 voting members from the Collaboration/Engagement Domain Task Force, excluding those members that were NIH employees. We identified 135 eligible PBRN Directors or other PBRN points of contact published on the Agency for Healthcare Research and Quality (AHRQ) PBRN Directory listserv.

Between December of 2013 and July of 2014, we distributed 3 sequential email requests for participation to CTSA CE and PBRN Directors. These requests explained the study premise and provided the appropriate online survey link (i.e., CTSA CE Director or PBRN Director) in Qualtrics survey software (www.qualtrics.com). Survey software was configured to prevent participants from taking the survey multiple times.

Survey Content

Separate web-based, anonymous surveys consisting of fixed and open-ended questions were prepared for CTSA CE Directors and PBRN Directors. The CTSA CE Director Survey consisted of 24 questions that explored the PBRN/CTSA relationship, support characteristics, and future expectations. The PBRN Director Survey consisted of 32 questions that explored similar topics with the addition of feedback on whether affiliated CTSAs understood the importance of PBRNs meeting their clinician members’ research needs/interests. CTSA CE Directors that reported they did not have any PBRN affiliation as well as PBRN Directors that were not affiliated with a funded CTSA grant/planning grant were directed to the end of the survey. The current study reports on CTSA CE and PBRN Directors who completed the full surveys describing their affiliations/working relationships with one another, support characteristics, and future expectations.

Two open-ended questions were asked of both CTSA CE and PBRN Directors: How has the collaborative relationship with your (CTSA/PBRN) changed since 2008? What expectations do you have for your future relationship between your organization and your (CTSA/PBRN)? One additional question was asked of PBRN Directors only: Do your CTSA leaders understand the importance of addressing the research needs and interests of your PBRNs’ clinician-members—please explain? One additional question was asked of CTSA CE Directors only: Does your PBRN have existing limitations to the CTSA mission—if yes, please describe? As described, participants could request a summary of the survey results by providing their identifying information in a separate link.

Data Management and Analysis

Quantitative data were downloaded into SPSS Version 22 (IBM, 2013), cleaned, then analyzed using descriptive statistics. When individual cells contained an expected count of <5, we conducted transformations when possible or utilized the Fisher exact test. To compare quantitative questions that contained >1 answer choice, we dichotomously coded each response option [i.e., Role of PBRN in CTSA: research methods (0=No; 1=Yes)]. We analyzed data using χ2 tests of independence to explore between (CTSA vs. PBRN responses on the 2014 survey) as well as within (2014 vs. 2008 responses)-group comparisons.

Qualitative data from open-ended questions were transferred to ATLAS-ti 7.0 (Scientific Software Development GmbH, 2014) for data management and analysis. We conducted semantic thematic analysis guided by the study’s research questions but were open to new information and perspectives [Reference Creswell33, Reference Elo and Kyngäs34]. Two investigators (M.R.-B., J.J.W.) jointly developed the codebook a priori and periodically met to clarify code definitions, discuss emerging themes, and check reliability [Reference Crabtree and Miller35]. In addition, narrative responses from PBRN Directors regarding whether their CTSA leaders understood the importance of PBRN(s) addressing the research needs and interests of their clinician members were grouped into 3 categories: “yes,” “yes and no,” and “no” for review. A small number of emergent codes were added during the analysis; a coding reliability threshold of 75% was met [Reference Cicchetti36]. In addition to identifying themes from open-ended questions on the 2014 survey, we compared emergent themes to published results from our 2008 study of CTSA CE and PBRN Directors [Reference Fagnan4].

Results

In total, 43% of all invited CTSA CE Directors (26/60) and 42% of all invited PBRN Directors (57/135) responded to the 2014 survey. Respondents indicating PBRN/CTSA affiliations were eligible to complete the full survey and included 70% (n=18) of the CTSA CE Directors and 73% (n=42) of the PBRN Directors. Participating PBRN Directors denoted that half (50%, n=21) of their PBRNs had been established between the years 2000 and 2013; participating CTSA CE Directors relayed that 56% (n=10) of their CTSA organizations were formed between the years 2006 and 2008. The majority of PBRNs (83%, n=50) were affiliated with only 1 CTSA. CTSAs were most often associated with 1 (40%, n=24) or 2 (40%, n=24) PBRNs. Both CTSA CE Directors and PBRN Directors reported that their CTSA renewals were due between 2015 and 2018 (CTSAs=81%; PBRNs=92%).

Quantitative Findings: Greater Alignment and Reduced Discordance

As summarized in Table 1, CTSA CE Directors survey responses in 2014 and 2008 varied less than the responses of PBRN Directors (significant difference in 4 items vs. 10 items, respectively). Comparing 2014 with 2008 responses, significantly more CTSA CE Directors reported no changes in budgetary support for PBRNs (63% vs. 4%) yet they provided significantly greater “in-kind” services to PBRNs in some areas (i.e., accounting and training). Responses of PBRN Directors in 2014 compared with 2008 also noted significantly greater use of CTSA resources (i.e., informatics, training) and mostly highlighted growth and improvement in the perceived relationships. For example, PBRN Directors indicated that significantly more PBRNs were involved with CTSAs (74% vs. 57%), were seen as very important to their CTSAs (51% vs. 24%), and that CTSAs were more often “very effective” in engaging with the PBRN (42% vs. 14%). However, in 2014, significantly more PBRN Directors indicated that their CTSAs had become less effective at referring investigators who had projects ready for implementation in a PBRN than in 2008 (52% rated poorly vs. 31%). Although more PBRN Directors in 2014 reported that research grants had been submitted or funded through NIH (44% vs. 12%), significantly more reported that no training grants had been submitted (73% vs. 36%).

Table 1 Quantitative responses from Clinical and Translational Science Awards (CTSA) community engagement (CE) Directors and Practice-Based Research Networks (PBRN) Directors on the 2008 and 2014 web-based surveys regarding PBRN/CTSA relationshipsFootnote *

NIH, National Institutes of Health; N/A, not asked; FET, Fisher exact test.

* Only respondents indicating PBRN/CTSA affiliations were eligible to complete the full survey (CTSA n=18; PBRN n=42).

Because of rounding or multiple response options, some percentage grouping totals do not add up to 100%.

1 χ2 (2)=6.75, p<0.05, Cramer’s V=0.29 (medium effect size).

2 χ2 (2)=8.00, p<0.05, Cramer’s V=0.31 (medium effect size).

3 χ2 (1)=7.35, p<0.01, Cramer’s V=0.26 (weak effect size).

4 FET, p<0.001, Cramer’s V=0.78 (strong effect size).

5 FET, p<0.01, Cramer’s V=0.59 (large effect size).

6 FET, p<0.01, Cramer’s V=0.68 (large effect size).

7 χ2 (1)=9.68, p<0.01, Cramer’s V=0.30 (medium effect size).

8 FET, p<0.05, Cramer’s V=0.42 (medium effect size).

9 χ2 (1)=8.88, p<0.01, Cramer’s V=0.29 (weak effect size).

10 χ2 (1)=4.01, p<0.05, Cramer’s V=0.18 (weak effect size)

11 χ2 (1)=14.87, p<0.01, Cramer’s V=0.37 (medium effect size).

12 χ2 (1)=14.04, p<0.01, Cramer’s V=0.36 (medium effect size).

13 FET, p<0.05, Cramer’s V=0.34 (medium effect size).

14 χ2 (1)=14.04, p<0.01, Cramer’s V=0.36 (medium effect size).

Perceptions of the CTSA-PBRN relationship appeared more similar than different in 2014 as reported by CTSA CE and PBRN Directors (see Table 1). Three significant areas of difference related to the types of support provided by CTSAs to their PBRNs focused on use of “IRBs,” “accounting,” and “consultation.” Nine additional items that significantly differed between CTSA CE and PBRN Directors in 2008 were no longer significantly different in the 2014 survey. For example, with respect to the roles of PBRNs within CTSAs, in 2008 CTSA CE Directors indicated that PBRNs were CTSA resources for study recruitment, research methods expertise, and research education significantly more often than did PBRN Directors; these differences were not observed in 2014. Similarly, in 2008 CTSA CE Directors compared with PBRN Directors reported that CTSAs provided significantly higher levels of financial support to PBRNs, significantly more biostatistical support, and submitted significantly greater numbers of CTSA-PBRN collaborative research and training grant applications; none of these differences were observed in 2014. Although in 2008 PBRN Directors as compared with CTSA CE Directors noted significantly higher effectiveness ratings related to how their PBRN engaged academic investigators, this item did not significantly differ for either respondent group in 2014.

Qualitative Findings: Divergent Experiences of CTSA/PBRN Programs Over Time

Qualitative responses on the 2014 survey indicated how perceptions of the CTSA/PBRN relationship were changing over time. As summarized in Table 2, both CTSA CE Directors and PBRN Directors indicated that there had been an improvement in collaboration and awareness of the role PBRNs could play in CTSA, increased value placed on PBRNs as infrastructure for community-engaged research, and an improved understanding of the benefits that PBRNs could provide. Some CTSA CE and PBRN Directors also noted, however, that their PBRN(s) lost the financial support of their CTSA(s) and that the CTSA(s) “lost interest” in working with the PBRN. In contrast, other respondents reported that their PBRNs had continued collaborating with their CTSA and were currently receiving financial, instrumental, or in-kind support.

Table 2 Clinical and Translational Science Award (CTSA) community engagement (CE) and Practice-Based Research Network (PBRN) Directors perceptions of PBRN/CTSA relationships over time and in the future

CTSA CE and PBRN Directors also displayed variability in their future expectations for CTSA-PBRN collaborative relationships (see Table 2). CTSA CE Directors acknowledged the need for continued flexibility in the research and healthcare environments, specifically the desire for PBRNs to expand into broader community engagement and to develop robust linkages for increased data sharing. PBRN Directors tended to describe their expectations for future CTSA-PBRN relationships with less certainty. Although some PBRN directors voiced positive expectations for continued collaboration and shared success, many indicated that meaningful collaborations will require increased levels of CTSA support and more shared decision making between the leaders of CTSAs and PBRNs.

PBRN Directors reported seeing significant variability in their CTSA CE Director’s understanding and prioritization of the research needs of the clinicians in their networks, see illustrative quotes in Table 3. Some PBRN Directors characterized CTSA leaders as understanding that the support of PBRN clinicians for local research was necessary in order to engage practices in larger studies, other CTSA leaders were reported to be less receptive to this message and continued to view PBRNs as resources for subject recruitment into clinical trials. Still others indicated that although their CTSA leaders believed the PBRNs to have much potential, the CTSA had not committed substantial infrastructure resources due to competing demands for financial support. As summarized in Table 4, when asked to describe the limitations of PBRNs with respect to the CTSA’s mission, CTSA CE Directors indicated that the priorities of CTSAs and PBRNs do not perfectly align, that CTSAs need to see PBRNs as more than a recruitment source for clinical trials, that the foundational ideologies might differ, and that it is often challenging to agree upon a common agenda that meets the needs of CTSAs and PBRNs within the confines of declining CTSA budgets. Overall, qualitative data suggest that the receptivity of CTSAs to PBRNs and vice versa was related to the level of congruence between CTSAs’ needs and of the goals of the associated PBRN(s).

Table 3 Practice-Based Research Network (PBRN) Directors’ perceptions of Clinical and Translational Science Award (CTSA) Leaders’ understanding of the importance of addressing/prioritizing the research needs and interests of PBRN clinician-members (n=42)

Table 4 Clinical and Translational Science Award (CTSA) community engagement (CE) Directors’ perceptions of the existing limitations of Practice-Based Research Network (PBRNs) to the CTSA mission (n=18)

Discussion

Despite the overlapping goals of CTSA and PBRNs, there are a paucity of studies that explore these relationships and how they have changed over time [Reference Fagnan4, Reference Haggerty5]. In fact, comparisons of the findings from our current study with the original 2008 surveys highlight the continually evolving relationship between PBRNs and CTSAs. Our quantitative findings indicate greater alignment between CTSA CE Director and PBRN Director perceptions in 2014 than in 2008 [Reference Fagnan4]. However, qualitative results suggest that there is growing discordance within respondent categories. For example, PBRN Directors described experiences with their CTSAs that were highly variable—some report strong collaboration and support while others described how their PBRNs were an afterthought for their CTSA and that they held little sway in leadership and decision making. Financial support of PBRNs by CTSAs was also wide-ranging—some CTSA Directors noted that PBRNs were spared from across-the-board budget cuts because the networks were so highly valued whereas other PBRNs lost CTSA financial support and were not provided with alternate funding to continue the collaboration.

These findings suggest that in some circumstances PBRNs are closely aligned with the CTSA structure and mission, while in other settings PBRNs are working to find new ways to fund their operations so they are not so reliant on the CTSA and changing organizational missions. These findings echo recent results from a survey of PBRN Directors in the USA and Canada which found that PBRNs affiliated with CTSA programs were less likely to report maintaining funding as a significant barrier [Reference Haggerty5]. Moreover, they suggest interesting changes as both CTSA and PBRNs evolve from programs that support “translational research” to those that are integral partners in a broader, community-focused learning healthcare system [Reference Leshner7, Reference Williams and Rhyne19, Reference Smith, Stuckhardt and McGinnis37]. Although early visions of translational research specified divisions between research and practice, learning healthcare systems facilitate an “iterative innovation process designed to generate and apply the best evidence for the collaborative healthcare choices of each patient and provider; to drive the process of discovery as a natural outgrowth of patient care; and to ensure innovation, quality, safety, and value in healthcare [Reference Kemp38].”

Despite evidence of growing congruence between CTSAs and PBRNs generally, both are responding to different environmental challenges as they seek to stay rigorous and relevant to their stakeholders and funders. Challenges for CTSAs include reductions in core funding and an increasing emphasis on the development of partnerships for research and collaboration across the CTSA consortium. Many changes in CTSAs have been driven by evolving priorities of the CTSA program and recommendations from a 2013 IOM report [Reference Leshner7]. In parallel, PBRNs face increasing demands on clinicians’ time, the need to make successful value propositions to healthcare systems for practices’ engagement in research, and pressure to develop the capacity to effectively utilize electronic health information and large administrative data sets in PBRN studies [Reference Davis13, Reference Werner and Stange39Reference Calmbach41]. As PBRNs have responded to the needs of their stakeholders, many have recognized that community partners and thus community-engaged research methods are critical to PBRN success [Reference Spears20, Reference Gaglioti42, Reference Tapp and Dulin43].

Community engagement provides an exemplar case for exploring how changes in funding and healthcare delivery may be shaping CTSA and PBRN relationships. As PBRNs shifted their focus “beyond the clinic walls” it provided an ideal opportunity to partner with CTSAs around community engagement, an area where many CTSAs expressed early challenges [Reference Leshner7]. For example, PBRNs had a long-standing history of linking academic investigators and community-based practices to identify, develop, and to conduct research and were expanding this approach to bridge with community. For example, the Community Health Improvement and Research Partnership model is one approach used by PBRNs to conduct community-engaged research and to mobilize research partnerships between academicians and diverse community stakeholders (e.g., health system, schools, business, service providers, engaged citizens) [Reference Davis44Reference Young-Lorion47].

Thus, it is possible that CTSAs could learn from and leverage existing PBRN expertise and infrastructure rather than to create parallel yet distinct programs covering the same regions [Reference Haggerty5]. However, in 2012 NCATS stopped requiring that a community engagement core was proposed within CTSA renewal applications, and many interpreted this to mean that CE had become a lower priority for NCATs. In reaction, some CTSAs de-emphasized their CE groups in renewal applications and prioritized core functions that were perceived as closer to the center of NCATS vision for clinical and translational research, which had deleterious effect on CTSA CE-supported PBRNs. Although the 2013 IOM report stressed the importance of community engagement within CTSAs, this did not result in NCATS’ reinstatement of CE as a required key function. Instead NCATS’ clarified that all CTSAs must have core resources across the full spectrum of translational research and emphasized that each CTSA program should emphasize it’s particular strengths [Reference Leshner7]. These changes may have prompted a shift away from CTSAs supporting PBRNs that are important developing assets but are not viewed as major strengths. Such changes would align with perceptions within the PBRN community that networks that were viewed as strengths within their CTSAs have received increased CTSA support over time, while PBRNs on the margins of their CTSAs have experienced funding cuts.

There are a few important limitations of this study. First, we only administered the survey instrument to CTSA CE Directors and PBRN Directors. It is possible that some PBRNs may be affiliated with the CTSA more broadly, or that other stakeholders (e.g., patients, investigators, community stakeholders) may have varied perceptions on CTSA/PBRN relationships. This approach was congruent with the original 2008 survey; participants were selected based on our understanding that these positions would have the most comprehensive view of CTSA/PBRN relationships. Second, less than half of all respondents approached completed the survey. However, this appears higher than the average response rate on web-based surveys [Reference Fan and Yan48]. Third, we conducted cross-sectional surveys in 2008 and 2014. In both surveys there was considerable heterogeneity in participant responses. Our analysis identified statistically significant differences between CTSA CE and PBRN Directors as well as across the 2008 and 2014 surveys. Qualitative findings reveal heterogeneity in participant responses that cannot fully be explored. Finally, because we deidentified participant responses we cannot link CTSAs or PBRNs to look at change over time within an individual settings. Despite these limitations, replication of the 2008 survey in 2013–2014 allowed us to understand incremental changes in PBRN/CTSA relationships and provides key considerations to inform for future research.

Our current findings indicate that even after 6 additional years, PBRN and CTSA relationships are still evolving. Although perceptions appear to be aligning in the quantitative data, qualitative themes revealed considerable variation across the participating sites. These variations may be due to historical relationships, federal changes to the CTSA mission or funding priorities for PBRNs, or to local competing priorities. A few case studies have described how CTSAs, PBRNs, and other regional programs have effectively partnered to promote research and community engagement [Reference Davis44, Reference Westfall49]. Future research is needed to explore how CTSA and PBRN relationships evolve over time and to identify factors that contribute to successful program alignment. We suggest that longitudinal as well as qualitative case studies may provide a rich opportunity to identify promising practices in CTSA and PBRN relationships. For example, conducting comparative case studies of “high” and “low” performing CTSA/PBRN collaborations could evaluate contextual factors and resource sharing structures over time. Qualitative methods could allow data collection from multiple CTSA and PBRN stakeholders (e.g., CTSA Directors, PBRN Network Managers, Affiliated Investigators, community-based stakeholders) and be used to identify promising practices in fostering collaborative rather than competitive partnerships for these programs. Conducting research and providing technical assistance to CTSAs/PBRNs in parallel could present an opportunity to share transportable lessons from what is working across the great variability in different CTSA and PBRN cultures, priorities, and settings. As increasing numbers of CTSAs submit for renewal of their applications under the modified RFAs, we may continue to see increasing variability in CTSA-PBRN relationships in the years ahead.

CTSAs and PBRNs have great overlap in their core missions of supporting the generation of relevant new knowledge to improve person and population health, and of speeding translation of research into practice for community benefit. Robust engagement of PBRNs has the potential to ensure that the questions asked by CTSA affiliated faculty serve the needs of “real world” practitioners and stakeholders across all phases of translational research, from basic research to population-based studies and back again [Reference Fort2]. Our 2008 study reported that in many cases PBRNs and CTSAs were in the discovery phase [Reference Fagnan4]. In the interim, PBRNs and CTSAs have undergone many changes in their internal and external contexts, yet a paucity of research has explored further evolution in these relationships [Reference Haggerty5]. Our updated 2014 survey results indicate that despite greater alignment in perspectives and a recognized value in a collaborative relationship, the future of CTSA and PBRN relationships is uncertain. Notably, the ability to respond with support to build the PBRN/CTSA relationship is limited as budgets are cut, primary care research is threatened, and the priority for community engagement by academic health centers changes. Our study shows growing recognition of the challenges and possibilities for continued evolution of collaborative opportunities between CTSAs and PBRNs to advance scientific knowledge, foster learning healthcare systems and to improve practice and community health.

Acknowledgments

The authors wish to thank the Clinical and Translational Science Award (CTSA) Community Engagement (CE) Directors and the Practice-Based Research Network (PBRN) Directors who completed the survey. The authors are also grateful to Amanda Ross for her help reviewing survey questions, identify the participant sample, and helping with study software.

Financial Support

This publication was made possible by the Clinical and Translational Science Collaborative of Cleveland, UL1TR000439 and to the Oregon Clinical and Translational Research Institute, UL1TR000128 from the National Center for Advancing Translational Sciences (NCATS) component of the National Institutes of Health and NIH roadmap for Medical Research. Dr Davis is supported in part by an Agency for Healthcare Research & Quality-Funded Patient Centered Outcomes Research (PCOR) K12 award (Award Number 1 K12 HS022981 01). Article contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH.

Disclosures

The authors have no conflicts of interest to declare.

Ethical Standard

The current study was approved by the Case Western Reserve University Institutional Review Board.

References

1. Westfall, JM, Mold, J, Fagnan, L. Practice-based research—“blue highways” on the NIH roadmap. JAMA 2007; 297: 403406.Google Scholar
2. Fort, DG, et al. Mapping the evolving definitions of translational research. Journal of Clinical and Translational Science 2017; 1: 6066.Google Scholar
3. Woolf, SH. The meaning of translational research and why it matters. JAMA 2008; 299: 211213.Google Scholar
4. Fagnan, LJ, et al. Linking practice-based research networks and Clinical and Translational Science Awards: new opportunities for community engagement by academic health centers. Academic Medicine: Journal of the Association of American Medical Colleges 2010; 85: 476483.Google Scholar
5. Haggerty, T, et al. Family medicine-specific Practice-Based Research Network Productivity and Clinical and Translational Sciences Award Program affiliation. Southern Medical Journal 2017; 110: 287292.Google Scholar
6. Zerhouni, E. Medicine. The NIH roadmap. Science (New York, NY) 2003; 302: 6372.Google Scholar
7. Leshner, AI, et al. CTSA Program at NIH: Opportunities for Advancing Clinical and Translational Research. Washington, DC: National Academies Press, 2013.Google Scholar
8. National Center for Advancing Translational Sciences (NCATS). About the center [Internet], 2017 [cited Jan 3, 2017] (https://ncats.nih.gov/about/center)Google Scholar
9. Feldman, AM. Clinical and Translational Science (CTS): 2005–2015. Clinical and Translational Science 2015; 8: 621622.Google Scholar
10. Clinical & Translational Science Awards (CTSA). [Internet], 2017 [cited July 22, 2017] (https://ctsacentral.org/)Google Scholar
11. Mold, JW, et al. Definitions of common terms relevant to primary care research. The Annals of Family Medicine 2008; 6: 570571.CrossRefGoogle ScholarPubMed
12. Agency for Healthcare Research and Quality. Practice-based Research Networks (PBRNS): About PBRNs [Internet], n.d. [cited July 9, 2016] (https://pbrn.ahrq.gov/about)Google Scholar
13. Davis, MM, et al. Characteristics and lessons learned from practice-based research networks (PBRNs) in the United States. Journal of Healthcare Leadership 2012; 4: 107116.Google Scholar
14. Agency for Healthcare Research and Quality. Practice-based Research Networks (PBRNs): Application of the PBRN Model to Non-Primary Care Settings. Rockville, MD: Agency for Healthcare Research and Quality, 2015, 9pp.Google Scholar
15. Mays, GP, et al. Public health research implementation and translation: evidence from practice-based research networks. American Journal of Preventive Medicine 2013; 45: 752762.Google Scholar
16. Tyler, CV, Werner, JJ. Community-engagement strategies of the developmental disabilities practice-based research network (DD-PBRN). Journal of the American Board of Family Medicine 2014; 27: 831838.Google Scholar
17. Mungia, R, et al. Generating national dental PBRN research ideas through the top consensus method workshop. Progress in Community Health Partnerships: Research, Education, and Action 2015; 9: 447456.Google Scholar
18. Gilbert, GH, et al. Purpose, structure, and function of the United States National Dental Practice-Based Research Network. Journal of Dentistry 2013; 41: 10511059.Google Scholar
19. Williams, RL, Rhyne, RL. No longer simply a Practice-based Research Network (PBRN): health improvement networks. The Journal of the American Board of Family Medicine 2011; 24: 485488.Google Scholar
20. Spears, W, et al. Use of community engagement strategies to increase research participation in practice-based research networks (PBRNs). Journal of the American Board of Family Medicine 2014; 27: 763771.Google Scholar
21. Westfall, JM, et al. Community-based participatory research in Practice-Based Research Networks. The Annals of Family Medicine 2006; 4: 814.Google Scholar
22. Pham, R, et al. “Finding the right FIT”: a community-led mixed methods study of rural patient preferences for fecal immunochemical test characteristics. Journal of the American Board of Family Medicine 2017; 30: 632–644.Google Scholar
23. Mold, JW, Peterson, KA. Primary care Practice-Based Research Networks: working at the interface between research and quality improvement. The Annals of Family Medicine 2005; 3(Suppl. 1): S12S20.Google Scholar
24. Ramanadhan, S, et al. Participatory implementation science to increase the impact of evidence-based cancer prevention and control. Under Review.Google Scholar
25. Bakken, S, et al. Barriers, enablers, and incentives for research participation: a report from the Ambulatory Care Research Network (ACRN). The Journal of the American Board of Family Medicine 2009; 22: 436445.Google Scholar
26. Gibson, K, et al. Physician perspectives on incentives to participate in practice-based research: a greater Rochester Practice-Based Research Network (GR-PBRN) Study. The Journal of the American Board of Family Medicine 2010; 23: 452454.Google Scholar
27. Collaborative Ohio Inquiry Network (COIN). The Certificate Program in Practice-based Research Methods [Internet], 2017 [cited July 22, 2017] (http://www.collaborativeohioinquirynetwork.com/pbrn-certificate-program.html)Google Scholar
28. Fagnan, LJ. Moving upstream-health extension and primary care. Journal of the American Board of Family Medicine 2017; 30: 1012.Google Scholar
29. DeVoe, JE, et al. Perspectives in primary care: a conceptual framework and path for integrating social determinants of health into primary care practice. The Annals of Family Medicine 2016; 14: 104108.Google Scholar
30. Heintzman, J, et al. Practice-based Research Networks (PBRNs) are promising laboratories for conducting dissemination and implementation research. The Journal of the American Board of Family Medicine 2014; 27: 759762.Google Scholar
31. Werner, JJ. Measuring the impact of Practice-based Research Networks (PBRNs). The Journal of the American Board of Family Medicine 2012; 25: 557559.Google Scholar
32. Peterson, KA, et al. Supporting better science in primary care: a description of Practice-based Research Networks (PBRNs) in 2011. The Journal of the American Board of Family Medicine 2012; 25: 565571.Google Scholar
33. Creswell, JW. Qualitative Inquiry and Research Design: Choosing Among Five Traditions, 2nd edition. Thousand Oaks, CA: Sage, 2007.Google Scholar
34. Elo, S, Kyngäs, H. The qualitative content analysis process. Journal of Advanced Nursing 2008; 62: 107115.Google Scholar
35. Crabtree, BF, Miller, WL. A template approach to text analysis: developing and using codebooks. In: Crabtree BF, Miller WL, eds Doing Qualitative Research in Primary Care: Multiple Strategies. Newbury Park, CA: Sage Publications, 1992, pp. 93109.Google Scholar
36. Cicchetti, D. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment. 1994; 6: 284290.Google Scholar
37. Smith, MRS, Stuckhardt, L, McGinnis, JM. eds Better Care at Lower Cost: The Path To Continuously Learning Health Care in America/Committee on the Learning Health Care System in America. Washington, DC: National Academies Press, 2013.Google Scholar
38. Kemp, KB. Using Evidence to Build a Learning Health Care System. Washington, DC: Academy Health, 2012.Google Scholar
39. Werner, JJ, Stange, KC. Praxis-based research networks: an emerging paradigm for research that is rigorous, relevant, and inclusive. Journal of the American Board of Family Medicine 2014; 27: 730735.Google Scholar
40. Carey, TS, et al. Practice-based research networks (PBRNs) in the era of integrated delivery systems. Journal of the American Board of Family Medicine 2015; 28: 658662.Google Scholar
41. Calmbach, WL, et al. Practice-based Research Networks (PBRNs): meeting the challenges of the future. Journal of the American Board of Family Medicine 2012; 25: 572576.Google Scholar
42. Gaglioti, AH, et al. Practice-based Research Networks (PBRNs) bridging the gaps between communities, funders, and policymakers. Journal of the American Board of Family Medicine 2016; 29: 630635.Google Scholar
43. Tapp, H, Dulin, M. The science of primary health-care improvement: potential and use of community-based participatory research by practice-based research networks for translation of research into practice. Experimental Biology and Medicine (Maywood, NJ) 2010; 235: 290299.CrossRefGoogle ScholarPubMed
44. Davis, MM, et al. Engaging the underserved: a process model to mobilize rural community health coalitions as partners in translational research. Clinical and Translational Science 2014; 7: 300306.Google Scholar
45. Davis, MM, et al. Milk Options Observation (MOO): a Mixed-Methods Study of chocolate milk removal on beverage consumption and student/staff behaviors in a rural elementary school. The Journal of School Nursing: The Official Publication of the National Association of School Nurses 2017; 33: 285–298.Google Scholar
46. McGinnis, PB, et al. Transitioning from CHIP to CHIRP: blending community health development with community-based participatory research. Family & Community Health 2010; 33: 228237.Google Scholar
47. Young-Lorion, J, et al. Rural Oregon community perspectives: introducing community-based participatory research into a community health coalition. Progress in Community Health Partnerships 2013; 7: 313322.CrossRefGoogle ScholarPubMed
48. Fan, W, Yan, Z. Factors affecting response rates of the web survey: a systematic review. Computers in Human Behavior 2010; 26: 132139.Google Scholar
49. Westfall, JM, et al. Engaging communities in education and research: PBRNs, AHEC, and CTSA. Clinical and Translational Science 2012; 5: 250258.Google Scholar
Figure 0

Table 1 Quantitative responses from Clinical and Translational Science Awards (CTSA) community engagement (CE) Directors and Practice-Based Research Networks (PBRN) Directors on the 2008 and 2014 web-based surveys regarding PBRN/CTSA relationships*

Figure 1

Table 2 Clinical and Translational Science Award (CTSA) community engagement (CE) and Practice-Based Research Network (PBRN) Directors perceptions of PBRN/CTSA relationships over time and in the future

Figure 2

Table 3 Practice-Based Research Network (PBRN) Directors’ perceptions of Clinical and Translational Science Award (CTSA) Leaders’ understanding of the importance of addressing/prioritizing the research needs and interests of PBRN clinician-members (n=42)

Figure 3

Table 4 Clinical and Translational Science Award (CTSA) community engagement (CE) Directors’ perceptions of the existing limitations of Practice-Based Research Network (PBRNs) to the CTSA mission (n=18)