Hostname: page-component-78c5997874-8bhkd Total loading time: 0 Render date: 2024-11-14T03:29:32.094Z Has data issue: false hasContentIssue false

Identifying strategies to promote team science in dissemination and implementation research

Published online by Cambridge University Press:  09 September 2019

Gregory A. Aarons*
Affiliation:
Department of Psychiatry, University of California San Diego, La Jolla, San Diego, CA 92093-0812, USA Child and Adolescent Services Research Center, San Diego, CA 92123, USA
Kendal Reeder
Affiliation:
Department of Psychiatry, University of California San Diego, La Jolla, San Diego, CA 92093-0812, USA Child and Adolescent Services Research Center, San Diego, CA 92123, USA
Christopher J. Miller
Affiliation:
Center for Healthcare Organization and Implementation Research (CHOIR), VA Boston Healthcare System, Boston, MA 02130, USA Department of Psychiatry, Harvard Medical School, Boston, MA 02115, USA
Nicole A. Stadnick
Affiliation:
Department of Psychiatry, University of California San Diego, La Jolla, San Diego, CA 92093-0812, USA Child and Adolescent Services Research Center, San Diego, CA 92123, USA
*
Address for correspondence: G. A. Aarons, PhD, Department of Psychiatry, University of California San Diego, 9500 Gilman Drive (0812), La Jolla, San Diego, CA 92093-0812, USA; Child and Adolescent Services Research Center, 3665 Kearny Villa Road, Suite 200N, San Diego, CA 92123, USA. Email: [email protected]
Rights & Permissions [Opens in a new window]

Abstract

Introduction:

Scientific endeavors are increasingly carried out by teams of scientists. While there is growing literature on factors associated with effective science teams, little is known about processes that facilitate the success of dissemination and implementation (D&I) teams studying the uptake of healthcare innovations. This study aimed to identify strategies used by D&I scientists to promote team science.

Methods:

Using a nominal group technique, a sample of 27 D&I scholars responded to the question, “What strategies have you or others used to promote team science?” Participants were asked to individually respond and then discuss within a small group to determine the group’s top three strategies. Through a facilitated consensus discussion with the full sample, a rank-ordered list of three strategies was determined.

Results:

A total of 126 individual responses (M = 9; SD = 4.88) were submitted. Through small group discussion, six groups ranked their top three strategies to promote team science. The final ranked list of strategies determined by the full sample included: (1) developing and maintaining clear expectations, (2) promoting and modeling effective communication, and (3) establishing shared goals and a mission of the work to be accomplished.

Conclusions:

Because of its goal of translating knowledge to practice, D&I research necessitates the use of team science. The top strategies are in line with those found to be effective for teams in other fields and hold promise for improving D&I team cohesion and innovation, which may ultimately accelerate the translation of health innovations and the improvement of care quality and outcomes.

Type
Research Article
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
© The Association for Clinical and Translational Science 2019

Introduction

Across the globe, a focus on teamwork and collaboration in the workplace is on the rise.1, Reference Salas, Stagl, Burke, Cooper and Robertson2 This growing trend is present in a variety of settings, including scientific research contexts.Reference Wuchty, Jones and Uzzi3 Team research efforts, referred to as team science, involve multiple scientists who carry out interdependent tasks to achieve a goal.Reference Stokols4 Compared to single investigators, science teams not only produce more academic publications, but also achieve higher impact factors, are cited more frequently, and are more likely to discover breakthroughs.Reference Wuchty, Jones and Uzzi3, Reference Singh and Fleming5 While scientific collaborations are often unidisciplinary in nature (i.e., within the same field), scientists are increasingly working in cross-disciplinary teams that bridge two or more fields and/or patients, community stakeholders, and other relevant stakeholders. However, engaging researchers and collaborators in implementation science teams and fostering the development of these teams can be a slow process.Reference Stokols4

Cross-disciplinary collaborations have been characterized as multidisciplinary, interdisciplinary, or transdisciplinary.Reference Rosenfield6 In a multidisciplinary team, researchers from different disciplines work within their respective fields and then apply their findings to a common research goal. For example, an epidemiologist might conduct a study while an anthropologist might address the same issues from a qualitative inquiry perspective, with their findings integrated post hoc. Interdisciplinary research involves more integration, such that researchers from separate fields (e.g., anthropology, medicine, social work) interactively work together to address a common problem (e.g., access to high-quality healthcare). Transdisciplinary research, the most collaborative form, involves a synthesis of distinct fields to develop models that transcend a single discipline and result in a hybrid discipline. For example, neuroscience, bioengineering, and human–computer interaction are hybrid disciplines that evolved from transdisciplinary efforts.Reference Muller7 We use the term cross-disciplinary to refer to science teams that collaborate at any of these levels.

Because cross-disciplinary teams may foster greater potential for innovation and creativity,Reference Lungeanu and Contractor8, Reference Kurtzberg9 they are especially important for tackling complex public health issues.Reference Disis and Slattery10Reference Hall12 As such, recent decades have seen a substantial increase in the amount of funding provided by both private foundations and federal agencies for team-based research efforts.Reference Cooke13 For example, National Institutes of Health (NIH) awards to projects with multiple principal investigators, first introduced in 2006, accounted for nearly 20% of all major grants funded in 2013.Reference Cooke13, 14 To further facilitate team science, NIH has established multiple cross-disciplinary programs and research centers.Reference Luke15, Reference Guise16 Team-based research efforts increasingly involve scientists from different organizationsReference Cooke13; multi-university collaborations have progressively become more common, and there have been recent calls to extend the scope of science team membership beyond academia.Reference Jones, Wuchty and Uzzi17Reference Selker and Wilkins19 Because many cross-disciplinary research initiatives ultimately aim to apply research findings to real-world settings to combat widespread public health issues,Reference Hall12, Reference Warnecke20, Reference Croyle21 collaborating with public stakeholders in team-based research efforts is a promising strategy to facilitate translation of these initiatives.Reference Altman22, Reference Hall, Frodeman, Thompson and Pacheco23

Moving from research to practice to improve health services and population health is an objective of the new science of dissemination and implementation (D&I).Reference Brownson24 There is a well-documented lag between the development of scientific evidence (e.g., new medications, psychosocial interventions) and their broad use in practice. This has resulted in a lag of roughly 17 years in the translation pipeline and then only about 14% of original research goes to the benefit of patient care.Reference Balas, Boren, Bemmel and McCray25 Teams are critical for D&I science, which naturally facilitates engagement between scientists and stakeholders (e.g., researchers within other disciplines, policymakers, community members) to accelerate the translation of new and effective evidence-based practices.Reference Yarborough26, Reference Proctor27 Cross-disciplinary D&I science teams have the potential to expedite the translation process through having the most relevant and innovative perspectives come together to determine the most efficient and effective ways to move research to practice. While there are challenges associated with co-production in science teams (e.g., disagreements, professional costs),Reference Oliver, Kothari and Mays28 research has described the richness and complexity of collaborative processes while also identifying multiple benefits that outweigh the challenges.Reference Aarons29 The requisite for teams to produce translatable outcomes, coupled with increased calls for the use of team science in D&I settings,Reference Lewis30Reference Tinkle33 creates a need for evidence-based strategies for facilitating successful D&I collaborations and synergistic transdisciplinary teams.Reference Hall34

Rich findings from research on team functioning suggest strategies for creating effective cross-disciplinary science teams.Reference Salas, Reyes and McDaniel35 This literature highlights strategies that enable successful teams, such as communication, the development of shared mental models (i.e., knowledge structures), trust, and shared leadership.Reference Marks, Mathieu and Zaccaro36Reference D’Innocenzo, Mathieu and Kukenberger40 While team effectiveness findings have typically been applied across settings, contextual factors may limit their generalizability.Reference Stokols11 Many findings come from studies of randomly composed teams in laboratory settings, where tasks and performance measures tend to be narrow. As such, literature on successful teams in real-world settings is on the rise, allowing strategies to be tailored to teams based on their setting and goals.Reference Salas, Reyes and McDaniel35 For example, studies on cross-disciplinary healthcare teams have found that team processes associated with team success include quick learning, intent listening, speaking up, and psychological safety.Reference Rosen41, Reference Edmondson42

The increasing calls for team science, however, have outpaced research on how to promote effective science teams. There are a number of questions relevant to science teams that have not been addressed, such as the effects of research structures and funding mechanisms on team functioning.Reference Cooke13 These questions and others are beginning to be addressed by the science of team science field, which aims to identify and understand strategies that are associated with successful science teams. Science of team science research has begun to uncover team processes that lead to effective team science, such as learning from other members and communicating face-to-face.Reference Hall34 Yet amidst calls for the greater use and study of team science in D&I contexts,Reference Lewis30Reference Tinkle33, Reference Chambers43 no studies have investigated strategies that facilitate success in D&I science teams.Reference Hall44 The aims of D&I science teams, which may include implementing evidence-based practices with fidelity,Reference Elliott and Mihalic45 sustaining these practices in community settings,Reference Altman22, Reference Israel46 or engaging public stakeholders in the research process,Reference Concannon47 introduce unique challenges not faced by cross-disciplinary teams that operate solely within academia.Reference Tebes and Thai18, Reference Hall44 The objective of this study was to identify and rank strategies to promote team science among scholars engaged in D&I research.

Methods

Procedures

The nominal group technique (NGT) was used in this study. The NGT is a commonly used group consensus method in health services research that involves a facilitated, multistep structured group process to generate and prioritize responses to a stimulus question among a panel who have expertise in a focal area,Reference Jones and Hunter48Reference Fink51 without relying on an a priori established conceptual model that might inadvertently bias results. This approach was used for pragmatic reasons (e.g., minimal preparation, only one meeting needed) and for fit with the goal of the meeting (i.e., to identify strategies used by D&I researchers to promote team science). In addition, we wanted to capitalize on the unique opportunity to learn from a diverse group of multidisciplinary researchers engaged in D&I research. We applied the recommendations to ensure methodological rigor when using consensus group methods to report the steps and results from the NGT approach used in this study.Reference Humphrey-Murto52

We used a modified NGT process that proceeded in the following stages. First, the facilitator (GAA) presented the nominal question: What strategies have you or others used to promote team science? Participants were first asked to generate ideas and write down their responses to this question individually. They were also asked to rank order their top three ideas. Next, small groups (consisting of 3–6 individuals) were formed and participants were asked to take turns sharing all of their individual responses with the group, to compile strategies within the group, and to rank order the top three strategies agreed upon by their small group (round 1). Following the first two rounds, the facilitator requested that each small group, in turn, report out their ratings and these were written out on a flip chart for all to view and consider (round 2). The facilitator then led all participants in a full group consensus ranking process to determine the top three strategies (round 3). After each round, the facilitator summarized responses and elicited participant confirmation of the summaries. Collective group consensus was defined as agreement indicated by verbal and/or nonverbal approval.

Setting

This study was conducted during the weeklong in-person training component of the National Institute of Mental Health (NIMH)/Veterans Affairs Health System (VA)/National Institute on Drug Abuse (NIDA)-sponsored Implementation Research Institute (IRI) on June 11, 2018 at the Knight Executive Education and Conference Center and was hosted by the George Warren School of Social Work and the Institute for Public Health at Washington University in St. Louis, Missouri. Data were collected as part of an hour-long interactive activity focused on team science and working on cross-disciplinary teams within D&I research. This study was approved by the Institutional Review Board of the University of California, San Diego as exempt research.

Participants

Twenty-one IRI fellows and six core faculty (n = 27) attended the presentation. The IRI is a 2-year interdisciplinary training program that provides in-depth training in D&I science through an annual learning collaborative, individualized mentoring, and a mentoring network of faculty and fellows to advance the research and careers of implementation science investigators. Participants were predominantly female (67%, n = 18). In terms of professional characteristics, the majority of participants had doctoral (i.e., PhD or equivalent) research degrees (82%, n = 22) and the remaining participants had medical degrees (19%; n = 5). There was a range of disciplines represented across the participants: Psychology (37%, n = 10), Social Work (7%, n = 2), Medicine (7%, n = 2), Psychiatry (7%, n = 2), Pediatrics (7%, n = 2), Public Health (7%, n = 2), Engineering (7%, n = 2), Anthropology (4%, n = 1), Sociology (4%, n = 1), Organizational Behavior and Management (4%, n = 1), Education (4%, n = 1), and Health and Public Policy (4%, n = 1). Participants were conducting implementation research in both domestic and international (i.e., low- and middle-income countries) settings and in a variety of healthcare contexts (e.g., primary care, mHealth, public systems, organizational level, schools, community mental health, prisons, Federally Qualified Health Centers). See Table 1 for participant characteristics.

Table 1. Sample characteristics

Note: n = 27

The facilitator (GAA) requested to collect individual and group written responses from those attendees who were willing to share their data. The NGT process typically includes 5–12 members, so our sample size of 27 exceeds this standard.Reference Humphrey-Murto52 Anonymity of participants was maintained by ensuring that no personally identifying information could be linked to a response, and no additional qualitative data were collected.

Data Analysis

Descriptive statistics were used to summarize the individual data and aggregated rankings. Data analysis and reduction was conducted as part of the NGT process so that data aggregation and rankings were complete at the end of the participatory group exercise.

Results

During the NGT exercise, in round 1, individual participants were asked to report, from their own written list, their top three strategies. In round 2, each of six small groups considered all of the individual written responses of all group members and reported out to the full group the top three strategies agreed upon within their small group. In round 3, group consensus was achieved from all small groups. After the third round, 14 of the 27 participants (response rate = 52%) submitted their individual written responses from round 1 to the research team for a total of 126 responses. The average number of individual responses was 9.00 (SD = 4.88; range = 4–18).

The three strategies from each of the six groups identified through the consensus process, along with their definitions, are reported in Table 2. Through full group discussion, the top strategies endorsed by each small group were consolidated into four primary categories. The categories were: (A) a focus on clear expectations, responsibilities, and well-defined roles of team members, (B) promoting effective communication, which includes respectful and assertive communication styles between team members and leadership, (C) fostering a culture of mutual respect and trust, and (D) developing a shared mission and/or set of goals. All six small groups submitted at least one strategy related to Category A, four of the small groups submitted at least one strategy related to Category B, four of the small groups submitted at least one strategy related to Category C, and three of the small groups submitted a strategy related to Category D. As shown in Table 3, the full group’s consensus process (i.e., all participants) identified the top three strategies for promoting team science in implementation and their respective ranks. Through the facilitated group consensus discussion (round 3), the final rank-ordered list of strategies for promoting team science were: (1) developing and maintaining clear expectations, (2) promoting and modeling effective communication, and (3) establishing shared goals and a mission of the work to be accomplished. Individual participant responses are provided in Table 4.

Table 2. Top three strategies identified by small groups and rank ordered within group

Table 3. Top three strategies identified in full group consensus ranking process

Table 4. Strategies identified by individuals

Discussion

We examined factors recommended to facilitate team science among a cohort of D&I researchers, including experienced faculty and early- to mid-career fellows of the NIMH/VA/NIDA-sponsored IRI. The top three strategies for fostering team science in D&I (by rank order) endorsed by the group were (1) setting clear expectations and roles in conducting research, (2) promoting and modeling effective communication, and (3) establishing shared goals and mission. These findings are broadly consistent with promoting team science in other fields,Reference Hall34, Reference Salas, Reyes and McDaniel35, Reference Marlow37 and with findings and conceptual models regarding team building more broadly.Reference Edmondson53, Reference Hackman and Tindale54 There are several possible reasons that these aspects of team functioning were seen as particularly relevant by our respondents. First, D&I science typically brings together disparate fields such as clinical services, implementation, organization, policy, and economics – in fact, many implementation science questions are impossible to answer without involving these diverse viewpoints. Each of these fields may bring different expectations regarding what research goals are most important and how best to communicate within the team. Thus, there is a need to reformulate research training to foster a more interprofessional perspective in favor of focusing too much on one’s own discipline.Reference Little55 Such an approach could lead to more professional empathy regarding other perspectives in team science. Second, because D&I science is a relatively new field, there will likely be variability among team members regarding the breadth and depth of familiarity with D&I concepts, frameworks, research designs, methods, and measures. This also suggests that clear communication may be especially important for D&I science teams. While the NGT resulted in the top three recommended strategies, it is important to acknowledge that there were many other recommendations from one or more of the six small groups that should also be considered. For example, relationships, psychological safety, and trust are important in research and also important in collaborative and mentoring relationships. Developing consensus while modeling respect and developing a sense of psychological safety can foster cross-disciplinary team science where there is a need for respect of differing perspectives and values. We also recommend that D&I research aspires to the level of “transdisciplinary” research to support effective collaboration and to promote synergy and innovation.

This recommendation naturally begs the question: What strategies might be used to develop transdisciplinary teams that will be innovative, effective, and productive in the field of D&I science? The answers are beyond the purview of the present study; however, we suggest that effective leadership and development of collaboration and mentoring networks can help to achieve these goals. These were identified in the NGT process and warrant further consideration. For example, the Leadership and Organizational Change for Implementation intervention,Reference Aarons56 while designed to create a climate for implementation of evidence-based practices, could be adapted to create a climate for team science in academic settings and in community academic partnerships.Reference Drahota57

There are many frameworks and theories that help to organize and describe implementation determinants and processes.Reference Tabak58, Reference Nilsen59 The Exploration, Preparation, Implementation, Sustainment (EPIS) framework describes outer (e.g., health system) and inner (i.e., organization) factors that might impact implementation as well as the four phases of the implementation process.Reference Aarons, Hurlburt and Horwitz60, Reference Moullin61 It is useful to consider how multidisciplinary teams of different configurations might be useful at different phases of the implementation process and also for understanding outer and inner context factors. For example, including an organizational psychology perspective can be useful in understanding operations in hospitals, clinics, or teams. Much of this work can also be considered to occur in the Exploration and Preparation phases of the EPIS framework. For example, the EPIS processes of collaboration or co-production as “bridging factors” that span outer and inner contexts can be considered to be active in bringing together diverse stakeholders including researchers and community or health system partners.Reference Moullin61 Such partnerships and collaborations can help to make implementation science more immediately relevant both theoretically and practically.

The call for team science is becoming the norm for research funders and academic institutions. For D&I research, team science and the need for collaboration to share and integrate perspectives, ideas, and ideals are essential and may be even more critical than in other fields. This is a function of the broad purview of concepts and approaches and crossing-cutting issues and contexts in D&I. These include different disease and health promotion areas, patient and service populations, service systems and settings that span geographical contexts (e.g., different continents, countries, municipalities), and technology platforms (e.g., eHealth, mHealth) where implementation occurs. Because of the inherently interdisciplinary and collaborative nature of the field of D&I science, it is imperative that we understand how to promote appropriate team science. This includes building teams with needed expertise and skills for professional empathy to both appreciate and apply the perspectives, theories, and methods of others working together in pursuit of solving D&I issues and advancing the field.

Even the process of funding invokes the need for team science. For example, in the USA, there are formal calls for D&I research that involves over 17 of the institutes within the NIH. In the Dissemination and Implementation Research in Health (DIRH) review committee, tasked with the scientific evaluation of D&I research, proposal members may have specific expertise in a given disease or public health category (e.g., cancer, addiction, mental health). However, the common language, concepts, and approaches of proposals being evaluated is that of D&I. The DIRH review panel itself could be considered an interdisciplinary evaluation team. While this is a promising approach to scientific review of these types of applications, there are still different and varied perspectives depending on disease and contextual issues (e.g., implementation in low- and middle-income countries) that require more effective team science in the conduct of science and, as in this committee, the evaluation of the science.

The findings presented in this study raise questions about the ease and difficulties of teams in implementation science. For example, the top-ranked strategy was related to clear expectations, responsibilities, and well-defined roles of team members. Implementing this strategy may present challenges where there are strong opinions or preference for leadership roles versus team member roles. Promoting effective communication can be dependent on the ability of leaders to engender respectful communication among team members, and the development of mutual respect and trust to create a shared mission and/or set of goals. In addition to the current study results, more detail about potential barriers to team science can be garnered from “lessons learned” from a team science project addressing multiple chronic health conditions.Reference Garg62 Reflecting on that process, the authors identified the need for funding for effective infrastructures to foster team science, taking a solutions-focused approach that supports in-person networking, matching collaborators, and having dedicated support for coordination of activities. Our findings and those from other studies are helping to build the knowledge base and promote testable theories for promoting, enhancing, and sustaining team science.

Limitations

It is important to note some limitations. First, the study was conducted with one group of implementation researchers whose work largely focused on behavioral health or substance use issues. However, this concern is mitigated by the fact that the D&I research of the group is fundamentally multidisciplinary in nature and spans diverse issues and settings. For example, issues addressed by faculty and fellows include mental health in emergency departments, behavioral issues related to gastrointestinal disease, testing multilevel implementation strategies in a range of service settings, implementation of evidence-based interventions in low- and middle-income countries, applying system dynamics and engineering approaches to D&I and use of health technologies. Related, while the participants represented a diverse array of disciplines, we acknowledge that not all disciplines (e.g., econometrics) were represented. Second, we did not use ancillary or other methods of data collection such as surveys. Thus, the results may not have identified all possible issues that would address team science in D&I research. However, the methods utilized allowed for participants to raise issues beyond what might have been included in a survey. Finally, we did not capture the degree to which responses were related to participants’ own use of strategies to promote team science versus strategies they had observed others using. However, members of team research initiatives can be and often are involved in creating, employing, and participating in team science promotion strategies, allowing for scoping observations from a wide variety of perspectives.

Conclusions

Team science is being called for and increasingly expected by funders and academic institutions to facilitate collaboration, innovation, and productivity in regard to translating health innovations to real-world settings. There is a strong argument for using team science in D&I research because of its inherently broad and complex nature. Thus, it is important to understand and develop approaches for fostering and improving team science in D&I research. Our study results indicate that key strategies for fostering team science in D&I include (1) setting clear expectations and roles in conducting research, (2) promoting and modeling effective communication, and (3) establishing shared goals and mission. Pursuing these strategies will likely improve team cohesion and innovation, and result in approaches to accelerate the adoption, implementation, and sustainment of health innovations across contexts and with the promise of improving quality of care and outcomes.

Acknowledgments

Dr. Aarons is core faculty and Drs. Stadnick and Miller are fellows with the Implementation Research Institute (IRI), at the George Warren Brown School of Social Work, Washington University in St. Louis. The IRI is supported through an award from the National Institute of Mental Health (R25MH08091607).

This project was supported in part by the US National Institute of Mental Health grants R25MH080916 (E. Proctor), K23MH110602 (Stadnick), R03MH117493 and R01MH072961 (Aarons), and National Institute on Drug Abuse R01DA038466 (Aarons). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Disclosures

The authors declare that they have no competing interests.

Ethics approval and Consent to Participate

Ethics approval was evaluated and determined to be exempt by the UC San Diego Human Research Protections Program.

References

Deloitte. Global Human Capital Trends 2016—The New Organization: Different by Design. Deloitte University Press; 2016 (retrieved from https://www2.deloitte.com/content/dam/Deloitte/global/Documents/HumanCapital/gx-dup-global-human-capital-trends-2016.pdf).Google Scholar
Salas, E, Stagl, KC, Burke, CS. 25 years of team effectiveness in organizations: research themes and emerging needs. In: Cooper, CL, Robertson, IT, eds. International Review of Industrial and Organizational Psychology. West Sussex, UK: John Wiley & Sons; 2004. 4792.Google Scholar
Wuchty, S, Jones, BF, Uzzi, B. The increasing dominance of teams in production of knowledge. Science 2007; 316(5827): 10361039.CrossRefGoogle ScholarPubMed
Stokols, D, et al. The science of team science: overview of the field and introduction to the supplement. American Journal of Preventive Medicine 2008; 35(2): S77S89.CrossRefGoogle ScholarPubMed
Singh, J, Fleming, L. Lone inventors as sources of breakthroughs: Myth or reality? Management Science 2010; 56(1): 4156.CrossRefGoogle Scholar
Rosenfield, PL. The potential of transdisciplinary research for sustaining and extending linkages between the health and social sciences. Social Science & Medicine 1992; 35(11): 13431357.CrossRefGoogle ScholarPubMed
Muller, MJ. Participatory design: the third space in HCI. Human-Computer Interaction: Development Process 2003; 4235: 165185.Google Scholar
Lungeanu, A, Contractor, NS. The effects of diversity and network ties on innovations: the emergence of a new scientific field. American Behavioral Scientist 2015; 59(5): 548564.CrossRefGoogle ScholarPubMed
Kurtzberg, TR. Feeling creative, being creative: an empirical study of diversity and creativity in teams. Creativity Research Journal 2005; 17(1): 5165.CrossRefGoogle Scholar
Disis, ML, Slattery, JT. The road we must take: multidisciplinary team science. Science Translational Medicine 2010; 2(22): 22cm29.CrossRefGoogle ScholarPubMed
Stokols, D, et al. The ecology of team science: understanding contextual influences on transdisciplinary collaboration. American Journal of Preventive Medicine 2008; 35(2 Suppl): S96S115. doi: 10.1016/j.amepre.2008.05.003.CrossRefGoogle ScholarPubMed
Hall, KL, et al. News from NIH: resources for team-based research to more effectively address complex public health problems. Translational Behavioral Medicine 2012; 2(4): 373375.CrossRefGoogle ScholarPubMed
Cooke, NJ, et al. Enhancing the Effectiveness of Team Science. Washington, DC: National Academies Press, 2015.Google Scholar
Establishment of Multiple Principal Investigator Awards for the Support of Team Science Projects. Secondary Establishment of Multiple Principal Investigator Awards for the Support of Team Science Projects 2006. https://grants.nih.gov/grants/guide/notice-files/not-od-07-017.html.Google Scholar
Luke, DA, et al. Breaking down silos: mapping growth of cross‐disciplinary collaboration in a translational science initiative. Clinical and Translational Science 2015; 8(2): 143149.CrossRefGoogle Scholar
Guise, JM, et al. Organizational and training factors that promote team science: a qualitative analysis and application of theory to the National Institutes of Health’s BIRCWH career development program. Journal of Clinical and Translational Science 2017; 1(2): 101107.CrossRefGoogle ScholarPubMed
Jones, BF, Wuchty, S, Uzzi, B. Multi-university research teams: shifting impact, geography, and stratification in science. Science 2008; 322(5905): 12591262.CrossRefGoogle Scholar
Tebes, JK, Thai, ND. Interdisciplinary team science and the public: steps toward a participatory team science. American Psychologist 2018; 73(4): 549.CrossRefGoogle Scholar
Selker, HP, Wilkins, CH. From community engagement, to community-engaged research, to broadly engaged team science. Journal of Clinical and Translational Science 2017; 1(1): 56.CrossRefGoogle ScholarPubMed
Warnecke, RB, et al. Approaching health disparities from a population perspective: The National Institutes of Health Centers for Population Health and Health Disparities. American Journal of Public Health 2008; 98(9): 16081615.CrossRefGoogle ScholarPubMed
Croyle, RT. The National Cancer Institute’s transdisciplinary centers initiatives and the need for building a science of team science. American Journal of Preventive Medicine 2008; 35(2): S90S93.CrossRefGoogle ScholarPubMed
Altman, DG. Sustaining interventions in community systems: on the relationship between researchers and communities. Health Psychology 1995; 14(6): 526.CrossRefGoogle ScholarPubMed
Hall, KL, et al. Understanding cross-disciplinary team-based research. In: Frodeman, R, Thompson, JK, Pacheco, RCS, eds. The Oxford Handbook of Interdisciplinarity. Oxford, UK: Oxford University Press; 2017. 338356.Google Scholar
Brownson, RC. Dissemination and Implementation Research in Health: Translating Science to Practice. Oxford University Press, 2017.CrossRefGoogle Scholar
Balas, EA, Boren, SA. Managing clinical knowledge for healthcare improvements. In: Bemmel, J, McCray, AT, eds. Yearbook of Medical Informatics 2000: Patient-Centered Systems. Stuttgart, Germany: Schattauer Verlagsgesellschaft; 2000. 6570.Google Scholar
Yarborough, M, et al. Relationships hold the key to trustworthy and productive translational science: recommendations for expanding community engagement in biomedical research. Clinical and Translational Science 2013; 6(4): 310.CrossRefGoogle ScholarPubMed
Proctor, EK, et al. Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Administration and Policy in Mental Health and Mental Health Services Research 2009; 36(1): 2434.CrossRefGoogle ScholarPubMed
Oliver, K, Kothari, A, Mays, N. The dark side of coproduction: do the costs outweigh the benefits for health research? Health Research Policy and Systems 2019; 17(1): 33.CrossRefGoogle ScholarPubMed
Aarons, GA, et al. Collaboration, negotiation, and coalescence for interagency-collaborative teams to scale-up evidence-based practice. Journal of Clinical Child and Adolescent Psychology 2014; 43(6): 915928.CrossRefGoogle ScholarPubMed
Lewis, CL, et al. Proceedings of the 3rd Biennial Conference of the Society for Implementation Research Collaboration (SIRC) 2015: Advancing efficient methodologies through community partnerships and team science. Implementation Science; 2016; 11: 85, BioMed Central.CrossRefGoogle Scholar
Puga, F, Stevens, KR, Patel, DI. Adopting best practices from team science in a healthcare improvement research network: the impact on dissemination and implementation. Nursing Research and Practice 2013; 2013.CrossRefGoogle Scholar
Glasgow, RE, et al. National Institutes of Health approaches to dissemination and implementation science: current and future directions. American Journal of Public Health 2012; 102(7): 12741281. doi: 10.2105/AJPH.2012.300755.CrossRefGoogle ScholarPubMed
Tinkle, M, et al. Dissemination and implementation research funded by the US National Institutes of Health, 2005–2012. Nursing Research and Practice 2013; 2013.CrossRefGoogle Scholar
Hall, KL, et al. The science of team science: a review of the empirical evidence and research gaps on collaboration in science. American Psychologist 2018; 73(4): 532.CrossRefGoogle Scholar
Salas, E, Reyes, DL, McDaniel, SH. The science of teamwork: progress, reflections, and the road ahead. American Psychologist 2018; 73(4): 593.CrossRefGoogle ScholarPubMed
Marks, MA, Mathieu, JE, Zaccaro, SJ. A temporally based framework and taxonomy of team processes. Academy of Management Review 2001; 26(3): 356376.CrossRefGoogle Scholar
Marlow, SL, et al. Does team communication represent a one-size-fits-all approach? A meta-analysis of team communication and performance. Organizational Behavior and Human Decision Processes 2018; 144: 145170.CrossRefGoogle Scholar
DeChurch, LA, Mesmer-Magnus, JR. Measuring shared team mental models: a meta-analysis. Group Dynamics: Theory, Research, and Practice 2010; 14(1): 1.CrossRefGoogle Scholar
De Jong, BA, Dirks, KT, Gillespie, N. Trust and team performance: a meta-analysis of main effects, moderators, and covariates. Journal of Applied Psychology 2016; 101(8): 1134.CrossRefGoogle ScholarPubMed
D’Innocenzo, L, Mathieu, JE, Kukenberger, MR. A meta-analysis of different forms of shared leadership–team performance relations. Journal of Management 2016; 42(7): 19641991.CrossRefGoogle Scholar
Rosen, MA, et al. Teamwork in healthcare: key discoveries enabling safer, high-quality care. American Psychologist 2018; 73(4): 433.CrossRefGoogle ScholarPubMed
Edmondson, AC. Speaking up in the operating room: how team leaders promote learning in interdisciplinary action teams. Journal of Management Studies 2003; 40(6): 14191452.CrossRefGoogle Scholar
Chambers, DA, et al. Mapping training needs for dissemination and implementation research: lessons from a synthesis of existing D&I research training programs. Translational Behavioral Medicine 2016; 7(3): 593601.CrossRefGoogle Scholar
Hall, KL, et al. Moving the science of team science forward: collaboration and creativity. American Journal of Preventive Medicine 2008; 35(2): S243S249.CrossRefGoogle ScholarPubMed
Elliott, DS, Mihalic, S. Issues in disseminating and replicating effective prevention programs. Prevention Science 2004; 5(1): 4752.CrossRefGoogle ScholarPubMed
Israel, BA, et al. Review of community-based research: assessing partnership approaches to improve public health. Annual Review of Public Health 1998; 19: 173202.CrossRefGoogle ScholarPubMed
Concannon, TW, et al. A systematic review of stakeholder engagement in comparative effectiveness and patient-centered outcomes research. Journal of General Internal Medicine 2014; 29(12): 16921701.CrossRefGoogle ScholarPubMed
Jones, J, Hunter, D. Consensus methods for medical and health services research. BMJ: British Medical Journal 1995; 311(7001): 376.CrossRefGoogle ScholarPubMed
Delbecq, AL, Van de Ven, AH, Gustafson, DH. Group Techniques for Program Planning: A Guide to Nominal Group and Delphi Processes. Glenview, IL: Scott Foresman Company, 1975.Google Scholar
Van de Ven, AH, Delbecq, AL. The nominal group as a research instrument for exploratory health studies. American Journal of Public Health 1972; 62(3): 337342.CrossRefGoogle Scholar
Fink, A, et al. Consensus methods: characteristics and guidelines for use. American Journal of Public Health 1984; 74(9): 979983.CrossRefGoogle ScholarPubMed
Humphrey-Murto, S, et al. Using consensus group methods such as Delphi and Nominal Group in medical education research. Medical Teacher 2017; 39(1): 1419.CrossRefGoogle Scholar
Edmondson, AC. Teaming: How Organizations Learn, Innovate, and Compete in the Knowledge Economy. John Wiley & Sons, 2012.Google Scholar
Hackman, JR. Why teams don’t work. In: Tindale, RS, et al. eds. Theory and Research on Small Groups. Boston, MA: Springer; 2002. 245267.CrossRefGoogle Scholar
Little, MM, et al. Team science as interprofessional collaborative research practice: asystematic review of the science of team science literature. Journal of Investigative Medicine 2017; 65(1): 1522.CrossRefGoogle Scholar
Aarons, GA, et al. Testing the Leadership and Organizational Change for Implementation (LOCI) Intervention in substance abuse treatment: a cluster randomized trial study protocol. Implementation Science 2017; 12(1): 29. doi: 10.1186/s13012-014-0192-y.CrossRefGoogle ScholarPubMed
Drahota, A, et al. Community-academic partnerships: asystematic review of the state of the literature and recommendations for future research. Milbank Quarterly 2016; 94(1): 163214.CrossRefGoogle Scholar
Tabak, RG, et al. Bridging research and practice: models for dissemination and implementation research. American Journal of Preventive Medicine 2012; 43(3): 337350.CrossRefGoogle Scholar
Nilsen, P. Making sense of implementation theories, models and frameworks. Implementation Science 2015; 10: 53.CrossRefGoogle ScholarPubMed
Aarons, GA, Hurlburt, M, Horwitz, SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research 2011; 38(1): 423. doi: 10.1007/s10488-010-0327-7.CrossRefGoogle ScholarPubMed
Moullin, JC, et al. Systematic review of the Exploration, Preparation, Implementation, Sustainment (EPIS) framework. Implementation Science 2019; 14(1): 1. doi: 10.1186/s13012-018-0842-6.CrossRefGoogle ScholarPubMed
Garg, T, et al. The AGING Initiative experience: a call for sustained support for team science networks. Health Research Policy and Systems 2018; 16(1): 41.CrossRefGoogle Scholar
Figure 0

Table 1. Sample characteristics

Figure 1

Table 2. Top three strategies identified by small groups and rank ordered within group

Figure 2

Table 3. Top three strategies identified in full group consensus ranking process

Figure 3

Table 4. Strategies identified by individuals