Mapping the evolving definitions of translational research

Abstract

Objective

Systematic review and analysis of definitions of translational research.

Materials and methods

The final corpus was comprised of 33 papers, each read by at least 2 reviewers. Definitions were mapped to a common set of research processes for presentation and analysis. Influence of papers and definitions was further evaluated using citation analysis and agglomerative clustering.

Results

All definitions were mapped to common research processes, revealing most common labels for each process. Agglomerative clustering revealed 3 broad families of definitions. Citation analysis showed that the originating paper of each family has been cited ~10 times more than any other member.

Discussion

Although there is little agreement between definitions, we were able to identify an emerging consensus 5-phase (T0–T4) definition for translational research. T1 involves processes that bring ideas from basic research through early testing in humans. T2 involves the establishment of effectiveness in humans and clinical guidelines. T3 primarily focuses on implementation and dissemination research while T4 focuses on outcomes and effectiveness in populations. T0 involves research such as genome-wide association studies which wrap back around to basic research.

Conclusion

We used systematic review and analysis to identify emerging consensus between definitions of translational research phases.

Introduction

Translational research as a concept has been widely used and applied in scientific literature for more than a decade. It is most broadly and simply defined as research steps to take discoveries “from the bench to the beside and back again.” What, precisely, this means in practice has been the subject of continuous, evolving discussion.

At the turn of the 21st century, advances in biomedical sciences and particularly genomics led to concerns that the volume of new discovery could not be “translated” into positive impacts on human health [1]. These concerns were captured by the Institute of Medicine in a series of roundtable discussions and workshops, and framed as 2 discrete “translational blocks” or “gaps” labeled T1 and T2, respectively, and described by Sung et al. starting in 2003 [2–6]. These workshops also provided the conceptual framework for the creation of the Clinical and Translational Science Award (CTSA) program by the National Institutes of Health in 2006 [7]. As institutions attempted to put translational research into practice, various authors began to modify and elaborate the original definitions. A T3 gap was split from T2 in 2007 [8], with the addition of a T4 and T0 soon following [9, 10].

The evolving number of steps, and changing definition of each step, reflect changing nature and understanding of basic bioscience research and clinical medicine. However, they also impact the description, design, conduct, and funding of research. Investigators and program coordinators need a common vocabulary to frame intent and significance of research. Simply put, translational researchers need to learn to speak the same language. Although a handful of papers have been instrumental in explicitly modifying the original definition, these alone are insufficient to understand how the concept of translational research is applied [11–13]. Outside of this handful, source definitions have been explained, adapted to different contexts (such as epidemiology) [14], and re-explained for yet others (such as medical education) [15]. Any review which does not take the broader context of how these definitions are applied will fall short.

An informal literature review of this topic by one of the authors (Starren) received significant interest from the CTSA community [16]. To expand on that preliminary work, we undertook a systematic literature review for definitions of the translational research phases and analysis to determine how these definitions have evolved over time. In this paper, we seek to better understand the differences between definitions of translational research, how they have changed over time, and which sources or authors were most influential in those changes.

Materials and Methods

Search

Research librarians (Shaw, Gutzman) were consulted to construct searches across several literature databases. The search strategy was developed in PubMed MEDLINE and adapted appropriately to conform to the differing controlled vocabularies and search syntax associated with each subsequent database. Databases searched were PubMed MEDLINE, Scopus, Web of Science, and Embase. In addition, a search of Google for non-journal literature, web pages, and presentations was conducted. Performance of search strings was evaluated with retrieval of a small gold standard corpus identified during manual review for preliminary work [16]. See Table 1 for database-specific search strings.

Table 1 Database-specific search strings

Research librarians were consulted to construct searchers across several literature databases. The search strategy was developed in PubMed MEDLINE and adapted to conform to the controlled vocabulary and syntax of each database. The order of brackets represents correct syntax for the search engines utilized rather than grammatical convention.

Bibliographic search identified 531 papers. Full text was retrieved for all English-language articles either digitally or through interlibrary loan. All initial papers were manually curated to select those which discussed and defined translational research phases, resulting in 68 papers for full reviewer attention. The 68 papers were each read by 2 primary reviewers. Of those, 35 papers were disqualified at this stage for various reasons such as a paper being a review itself rather than a novel definition, or because it only replicated a pre-existing definition (eg, with a referenced figure). In the instance where a paper cited a qualifying definition of translational research phases which was not in the corpus, the original defining paper [8] was substituted for the citing paper. The final corpus comprised of 33 papers [8–10, 14, 15, 17–44]. See Fig. 1 for a flow chart summarizing search, filtering, and review.

Fig. 1 Systematic search and filtering flow chart. Bibliographic search in 4 sources, deduplication, manual curation, and dual reviewer filtering produced a final corpus of 33 papers.

Review

Each paper in the corpus was read by at least 2 reviewers (Fort, Herr). Reviewers mapped each paper’s translational phase definitions to a set of research activities defined for this effort. In instances of broad disagreement or where consensus over minor differences could not be reached, a third reader (Starren) was used for arbitration.

Categories

Common process categories were developed through an iterative approach which started with all unique translational gap definitions and followed by abstractive refinement into a common set. The first subset of processes (basic research through Phase IV clinical trials) are assumed to be continuous such that the phrase “basic research through Phase IV trials” maps all intervening progress categories. All remaining processes must be explicitly mentioned to receive a label. However, a similar “continuum” of later stage research (comparative effectiveness research through disease modeling and -omic studies) has been assigned post hoc based on most common labeling and the assumption that translational phases imply order (ie, processes associated with T4 follow those in T3). Finally, 3 early categories (target validation, lead optimization, and process development) were collapsed into 1 category (target development) for final presentation as there was no variation in their labeling across the entire corpus.

Citation Analysis

Citation data were retrieved from Scopus title and PubMed identifier (PMID) of each paper in the corpus. Annual global citations for each paper were compiled to indicate relative influence of each paper over time. Intracorpus citations (ie, which paper in the corpus cited which other papers in the corpus) were compiled as a directed network and manually arranged to indicate chains of acknowledged influence within the corpus. Nodes represent papers and directed edges indicate a citation of the target by the source node. Node size and color are proportional to the node’s in-degree, in this case the number of citations of that paper by other papers within the corpus. In a handful of incidents, recorded citations predate official publication of a paper and indicate prior online availability. In order to clarify chains of influence, date of original availability, be it online or in official publication, was used for this analysis.

Consensus Analysis

An emerging consensus definition of translational research phases was derived from the label results of the primary review. Label definitions were “horizontally summed” across processes to determine most common label for each process. Results are displayed as fraction of papers in the corpus and the final consensus reflects the most common label for any research activity regardless of how many papers used the given research process. Early clinical trial phases are labeled as T2** to reflect the clear shift in labeling following 2010 despite the historic majority of T1 labels.

Similarity Analysis

Labeled processes for each reviewed paper were compiled as vectors of nominal variables. Dissimilarity matrix calculation and agglomerative clustering were performed using daisy and agnes functions of the default clustering package in R. The goal of this clustering is to evaluate chains of influence within the corpus based on definition similarity rather than the citation analysis performed above.

Results

Primary Review, Consensus, Clustering, and Total Citations

Our final corpus was comprised of 33 papers, filtered from 68 strong candidates out of an initial returned pool of 531 papers [8–10, 14, 15, 17–44]. Labeling of translational phase definitions and total citations for each paper in the corpus are summarized in Fig. 2. Overall, the papers identified 25 discrete research activities. Early research activities (basic research through Phase IV clinical trials) are assumed to be continuous, whereas later categories were ordered based on common labeling and the assumption that translational phases imply continuity (ie, T4 follows T3). In the figure, papers are horizontally ordered by similarity as defined by the agglomerative clustering. In instances where definitions uniquely labeled parts of the research continuum as something other than a translational phase (eg, “Clinical Research” in Sung et al. [17]), these labels have been preserved. Alongside the table, consensus labeling for each translational phase is presented as a line graph of the fraction of processes assigned to each label and results in an emerging consensus categorization.

Fig. 2 Primary review results with consensus, clustering, and total citation information. The center of the figure shows the results of primary definition labeling. Blank cells indicate that the particular paper did not mention that research activity. Target development includes 3 named activities that were categorized the same by all papers (target validation, lead optimization, and lead development). The top of the figure shows a dendrogram representing the results of agglomerative clustering on the activity categories, resulting in 3 main definition families and a set of outliers (the “Other” grouping and Blumberg on the right), and also defines the order of papers for presentation. The far right side of the figure includes a consensus categorization and graph showing the frequency of assignment of each process to each T-phase as a fraction of all papers in the corpus. Early clinical trial phases are labeled as mixed T2**. Although historic majority labeling is T1, since 2010 the predominant and emerging consensus label for these processes is T2. Citation counts for each paper are included below as a bar graph overlaid with the actual citation count for each paper.

The result of agglomerative clustering is visualized as a dendrogram and defines the order of the presentation of definitions. Here, depth of matched pairs in the dendrogram denotes higher similarity between source definitions, and the branches denote “families” or “lineages” of similar definitions. This process identified 3 major families of definitions with an additional set of outliers for discussion. These families are the “gap” model originated by Sung et al. [17], where translational research is conceptualized as bridging gaps in a more traditional research process; the “continuum” model originated by Khoury et al. [9], where the same phases are relatively continuous across all research processes; and the “mixed” model originated by Woolf [22], which appear to match the gap definitions in early structure and the continuum definitions in the inclusion of later phases. With the exception of Shekhar et al. [35], the mixed definitions are notable for not mentioning clinical trial phases at all. As will be expanded on later, the originating paper of each family has been cited ~10-fold more than any other paper in the family, suggesting that each family represents a distinct school of thought with a clear anchoring work.

Citation Frequency

Annual citation counts for each paper in the corpus are compiled in Table 2 as a heat map. The 33 papers in the corpus have been cited 2782 times (average 82 citations per paper). Sung et al. [17] and Woolf [22] are the most-cited papers, despite Sung et al. (2003) predating Woolf (2008) by 5 years. These citation data strongly suggest an explosion of interest and discussion on the topic of translational research gaps in 2008 and 2009, with total annual citations of the corpus doubling each of these years. Overall, 67% of the citations of the corpus, including 4 of the 5 most-cited papers, were published in the Journal of the American Medical Association.

Table 2 Annual citation frequency and journal summary

Annual number of citations for each paper are presented as a heat map, illustrating an explosion of interest in this topic in 2008/2009. Four out of the top five papers, accounting for 67% of total citations, were published in the Journal of the American Medical Association. Bolded values in the Citation total column denote the top five most frequently cited papers in the corpus. Note that four of these top five are also the most internally cited papers in Figure 3.

Directed Citation Network

Citations within the corpus were converted into the directed network in Fig. 3 to visualize influence within the published literature. We hypothesized that larger and more strongly colored nodes represent papers with greater acknowledged influence upon the evolving definition of translational research phases. As with the citation heat map, some included papers are poorly cited or uncited. However, there is evidence of chains of influence within the corpus. Sung et al. [17], Westfall et al. [8], Woolf [22], and Dougherty and Conway [20] are notable for their influence within the corpus.

Fig. 3 Directed citation network. Nodes represent papers in the corpus. Directed edges represent a citation of the target by the source. Size and color of each node reflects the number of times that paper was cited by other papers in the corpus (red, large—high citation count; yellow, small—low citation count; and green, tiny—no citation count). Height of a node corresponds to year of first availability either in print or online.

Discussion

The definition of translation phases has shown remarkable evolution in a relatively short time. Not only have the number of translation phases increased from 2 to 5, but the activities assigned to each phase have also changed. This analysis makes equally clear that the definition of translational research phases remains an area of disagreement within the translational research community. In spite of the lack of unanimity regarding translational research phases, a number of consensus patterns do emerge.

Emerging Consensus Definition of Translational Research

The definition of T1 translational research demonstrates the highest degree of consensus, with 75% of papers agreeing that T1 research comprises processes from basic research to initial testing in humans. Approximately half of these agree that T1 continues through early clinical trial phases, whereas the remainder put even these early clinical trial phases in the realm of T2. Most definitions put the end of T1 at the establishment of clinical efficacy of an intervention, or the Phase II clinical trial. While the T1 label is historically dominant, T2 has emerged as the most common label for these research processes after 2010. Therefore we have labeled early phase clinical trials as T2** in our emerging consensus definition.

Following early clinical trial phases, T2 is broadly agreed upon to relate to the establishment of effectiveness of an intervention and particularly the establishment of clinical guidelines. T3 is broadly agreed to focus on implementation and dissemination research. T4, when it appears in definitions, is concerned with outcomes and effectiveness research. Definitions including a T0 phase are relatively rare, but define it as steps which close the research cycle back to T1, such as genome-wide association studies. Although a few CTSA institutions have included a T5 phase in their descriptions [45], we were unable to locate a mention of T5 in the peer-reviewed literature using our search strategy. As originally conceived, T1 and T2 translational research bridged the “gaps” between the endpoints of traditional bench and clinical research and this is evident in the early papers by Sung et al. [17], Hait [18], and Westfall et al. [8]. These definitions persist into later discussions by Morris et al. [33] and Rubio et al. [27], and are also supported by heavy ongoing citation of these original papers. However, by the time discussion of the topic exploded in 2008/2009 the consensus definition of translational research had evolved to a “continuum” of translational research.

In the newer definitions, traditional bench and clinical research become part of a process where scientific ideas are translated across a continuous research spectrum and phases in this continuum are labeled by common setting or research methods. Although there is still significant disagreement in labeling of these phases, dating back to their originators (eg, Khoury et al. [9] vs. Chesla [19]), continuum definitions of translational research (n=13) are more prevalent than the original gap definitions (n=8).

Of further interest is that the difference between these 2 approaches is readily visible in an agglomerative clustering of definitions. The same clustering also reveals an almost hybrid group of definitions, labeled as the mixed model family. These are interesting for matching the gap definitions in early structure where they exclude clinical research from all labeling (particularly notable in the transition from Sung et al. [17] to Woolf [22]), but better resemble the continuum definitions in terms of later translational research phases.

Evolution of Translational Research Definitions

The evolution from gap to continuum definitions of translational research represents the single most obvious step in the discussion of this topic. Beyond that commonality, however, there are detectable points of consensus regarding definitions of individual translational research phases discussed above. Also notable is that while additional translational phases (T3, T4, T0) are widely understood to have been added over time, a 4-phase continuous definition from Khoury et al. appears as early as 2007 [9], roughly concurrent with the better-cited papers by Woolf [22] and Westfall et al. [8], and predates the explosion in discussion on this topic around 2008/2009.

The addition of higher translational research phases appears to serve 2 purposes. Points where agreement is muddy, such as the range of outcome and effectiveness research processes, demonstrate where the addition of an extra phase (T4) has added clarity. Early T2 and T3 definitions are evenly reported for these processes, demonstrating a lack of clarity which was apparently solved by assigning these processes to a fourth translational phase. This is in contrast to the addition of step (T0) which adds a fundamentally new idea to the research continuum. Before the appearance of the T0 translational research phase, there is very little apparent discussion of closing the research cycle back to T1.

Finally, Phase IV clinical trials and comparative effectiveness research, the processes at which research moves into establishing real-world effectiveness of interventions, represent a point of almost maximum disagreement or flux within our results. Most definitions before 2011 put Phase IV clinical trials as part of T2 or T3 research where afterwards it is more likely to appear as T4. We hypothesize that this effect may be an artifact of the Patient-Centered Outcomes Research Institute (PCORI) publicizing comparative effectiveness research both as an important research topic and as subtly distinct concept than what it had been before [46]. However, there was not enough momentum in these changes for us to deviate from the historic majority label on these processes at this time.

Citation Patterns and Influence

The originating paper in each definition family has been cited ~10-fold more than any other paper, suggesting an acknowledged lineage and anchor within each family. This lends credence to the idea that the mixed model family is as defined as the gap and continuum models. What also stands out is that 2 of the 5 most-cited papers (Westfall et al. [8] and Dougherty and Conway [20]) have no corresponding families. As seen in the citation network and in total citations, these papers have an acknowledged historical influence on the discussion around translational research, but the influence never extended to propagating their specific conceptual definitions.

The results pertaining to citations, influence, and similarity also lend themselves to minor commentary on the publication and dissemination of new ideas. The paper by Sung et al. [17], a report on a series of workshops held by the then Institute of Medicine, is widely considered the originating manuscript on this topic. However, it is the later paper by Woolf [22] in the same journal which is cited most frequently even though Woolf repeats nearly the exact same definition. The reason for this difference is not obvious. It may be that Sung’s paper was overlooked as a workshop report. Perhaps Woolf’s paper appeared at a more opportune time. Finally, Woolf’s paper may have been more prominent in electronic searches because the title contained the words “translational research.”

Also notable is that 4 of the top 5 most-cited publications appear in a single journal—the Journal of the American Medical Association. The exception, Khoury et al. [9], also serves as something of a cautionary tale. In 2007, predating both Woolf [22] and Westfall et al. [8], Khoury presented a 4-phase translational research continuum which highly predicts what would emerge as the later consensus on translational research. Yet this first Khoury paper shows little evidence of direct influence within our corpus and 4 out of 5 of the citing papers feature Khoury as first or senior author [10, 26, 32, 44]. It is not for 4 years (2011), and appearance of these additional papers later, that we observe adoption of these ideas. Again, we can only speculate whether the original Khoury paper found publication in a less visible journal or was simply ahead of its time.

Limitations

This work has 4 primary limitations. First, as with any systematic review, our analysis was limited to those papers we retrieved and, therefore, relied entirely on the strength of our search strategy. With that in mind, we designed our search strategy in consultation with professional research librarians and evaluated it using a gold standard set which was manually identified during preliminary work [16]. The second limitation involves our research process categories and labeling. Categories were derived through an iterative approach where research processes were abstracted from definitions in our final corpus. A limitation of this is that 2 papers may use slightly different words to describe the same process and synonymy is based on human judgment. To minimize variation, we employed 2 independent reviewers with a third acting as an adjudicator to facilitate consensus categorization. Third, our conclusions about citation frequency and dissemination of ideas do not take into account citation context. We contend that the intersection of agglomerative clustering and citation frequency are sufficient for our conclusions, but our results are limited by not examining citation context. Finally, our consensus assignments of processes to categories represent, primarily, a voting based on simple majority labeling rather than a formal consensus development process involving active participation of the various authors. Thus, it is possible that the more common, rather than the more persuasive, assignment for a particular category may have been chosen. Such a process was outside the scope of this investigation, though exceptions such as the T1/T2 overlap in early clinical research phases have been noted. We hope that this analysis could provide a starting point for such an exercise.

Conclusions

We used systematic review and analysis to identify emerging consensus between definitions of translational research phases. T1 involves processes that bring ideas from basic research through early testing in humans. T2 involves the establishment of effectiveness in humans and clinical guidelines. T3 primarily focuses on implementation and dissemination research while T4 focuses on outcomes and effectiveness in populations. T0 involves research such as genome-wide association studies which wrap back around to basic research. Within the field of translational research, we have also been able to describe evolution of definitions over time and families of definitions based on similarity. In addition, we have demonstrated that while citations are an important tool to describe the influence of any particular paper, acknowledgment of this influence does not mean dissemination of the ideas of the paper. Finally, while our techniques have been useful within the field of translational research, we do hope they prove useful in similar analysis of other complex topics.