Education and training of clinical and translational study investigators and research coordinators: A competency-based approach

Abstract

Introduction

Training for the clinical research workforce does not sufficiently prepare workers for today’s scientific complexity; deficiencies may be ameliorated with training. The Enhancing Clinical Research Professionals’ Training and Qualifications developed competency standards for principal investigators and clinical research coordinators.

Methods

Clinical and Translational Science Awards representatives refined competency statements. Working groups developed assessments, identified training, and highlighted gaps.

Results

Forty-eight competency statements in 8 domains were developed.

Conclusions

Training is primarily investigator focused with few programs for clinical research coordinators. Lack of training is felt in new technologies and data management. There are no standardized assessments of competence.

Introduction

This report reflects the consensus of the Clinical and Translational Science Awards (CTSA) Consortium Enhancing Clinical Research Professionals’ Training and Qualification (ECRPTQ) project that responded to a mandate to improve the efficiency of clinical trials through educating principal investigators (PIs) and clinical research coordinators (CRCs) in core clinical trial competencies. The objectives of this project were to codify the core competencies into a single high-level set of standards that could serve as the framework for defining professional competency across the clinical research continuum. There is pressure to provide institution-specific training for some aspects of job performance, but we believe there is significant value and efficiency to be gained by a uniform curriculum that trains to a standard set of competencies.

The Institute of Medicine (IOM) 2012 workshop report entitled Envisioning a Transformed Clinical Trials Enterprise in the United States, stated that the more traditional areas of mechanistic research and efficacy trials call for specialized workforces that until now have all too often depended on ad hoc, “on-the-job” learning as opposed to the prospective training and education that defines a mature discipline [1]. The data from the US Food and Drug Administration Center for Drug Evaluation and Research inspections from fiscal years 2004–2011 (n=2325) of US clinical trial sites show the following clinical investigator deficiencies: 42% protocol violations, 30% record-keeping deficiencies, 12% informed consent deficiencies, 10% of drug accountability violations, 9% Institutional Review Board (IRB) communication, and 5% with problems reporting adverse events (AEs) [2]. Moreover, despite recent requirements for Good Clinical Practice (GCP) training of investigators and clinical research staff, a decrease in these deficiencies and violations have persisted (see http://bit.ly/1maLi8W [3]). These data suggest that there is a need for an intervention to increase clinical investigator and coordinator competence and improve clinical trial performance metrics.

We hypothesize that clinical trial (and particularly multisite clinical trial) performance will be significantly improved by a standard set of systematically harmonized competencies that equip PIs and CRCs with the necessary skills to more effectively, efficiently, and safely execute clinical trials.

Current State of PI and CRC Training

The IOM has issued a challenge to create a clinical research workforce that can address increasing complexity in clinical and translational research and generate study results that reach the community more efficiently [1]. This challenge has focused attention on a clinical research workforce that includes not only PIs, but also CRCs and staff members. To date, education and training for PIs has evolved through the CTSA, but this training did not reach all investigators, nor the many clinical research professionals, and other team members who were external to the local CTSA funding mechanism. Mandates for GCP training by industry sponsors and IRBs have provided a minimal training activity for investigators and their clinical research teams. Academic and nonacademic sites provide local training and educational links, but often these efforts are unfunded and/or institution-specific activities. A formalized education and training requirement is lacking for individuals working in this profession.

Competency-based Education (CBE)

Over the past decade, various professional societies and institutes have supported a CBE approach for clinical research professionals. CBE identifies specific learning outcomes for knowledge and its application and is often referred to as outcomes-based education. Part of an educational trend that emerged in the 1970s, this approach has been endorsed by the IOM in its 2005 report. Characteristics that distinguish CBE include the following:

• ∙ Learner outcomes that are based on analysis of typical job responsibilities of practitioners.

• ∙ A curriculum focused on what learners need to learn to perform specific job tasks, not necessarily on traditional subject matter.

• ∙ Hierarchically sequenced modules that allow learners to proceed at their own pace.

• ∙ Educators employing assessment techniques that measure learner performance in settings that approximate the real environment [4].

The evolution of CBE/competency-based training and the publication of core competencies in clinical research offer a pathway for achieving workforce development goals [5]. CBE/competency-based training promise a skilled workforce to a variety of stakeholders by mapping core competencies to educational and training curricula. The competency-based approach ultimately defines competences and qualifications in a systematic learner-centric pathway [6]. Competencies represent not only basic knowledge, but higher levels of knowledge, skills, and attitudes (KSAs) that embody the profession.

Core competencies for clinical research nurse coordinators evolved from several works generated by the Royal College of Nursing in the United Kingdom, Oncology Nursing Society, and a National Institutes of Health (NIH) Clinical Research Nurse Working Group [7–9]. Other role delineation work for clinical research nurses have continued to study these domains and specific KSAs in practice [10–13]. Many of the role delineations attributed to clinical research nurses can also be attributed to non-nurse CRCs [14].

In addition, core competencies for clinical and translational PIs were featured on the CTSA Web site [15] and formed the basis for curriculum development for master-level courses under the CTSA, beginning with the K-30 awards. The role of pharmaceutical physicians, which is primarily a role found in Europe and South America, have also resulted in an evolved core competency framework [16].

Materials and Methods

Criteria for Selecting a Competency Framework

The ECRPTQ leadership team considered a variety of competency frameworks for this phase:

• ∙ The CTSA Education and Career Development Key Function Committee developed the CTSA master-level competencies, approved in 2011 (http://bit.ly/21cX5n8), to define the training standards for individuals functioning at the master’s level in clinical and translational research [15]. This framework includes 14 thematic areas that are intended to shape the training experiences of early career investigators and it represents the foundation for many graduate programs in clinical research across the CTSA Consortium. Although the ECRPTQ leadership team felt these competencies to be highly relevant for investigators, they did not fully address the necessary qualifications and skills for other team members. The ECRPTQ leadership team also examined specialty competencies in a variety of areas developed by CTSA key function committees, including bioinformatics.

• ∙ The NIH Clinical Research Nursing Domains of Practice for the Specialty of Clinical Research Nursing (http://cc.nih.gov/nursing/crn/DOP_document.pdf) was another framework that was reviewed and considered, but also did not necessarily address the qualifications and skills for all team members.

• ∙ Competencies outlined by the Oncology Nursing Society (https://www.ons.org/sites/default/files/ctncompetencies.pdf) were also reviewed but the group felt they were rather limited, focusing only on oncology studies.

• ∙ The ECRPTQ leadership also acknowledged the work of the National Research Coordinator Consortium, formerly known as the CTSA Research Coordinator Taskforce. This outlined job description recommendations and identified critical training needs and resources for CRCs. Like the other domains listed previously, the ECRPTQ leadership felt a framework that would be inclusive of all study team members was needed.

Ultimately, the framework proposed by the Joint Task Force for Clinical Trial Competency (JTF) [5] was selected that identified 8 broad domains of competence:

1. (1) Scientific concepts and research design

2. (2) Ethical and participant safety considerations

3. (3) Medicines development and regulation

4. (4) Clinical trial operations

5. (5) Study and site management

6. (6) Data management and informatics

7. (7) Leadership and professionalism

8. (8) Communication and teamwork

The ECRPTQ leadership team selected this framework because of its comprehensive applicability and its widespread uptake by numerous other stakeholders in the clinical trial enterprise. An additional consideration was the knowledge that the JTF mapped its competency framework to those mentioned previously, along with additional competency frameworks identified by Consortium of Academic Programs in Clinical Research (CoAPCR) and the UK National Health Service [17]. The JTF model is displayed in Table 1.

Table 1 JTF Core Competencies

GCP, Good Clinical Practice.

Competency Domain Working Groups: Recruitment and Composition

This CTSA-wide endeavor drew upon expertise across the consortium for this important and complex project. The ECRPTQ leadership team invited members of the CTSA Consortium with relevant expertise to participate in each of the Competency Domain Working Groups, which were created based on the 8 competency domains identified by the JTF. Two co-leads were appointed to each group. The expectations of the groups included a desire to focus on 2 roles: PIs and CRCs conducting clinical trials. The deliverables also included the charge to review and refine the JTF competency statements, identify assessment areas for each competency statement, and determine gaps in existing training.

As part of the first phase of the ECRPTQ project, a Social/Behavioral Research (S/BR) Working Group was created in response to a recognized need to address GCP in an appropriate and meaningful way for researchers conducting clinical trial testing behavioral interventions. This work is described in a separate paper authored by Murphy and her colleagues in this journal. In this phase of the ECRPTQ project, members of the S/BR Working Group were invited to participate in each of the Competency Domain Working Groups, contributing their expertise.

Competency Domain Working Group Process

Workgroups began working as soon as co-leads were identified and workgroup membership assigned. S/BR Working Group members were embedded within the Competency Domain Working Groups to provide feedback to ensure that the competencies were inclusive of S/BR. After a series of conference calls, email exchanges, and 2 working meetings in 2015, all Competency Domain Working Groups submitted their deliverables to the leadership team. Following this submission, a review team comprised of individuals from across the consortium conducted a thorough appraisal of this work to synthesize and collate the materials and provide a final draft to be reviewed by the Project Leadership Team before being forwarded to the CTSA Steering Committee. The core competencies were then reviewed by the JTF, CoAPCR, and Association of Clinical Research Professionals (ACRP). The final meeting was attended by project leadership, the Competency Domain Co-leads, JTF, and CoAPCR.

At the final meeting, the attendees revisited the importance of focusing on drafting competency statements that represent clear and measurable expressions of performance for professionals involved in clinical trials. The attendees also discussed areas of potential overlap within the framework, with some competencies appearing in more than 1 general competency domain. The group agreed that some overlap was acceptable and probably necessary under some circumstances.

Clinical Trial Competency Statements

The 51 competency statements written by the JTF were carefully reviewed by Competency Domain Working Groups and review teams. Of the 51 JTF statements, 34 were modified to enhance meaning and to reflect a focus on clinical trials. Five of the JTF competency statements were removed as stand-alone statements and were rewritten as assessments for other competencies, and 3 new ECRPTQ competency statements were added. In total, 48 ECRPTQ competency statements reflect the work of these groups (see Appendix 1).

Several items are particularly noteworthy regarding the overall work:

• ∙ First, the Competency Domain Working Group reviewing the Medicines Development and Regulation domain called for a renaming of that domain to Investigational Products Development and Regulation to be more inclusive of device research.

• ∙ Second, the domain Communication and Teamwork was separated into 2 domains at the request of the ECRPTQ leadership team early in the Phase II process, believing the concept of team science to be of critical importance for the CTSA Consortium. Because the field of team science is still emerging and that the relevant skills are still being defined, the decision was made that the domain should be combined with the Leadership and Professionalism domain to become Leadership, Professionalism, and Team Science. As team science competencies emerge and are refined, it may make sense later to separate the domains.

• ∙ Third, there are purposefully some areas of overlap across domains, as the group agreed that understanding concepts can be very related but still highly nuanced.

• ∙ The S/BR Working Group suggested additional edits to 6 competency statements in 3 competency domains. These suggestions are relevant to study teams specifically conducting clinical trials involving behavioral interventions or assessments.

• ∙ Echoing a sentiment expressed by the JTF at its meeting in April 2015, the group recognized the importance of regular updates and revisions to this work, corresponding to advances in science and concomitant regulations.

Competency Domain Assessments

Competency statements are broad and meant to be generally applicable to both PIs and CRCs; however, in thinking of how to apply these statements to different roles within a study team, Competency Domain Working Groups identified areas of assessment specific to investigators and CRCs, with a principal focus on entry-level individuals.

Competency Domain Working Groups were provided with an overview of Bloom’s taxonomy [18] to assist them in writing measurable and appropriately leveled assessments that are at higher level KSAs. Most assessments proposed by the Competency Domain Working Groups are specific and measurable, such as “Prepare a research question” and “Demonstrate knowledge of appropriate control, storage, and dispensing of investigational products.” Groups identified a total of 429 potential assessments across all domains, with 220 identified as appropriate for investigators and 209 identified as appropriate for CRCs, a list that is in no way exhaustive. A considerable number of these assessments are identical for investigators and for CRCs, so the total does not represent unique assessments.

In terms of methods, many Competency Domain Working Groups suggested the use of case studies and observation of behavior, indicating that simply passing a quiz or multiple-choice exam is not indicative of mastery; some also recommended pre/post testing. It is important to acknowledge that while the creation of detailed assessment methods was not within the scope of this work, some working groups did suggest methods that might be employed. (See http://bit.ly/1Qfm5qr for more details about the assessments by competency domain.)

Existing Clinical Trial Training

Competency Domain Working Groups were also tasked with identifying and examining existing training believed to address the competencies identified within the groups’ respective competency domains. They took a broad approach to this task, based on the expertise and experiential knowledge of their members. It was not possible to examine every training program that might exist, but the groups reviewed a reasonably representative sample of offerings. In general, included were training and education offered by CTSA institutions; by professional organizations devoted to education and resources; by industry; and by government units (see http://bit.ly/1IV821x, entitled training gaps by competency domains). Collectively, Competency Domain Working Groups examined the following number of education and training offerings (with some offerings identified as appropriate for multiple competencies) (see Table 2).

Table 2 Summary of existing education and trainings

CTSA, Clinical and Translational Science Awards.

From Table 2, of the 343, 219 unique education and training offerings were identified.

Every competency domain had associated training with a few exceptions, noted below:

• ∙ Leadership, Professionalism, and Team Science: identify and apply professional guidelines and codes of ethics as they relate to the conduct of clinical trials. No training identified.

• ∙ Leadership, Professionalism, and Team Science: Describe the methods necessary to work effectively with multidisciplinary and interprofessional research teams. No training identified.

• ∙ Communication: Describe the component parts of a traditional scientific publication. No training identified.

Just over half of the offerings identified appear to be available online; not all of those offerings are readily accessible in the public domain, or free of charge, however. In addition to the trainings identified as meeting-specified competencies, groups cited 214 supplemental resources, including Web sites, reports, books, and published articles. See http://bit.ly/1U3PYbS for the catalog of Identified Existing Education and Trainings organized by competency domain.

Sample online offerings:

• ∙ Northwestern University, Clinical and Translational Sciences Institute Introduction to Clinical Research Online Modules

• ∙ University of Washington, Institute of Translational Health Sciences (ITHS), Self-Directed Learning Center

• ∙ Office of Research Integrity: The Lab, The Research Clinic

• ∙ NIH: Teaching the Responsible Conduct of Research

• ∙ ACRP: GCP—An introduction to ICH GCP Guidelines

• ∙ Collaborative Institutional Training Initiative (CITI): Populations in Research Requiring Additional Consideration

• ∙ UC Davis: Strengthening Provider Patient Communication Skills in Clinical Trials.

Competency Domain Working Groups were not tasked to consider cost as a factor in the groups’ work, but it is certainly an important consideration and some groups did so. The cost of these offerings ranges from free to a significant financial investment by the department or individual.

Similarly, the working groups were not tasked to consider training quality, but many did so nevertheless identifying the need for training that actively engages the learner, going beyond mere rote memorization. A number of Competency Domain Working Groups noted that existing training focuses too heavily on theoretical concepts and historical events and not enough on the application of knowledge. Competency Domain Working Groups frequently noted that online offerings are insufficient alone and must be supplemented by local institutional education and training. The education and training offerings that are listed in this report (see http://bit.ly/1U3PYbS) are not endorsed by the ECRPTQ leadership team. It was outside the parameters of this project to conduct a thorough evaluation of the suggested trainings from the Competency Domain Working Groups. There was broad agreement that the assessment of their quality should be undertaken as a future phase of the project.

Results

Identified Training Gaps

Competency Domain Working Groups identified 115 specific training gaps across the 8 reviewed domains. In the review of identified trainings, 5 broad categories of gaps were identified: (1) training that is needed but does not currently exist; (2) existing training that is not adequate; (3) certification, documentation of skill, or formal assessment is needed; (4) a core training curriculum needs to be defined and/or developed; and (5) there is inadequate training at a level for CRCs. Overwhelmingly, groups concluded that some training exists for most ECRPTQ competency domains, but that this training is not adequate to fully meet the needs of the investigators and CRCs. This finding is consistent with the data collected in the JTF Core Competency Survey; participants were asked whether they felt a need for training in these domains, with the majority of respondents indicating they felt training was needed [19].

Overall, the education and training identified is primarily investigator focused, particularly offerings provided by CTSA institutions. There are few organized curricular programs available for CRCs. Lack of training is keenly felt in the adoption of new technologies and in areas such as data management. Very importantly, there are no standardized assessments of competence in the domains. Training is not generally organized by level of expertise, and often does not distinguish roles and responsibilities between investigators and CRCs. Some Competency Domain Working Groups, such as Leadership, Professionalism, and Team Science, and Communication were forced to draw from offerings completely outside the context of clinical research, because they were unable to find relevant training within clinical research. As noted previously, many of the trainings cited in this report do not provide opportunities to apply knowledge and do not incorporate learning strategies known to be effective in promoting the development of competence. See http://bit.ly/1IV821x for more details about training gaps by competency domain.

S/BR Working Group Considerations

The S/BR Working Group reviewed all the Competency Domain Working Groups’ deliverables and offered suggested edits to the ECRPTQ competencies in some domains to promote greater inclusion of research teams carrying out clinical trials involving behavioral interventions and assessments. Additional comments made by the group regarding trainings and assessments for select domains were made (reference to http://bit.ly/1RaeSIk).

Recommendations

• ∙ The recently completed JTF Core Competency Survey that is currently being analyzed will provide important data that will help validate the JTF framework. The survey asked individuals to self-assess their own level of competence in the framework’s domains, as well as the relevance of those domains and their perceived needs for additional training. More objective measures of competency are needed to complement these data. This organization is also currently exploring mechanisms to revise and update its competency statements, particularly as new scientific fields and technologies emerge.

• ∙ Building upon the identification of assessment areas by Competency Domain Working Groups, specific assessments must be developed to assess competence. Such assessments should focus not only on different study team member roles, but levels of mastery as well.

• ∙ An evaluation of the quality of existing training should be undertaken, as well as an expansion of the catalog of training that emerged from the working groups. This catalog does not include all available educational opportunities and should be expanded to include additional relevant training.

• ∙ A deeper exploration of training gaps should be undertaken to determine whether new training modules are needed. If so, the development of this education and training should be undertaken by individuals skilled in instructional design and curriculum development, built upon the principles of adult learning.

• ∙ Examination of a cloud-based learning management platform to support individuals seeking and tracking their CBE and assessment, based on the competency framework developed in this work may be warranted.

• ∙ An ePortfolio system would be identified to allow PIs and CRCs to collect, organize, and share their completed trainings, demonstrate learning, and have a portable record of their achievements. Such a system would allow individuals to upload artifacts of competence that could be made available to relevant institutions, sponsors, and other regulatory bodies.

• ∙ Because of the extraordinary opportunity of working with external stakeholder organizations to identify standard competencies for these 2 cohorts (investigators and CRCs), these continued activities should be undertaken in partnership with individuals from the JTF, CoAPCR, ACRP, Clinical Trials Transformation Initiative (CTTI) and other organizations invested in clinical research professional training. These organizations are committed to study team education and training, and the JTF framework is gaining significant traction nationally and internationally through their efforts.

• ∙ Recommendations for institutional policies on clinical research training should be expanded beyond basic IRB and GCP training, with step-wise approaches to training personnel.

• ∙ Job descriptions should be modified to reflect specific competencies by levels.

• ∙ Inter-institutional training courses should be developed and shared as cost-free, easily accessible, web-based formats with additional train-the-trainer mechanisms for onsite training and continuing education.

• ∙ Academic pathways for baccalaureate and graduate degrees in clinical research should be endorsed and more highly accessible to clinical research professionals working in academic medical centers and hospitals. If an institution does not offer a clinical research program of study, then clinical research staff should be able to transfer educational benefits to other institutions.

Conclusions

This document is the result of the CTSA Consortium ECRPTQ supplement and includes discussion and consensus documents with iterative revisions. The competencies and assessments generated in Phase II of the ECRPTQ supplement have been approved by the CTSA Consortium and have the support of JTF, CoAPCR, and ACRP. The competencies and assessments have been submitted to National Center for Advancing Translational Sciences (NCATS) for their consideration. Although these documents provide a framework for investigator and CRC training it does not provide direction on how to implement a training program. The purpose of this document is to provide a standard set of core clinical trial competencies and KSA assessments that equip investigators and CRCs with the necessary skills to more effectively, efficiently, and safely execute clinical trials. Education and training should be patterned to these competencies and ultimately lead to specific formative and summative evaluations of learning.