157 Development of a skills assessment for the formative evaluation of clinical research professionals working in behavioral and social science research contexts

Abstract

Objectives/Goals: To support the growing number of clinical research professionals (CRP) working in behavioral and social science contexts, relevant formative assessments of research skills are needed. This study examines the development of an objective formative assessment designed to assess skills relevant to conducting social and behavioral research. Methods/Study Population: A multidisciplinary group at the University of Michigan was convened to develop the assessment. Case studies depicting clinical and translational research conducted in behavioral and social science contexts were used to measure proficiency in seven of the eight ECRPTQ competencies: Scientific Concepts & Research Design, Ethical & Participant Safety Concerns, Clinical Trial Operations (Good Clinical Practice), Study & Site Management, Data Management & Informatics, Leadership & Professionalism, Communication, and Teamwork & Team Science. Three difficulty levels of questions were developed: basic, intermediate, and advanced. To reinforce knowledge and skill development, the assessment was designed to give respondents formative feedback after responding to each question. Results/Anticipated Results: A preliminary “pre-pilot” test was conducted with three postdoctoral scholars to ensure that the assessment items were understandable. The assessment was then pilot tested with a larger group of 40 clinical research professionals (CRP) to test for the clarity and difficulty level of the items. A smaller group of 20 of these CRPs agreed to participate in focus groups to obtain feedback on their user experience. Data regarding years of experience as a CRP, types of studies engaged with, and information regarding professional certification were collected and used in the analyses. Demographic data collected were not connected to user responses. Results of the pilot test and focus groups were used to revise the questions on the final version of the assessment. Discussion/Significance of Impact: To our knowledge, this is the first objective assessment of research skills for CRP working in behavioral and social science contexts. We will discuss how other institutions can use this instrument to evaluate the training needs of their social and behavioral research workforce.