386 Developing an assessment tool for NIH data management and sharing plans to understand current data practices and needs

Abstract

Objectives/Goals: NIH requires researchers submit Data Management and Sharing (DMS) Plans with their grant applications. Librarians developed an assessment tool for the plans and completed a pilot assessment in order to leverage the plans and understand current institutional research data management and sharing. Methods/Study Population: The assessment tool includes questions related to evaluations of DMS Plans as well as questions related to the content of the plans. Evaluation questions were adapted from the Federation of American Societies for Experimental Biology evaluation rubric developed for the DataWorks! Data Management Plan (DMP) Challenge. Fields were added to collect information on the content of DMS Plans, including data type, institutional resources, data repositories, data standards, and data dissemination timelines. The assessment tool was tested in a pilot implementation. Seven library workers were trained and completed paired review samples of 27 DMS Plans (54 evaluations total) in order to test for tool reliability. Results/Anticipated Results: Results include findings on the reliability of the tool as well as preliminary results from an assessment of DMS Plans. Findings on the reliability of the tool include assessments of the paired reviewers for each question included in the tool. Paired reviewers generally agreed, but tended to differ on specific questions, including questions pertaining to the data types generated or used in a research project. Questions with high levels of agreement included subjects of study and code sharing practices. Results on the content of the DMS Plans include information such as data repositories used, data oversight responsibilities, and data and metadata standards employed. Discussion/Significance of Impact: DMS Plans present an opportunity to better understand data management and sharing practices, and good data management supports high-quality, reproducible research. Developing and testing assessment tools for these plans is a key step toward understanding and improving current research data management practices.