Response to: Request for Information To Improve Federal Scientific Integrity Policies

Below is our response to the Request for Information To Improve Federal Scientific Integrity Policies which “seeks information to help improve the effectiveness of Federal scientific integrity policies to enhance public trust in science” (Executive Office of the President, 2021).

Name: Leslie D. McIntosh

Name of Organization: Ripeta, a Digital Science Company

Type of Organization: Other

Other Type of Organization: Technology Company

Role: Researcher and Executive Leader

Prepared by: Leah Haynes, Mary Uhlmansiek, and Leslie D. McIntosh

Introduction

Ripeta thanks the Office of Science and Technology Policy (OSTP) for the opportunity to submit comments on improving Federal scientific integrity policies. Ripeta is a company led by a team of researchers and data scientists dedicated to making better science easier. Our tools and technology automatically scan manuscripts for key scientific quality indicators and provide critical feedback to authors, publishers, funding agencies, and institutions on the quality of reported research. 

We applaud the work of OSTP and Federal agencies in the development and implementation of policies to promote trust in federally funded science. OSTP has been central in setting policies and practices for Federal agencies to put in place to advance scientific integrity, including the 2013 memo on Expanding Public Access to the Results of Federally Funded Research and the 2009 Presidential memo on Scientific Integrity. These memos have strengthened federal funding and research integrity policies, ultimately increasing the availability of open data, software, and code. Additionally, the work of OSTP on Desirable Characteristics of Repositories for Managing and Sharing Data Resulting from Federally Funded Research is critical for ensuring the locations where data is shared are sustainable and reputable.  

Especially in light of the novel COVID-19 pandemic, we believe it to be essential to further improve the efficacy and integrity of scientific research by ensuring the scientific reproducibility, availability, and quality of data. As a result, our recommendations center around the topics of the effectiveness of Federal scientific integrity policies in promoting trust in Federal science, effective policies and practices Federal agencies could adopt to improve the communication of scientific and technological information, and effective practices Federal agencies could adopt to improve training of scientific staff about scientific integrity and transparency into their scientific integrity practices.

#1 The effectiveness of Federal scientific integrity policies in promoting trust in Federal science:

Promoting trust in science means developing a system that verifies:

• Individuals conducting science;
• Institutions funding science;
• Independent scientific research.

Federal scientific integrity policies must implement mechanisms to monitor science and stakeholders within the scientific ecosystem. While not altogether new, the internet has facilitated the reach of misinformation, disinformation, and fraud. General trust in scientific information will continue to deteriorate without checks in place to establish authority in scientific research and communication.

This authority should be defined by well-informed and trained individuals and institutions that reflect the demographic makeup of this country. We are in need of a deliberate and intentional support system that both fosters and funds a more diverse workforce in which all individuals are able to contribute to the pursuit of better and more trustworthy science.

#2 Effective policies and practices Federal agencies could adopt to improve the communication of scientific and technological information:

Improving scientific communication and technological information recommendations:

• Verify the expertise of scientific researchers and authors;
• Use persistent identifiers;
• Enforce and check data sharing practices (specifically the use of trustworthy repositories). 

We believe that a fundamental aspect of improving scientific communication involves developing a system of checks that verifies the validity and integrity of transmitted information. This would include verification of the expertise of scientific researchers and authors of scientific publications as well as verification of their processes. Enacting a system of routine and automated checks on quality indicators for scientific research would improve not only communication of scientific and technological information but also trust in the information itself. This may be facilitated by increased use of persistent identifiers (e.g. ORCIDs, DOIs, RORs, RRIDs) in order to better document and track information throughout the process of scientific research and publication. The use of registered reports and protocols may also increase trust in the integrity of the research process and methods.

More fundamental than checks such as these, however, is the actual availability of information (e.g., data, code, software, protocols, and materials) necessary to replicate scientific research. Ripeta’s analyses have shown significant variability in scientific researchers’ data sharing methods. Researchers share data many different ways including within their papers, upon request, through a repository, or not at all. The ease of accessibility to data thus varies significantly. Many government funders have clearly stated their preference for data to be shared through trustworthy repositories yet this occurs less than 15% of the time in data availability statements. We believe that Federal agencies should not only encourage but actually enforce and verify the sharing of data. 

#4 Effective practices Federal agencies could adopt to improve training of scientific staff about scientific integrity and the transparency into their scientific integrity practices:

In order to ease the implementation of policies and practices as previously discussed, it is important to train graduate students on quality research practices and the importance of scientific integrity. By standardizing scientific reporting practices and training students accordingly, we may normalize more robust reporting which would facilitate the aforementioned system of automated checks on quality indicators and improve scientific communication and integrity. Important parts of standardized scientific reporting practices would include the inclusion of individual statements addressing data availability, code and software availability, authorship, ethics, and more as well as a generally accepted structure for scientific publications.