I am delighted to introduce Dr. Larry Tabak, NIH's Principal Deputy Director, who recently spoke to the NIAMS Advisory Council about an important new initiative in which all of the NIH Institutes and Centers will participate. His presentation invited considerable discussion on enhancing research reproducibility and transparency, a topic of great interest to the NIH, the research community, and the public. In this month's Director’s Letter, Dr. Tabak discusses this topic and invites wide participation in the process. We will continue to update you as this initiative progresses and we look forward to your feedback.
If you have comments or questions, please send them to Anita Linde, M.P.P. firstname.lastname@example.org.
Stephen I. Katz, M.D., Ph.D.
Director, National Institute of Arthritis and Musculoskeletal and Skin Diseases
In rare circumstances, scientific breakthroughs happen quickly and unexpectedly. More often, the scientific enterprise is an incremental process with new discoveries based upon the foundation of previous findings. Recently, a growing number of studies examining biomedical and behavioral research have drawn attention to a lack of methodological transparency and reproducibility of results in published research, which can affect this process. This issue may stem from a number of factors, including:
- deficiencies in reporting the experimental methodology and a lack of core standards that should be reported,
- bias in the randomization and blinding of experimental subjects,
- insufficient numbers of study participants to detect meaningful differences between the treatment and control groups, and
- problems in data handling and analysis which can lead to false positive results.
These deficiencies are compounded when flawed or non-reproducible basic research findings are used to formulate pre-clinical and clinical studies. By addressing and reducing these issues, we set ourselves on the path toward improving the predictability of disease models and the validity of drug targets, and ultimately improving the success rates in clinical trials.
Building upon efforts begun in the National Cancer Institute and the National Institute of Neurological Disorders and Stroke, the NIH is taking steps to enhance the reproducibility and transparency of research findings. An ad hoc working group was convened to examine the problem and develop principles for an NIH-wide initiative to address it. These principles include:
- increasing community awareness,
- enhancing formal training of investigators,
- improving the review of grant applications while protecting the integrity of the science, and
- increasing funding stability for investigators, enabling them to use more appropriate and complex study designs.
NIH will be taking several steps over the coming months and years to address this important issue. First, all NIH Institutes and Centers have been asked to discuss the reproducibility and transparency of research findings with their Advisory Councils and stakeholder communities and to solicit feedback. To this end, I have recently spoken to the NIAMS Advisory Council about this initiative and the recommendations of the ad hoc working group.
While we are reaching out to our stakeholder communities, we will also be working within NIH to develop a pilot training module on research integrity as it relates to experimental biases and study design. Once finalized, this module will be made widely available to the research community. Selected Institutes and Centers, including the NIAMS, will also be conducting a pilot evaluation of the scientific premise of grant applications for Phase I clinical trials. In addition, we will be exploring changes to the bio-sketch portion of grant applications so that investigators are able to more fully describe their contributions to cited publications. We will be investigating methods to provide substantial, longer-term support to experienced investigators. Finally, NIH will continue to collaborate with scientific journals and the research community on efforts to improve the rigor of the review process for publications.
We are immensely proud of the research NIH conducts and supports, and the impact that it has had in improving the health of this country. However, we also have a responsibility to remain vigilant and to quickly identify and address aspects of our work that would benefit from change. Improving the reproducibility and transparency of the scientific research that we support is of paramount importance to the NIH, and we look forward to your input and feedback as we move forward on this initiative.
Lawrence A. Tabak, D.D.S., Ph.D.
Principal Deputy Director
National Institutes of Health