Two exciting opportunities in the works: First, the National Institute of Neurological Disorders and Stroke (NINDS) released a “Request for Information on Potential Solutions for Reducing Publication Bias Against Null Studies." Responses are due on April 1, 2024.
Excellent piece Stuart! The long-standing bias against publishing studies with null results creates a hugely distorted science literature where meta-analyses are computing effect sizes based on cherry-picked studies. Add to this the fact that most for-profit companies bury their studies with null results so they never see the light of day. To show that an idea/hypothesis is WRONG is as or more important to scientific progress than getting a significant result.
A “strategic plan” means that the NIH consciously moves science in their preferred direction. That’s different than just reviewing grant applications and supporting those that seem promising. The latter allows for a wider range of study. The former by design narrows the range. Which should we expect will produce better results?
Excellent piece Stuart! The long-standing bias against publishing studies with null results creates a hugely distorted science literature where meta-analyses are computing effect sizes based on cherry-picked studies. Add to this the fact that most for-profit companies bury their studies with null results so they never see the light of day. To show that an idea/hypothesis is WRONG is as or more important to scientific progress than getting a significant result.
A “strategic plan” means that the NIH consciously moves science in their preferred direction. That’s different than just reviewing grant applications and supporting those that seem promising. The latter allows for a wider range of study. The former by design narrows the range. Which should we expect will produce better results?