Three thoughts on "high-risk" research

The best scientific research can be unpredictable and not easily measurable by the usual metrics (citations, publications, etc.). Indeed, when agencies like NIH or NSF ask that researchers produce “transformative” research on command, that request can backfire, resulting in tons of proposals with inflated claims while great scientists are left with less time to focus on their actual research.

Can you tell this is AI-generated?

Three relevant quotes I came across recently:

First, Fred Sanger is one of the very few people to win 2 Nobel Prizes. He was part of the cohort of geniuses who worked at the Laboratory of Molecular Biology at Cambridge in the mid-20th century.

According to a 2014 retrospective article by Sydney Brenner, Sanger would be eaten alive in today’s university and NIH system:

A Fred Sanger would not survive today’s world of science. With continuous reporting and appraisals, some committee would note that he published little of import between insulin in 1952 and his first paper on RNA sequencing in 1967 with another long gap until DNA sequencing in 1977. He would be labeled as unproductive, and his modest personal support would be denied. We no longer have a culture that allows individuals to embark on long-term—and what would be considered today extremely risky—projects.

Second, from an article on “transformative research” in ecology:

Our results suggest that predicting [transformative research] at the proposal stage is not only difficult, but also that prioritizing it can be potentially harmful. Survey-takers expressed strong consensus that emphasizing the importance of [transformative research] in proposals causes researchers to overstate the potential importance of their work … .

With limited funding, incentivizing ‘promise inflation’ can be harmful to the ‘honest scientists who . . . propos[e] to [make] an important contribution to the solution of an important problem. They risk being dismissed as small-timers with no vision’ [27].

One respondent commented that transformative statements can be more of an ‘essay contest’ than truthfully representing the likely impact of proposed research.

Survey results echo this idea. Respondent Dr Lauri Oksanen stated, ‘If folks are requested to emphasize the transformative aspects of a proposal, the likely result is self-bragging and a soup of fashionable terms.’

Holbrook, an organizer of an NSF workshop on [transformative research], expressed the opinion that grant writing has turned into a game where ‘transformative’ is a buzzword not taken seriously by proposers, reviewers, or authors.

Third, here’s some testimony given by Nobel winner Roger Kornberg to Congress in 2007 (emphasis in original):

Our work has been supported almost entirely by the NIH. The cost was about $20 million over 30 years, mostly for the stipends of the more than 80 graduate and postdoctoral trainees involved. Due to current constraints on the NIH budget, virtually none of our work would be funded today. I can say with certainty that a grant application for the research leading to the discovery of the nucleosome, fundamental particle of the chromosome, would not be approved. The reason is simple: I had no idea at the outset of what I might find, and no good idea of how to go about it. Our RNA polymerase structure work was supported by NIH only after it became clear it would succeed. When we began, the prospects for success were virtually nil – no way of producing the RNA polymerase, no hope of forming the crystals needed for imaging, and no technology for deriving the image.

The reason for the disconnect between funding and discovery is clear: funds are awarded for compelling ideas, supported by preliminary evidence, creating a high likelihood of success. But discoveries are by their nature unanticipated, completely unknown. They cannot be sought out in a deliberate manner. They cannot be proposed to granting agencies or evaluated by review groups. So how are discoveries made in the American system? The answer is by risk-taking. Scientists supported to do straightforward research may divert some of their funds for testing new ideas. If they succeed, then the results form the basis for new grant applications. If they fail, they may be in trouble and be unable to continue even with their original research.

Put it all together, and we have the paradox: When funding agencies ask for high-impact research, that can be the very thing that prevents high-impact research. Good scientists who just needed some breathing room and some time to think are forced to spend way too much time writing over-hyped proposals.