Investing In The Science Of Science: What Medicare Can Teach The NIH About Experimentation

That’s the title of a new piece released today in the journal Health Affairs. It’s by me and Kushal Kadakia, a former Rhodes Scholar who has spent time at both CMS and FDA. For years, meta-science scholars have said that we should “turn the scientific method on ourselves,” to quote Pierre Azoulay. But a challenge is that doing pilot experiments with new models of science funding, peer review, etc., isn’t anyone’s job at the moment. Indeed, a flood of outsider ideas can come across as, at best, a distraction from agency employees’ day-to-day duties.

As the Center for Medicare and Medicaid Services (abbreviated “CMS” for a reason I find hilarious*) found, one way for an agency to actually try out new ideas is to create a new team of people whose entire job is to do so—namely, the Center for Medicare and Medicaid Innovation.

To be sure, CMMI hasn’t singlehandedly revolutionized Medicare and Medicaid, but it has sponsored over 50 pilot tests that have generated many insights about better ways to pay for (or measure the quality of) health care.

A Center for NIH Innovation could replicate this model of institutional innovation. With the right authority and budget, it could begin to answer long-standing questions about how peer review works, how better to fund early-career researchers or “high risk” research, and much more.

Read the whole thing here. And a bonus at the end of this email: sample legislative text based on the CMMI statute.

PS: Why is it called “CMS”? Isn’t that missing one of the Ms?

Answer from an interview between Uwe Reinhardt and Tom Scully (the first CMS administrator in 2003):

Reinhardt:

Let me end by asking you about something that has puzzled me for some time. Your first act as the newly appointed CMS administrator (or HCFA, as it was then called) was to field a contest for a new name for your agency. I had submitted to your staff the name “Senior Health Insurance Trust,” but for some reason that mellow name got no response. You chose instead “Centers for Medicare and Medicaid Services” and collapsed it into the acronym CMS. What has intrigued me ever since is this: When you chose that acronym, CMS, which of the two Ms—Medicare or Medicaid—didn’t you give a dang about?

Scully:

[HHS] Secretary [Tommy] Thompson and I had an employee contest, and we picked the name from a group of finalists. He liked Medicare and Medicaid Administration, but our focus groups showed that MMA—pronounced “Mama”—would not be funny to half the population. So we went with CMMS—but since I already mumble anyway, I suggested that we shorten it to CMS. He agreed.

Appendix: Sample Legislative Text

Section 401 of the Public Health Service Act (42 U.S.C. § 281) is amended by inserting the following new subsection (g) after subsection (f) (with the current subsections (g) and (h) being renamed (h) and (i) respectively):

“(g) Center for NIH Innovation.

(1) In General—There is hereby created a Center for NIH Innovation (“CNI”) at the National Institutes of Health to carry out the duties described in this section, with the overall goal of accelerating the pace of biomedical advancement. The purpose of CNI is to work with other NIH Institutes and Centers to test new ways of funding grants and contracts; measure the impact of pilot projects and experiments; and scale up innovations that are successful.

(2) Deadline—The Secretary of HHS shall ensure that CNI launches and is able to carry out its statutory mission by [date].

(3) Organization—The Office of Evaluation, Performance and Reporting and the Office of Portfolio Analysis are hereby consolidated with CNI and shall report to the CNI Director.

(4) Consultation—CNI shall create an advisory council with 5 or more representatives from other NIH Institutes/Centers, 5 or more representatives from universities, and 3 or more researchers with expertise in meta-science. This advisory council shall meet at least twice each calendar year, and shall provide CNI with expert advice on ideas for experimentation and evaluation of NIH’s processes. CNI shall also make an email address available to the public for suggesting other ideas (including anonymously); relevant submissions shall be considered by CNI and its advisory council at the biannual meetings.

(5) Selection of Ideas to Be Tested—CNI shall select ideas to tested by gathering evidence as to promising methods for improving science funding from other national science funders, the advisory council, the academic literature, or other submissions of ideas.

(6) Evaluation—

(A)    Where possible, CNI shall attempt to use randomization or cutoff-based methods to pilot new science funding models and to determine their effects.

(B)    CNI shall evaluate the success of alternative scientific funding models by a diversity of outcomes, including qualitative evidence, citations, patents, prominence of new discoveries, and other signals of scientific achievement. CNI shall additionally explore funding meta-scientific work to determine how best to measure the outcomes of scientific funding, and whether various short-term outcomes are indicative of longer-term outcomes.

(7)    Waiver Authority-CNI may waive the requirements of the Public Health Service Act as regards peer review or any other issue as may be necessary for purposes of carrying out this section with respect to testing new models of NIH funding;

(8)    Limitations on Review—There shall be no administrative or judicial review of –

(a)    The selection of NIH funding models for testing or expansion under this section;

(b)    The selection of organizations, sites, or participants to test those models; or,

(c)    The elements, parameters, scope, and duration of such funding models.