The White House (OMB) Arguably Needs More Authority Over NIH
As readers may know, the Good Science Project has taken the position that the Paperwork Reduction Act needs reform so that it accomplishes its own purpose of streamlining government. Right now, if an agency does anything that arguably affects paperwork from a mere 10 Americans, it has to go through an onerous and lengthy review process that often creates more paperwork than it reduces.
It’s basically a no-brainer to raise this threshold to at least 1,000 people, or at least to a number based on OMB’s criteria for having significant regulatory impact on the public or an industry.
That said, the Paperwork Reduction Act also embodies some key authorities for the White House Office of Management and Budget (OMB) to make sure that federal agencies are sharing data for evaluation and other purposes. In part, this is because the OPEN Government Data Act (part of the Evidence Act) consisted of amending the Paperwork Reduction Act!
For example, 44 U.S.C. § 3504 now says that the OMB Director shall “provide direction and oversee” “agency dissemination of and publication to information; statistical activities; records management activities; privacy, confidentiality, security, disclosure, and sharing of information.”
That same statute now says that the OMB Director shall “foster greater sharing, dissemination, and access to public information, including through the use of comprehensive data inventories.”
It also says that OMB shall “issue guidance for agencies” that takes into the “risks and restrictions related to the disclosure of personally identifiable information,” the “determination of whether a data asset is subject to intellectual property rights” or whether it “contains confidential business information” that could be withheld under FOIA, and more.
And it creates the role of Chief Statistician of the United States, which is critical to the overall federal effort to create better evidence and data.
SIDE NOTE: Even though the above has been the law for well over 5 years now, OMB has yet to release key guidance for federal agencies that should have been completed within one year.
This newsletter has many readers who work for Congress, GAO, and the White House. Someone might want to ask OMB what is taking so long.
In short, the Paperwork Reduction Act (as amended by the OPEN Government Data Act) contains lots of key provisions for how OMB can tell federal agencies how to allow statistical activities, how to protect privacy, how to make data open while respecting intellectual property, and more! That is all very important.
But there’s a problem with regard to NIH.
What’s the problem?
An exemption created by the 21st Century Cures Act (P.L. 114-255), which amended 42 U.S.C. § 241 to say this:
“Subchapter I of chapter 35 of title 44, United States Code [this refers to the entire Paperwork Reduction Act], shall not apply to the voluntary collection of information during the conduct of research by the National Institutes of Health.”
Hmmm, what does this mean?
Our only clue so far is the NIH Policy Manual, which says:
21st Century Cures Act Research Exemption
Section 2035 of the 21st Century Cures Act, enacted December 13, 2016 (P.L. 114-255), amended Section 301 of the Public Health Service Act (42 U.S.C. § 241) by adding:“(g) The PRA [sic] shall not apply to the voluntary collection of information during the conduct of research by the National Institutes of Health.”
This new provision is explained below.
For the purposes of this policy, ICs must answer the following questions…:
Will the program analyze the project outcomes in a systematic way?
Will the program publicize the results of your systematic analyses?
If the answer to both questions is yes, then the program is conducting NIH research and is exempt from the PRA requirements as outlined by 21st Century Cures Act.
As an initial matter, NIH’s policy manual hasn’t been reviewed by OMB, and there is no guarantee that anyone else in government (or in the court system) would agree with this exact interpretation of the Paperwork Reduction Act exemption. It’s not obvious that “publicizing results” is necessary in order for something to count as “conduct of research,” for example.
Beyond that, there are some problems that I have come across through discussions with policymakers.
First, this NIH exemption from the entire Paperwork Reduction Act means that OMB may lack some key abilities to exercise oversight and budgetary authority over risky research that NIH might support.
This issue has come to head since Covid, most recently with the introduction of the Risky Research Review Act sponsored by Sen. Rand Paul. The aims of that Act are to protect public health by exercising more oversight over risky biological research that could lead to new pandemics, new genetic engineering techniques, and the like.
But here’s the problem: To the extent that the NIH does its own internal research on the implementation and outcomes of such programs, OMB arguably won’t be able to exercise oversight over such research (such as when and how the internal data must be shared).
Bottom line: If you want OMB and the public to have more oversight over NIH research, rethink the Paperwork Reduction Act exemption that was adopted in the 21st Century Cures Act and codified at 42 U.S.C. § 241(g).
***
Second, to the extent that NIH engages in evaluation activities under the Evidence Act (including its engagements with external researchers), NIH could argue that it is exempt from the Paperwork Reduction Act. But that would mean NIH could ignore OMB’s (hopefully forthcoming!) guidance on how much data needs to be released openly, how to protect intellectual property, and so forth.
That is problematic! Out of all federal agencies, why would NIH alone be given an exemption from the White House’s oversight here?
Consider how NIH has acted in one critical context: The small business and technology transfer programs created under the Small Business Act (15 U.S.C. 638 et seq.)—that is, SBIR and STTR programs.
These programs are required at any federal agency that spends above a certain amount on research every year, and the goal is to commercialize research to the benefit of the American public.
How well are the SBIR and STTR programs actually working?
No one knew, so in 2012, Congress included “Title LI—SBIR and STTR Reauthorization” in the National Defense Authorization Act for FY 2012 (P.L. 112-81). Section 5137 of that Act (which is codified at the note to 15 U.S.C. 638) stated that each agency with SBIR/STTR programs has to “enter into an agreement with the National Academy of Sciences” to do a “comprehensive study” of those programs every four years. [Apologies for the detailed citations, but it is very difficult to figure them out from scratch, so I erred on the side of giving more detail.]
The most recent evaluation of the SBIR/STTR programs at NIH was published by the National Academies in 2022. You can see that the National Academies committee was frustrated at NIH’s refusal to hand over data to help figure out how well the programs worked
Absent active randomization, the most informative study design would rely on the priority scores of each application. As part of the standard NIH review protocol, each proposal submitted to the program is evaluated by an expert review panel and assigned a priority score that reflects project quality (see Chapter 3). For some ICs, proposals are then ranked and generally funded in order of priority scores until financial resources have been exhausted. Because proposals just above and just below the payline threshold (the cutoff point for scores below which applications generally are not funded) are very similar, one can compare the outcomes of these two groups to enable credible causal estimates.
Previous analyses of NIH programs (Azoulay et al., 2019; Jacob and Lefgren, 2011; Li and Agha, 2015) and the SBIR/STTR program of other agencies (Howell, 2017) have used priority score–type information in regression discontinuity designs to compare “just funded” to “just not funded” applicants, and provide estimates of the program effects. In addition, a previous National Academies report assessing the Advanced Research Projects Agency-Energy (ARPA-E) program used individual reviewer scores to evaluate that program’s funding selection processes (NASEM, 2017).
Although the committee requested priority score information from NIH, this information was not provided because of confidentiality concerns. If future analyses are to be more robust and enable stronger statements on program impact, NIH will need to find a way to provide this information to researchers, as it (Ginther et al, 2011; Jacob and Lefgren 2011; Li and Agha, 2015) and other agencies (Howell, 2017; NASEM, 2017) have done in the past.
(page 155).
If you’re used to reading anodyne National Academies reports, the language above is scathing by comparison.
That is, the National Academies committee was basically saying:
We needed this key data to do the job that Congress asked us to do, but NIH refused to give us the data because of supposed “confidentiality concerns,” even though NIH has already given this exact same data to other researchers (and other agencies have done so as well).
Does anyone at NIH actually think that you can’t trust a National Academies committee commissioned by Congress to be careful in handling confidential data?
I’d say no. Someone had to have known what it meant to refuse to provide data here: thwarting oversight and evaluation. And this will always be the case when agencies have risk-averse lawyers who can point to the pretext of “confidentiality” as a [paper-thin] excuse.
This is yet another reason that Congress should rethink the NIH’s Paperwork Reduction Act exemption that was adopted in the 21st Century Cures Act and codified at 42 U.S.C. § 241(g).
The White House OMB needs to be able to exercise federal-wide authority to tell agencies what data they should release under FOIA and the OPEN Government Data Act, what limited set of data truly is confidential (and how to protect confidentiality without blocking all evaluation), and so forth.
PS:
NIH isn’t alone here. The National Academies also released an evaluation in 2023 of the National Science Foundation’s SBIR and STTR programs. Throughout, the National Academies committee was obviously frustrated at NSF’s refusal to provide enough data. From the Introduction alone:
The committee’s lack of data on applicants impeded its ability to assess the efficacy of NSF’s current outreach effort in increasing the number of applications from small businesses that are (1) new to the SBIR/STTR programs, (2) from underrepresented states, (3) woman-owned, or (4) minority-owned. Those analyses were therefore limited in focus to the changing demographics of the awardee pool. . . .
The lack of systematic data collection, data access, and preregistered evaluation approaches (i.e., one where the evaluation is planned and made public) at NSF limited the committee’s ability to fully assess the agency’s programs. In particular, the committee was unable to determine which of a number of programmatic changes to the NSF SBIR and STTR programs have been effective at improving selection outcomes for woman- and minority-owned businesses, nor could it determine differences in commercial success between awardees and unsuccessful applicants. Granting access to data on all applicants for program assessment purposes, as is called for in the legislation mandating this review, and establishing processes that would allow for structured evaluation of policies and procedures would help NSF understand the effectiveness of its initiatives and how its programs could be improved. [pages 2-3]
. . .
Neither NSF nor SBA has a systematic dataset with uniform commercialization outcomes, which made the committee’s assessment of commercialization outcomes difficult. The committee was unable to obtain access to internal scoring for any of the SBIR/STTR applicants, nor did it gain access to information about applicants that were unsuccessful. The committee also lacked access to any demographic information about reviewers. These limitations impeded the committee’s ability to determine differences in commercial success between awardees and unsuccessful applicants or the root causes of changes in awardee diversity.
Finally, lack of access to the entire set of applicants limited the committee’s ability to analyze collaborations between small businesses and their research partners. Thus the committee’s analysis was limited to such collaborations within the STTR program. [page 5]
Note that the National Academies committee accused NSF of refusing to hand over the exact data that is “called for in the legislation mandating this review”! Amazing.
This newsletter has multiple subscribers who work on Congressional oversight or appropriations for NSF. They should pay attention to this issue, and should work to clarify what NSF is and isn’t allowed to do (rather than allowing important agency evaluations to be effectively vetoed by risk-averse legal counsel).
PPS:
This is very much a side note, but look at the 2012 bill linked above. If you’ve ever heard anyone (including members of Congress) complain that no one had time to read a bill . . . well, no one reads the full text of bills anyway. Actual bills are full of pages that look like this, where the legislative counsel’s office comes up with bill text that rewrites the United States Code to do what Congress intended.
Legislative text like that is much less important than knowing the overall gist of what is being accomplished. Do you think that a single member of Congress or the administration ever reads (and personally follows the cross-references for) hundreds of pages of that? They’d never have time for anything else.