Noni Byrnes wants to bring more transparency to grant review at the NIH

For many scientists, grant review at the US National Institutes of Health is a black box.

But Noni Byrnes, the director of the NIH’s Center for Scientific Review (CSR), wants to shine a light into a process that affects the lives of thousands of scientists every year and sets the direction of health-related research in the US.

“When you don’t know what’s going on, you’re much more likely to assume the worst,” Byrnes says from the CSR’s offices in Bethesda, Maryland. “Any attempt we have to make the process transparent, to bring more people into our system, is going to help that.”

The CSR is the prime facilitator of peer review at the NIH, with its staff of 500 handling 77% of the agency’s grant applications. The staff manage more than 18,000 reviewers evaluating 62,000 grants from over 200 topical study sections. Byrnes is relatively new to the job—she officially became director in February 2019—but she has worked at the CSR for 19 years. So she knows that the agency and its review process are often the target of both fascination and criticism.

Byrnes wants to tackle head-on some of the common critiques of the grant review process, including ensuring diversity among reviewers and grantees, easing the sometimes-heavy burden on reviewers, and making sure that the review process is doing its best to pick the most important science.

“You don’t want bureaucrats or government officials making these decisions,” Byrnes says.

A large part of Byrnes’s efforts involves making sure the scientific community actually knows how the CSR works. She’s expanded the agency’s advisory council, launched a systematic critique of its study sections, and increased the agency’s outreach and social media efforts.

But Byrnes knows she’s not going to make everyone happy. “At any given time, 80% of the people are not getting funded. That’s very difficult.”

Born and educated in Pakistan, Byrnes came to the US for college. “In reality, I just wanted to get away,” she says. Her older sister wed at age 20 in an arranged marriage. “I just didn’t want that.”

Her older brothers were engineers. Byrnes thought she might be a doctor, so she initially majored in biology at Allegheny College. Then she realized that “chemistry to me made just so much more sense,” she says. “It wasn’t subjective. I really liked that aspect of it.” Byrnes went on to graduate school at Emory University, where she got her PhD in analytical chemistry in 1994.

After graduate school, she took a job in Procter & Gamble’s pharmaceutical division, where she developed and evaluated analytical methods as part of drug development. “That was a very hard-core chemist analytical role,” she remembers.

When she had children, she decided she wanted to stay home until her youngest child was 3 years old. “I think for women and even for men these days, it is kind of a struggle,” she says of her decision to take time off. “It just felt right to me.”

Around the time she was ready to go back to work, her PhD adviser let her know about a job opening at the NIH. The CSR was starting a new study section, and they wanted an analytical chemist to lead it. “I said, ‘What’s a study section?’ ” Byrnes remembers. She hadn’t applied for a grant before and didn’t know the ins and outs of the application or review process.

But the position was to oversee the evaluation of technology development proposals, which was a good fit for Byrnes’s background in industry. She applied and got the job running the Enabling Bioanalytical and Biophysical Technologies Study Section in 2000.

There is probably no better place to learn peer review than the CSR, Byrnes says. Unlike other parts of the NIH, which also run programs or monitor projects, “that’s all we do. Everyone here, from top to bottom, is all about peer review.”

She quickly realized that the process is critically important to the research community, she says. “What comes out of the grant process really drives the science,” Byrnes says. “A lot of human beings depend on this thing to work and to be fair.”

After 5 years running that section, Byrnes was promoted to run the Cell Biology Integrated Review Group. As a chemist, she wasn’t an obvious choice, she says. But she suspects she was chosen in part because of her interest in adding more interdisciplinary reviewers to panels.

In 2011, she moved up to lead the CSR’s Division of Basic and Integrative Biological Sciences. In 2018 she became the CSR’s acting deputy director, then acting director; she became director in February 2019.

Issues of transparency aside, in Byrnes’s nearly 2 decades at the CSR, the biggest challenge has been the agency’s gradually declining ability to fund research, she says. Byrnes started during heady times, when Congress doubled the NIH’s budget, and money was flowing in. In the past few years, funding levels have flattened, but the number of scientists seeking funding hasn’t. The resulting low funding rates have created a lot of pressure on the community, Byrnes says.

“I feel for the scientific community,” she says. For the CSR, the tight times make it “even more critical that the job that we do is based on actual data and is based on input from our stakeholders.”

One constant question for the NIH is whether the grant review system can actually distinguish one top grant from another, especially at such low funding levels. Overall, the NIH funded 20.2% of individual researcher grant applications in 2018.

Right now, reviewers submit application scores to a study section, and the numbers fall where they may. Byrnes wants to look at ways to make sure a grant’s score reflects its place in the application pool. For example, reviewers or committees could sort reviews themselves, either by ranking each application or grouping them by importance. “It will force people to make some distinctions,” Byrnes says.

She also wants to examine who is serving on the CSR’s study sections. To start, she is looking to broaden the pool of reviewers to make sure panels are diverse in many ways, including race, gender, scientific background, and years of experience.

One idea is to find a way for institutes and scientific societies to put forward potential reviewers in different subject areas. Another is to make better use of a database of thousands of early-career scientists who are interested in serving on panels. Currently, the CSR can include only a fraction of those who are interested because there are not enough spaces on panels.

“Some of the younger faculty tend to be more diverse, including more women,” she says. “But the most important thing is that they think differently” from older scientists in part by being more open to using new technologies.

Some study sections also have trouble getting enough senior scientists to serve, so Byrnes is looking at ways to motivate scientists who have received NIH funding to step up. “We do want a balance of senior people on each committee,” Byrnes says.

One thing that could help is making reviewing easier by focusing only on questions about the science. Right now, reviewers often answer administrative questions about the grant along with evaluating the research itself. “The most critical role of our reviewers is to provide us with their unfettered, honest, candid scientific input,” she says.

Another area of increasing focus for the CSR is the ethics of peer review. “We’re seeing some concerns regarding violation of confidentiality in peer review,” she says. Byrnes couldn’t quantify how many violations staff members have seen, but “it’s very few,” she says. “Most of our reviewers I think take this seriously and are trying to do the right thing.”

To help, the CSR is improving its integrity training to remind reviewers that they cannot share applications, reviews, or scores. Even something that seems innocuous, like a hallway discussion or having a postdoc review a grant application, can lead to larger violations, she says. “It’s a slippery slope.”

In addition, Byrnes is taking steps to make sure anyone accused of a research integrity violation, including sexual harassment, is not on a panel. The agency needs to protect the process and the scientists involved in it, she says.

Overall, Byrnes wants to take a hard, data-driven look at all aspects of NIH peer review. As part of that, the CSR is bringing in external reviewers to assess the quality of science coming out of each study section. That can include the number of publications coming out of funded grants or other metrics, depending on the field’s own measures of success. “My goal is to look systematically at our process through data and through input from our stakeholders,” she says.

And she wants to make as much of the data available for the community as possible so people can analyze them themselves. “I’m not worried about criticism. I actually think I don’t have all the answers,” Byrnes says. “I want to know if this system isn’t working.”