Measuring Chemistry’s Impact

Calculating the return on investment from taxpayer-funded research is complicated, mainly because the economic value of basic research may take decades or longer to become evident. Even so, the National Science Foundation hopes to find ways to measure that value by funding four projects that will look at research in the chemical sciences.

Supported jointly by NSF’s Science of Science & Innovation Policy (SciSIP) program and its Chemistry Division, the projects will conduct an input-output analysis of academic research, study the sociology of interdisciplinary collaborations, examine the impact of changing attitudes toward intellectual property, and evaluate the effect of regulation on businesses spun off from research.

NSF is interested in similar assessments in all sectors, according to an announcement last year from the agency about the funding opportunity. It is starting with the chemistry sector because, with its far-ranging impacts on many sectors, chemistry provides “an exemplary test bed” on which to base subsequent investigations.

The two-year awards, which total nearly $2 million for 2011, provide a unique data collection opportunity. “Instead of collecting evidence that is far removed from the action, what you’ll see in these grants are social scientists and chemists getting together to figure out at a grassroots level what the impact of research is and how investments in the chemical sciences work their way through the economy,” explains Julia I. Lane, program director at SciSIP. This process, she continues, will begin to provide an understanding that investments in basic research yield a wide variety of economic impacts.

The need to better understand the impact of federally funded R&D was recently underscored by Federal Reserve Chairman Ben S. Bernanke. In a speech last month during a conference on new building blocks for economic and job growth in Washington, D.C., he emphasized the importance of continued and sustained federal support of R&D in the U.S., but he noted that more needs to be done to understand how to make the most of limited dollars.

“In the abstract, economists have identified some persuasive justifications for government policies to promote R&D activities, especially those related to basic research,” Bernanke said. “In practice, we know less than we would like about which policies work best.”

Bernanke cited NSF’s efforts to fill the gap by finding better ways to measure or account for innovation, R&D activity, and intangible capital. “We will be more likely to promote innovation activity if we are able to measure it more effectively and document its role in economic growth,” he said.

One measure of impact is the rate of return on investment. A 2005 report by the Council for Chemical Research found that each federal dollar invested in the chemical sciences gives back $40 to the economy. This information argues for continued investment, but it doesn’t say how or where to put money to maximize the rate of return.

“We’re not so much concerned with the magic multiplier of finding out the ratio of benefits to costs,” says Kelsey D. Cook, program officer at the Chemistry Division at NSF. “Instead, we’d like to look forward at how we can get the most out of the next billion dollars that is going to be put into basic R&D in chemistry.”

The four projects not only will help determine where to allocate funding but also will provide insight into how research organizations should be set up to make the most of the limited amount of taxpayer funds, explains David C. Croson, program director at SciSIP. “The two complement each other,” he notes.

“NSF is taking a proactive route to trying to better characterize and better understand how federal research dollars are impacting scholarship, scientific disciplines, and local economies,” explains Joseph A. Heppert, associate vice chancellor for research and graduate studies and a chemistry professor at the University of Kansas. Heppert is one of the principal investigators on the new grants.

“The merit review system has a great deal of utility,” notes Heppert’s colleague Joshua L. Rosenbloom, a professor of economics at Kansas, but “the federal government doesn’t know a whole lot about the impact of the distribution of award dollars across different universities and departments.” Rosenbloom, Heppert, and Kansas economics professors Ted Juhl and Donna K. Ginther are leading a project to study how to measure research outputs.

“At a modest level, we expect to be able to say something about the econometric relationship between taxpayer dollars as inputs and various scientific outputs,” Rosenbloom explains. Additionally, the group hopes to provide a nuanced interpretation of that relationship.

To that end, the project will also look at cultural differences among various departments and universities and how they affect output. When the project concludes, Rosenbloom says, “we’d like not just to be able to provide guidance to federal agencies supporting research but also to be able to provide feedback to academic disciplines about how their structure affects what they are able to do.” Having a chemist on the team is essential to achieving the second goal, Rosenbloom adds, because without an “understanding of the culture within the discipline in which these activities are taking place, we as economists would be clueless.”

The challenge facing this team is how to measure outputs that can take 15 or more years to yield their ultimate value. Instead, the team must find a way to measure and evaluate impacts by studying “intermediate products,” Rosenbloom points out.

“Making the connection between those intermediate products and ultimate outcomes—for example, a highly skilled and effective workforce, the commercialization of intellectual property, and economic growth through the development of companies—is going to be very important,” Heppert says.

Before research outcomes can be measured, scientists must do the work. To gain a better understanding of innovation at the laboratory level, Laurel Smith-Doerr, a sociology professor at Boston University, and Jennifer L. Croissant, a women’s studies professor at the University of Arizona, will use a SciSIP-Chemistry grant to study how labs operate.

“Research funding models in chemistry differ,” Smith-Doerr tells C&EN. Research can be funded by individual faculty investigator awards, private funding from for-profit companies, interdisciplinary research centers, and increasingly by interdisciplinary collaborations, which will be the focus of the new study by Smith-Doerr and Croissant.

“A greater understanding of collaboration is a key to knowledge production, and investigating what scientists view as worth pursuing and how they manage collaborations is important,” Smith-Doerr says. “These data will contribute to building basic social science knowledge about collaborations.”

Smith-Doerr and Croissant plan to do case studies of chemistry labs and are looking for participants from both academic and industrial labs. “We are conducting interviews to examine how researchers in chemical science fields come to form research interests and collaborations—including the role of funding, how women and men collaborate, and how researchers understand their professional and societal responsibilities in collaborations,” Smith-Doerr explains. She adds that chemical researchers who are interested in taking part in this study should contact her.

Regardless of the research model, scientists have become mindful of the economic value of their work. Spurred by the Bayh-Dole Act of 1980—which, among other things, gives universities control of inventions arising from federally funded research—universities are holding more patents and aiming to transfer technologies to the marketplace. The impact of this shift in attitude toward intellectual property is the subject of the third SciSIP-Chemistry grant.

“Although university chemists primarily focus on publications, many of them try to foster the commercialization of their inventions by patenting them, licensing them, or even forming start-ups of their own,” points out Zhen Lei. Lei, an assistant professor of energy and environmental economics at Pennsylvania State University, is working with Brian D. Wright, an economics professor at the University of California, Berkeley, on a project to study publishing, patenting, and licensing of chemical research at universities in the post-Bayh-Dole era, including the types of funding sources supporting academic research.

“The insights from our project will help us better understand the nexus of government funding and industrial funding and the impact of the Bayh-Dole Act,” Lei says. “They will also inform an ongoing debate about university patenting policy and the proper roles of universities, corporations, and government in fostering research.”

The fourth grant will look at the emerging market for generic biologic drugs, also called follow-on biologics or biosimilars, and evaluate how government policies affect competition.

“Although biologics encompasses an increasing share of the innovation and new products in the pharmaceutical industry, the ability to ensure access to these therapies is limited in the absence of an effective regulatory framework,” explains Scott Stern, a professor of technological innovation, entrepreneurship, and strategic management at Massachusetts Institute of Technology. “Understanding the impact of alternative policies for biologics can significantly enhance both the incentives for innovation and access to important treatments.”

To achieve this goal, Stern and Fiona M. Scott Morton, a professor of economics at Yale University, are doing an assessment of the cost, competition, entry, and social welfare impacts associated with the European experience with biosimilars, which is further along than the U.S. with respect to regulations. “Our project is important because the exact form of U.S. biosimilars regulations will depend importantly on precisely what types of approval, entry regulations, and intellectual property requirements are put in place for follow-on products,” Stern says.

As the four projects get under way, NSF’s Cook acknowledges, there is a chance—albeit a negligible one—that the research will show that the U.S. is overinvesting in the chemical sciences.

“It wouldn’t be necessary to have as sophisticated a group studying the question if the answer was obvious,” Cook says. “But I’m pretty sure the answer will benefit our community.” ◾