A White House plan for standardizing risk assessments across the U.S. government is well-intended, representatives of federal agencies told a National Research Council (NRC) committee late last month. But as laid out in January by the President's Office of Management & Budget (OMB), the proposal is likely to snarl government action, they warned, giving plenty of examples.
Adherence to the proposed guidelines, they said, could deter the Food & Drug Administration from telling the public about serious health effects from a prescription drug. Or impede the National Institutes of Health from providing guidance to health professionals about handling blood from patients with HIV. Or slow down the Department of Agriculture's assessments of whether a low-income pregnant woman, infant, or young child is at risk of insufficient nutrition and thus qualifies for supplemental foods provided by the government. Or hamper the Occupational Safety & Health Administration's ability to issue informational bulletins about potential workplace hazards. Or cause the National Aeronautics & Space Administration to ditch years of work compiling detailed risk assessment procedures that are highly specific to space missions.
OMB's draft bulletin is designed to set minimum standards for the scientific quality of federal agency assessments of risk to human health, safety, or the environment. The guidance would require agencies to describe the methods, assumptions, and uncertainties in calculating a risk assessment and to provide a range of risk estimates. It would apply to risk assessments performed for regulation of pollutants, approval of new pharmaceuticals, and development of safety systems for nuclear and chemical plants, liquefied natural gas terminals, and space shuttles (C&EN, Jan. 16, page 6).
OMB asked NRC to review the proposal and to make suggestions for improving it. The NRC committee reviewing the plan held its first public meeting on May 22 to gather information. Members of the panel got an earful from agency officials as well as from representatives of academia, industry, and public interest groups.
John D. Graham, credited as the chief architect of the risk assessment proposal, said the guidelines would apply to risk assessments done for regulation, such as Environmental Protection Agency controls on chemicals. They would also cover deregulation, such as the U.S. decision to accept shipments of Canadian beef again after Canada put systems in place to prevent mad cow disease, he said. The bulletin is an internal guidance document that sets out OMB's expectations of how agencies are to operate, he said.
Graham, who was the administrator of OMB's Office of Information & Regulatory Affairs (OIRA) from 2001 until earlier this year, is now dean of the RAND Graduate School in Santa Monica, Calif.
Representatives from federal agencies said they supported the broad goals of the bulletin, but most also laid out specific concerns about how the proposal could inhibit key activities.
Steven K. Galson, director of FDA's Center for Drug Evaluation & Research, said OMB's bulletin would enhance the quality of his agency's risk assessments. But it also may deter the agency from activities that are important for protecting public health, he said.
For instance, the proposal's definition of risk assessment is too broad, capturing activities that are not intended as formal risk assessments, Galson said.
This broad definition could have serious impacts on FDA's getting critical information to physicians and patients quickly about drugs, medical devices, or foods, Galson said. He pointed out that this kind of risk information is used not by federal regulators but by doctors and patients to select the best course of action for patients.
For instance, FDA may, in the interest of protecting public health, wish to inform the public about an adverse effect from a drug before the agency can, through a formal risk assessment, define the relationship between dose and outcomes, he said. Such a situation occurred in June 2005 when FDA alerted the public about potential suicidal thoughts and behavior in adults being treated with antidepressant medications. The agency did so before it had completed a formal review of the issue, which is still in process.
In addition, the agency is concerned that the OMB bulletin could hamper FDA response to diseases that require rapid action, such as a pandemic flu, Galson said.
The bulletin needs to distinguish "discovery research," such as studies conducted by NIH, from risk assessments prepared by regulatory agencies, said Christopher J. Portier, associate director for risk assessment at the National Institute of Environmental Health Sciences. Because they identify hazards associated with chemicals, human health studies and toxicology tests may eventually feed into risk assessments conducted by regulatory agencies, he pointed out. But the studies themselves, as well as National Toxicology Program products, such as the congressionally mandated report on carcinogens, need to be explicitly excluded from the bulletin's sweep, he said.
Meanwhile, many of the risk assessments done by USDA affect regulatory actions pertaining to pests, diseases, safety of meat and poultry, and release of genetically engineered organisms and would fall under the proposed guidelines, said Linda Abbott, USDA senior scientist for risk modeling. The department, however, is most concerned about whether the OMB bulletin would apply to evaluations of people who may be eligible for benefits under WIC, the Special Supplemental Nutrition Program for Women, Infants & Children.
The proposal could also apply to assessments done to determine the risks of individual farmer's fields to erosion and for other environmental impacts. If the planned standards cover assessments of such individual needs, some activities of the department could face serious delays, she told the NRC committee.
Similarly, OSHA uses risk assessments almost exclusively for its regulatory actions, said William Perry, acting deputy director of the agency's standards and guidance directorate. The bulletin would apply to those risk assessments. However, OSHA is unsure whether the guidelines would also affect the nonregulatory, informational bulletins that it issues occasionally about workplace hazards, Perry said. For instance, OSHA and EPA in 2000 jointly issued an informational bulletin warning about a little-known risk from molten 2,4-dichlorophenol (C&EN, May 1, 2000, page 49).
EPA is the agency rumored to be the main target of the OMB bulletin. George M. Gray, the agency's science adviser, said, though, that EPA "is in pretty good shape" for complying with the planned risk assessment guidelines. The proposal, however, does not address how interagency disputes over risk assessments will be resolved, Gray told the NRC committee.
EPA has found itself at odds with other agencies in some of its chemical-specific assessments. Notably, the agency's preliminary drinking water standard for perchlorate came under heavy fire from the Pentagon, the Department of Energy (DOE), NASA, and their contractors, all of which face liability for pollution from this component of rocket fuel and wanted a more relaxed standard. Eventually, NRC got involved in the dispute and offered a safe daily dose number that was higher than EPA's but would translate into a lower drinking water standard for perchlorate than suggested by the military (C&EN, Jan. 17, 2005, page 13).
Gray and Homayoon Dezfuli, manager for system safety at NASA's Office of Safety & Mission Assurance, pointed out that their agencies have developed their own policies and procedures for risk assessments. OMB's draft bulletin is silent on the role of these agency-specific guidelines.
Dezfuli added that some of the proposal's standards are of limited applicability to the agency's risk assessments or are inconsistent with NASA's formal risk assessment procedures, which represent years of effort. Harmonizing NASA's procedures to the ones proposed by OMB would be a difficult task, he added.
In contrast with many agencies' worries about OMB's bulletin, Shannon E. Cunniff, director of emerging contaminants at the Department of Defense (DOD), indicated no concerns about the proposed standards to the NRC panel.
The plan would apply to many of the military's risk assessments, she said. Most focus on occupational issues, ranging from explosive safety to the air quality inside submarines, but DOD also does site-specific assessments of the risks from contaminants at its facilities, Cunniff said. She added that the military is still trying to determine whether the bulletin would be applicable to weapons safety.
Al Cobb, senior policy adviser in DOE's Office of Policy & International Affairs, said his department probably has "the least interest" in the bulletin among federal agencies because it has virtually no regulatory role. However, DOE is interested in seeing consistency among risk assessments in the government, he said.
John F. Ahearne, chair of the NRC committee, suggested that the bulletin may affect DOE risk assessments done in connection with the cleanup of sites such as the Hanford Nuclear Reservation in Washington. Ahearne is director of the ethics program at Sigma Xi, the scientific research society.
Meanwhile, it is unclear how OMB would ensure that agencies comply with its risk assessment guidelines. The committee asked Nancy Beck, a toxicologist and risk assessor at OIRA, about the consequences for an agency that does not follow the guidelines. She responded, "We haven't gotten that far."
Graham, the former OIRA chief, told the panel that OMB is "not toothless" in ensuring that agencies follow its bulletins because OMB determines agencies' annual budgets.
The committee also heard critiques of the proposal from those outside the government. The draft bulletin uses a definition of risk assessment that is broader than the standard one, which is "the probability as well as the severity of risk," said Gilbert S. Omenn, professor of medicine, genetics, and public health at the University of Michigan Medical School. He chaired a congressionally mandated commission that produced a 1997 report delineating the appropriate uses of risk assessment and risk management in federal regulation.
"OMB has tried to put too much in there," agreed Alan J. Krupnick, a senior fellow at Resources for the Future. He and Omenn, however, endorsed the draft bulletin's intent of improving risk assessment throughout the government.
Risk assessment, Omenn said, is distinct from the identification of hazard presented by a commercial chemical, drug, or other agent, Thus, he said, risk assessment does not include identification of hazards, toxicology studies, structure-activity relationships, modeling, or epidemiology studies. All of these, he said, are "preliminary" to risk assessment. Omenn urged that the final version of the bulletin clearly state that research on hazards does not fall under the proposed guidance.
The NRC committee also heard a variety of arguments about one particular provision in the draft guidelines. This part would require federal risk assessments to provide a range of risk estimates, from what some describe as the "worst case" to the "best case."
Cunniff endorsed ranges instead of a "point estimate," a single number in an assessment reflecting the best judgment on what the risk is most likely to be. Although risk managers tend to prefer a single point estimate, this provides "a false sense of precision that can be misleading to the public," she said.
But point estimates are appropriate because they are provided through the expert judgment of risk assessors, said Robert Shull, director of regulatory policy at watchdog group OMB Watch. Providing "mushy ranges when point estimates are needed" inappropriately hands over part of the risk assessor's job to policymakers deciding how to manage risk, he said.
Judith A. Graham, managing director of the American Chemistry Council's long-range research initiative team, said ranges of risk can be useful to risk managers. "I don't see how information is bad as long as it doesn't cause paralysis by analysis," she said.
Jennifer Sass, a senior scientist at the Natural Resources Defense Council, said scientists within agencies often have expertise specific to a narrow subject or location. The bulletin, by setting government-wide standards, would strip these scientists of their ability to provide expert judgments to agency decisionmakers, she said. Shull agreed, saying the proposal would shift such judgments from agencies to OMB.
The bulletin puts the onus on agencies to justify the risk assessments underlying their regulatory decisions. Omenn and Sass pointed out, however, that it explicitly excludes analyses by companies seeking licenses, approvals, permits, and registrations from the federal government. "I think that might be revisited," Omenn said.
ACC's Graham argued that the proposed standards should cover risk assessments for agencies' overall policies for licensing, permitting, or other approval, but not for those affecting decisions on companies' products.
Lorenz Rhomberg, a principal at Gradient Corp., an environmental consulting company, expressed concern about the proposal's broad call for agencies to describe uncertainties in risk assessments. It is difficult to deal with uncertainty comprehensively, he said, because "someone can always think up" another factor that the assessment has not addressed. Also, no scientific standards exist for analysis of uncertainty. Rhomberg added that simply laying out uncertainties without suggesting which are the most important is not helpful to decisionmakers.
Sass noted that although the bulletin contains some positive aspects, it lays out a burdensome approach that will hamper agencies from taking regulatory action.
ACC's Graham, in contrast, had only praise for the bulletin. "The bulletin is essential," she said, adding that it "will invigorate the scientific community that has advanced the knowledge of chemicals and ways to assess them only to be frustrated when that knowledge and more sophisticated practice was not used" by agencies.
John D. Graham, the former OMB official, said the draft risk assessment bulletin is likely to be revised in light of the NRC committee's comments. Those are expected later this year.