Synthesis Sees Inflationary Push

Most chemists have gone into the lab at some point in their careers with a new chemical recipe in hand, only to be per­plexed when it doesn’t turn out as described in the paper from which it was torn. Organic chemists Tomas Hudlicky and Martina Wernerova of Brock University, in St. Catharines, Ontario, recently decided to dig a little deeper into one facet of this phenomenon: inaccurate reporting of isolated product yields, ratios of diastereomers, and enantiomeric excess values in organic synthesis papers (Synlett, DOI: 10.1055/s-0030-1259018).

In comparing organic papers published from 1955 to 1980 and from 1980 to 2005, Hudlicky determined that reports of nearly perfect yields and remarkable enantioselectivities were common in the recent period but rare before 1980. To support the findings, Hudlicky and Wernerova conducted volumetric experiments on sample reactions, measuring differences between the actual content of materials and the experimentally measured values for yields and isomer ratios to determine the practical limits and relative errors in reporting such data.

The inflationary trend in yield and other data is occasionally a topic of informal conversation at chemistry conferences, Hudlicky tells C&EN. Most chemists agree that reported yields above 95%, isomer ratios above 200:1, and enantiomeric excesses above 99% aren’t realistic. For example, a couple percent yield can be lost just in extractions and product purifications, let alone from incomplete reactions and weighing errors.

“Such reports at best may be attributed to the inability of the authors and research personnel to determine accurately the true content of the sample under scrutiny, and at worst to deliberate manipulation of research data,” Hudlicky and Wernerova write.

The reasons for inflated yields and ratios are many, Hudlicky says, but the key ones include pressure on the research community to produce high-quality, high-impact research on smaller scale reactions at a fast pace. He suggests that principal investigators may not be adequately reviewing the research conducted in their labs, academic researchers may not have the training needed or the equipment and financial resources needed, and referees and editors may not be adequately scrutinizing submitted papers.

“People continue to engage in self-deception regarding the reality of experimental work,” Hudlicky adds. “Most of the time they have no idea what they actually are measuring.” For example, reporting 99% yields following chromatography “is an outright ridiculous situation,” he points out.

The Synlett analysis “has made quite an impact,” Hudlicky says. He has received hundreds of e-mails, he notes, that agree something needs to be done about this issue. There has also been discussion about the paper in chemistry blogs.

Trevor Laird, editor of the American Chemical Society journal Organic Process Research & Development and founder of industrial chemistry consulting firm Scientific Update, agrees with Hudlicky and Wernerova’s assessment and commented on it in an editorial (Org. Process Res. Dev., DOI: 10.1021/op2000404). Laird says researchers should be wary of literature data claiming quantitative yields.

“In industry we know how easily final products can be contaminated with inorganic salts, which do not show up in NMR or HPLC nonquantitative analyses, and this is why residue on ignition or sulfated ash tests are routine for final products in industry,” Laird writes. But such tests are rarely carried out in academic laboratories, he adds.

Laird suggests that most papers in OPR&D come from industrial process chemistry labs that are more likely to report quantitative measurements that have been repeated many times by different experimenters. As a consequence, he says, they should be reproducible in the lab and on a larger scale than experiments reported in most journals.

For their part, Hudlicky and Wernerova used as a yardstick Organic Syntheses, the highly respected independent open-access journal in which experiments are separately checked by referees before publication and yields rarely exceed 95%.

Hudlicky and Wernerova offer a few remedies for overstated yields and ratios: Authors could either provide evidence of a standard calibration and the method used or insert a disclaimer indicating that the values have not been validated, authors could report a range of yields when multiple experiments are carried out, and journal editors could reject papers when the reported yields are deemed unrealistic.

Amos B. Smith III, editor of the ACS journal Organic Letters and a chemistry professor at the University of Pennsylvania, finds Hudlicky and Wernerova’s observations “timely and important,” and he is taking their message to heart. After reading the Synlett paper, Smith asked his associate editors and members of his editorial advisory board to read it and consider how they might follow up on the suggestions.

The issue of overstating yields and isomer ratios “is not a revelation in the organic research community,” Smith says. “It is bogus to report 100% yields and enantiomerically pure products. But we understand that when someone reports very high yields and ratios, it means the results are pretty good.”

Smith adds that, at least in the academic community, which is the major source of research papers, no one has the resources anymore to spend a huge effort to nail down yields and ratios with high precision. He says that in lieu of costly elemental analyses, using NMR or chromatographic techniques is reasonable. But researchers do need to clearly state how they obtained their data, Smith notes; that information typically appears in the fine print of data tables or in the online supporting information.

“This issue is well worth giving additional consideration and having our students think about it,” Smith says. “Where possible we should tighten up reporting data, so long as the process is not so onerous that it impedes new discoveries.”

It turns out Hudlicky is well-known in the organic community for voicing his opinions. Several years ago he commented in a C&EN letter to the editor that the use of catchy titles and adjectives that hype up the content of organic synthesis papers had “gone through the roof” (C&EN, April 4, 2005, page 6). Back then, Hudlicky did a literature search and found that descriptors such as new, novel, efficient, concise, facile, and practical were proliferating.

“If 1% of such claims were true, there would be no problems in organic chemistry left unsolved,” Hudlicky wrote. “Such claims are in stark contrast to the actual content of articles so advertised.” He suggested that it would be prudent of journals to agree on a policy to ban the use of these descriptors. Hudlicky has now incorporated the hyperbole and the inflated yield and ratio issues into a one-day short course on research integrity that he teaches.

Hudlicky says his intentions are only for the good—he pleads guilty himself of reporting unrealistic yields in the past. Getting these issues more out in the open so the organic community can reflect on them in greater detail, he says, should help improve experimental reproducibility and facilitate chemical advances, as well as improve the credibility of researchers.