Pandemic puts preprints first

Before the COVID-19 pandemic, Max Crispin had been a little wary of preprints. He’d worked with collaborators who submitted their joint work to preprint servers, which allow public access to papers that have not yet been peer reviewed or published in a journal. But the University of Southampton biochemist hadn’t submitted any of his papers to one.

Then in February, Crispin sent a paper to Nature Communications on coronaviruses. The paper mentioned SARS-CoV-2, the novel virus that causes COVID-19. The editor told him that all papers related to the virus had to be submitted to a preprint server to speed access to the research.

“That was pretty impressive,” Crispin says. “So I thought ‘Let’s do it.’ ” He submitted the paper to the life science preprint server bioRxiv. He added a second paper a few weeks later, this time one focused on SARS-CoV-2.

Preprint servers in almost all disciplines are seeing an influx of papers surrounding COVID-19, pushed by a need for speed during a global pandemic. Simultaneously, they are dealing with an immense increase in views from the public and nonscience media, who aren’t always familiar with what a preprint server is or how it is different from a research journal.

Preprint servers are taking that newfound attention seriously. They have placed prominent notices on their home pages and added extra screening for SARS-CoV-2 papers. At the same time, many journals are pushing authors like Crispin toward preprints as a reflection of the urgency of the pandemic. Whether this will change scientific publishing and the public’s understanding of science is still up in the air.

But everyone knows the public is watching. “For obvious reasons, and rightly so, all eyes are on COVID-related research,” says Magdalena Skipper, editor in chief of Nature. “In many ways it’s an unprecedented situation for science, that it’s so center stage, round the clock, all the time.”

Preprints step up

John R. Inglis remembers waking up one Saturday morning in February to discover an enormous amount of traffic on bioRxiv. The views were centered on one particular paper that had been posted the previous evening saying that the similarity of certain SARS-CoV-2 and HIV protein sequences was “uncanny.” Much of the traffic came from sites pushing a conspiracy theory that the virus had been made in a lab.

It was the first evidence that the pandemic was bringing in readers “who are not professional scientists and who had a limited understanding of the aim and purpose of a preprint,” says Inglis, executive director of Cold Spring Harbor Laboratory Press, which runs bioRxiv and its health science–focused counterpart, medRxiv.

Within 2 h, Inglis and his colleagues decided to strengthen an existing statement that preprints are not peer reviewed and post it more prominently on both websites, including on a page collecting COVID-19-related research and on individual articles. Other preprint servers have followed suit.

bioRxiv and medRxiv’s leadership has always had concerns that posted research might be used for bad purposes. Those could be developing a biological weapon or “encouraging people to take something that might be dangerous, because the preprint says it has some therapeutic value,” Inglis says. “Those [concerns] became much more acute in this time of the pandemic.”

The particular paper comparing SARS-CoV-2 and HIV proteins was attacked on social media for its bad science and quickly withdrawn by its authors, a fairly rare occurrence for bioRxiv preprints, Inglis says. It put all preprint servers on alert that the COVID-19 pandemic was not a normal time.

ChemRxiv product manager Marshall Brennan noticed a readership spike in March, especially from Google. He also got emails from several reporters who didn’t realize the research wasn’t peer reviewed. Stories referencing ChemRxiv papers were published in CNN and the New Yorker, showing the unusual extent of interest in coronavirus research.

On April 20, Brennan added a banner to the ChemRxiv home page explaining what a preprint is. Now, the same wording appears on every paper.

“We recognize that it’s a weird thing for us as a preprint server to do, because preprint servers are supposed to be relatively hands off,” Brennan says. “We think it’s justified” given the global health situation.

Queen’s University chemist Cathleen Crudden, who is on ChemRxiv’s board of directors, agrees. “We do have to be very careful in these days of fake news and rapid dissemination of information that the general public understands the difference between a preprint and a paper,” she says. (The American Chemical Society, which publishes C&EN, is one of five organizations that support ChemRxiv. Crudden is also a member of C&EN’s advisory board.)

Many preprint servers are also instituting more screening for papers related to COVID-19. ChemRxiv already screened papers to ensure they were actually science. Now it looks for any language that implies a therapeutic use. That means considering whether someone without a science background who googles COVID-19 and reads a paper would take an action they probably shouldn’t without medical advice, Brennan says.

“A lot of this really comes down to folks trying to make drug recommendations off of computational data,” Brennan says. For example, he asked authors to rethink language in a docking study of a home remedy to SARS-CoV-2. “Look, that is coming way too close to the line of trying to recommend that people treat themselves using this,” he says he told them. They resubmitted it without the problematic language.

bioRxiv already had up-front screening to make sure papers were scientific research and hadn’t been published before or plagiarized, Inglis says. As the pandemic unfolded, the leadership decided that all papers must include biological experiments; the servers will no longer take COVID-19 papers that are only computer-based analyses. “We felt that they weren’t very useful at this point in the proceedings,” he says.

Before the pandemic, medRxiv’s prescreening had more extensive checks than bioRxiv to make sure patient consents are signed, clinical trials are registered, and more. Those were checked by a pool of scientific affiliates of the preprint servers.

“With the advent of COVID-19, we felt that we needed to strengthen the pool of scientific affiliates to include a group of people who have outbreak science expertise,” Inglis says. Those experts can reject papers for making sensationalized health claims or making claims that the data don’t justify. “We are more careful about the health claims that are being made in these papers, knowing that the public and the media are paying a great deal of attention,” Inglis says.

Journals respond

Journals have long had an uneasy relationship with preprint servers. So it was a big step when many journals required that authors post papers related to COVID-19 on preprint servers.

The requirement came out of a coronavirus research openness agreement, organized by the Wellcome Trust in January, that was signed by dozens of journals, universities, scientific funders, and societies, including ACS.

One of the Wellcome statement signatories is the New England Journal of Medicine, which historically has been reticent about allowing authors to submit papers to preprint servers before they are published in the journal. But in a February editorial, the journal’s editors said they would encourage authors to submit COVID-19 research to preprint servers.

“The current outbreak of coronavirus infection is a threat to the health of the public and a breaking news story that changes hour by hour,” the editorial says. “We can eventually take a long view of how to manage and prevent epidemics, but today practitioners and public health authorities need actionable information as soon as possible.”

Nature has always been one of the publishers more open to preprints for all of its journals, Skipper says. It has allowed scientists to use them for around 20 years. In December, its parent company, Springer Nature, decided to encourage—but not require—authors to post pandemic-relevant papers on preprint servers before submitting to journals. Springer Nature subsequently signed on to the Wellcome Trust agreement.

“In the pandemic, it’s absolutely paramount that research results and data are available straight away,” Skipper says.

Skipper still thinks that research papers are better after they go through a journal’s review process than they are if feedback only comes from online comments on a preprint. Journals expect reviewers to look at all parts of a paper and make sure the data match the conclusions.

“When you are asked to peer review a paper for a journal, and you agree to it, you have a sense of responsibility, you have a sense of duty to your colleagues, to your community, to the journal,” she says. “In that sense, it’s a complementary process” to preprints.

Crudden, who is also an editor at ACS Catalysis, agrees that when done properly, “peer review really improves papers, and we would hope that can prevent incorrect information from getting into the literature.”

But that process takes a long time. “It’s not unusual for some high profile papers to be in review for a year,” Crudden says. That’s why she sees an important role for preprints. “As an author, you want your work to get out there, because you want people to know that this is what you’re doing.”

Future of publishing science

The pandemic could help chemists see the value of preprints, says Henrik Pedersen, a materials scientist at Linköping University who has been submitting his papers to ChemRxiv for several years.

“It’s taken the whole chemistry community quite some time to get adjusted to the idea of preprints,” he says. But with the coronavirus, “everybody knows that peer review can take months, and we don’t have months to get access to new resources.”

Crispin saw the benefits of early exposure when he submitted his coronavirus papers as preprints. “What I hadn’t appreciated is just the synergy between the preprint servers and social media,” he says. “You get a really good feel for the level of interest on your paper.”

Several journals also approached him about publishing his papers. That was “very valuable in knowing that they’d be interested in that type of work,” Crispin says. Going forward, he might not default toward submitting preprints for every paper, but he would for papers that relate to public health or face other time pressure, he says.

Virologist Andrew Ward at Scripps Research in California says that young scientists like himself have been the ones who have pushed journals to accept preprints. But he is skeptical a pandemic-induced turn toward preprints will lead to immediate change. He thinks the current push “might just be a blip” mostly driven by “nonexperts and people who have never worked on coronaviruses before who want a fast way in to the marketplace,” he says. Nevertheless, he thinks the long-term trend points to more preprint publishing, “especially as younger scientists take on more senior leadership roles.”

Ward does worry about the quality of the science on COVID-19. “To be honest, there’s a ton of garbage out there, but it’s hard to filter that out,” he says. Someone can’t rely just on interest or downloads to gauge importance—Ward points out that the “completely bogus study” comparing SARS-CoV-2 and HIV proteins has thousands of downloads, “but it obviously doesn’t positively contribute to science or knowledge of the virus.”

Ivan Oransky, cofounder of the research integrity blog Retraction Watch, hopes that scientists don’t change their writing process because of concerns about the public reading their papers. “I would hate for scientists to self-censor because of that,” he says.

For now, the pandemic is clearly accelerating scientists’ use of preprints, Oransky says. Bad papers show up on preprint servers but can also be published in even the best journals, he points out.

He would like to see journals be more transparent about their processes with notices such as those ChemRxiv and other preprint servers have posted for the public. For example, they could clarify what peer review means by detailing how many researchers reviewed a paper and how those reviewers were chosen. “It remains to be seen what world we end up with,” Oransky says. “If this allows for a scientific publishing process that better reflects how science actually works, I think that would be great.”

Overall, the pandemic has really shown the value of preprints, Inglis says. “Having hundreds, possibly even thousands of pairs of eyes on a particular piece of work—that benefits the author and it benefits science,” he says.

But he thinks the real change might come from the public attention on science. The public is “seeing the process in action. I tend to think that there are going to be ultimate benefits to that,” like attracting young people who want to work on important problems, Inglis says.

At the same time, because of the spotlight on science, “the potential damage of stories which are clearly incorrect is enormous,” Nature’s Skipper says. Incorrect or misinterpreted science may go beyond merely an attention diversion and waste of time, she says, such that “the general public may begin to doubt the veracity of scientific research.”