Materials scientist explains why he started commenting on PubPeer

When Angus Wilkinson was looking for case studies on research integrity to include in his presentation for a lab group meeting in November, he came across something interesting.

Wilkinson, a materials scientist at the University of Oxford, saw a March 2023 post on the website PubPeer, where scientists often discuss the rigor of scientific studies. The post described a July 2022 advertisement on the messaging service Telegram offering to sell citations in a study that, at the time, was going through peer review at a journal.

The post was by Nick Wise, an early-career fluid dynamics researcher at the University of Cambridge and a research integrity sleuth whose work has led to the retractions of hundreds of studies. Wise noted in his post that the ad had contained keywords that were prevalent in a study published in December 2022 (Theor. Appl. Fract. Mech., DOI: 10.1016/j.tafmec.2022.103573).

Of course, citations should be earned, not sold. And Wilkinson realized that the study in question aligned with one of his lab’s research areas. “It’s quite useful to make it clear to the group that this stuff happens within the area that we operate in,” he says. “It’s not just a problem that other bits of science might have.”

When he took a closer look at the published article, Wilkinson realized that, in keeping with the concerns raised by Wise, a large number of citations in the study were unrelated to the topic of the study. So he posted his observation on the same PubPeer thread.

“I’ve looked through the introduction section of this paper and suggest that ~65% of the papers cited are inappropriate for the subject area of the text,” he wrote on PubPeer in November 2023. “It would be great if the authors could respond to any of the points detailed below and provide a rebuttal.”

Theoretical and Applied Fracture Mechanics has since retracted the study. The retraction notice reads: “the Editor in Chief has concluded that this article must be retracted due to serious errors, such as including 24 inappropriate references, a duplication of Figure 13 from Figure 14 and changes of authorship without the approval of the Editor in Chief.”

The last author of the study is Filippo Berto, a mechanical engineer at the Sapienza University of Rome. As the Retraction Watch website reported last month, an independent research integrity committee appointed by the Norwegian University of Science and Technology, where Berto had been based previously, has recommended a probe into his work. According to Retraction Watch, five studies coauthored by Berto have been retracted so far.

Wilkinson says that the PubPeer thread led to an interesting discussion in his lab about what steps should be taken if one comes across potentially inappropriate research or publishing behavior. “It was quite sobering as well just to reflect on my own practice,” Wilkinson says. He was left wondering whether he was doing enough to highlight such questionable behavior.

“Is it enough just to kick it around within the group, or should I be taking something a bit further?” he asked himself. So he started commenting on PubPeer every time he came across something fishy.

Since his initial posts last November, Wilkinson has become more active on PubPeer, commenting on more posts and creating new threads about problematic papers he comes across. While many research integrity sleuths use pseudonyms on PubPeer, Wilkinson, like Wise, posts using his own name.

A search of Wilkinson’s name on PubPeer reveals he has posted comments on dozens of threads about different papers. When a study is flagged on PubPeer, often questions are being raised about its integrity, though some PubPeer threads are just discussing research.

Such research integrity sleuthing is not risk free. In 2014, Fazlul Sarkar, formerly a pathologist at Wayne State University, sued PubPeer and subpoenaed the site for names of researchers who commented about his work, claiming that they cost him a job offer from the University of Mississippi. Sarkar was later found guilty of systemic misconduct.

In part to avoid threats or legal ramifications for his sleuthing activities, Wilkinson says he tries to stick to the facts. “It’s nothing personal against the individual here,” he says. “It’s against the practices.”

In addition to posting on PubPeer, Wilkinson contacts journal editors, alerting them to concerns about certain studies. He also encourages others to take up part-time sleuthing and flag questionable practices when they find them. Similarly, he advocates for researchers to use the PubPeer browser plugin—which automatically alerts readers if the paper they are reading has an associated PubPeer thread—so that they are aware of concerns or questions others have raised.

Once you spot something questionable by a researcher or a particular group of authors, Wilkinson says, the obvious thing to do is to take a look at their other publications. That has led him to branch out his sleuthing to papers from fields other than fatigue and structural materials, which are his areas of expertise.

Wilkinson has many concerns about academic publishing, including the fact that problematic research often sneaks through peer review. Instead of relying on the efforts of part-time sleuths like Wilkinson and Wise, he says, the emphasis should be on making peer review more rigorous and on changing the system of incentives and rewards in science so it doesn’t spawn problematic papers in the first place.

Wilkinson says the members of his lab have been having discussions on research integrity for a number of years on topics such as what contributions warrant a researcher being on a manuscript’s author list; the intricacies of peer review; and export controls—rules that govern the international transfer of goods, software, and technology.

“Most of the technologies that we have in materials science are dual-use technologies, so you could end up exporting knowledge that shouldn’t be exported,” Wilkinson says. “Making sure that everybody in the group is aware of those sorts of things is really quite important.”

Being established in his career and in a secure position, Wilkinson says it’s been “relatively easy” for him to become a part-time research sleuth without worrying about career ramifications or upsetting future colleagues or collaborators. “If I don’t feel that I could do it,” he says, “then I can’t really point to anybody else and criticize them for not calling this sort of practice out.”