Brazilian effort highlights struggle to replicate research results

Fifty-six laboratories across Brazil have attempted to complete 96 replications total, covering 47 experiments in a bid to quantify how reproducible biomedical research is in the country. The experiments included reverse transcription polymerase chain reactions, colorimetric assays, and behavioral assays for rodents.

According to the study results, which were posted on the preprint server bioRxiv in April, 6 years after the effort was initially announced, only 15–45% of the experiments were found to be reproducible, says study coauthor Olavo Amaral, a metascience researcher at the Federal University of Rio de Janeiro.

“I am worried about that,” Amaral says, pointing out that solid science is difficult to build on if the literature is so difficult to verify. He notes that his results aren’t much different from those of previous replication efforts in the biomedical sciences. “I don’t think we have a number to compare whether our local reality is better or worse than other places.”

Amaral says researchers attempted to replicate each experiment multiple times. If the primary literature were wrong because of cherry-picked data, biases, or fraud, then you’d expect all the replications to fail to reproduce the original results, he explains.

“I do think that a lot of the issue here is technical variability between different laboratories, or perhaps human error,” Amaral says. “If we can’t replicate our own results as a community, I think we have a problem, and it doesn’t matter if the replication estimates or the original estimate is closer to reality.”