Researchers uncover weaknesses in AlphaFold 3

The artificial intelligence–based structure-prediction software AlphaFold, originally developed by Google DeepMind, has quickly become an important tool for drug discovery. Its original iteration wowed scientists with its speed and accuracy in 2020, and two of its developers shared part of the 2024 Nobel Prize in Chemistry. One of the improvements in AlphaFold 3, which DeepMind made together with its drug-discovery spin-off Isomorphic Labs, is more-accurate structure predictions for protein and nucleic acid complexes.

But when U.S. National Institute of Standards and Technology researchers Christina Bergonzo and Alexander Grishaev put AlphaFold 3 to the test, they discovered that there are cases where the tool’s predictions don’t match experimentally determined structures (J. Chem. Inf. Model. 2024, DOI: 10.1021/acs.jcim.5c00245).

The researchers asked the program to predict the structures of a number of RNA and DNA sequences, with some of the RNA sequences coordinated to metal ions. They also selected two sequences—each with structures that change dramatically with a single mutation—and asked AlphaFold to predict the structures before and after each mutation. The researchers compared those and other AlphaFold-predicted structures with ones drawn from the literature that had been deduced using nuclear magnetic resonance spectroscopy. AlphaFold tended to perform best when asked to predict more-common structures.

For instance, when given a section of an RNA ribozyme coordinated to monovalent sodium ions, AlphaFold 3 suggested the section forms a tighter bend than experimental evidence has found. The AlphaFold-predicted shape looked more like the same sequence’s structure when coordinated to divalent ions like manganese ions. The tighter bend found with divalent ions is more common in RNA complexes and would be better represented in the Research Collaboratory for Structural Bioinformatics Protein Data Bank, from which AlphaFold drew much of its training data, Bergonzo says.

The researchers say that is a theme in their results: where oligonucleotide structures obey the most-basic rules—think of adenine-uracil and guanine-cytosine pairing in RNA—AlphaFold 3 makes accurate predictions. But it struggles when it encounters less common motifs.

Grishaev says the results show how important it is that researchers validate AlphaFold 3’s predictions with experimental evidence.

It wasn’t all bad news for AlphaFold 3. Bergonzo says she was “delighted and shocked” that AlphaFold 3 accurately predicted a dramatic difference in structure—a flat plane rather than a hairpin fold—when another RNA sequence was coordinated to mono- and divalent ions.

DeepMind had not responded to C&EN’s request for comment about the new paper at the time of publishing. But the AlphaFold 3 team has acknowledged their tool has limitations, noting in particular that it is stronger in predicting static protein structures—which are captured well in the crystallographic data it was trained with—than more-dynamic molecules, such as RNA and DNA (Nature 2024, DOI: https://www.nature.com/articles/s41586-024-07487-w).

Knowing the correct structure is vital for researchers developing new drugs or trying to understand mutations, says structural biologist Julien Bergeron of King’s College London. Bergeron played no part in developing AlphaFold 3, but he was among its testers and says his laboratory uses it daily. He describes the paper as an example of “science working the way it should” and alerting scientists to some of AlphaFold 3’s specific shortcomings. He suspects DeepMind will be glad to see the data, as they will help the company improve the tool.