Evaluating Deep Learning Based Structure Prediction Methods on Antibody-Antigen Complexes

Motivation

AlphaFold2 significantly improved the prediction of protein complex structures. However, its accuracy is lower for interactions without co-evolutionary signals, such as host-pathogen and antibody-antigen interactions. Two strategies have been developed to address this limitation: massive sampling and replacing the evoformer with the pairformer, which does not rely on co-evolution, as introduced in AlphaFold3.

Results

In this study we benchmark structure prediction methods on unseen antibody-antigen complexes reveals that: (i) increased sampling improves the chances of generating correct models in a roughly log-linear way, (ii) for all methods, a significant challenge is identifying the correct models among those generated, (iii) AlphaFold3 outperforms AlphaFold2, Boltz-1 and Chai-1, and (iv) AlphaFold3’s performance declines significantly for complexes lacking structural similarities to the training set, indicating that it has primarily learnt to detect remote structural similarities rather than to learn the physics of interactions.

Availability and Implementation

All data is available from <ext-link xmlns:xlink="http://www.w3.org/1999/xlink" ext-link-type="uri" xlink:href="https://gitlab.com/ElofssonLab/abag-benchmark-dataset">https://gitlab.com/ElofssonLab/abag-benchmark-dataset</ext-link> and DOI;10.5281/zenodo.15764903. Code is available at <ext-link xmlns:xlink="http://www.w3.org/1999/xlink" ext-link-type="uri" xlink:href="https://github.com/samuelfromm/abag-benchmark-set">https://github.com/samuelfromm/abag-benchmark-set</ext-link>.

Supplementary information