[Paper Review] ABodyBuilder3: Improved and scalable antibody structure predictions

Henry Kenlay, Frédéric A. Dreyer+3 more|arXiv (Cornell University)|May 31, 2024

Monoclonal and Polyclonal Antibodies ResearchMedicine3 citations

TL;DR

ABodyBuilder3 is a state-of-the-art, scalable antibody structure prediction model that improves accuracy—particularly for the highly variable CDRH3 loop—by integrating protein language model embeddings and a learned pLDDT uncertainty head. It achieves superior performance over prior methods with a single prediction, eliminating the need for computationally expensive ensembles while enabling precise uncertainty estimation and faster, more efficient inference via mixed-precision training and vectorized implementation.

ABSTRACT

Accurate prediction of antibody structure is a central task in the design and development of monoclonal antibodies, notably to understand both their developability and their binding properties. In this article, we introduce ABodyBuilder3, an improved and scalable antibody structure prediction model based on ImmuneBuilder. We achieve a new state-of-the-art accuracy in the modelling of CDR loops by leveraging language model embeddings, and show how predicted structures can be further improved through careful relaxation strategies. Finally, we incorporate a predicted Local Distance Difference Test into the model output to allow for a more accurate estimation of uncertainties.

Motivation & Objective

• To improve the accuracy of antibody structure prediction, especially for the highly diverse CDRH3 loop, which is critical for antigen binding.
• To develop a scalable and efficient inference pipeline that reduces computational overhead compared to ensemble-based methods.
• To replace ensemble-based confidence estimation with a learned per-residue pLDDT score for more accurate and efficient uncertainty quantification.
• To enhance model performance through improved data curation, sequence representation, and structure refinement strategies.
• To enable high-throughput, precise prediction of therapeutic antibody structures for rational drug design.

Proposed method

• The model uses a sequence of eight structure modules based on the ABodyBuilder2 architecture, updated via invariant point attention and backbone coordinate refinement.
• It employs protein language model embeddings (e.g., ProtT5) as input representations to better capture sequence-structure relationships, especially for CDRH3.
• A learned pLDDT head predicts per-residue confidence scores using a multilayer perceptron with softmax activation, trained via cross-entropy loss against ground-truth lDDT-Cα scores.
• The model is trained using mixed-precision (bfloat16) and vectorized operations to improve hardware utilization and accelerate training and inference.
• Structure refinement is applied post-prediction using careful relaxation strategies to further improve model accuracy.
• Uncertainty estimation is derived directly from the pLDDT output, replacing the prior ensemble-based approach in ABodyBuilder2.

Figure 1: Left: Overview of an antibody structure, with the variable region and CDR loops shown. Right: Schematic representation of the ABodyBuilder3 architecture, with 8 sequential and independent update blocks providing the final atomic coordinates and uncertainty predictions from an embedding rep

Experimental results

Research questions

• RQ1 Can protein language model embeddings significantly improve the accuracy of CDRH3 loop prediction in antibodies?
• RQ2 Can a single-model pLDDT head provide more accurate and efficient uncertainty estimation than ensemble-based methods?
• RQ3 How does vectorized, mixed-precision training improve the scalability and computational efficiency of antibody structure prediction models?
• RQ4 To what extent does improved data curation and structure refinement enhance overall prediction accuracy?
• RQ5 Can the pLDDT score serve as a reliable threshold for identifying high-confidence predictions, particularly for CDRH3?

Key findings

• ABodyBuilder3 achieves a mean CDRH3 RMSD of 0.61 Å, representing a significant improvement over ABodyBuilder2’s 0.57 Å when using a single prediction.
• The model with ProtT5 embeddings (ABodyBuilder3-LM) achieves a CDRH3 RMSD of 0.73 Å, demonstrating the benefit of advanced sequence representations.
• The pLDDT-based uncertainty estimate shows a Pearson correlation of 0.58 with RMSD across the CDRH3 region, outperforming the ensemble-based method in ABodyBuilder2.
• Using a pLDDT threshold of 85, 32% of CDRH3 structures are retained, with over 80% of those having a CDRH3 RMSD below 2.0 Å.
• The model is over three times faster than ABodyBuilder2 due to vectorized, mixed-precision training and optimized hardware utilization.
• The pLDDT head enables accurate uncertainty estimation with minimal computational overhead, eliminating the need for multiple model inferences.

Figure 2: Left: Structure predicted by ABodyBuilder3, with colouring indicating the pLDDT uncertainty estimate. The ground truth (7T0J) is shown in grey. Right: Distribution of CDRH3 RMSD across different bins of the CDRH3 pLDDT score.