[论文解读] Deciphering antibody affinity maturation with language models and weakly supervised learning

Jeffrey A. Ruffolo, Jeffrey J. Gray+1 more|arXiv (Cornell University)|Dec 14, 2021

Monoclonal and Polyclonal Antibodies Research被引用 91

一句话总结

本论文提出 AntiBERTy，一种在 558M 序列上训练的抗体特异语言模型，并使用弱监督的 MIL 框架在重库中识别结合残基。结果表明，重库嵌入揭示亲和力成熟轨迹，且 MIL 注意力定位于 VRC01 类抗体的对抗位残基。

ABSTRACT

In response to pathogens, the adaptive immune system generates specific antibodies that bind and neutralize foreign antigens. Understanding the composition of an individual's immune repertoire can provide insights into this process and reveal potential therapeutic antibodies. In this work, we explore the application of antibody-specific language models to aid understanding of immune repertoires. We introduce AntiBERTy, a language model trained on 558M natural antibody sequences. We find that within repertoires, our model clusters antibodies into trajectories resembling affinity maturation. Importantly, we show that models trained to predict highly redundant sequences under a multiple instance learning framework identify key binding residues in the process. With further development, the methods presented here will provide new insights into antigen binding from repertoire sequences alone.

研究动机与目标

• Develop an antibody-focused language model to learn repertoire representations.
• Explore affinity maturation trajectories through embeddings of antibody sequences.
• Identify binding/paratope residues using a multiple instance learning framework.
• Demonstrate that MIL attention aligns with known paratope regions in VRC01-class antibodies.

提出的方法

• Train AntiBERTy (BERT-style transformer encoder) on 558M non-redundant antibody sequences using masked language modeling.
• Construct k-NN graphs and evo-velocity analyses from AntiBERTy embeddings to study repertoire evolution.
• Use a multiple instance learning framework to predict bags containing highly redundant (likely binding) sequences.
• Encode sequences with AntiBERTy to produce per-residue embeddings, followed by attention-based pooling and a MIL classifier.
• Train MIL models on bags of 64 sequences with equal positive/negative sampling and use cross-entropy loss.
• Analyze attention maps to identify residues contributing to binding, comparing with known VRC01 gp120 complex structures.

实验结果

研究问题

• RQ1 Can an antibody-specific language model reveal affinity-maturation trajectories within immune repertoires?
• RQ2 Does a MIL framework using AntiBERTy embeddings identify paratope residues in antibodies from repertoire data?
• RQ3 Do MIL attention scores localize to known binding regions (paratopes) in HIV-1VRC01-class antibodies?
• RQ4 What are the quantitative performance characteristics of MIL models on donor-specific and combined repertoires?

主要发现

• AntiBERTy embeddings reveal continuous trajectories from germline to highly mutated derivatives in repertoire embeddings, consistent with affinity maturation.
• MIL models identify bags containing highly redundant sequences with reasonable accuracy across donors (AUROC up to 0.96 for some repertoires).
• Attention maps from MIL embeddings localize to known paratope regions (H2 loop and following beta strand) in several VRC01 antibodies.
• Single-instance MIL predictions correctly flag ten crystallized VRC01 antibodies in binding contexts, validating the approach.
• Combined repertoire MIL model achieves AUROC 0.84 and F1 up to 0.77, indicating transferable signals across donors.