[Paper Review] Protein-Nucleic Acid Complex Modeling with Frame Averaging Transformer

Tinglin Huang, Zhenqiao Song+2 more|arXiv (Cornell University)|Jun 13, 2024

Bacteriophages and microbial interactionsEnvironmental Science3 citations

TL;DR

This paper introduces FAFormer, an equivariant Transformer architecture that integrates frame averaging within each transformer block to model protein-nucleic acid complexes. By predicting residue-nucleotide contact maps in a geometrically aware manner, FAFormer achieves over 10% relative improvement in contact map prediction and enables unsupervised aptamer screening with 20–30x faster inference than RoseTTAFoldNA, outperforming it on real-world aptamer datasets.

ABSTRACT

Nucleic acid-based drugs like aptamers have recently demonstrated great therapeutic potential. However, experimental platforms for aptamer screening are costly, and the scarcity of labeled data presents a challenge for supervised methods to learn protein-aptamer binding. To this end, we develop an unsupervised learning approach based on the predicted pairwise contact map between a protein and a nucleic acid and demonstrate its effectiveness in protein-aptamer binding prediction. Our model is based on FAFormer, a novel equivariant transformer architecture that seamlessly integrates frame averaging (FA) within each transformer block. This integration allows our model to infuse geometric information into node features while preserving the spatial semantics of coordinates, leading to greater expressive power than standard FA models. Our results show that FAFormer outperforms existing equivariant models in contact map prediction across three protein complex datasets, with over 10% relative improvement. Moreover, we curate five real-world protein-aptamer interaction datasets and show that the contact map predicted by FAFormer serves as a strong binding indicator for aptamer screening.

Motivation & Objective

• To address the challenge of limited labeled data in protein-aptamer binding prediction by developing an unsupervised learning approach.
• To improve contact map prediction for protein-nucleic acid complexes using geometric deep learning.
• To enable large-scale, efficient aptamer screening without requiring costly experimental data.
• To integrate geometric invariance into the Transformer architecture while preserving spatial semantics of 3D coordinates.

Proposed method

• FAFormer employs a novel equivariant Transformer architecture that embeds frame averaging (FA) within each transformer block to preserve spatial geometry.
• It uses a Local Frame Edge Module to encode local pairwise interactions between nodes and their neighbors using geometric frames.
• A Biased MLP Attention Module integrates relational edge features into attention mechanisms, enabling equivariant coordinate updates.
• A Global Frame FFN layer fuses geometric information into node representations across the global context.
• The model is trained end-to-end to predict residue-nucleotide contact maps from 3D structures, with binding affinity estimated as the maximum contact probability.
• Inference is accelerated by using ESMFold-predicted unbound structures instead of MSA-dependent models.

Figure 1: (a) The pipeline of contact map prediction between protein and nucleic acid, and applying the predicted results for screening in an unsupervised manner. The affinity score is quantified as the maximum contact probability over all pairs. (b) Comparison between Transformer with vanilla frame

Experimental results

Research questions

• RQ1 Can frame averaging integrated within each Transformer block improve geometric modeling of protein-nucleic acid complexes?
• RQ2 Does the proposed FAFormer architecture outperform existing equivariant models in contact map prediction across diverse protein complex datasets?
• RQ3 Can a contact map predicted by FAFormer serve as an effective unsupervised indicator for aptamer screening?
• RQ4 How does FAFormer compare in speed and accuracy to large pretrained models like RoseTTAFoldNA for aptamer screening?

Key findings

• FAFormer achieves over 10% relative improvement in contact map prediction across three protein complex datasets compared to state-of-the-art equivariant models.
• On five curated real-world protein-aptamer interaction datasets, FAFormer outperforms RoseTTAFoldNA in aptamer screening, with higher Top10 and Top50 precision and PRAUC scores.
• For the same screening task, FAFormer achieves 20–30x faster inference than RoseTTAFoldNA, with average inference times of 32.65 seconds (protein-DNA) and 51.75 seconds (protein-RNA).
• In contact map prediction, FAFormer matches RoseTTAFoldNA’s performance on the test set while maintaining superior generalization on unseen targets.
• Case studies on PDB IDs 7DVV and 7KX9 show that FAFormer’s predicted contact maps closely align with ground truth, even with sparse contact patterns.
• The integration of frame averaging within each transformer block enables better geometric expressivity than standard FA or spherical harmonics-based methods, without incurring high computational overhead.

Figure 2: Overview of FAFormer architecture. The input consists of the node features, coordinates, and edge representations, which are processed by a stack of (b) Biased MLP Attention Module, (c) Local Frame Edge Module, (d) Global Frame FFN, and (e) Gate Function. $\sum$ deontes aggregation, $\cdot