[Paper Review] AtomSurf : Surface Representation for Learning on Protein Structures

Vincent Mallet, Souhaib Attaiki+3 more|arXiv (Cornell University)|Sep 28, 2023

Machine Learning in Materials ScienceMaterials Science3 citations

TL;DR

This paper introduces AtomSurf, a novel surface-based representation for protein structures that synergistically combines graph and surface learning in a unified architecture. By integrating geometric message passing across both modalities, the method achieves state-of-the-art performance on all Atom3D benchmark tasks, outperforming standalone surface or graph approaches, particularly in ligand-binding pocket classification and protein structure prediction.

ABSTRACT

While there has been significant progress in evaluating and comparing different representations for learning on protein data, the role of surface-based learning approaches remains not well-understood. In particular, there is a lack of direct and fair benchmark comparison between the best available surface-based learning methods against alternative representations such as graphs. Moreover, the few existing surface-based approaches either use surface information in isolation or, at best, perform global pooling between surface and graph-based architectures. In this work, we fill this gap by first adapting a state-of-the-art surface encoder for protein learning tasks. We then perform a direct and fair comparison of the resulting method against alternative approaches within the Atom3D benchmark, highlighting the limitations of pure surface-based learning. Finally, we propose an integrated approach, which allows learned feature sharing between graphs and surface representations on the level of nodes and vertices across all layers. We demonstrate that the resulting architecture achieves state-of-the-art results on all tasks in the Atom3D benchmark, while adhering to the strict benchmark protocol, as well as more broadly on binding site identification and binding pocket classification. Furthermore, we use coarsened surfaces and optimize our approach for efficiency, making our tool competitive in training and inference time with existing techniques. Code can be found online: https://github.com/Vincentx15/atomsurf

Motivation & Objective

• To evaluate the effectiveness of surface-based representations in protein structure learning, especially in comparison to graph and grid-based methods.
• To address the limitation of standalone surface learning, which underperforms relative to other modalities despite theoretical advantages for surface-sensitive tasks.
• To design a unified, learnable architecture that synergistically combines graph and surface representations to leverage their complementary strengths.
• To benchmark the proposed method on the Atom3D benchmark and binding pocket classification, setting new performance standards.
• To investigate architectural choices in multimodal fusion, including message-passing mechanisms and network depth, for optimal performance.

Proposed method

• Represents proteins as 3D molecular surfaces using a triangulated mesh, preserving geometric properties like curvature and geodesics.
• Introduces a bipartite message-passing framework that enables cross-modality information exchange between graph-based atomic nodes and surface-based mesh vertices.
• Employs three distinct message-passing mechanisms: self-attention (Att.), Graph Convolutional Networks (GCN), and Graph Attention Networks (GAT), with Flash Attention for computational efficiency.
• Uses DiffusionNet as the surface encoder to improve surface representation learning, replacing the less effective MeshCNN used in prior work.
• Applies a shared, learnable projection head to fuse features from both modalities before final prediction head for downstream tasks.
• Conducts ablation studies by varying network depth (3 or 4 blocks) and message-passing strategy while keeping parameter count constant.

Experimental results

Research questions

• RQ1 Can surface-based representations alone achieve competitive performance on protein structure learning benchmarks compared to graph or grid-based methods?
• RQ2 How does combining graph and surface representations improve performance on protein structure prediction and binding site classification tasks?
• RQ3 Which message-passing mechanism (e.g., GAT, GCN, self-attention) yields the best performance in a hybrid graph-surface architecture?
• RQ4 Does deeper architecture or cross-modality attention enhance the model’s ability to capture long-range dependencies in protein structures?
• RQ5 Can a unified, learnable architecture outperform sequential or parallel fusion strategies in multimodal protein representation learning?

Key findings

• Standalone surface-based methods, despite promising theoretical properties, underperform compared to graph and grid-based models on the Atom3D benchmark.
• The proposed hybrid architecture achieves state-of-the-art performance across all tested tasks, including MSP (AuROC: 0.707), PIP (AuROC: 0.859), and PSR (AuROC: 0.833).
• The GAT-based bipartite message-passing mechanism with depth 3 achieves the best overall performance, particularly excelling in PSR, where surface-only models fail.
• The self-attention mechanism (Att.) outperforms localized message passing in most tasks, suggesting long-range information flow is critical for surface-guided learning.
• Deeper networks (4 blocks) improve performance on PIP and PSR, indicating that increased capacity enhances feature refinement in the hybrid setting.
• The sequential and parallel fusion baselines perform poorly, indicating that naive fusion strategies are suboptimal and that architectural design is crucial for success.