[Paper Review] Learning from Protein Structure with Geometric Vector Perceptrons

Bowen Jing, Stephan Eismann+3 more|arXiv (Cornell University)|Sep 3, 2020

Computational Drug Discovery Methods41 references193 citations

TL;DR

Introduces geometric vector perceptrons (GVPs) to extend GNNs for dual geometric and relational reasoning on protein structures, achieving state-of-the-art results in computational protein design and model quality assessment.

ABSTRACT

Learning on 3D structures of large biomolecules is emerging as a distinct area in machine learning, but there has yet to emerge a unifying network architecture that simultaneously leverages the graph-structured and geometric aspects of the problem domain. To address this gap, we introduce geometric vector perceptrons, which extend standard dense layers to operate on collections of Euclidean vectors. Graph neural networks equipped with such layers are able to perform both geometric and relational reasoning on efficient and natural representations of macromolecular structure. We demonstrate our approach on two important problems in learning from protein structure: model quality assessment and computational protein design. Our approach improves over existing classes of architectures, including state-of-the-art graph-based and voxel-based methods. We release our code at https://github.com/drorlab/gvp.

Motivation & Objective

• Bridge geometric and relational reasoning for learning from biomolecular structures by unifying CNN-like geometric processing with GNNs.
• Develop a rotation-equivariant network layer that directly operates on vector features to preserve geometry during graph propagation.
• Demonstrate the approach on two protein-structure learning tasks: computational protein design (CPD) and model quality assessment (MQA).
• Show that GVP-GNN outperforms existing architectures on CPD and MQA benchmarks.

Proposed method

• Introduce geometric vector perceptrons (GVPs) that operate on scalar features s in R^n and vector features V in R^{ν×3}, producing s' in R^m and V' in R^{μ×3}.
• GVPs concatenate the L2 norms of transformed vectors to extract rotation-invariant information, then apply separate linear and nonlinear transformations to scalars and vectors.
• Prove equivariance/invariance of GVPs under 3D rotations/reflections and show universal approximation properties for rotation- and reflection-invariant scalars.
• Incorporate vector channel dropout and vector-layer normalization to stabilize training.
• Construct GVP-GNN by using GVPs in message-passing steps with scalar and vector edge/node features representing protein backbones.
• Represent proteins as proximity graphs with backbone atoms as nodes and neighbor-edge features that include directional vectors, distances via Gaussian basis, and backbone-relative encodings.

Experimental results

Research questions

• RQ1 Can GVPs enable a graph neural network to reason jointly about geometric and relational aspects of protein structures?
• RQ2 Do GVP-GNNs improve performance on CPD and MQA benchmarks compared with state-of-the-art structure- and sequence-based methods?
• RQ3 What is the impact of including both scalar and vector features versus using only scalars or only vectors on performance?
• RQ4 How do architectural ablations (e.g., removing Wμ, using only scalars/vectors) affect CPD and MQA outcomes?
• RQ5 How does GVP-GNN compare to existing GNNs and 3D CNNs across CASP-era datasets?

Key findings

• GVP-GNN achieves state-of-the-art perplexity and sequence recovery on the CATH 4.2 CPD benchmark, outperforming Structured Transformer and Structured GNN variants.
• On TS50, GVP-GNN achieves 44.9% recovery, exceeding Rosetta's 30% and outperforming methods based on the three representation classes.
• In MQA, GVP-GNN attains higher global and per-target correlations than single-structure, structure-only methods on CASP 11-12 benchmarks, and outperforms top structure-only methods on CASP 13.
• GVP ablations show that removing either scalar or vector pathways degrades performance, with the dual scalar/vector design being essential for best results; removing Wμ also slightly reduces performance.
• Overall, GVP-GNN provides a lightweight equivariant alternative to higher-order tensor methods and demonstrates strong performance on CPD and MQA tasks.