[Paper Review] Autoregressive Enzyme Function Prediction with Multi-scale Multi-modality Fusion

Dingyi Rong, Wenzhuo Zheng+4 more|arXiv (Cornell University)|Aug 11, 2024

Machine Learning in BioinformaticsBiochemistry, Genetics and Molecular Biology3 citations

TL;DR

MAPred is a novel autoregressive deep learning model that predicts enzyme EC numbers by fusing multi-scale protein sequence and 3Di-structured representations through a dual-pathway network. It outperforms existing methods on benchmark datasets (F1: 0.610 on New-392, 0.493 on Price, 0.680 on New-815), demonstrating superior accuracy by leveraging hierarchical EC number structure and attention to functional sites.

ABSTRACT

Accurate prediction of enzyme function is crucial for elucidating biological mechanisms and driving innovation across various sectors. Existing deep learning methods tend to rely solely on either sequence data or structural data and predict the EC number as a whole, neglecting the intrinsic hierarchical structure of EC numbers. To address these limitations, we introduce MAPred, a novel multi-modality and multi-scale model designed to autoregressively predict the EC number of proteins. MAPred integrates both the primary amino acid sequence and the 3D tokens of proteins, employing a dual-pathway approach to capture comprehensive protein characteristics and essential local functional sites. Additionally, MAPred utilizes an autoregressive prediction network to sequentially predict the digits of the EC number, leveraging the hierarchical organization of EC classifications. Evaluations on benchmark datasets, including New-392, Price, and New-815, demonstrate that our method outperforms existing models, marking a significant advance in the reliability and granularity of protein function prediction within bioinformatics.

Motivation & Objective

• To address the limitations of existing EC number prediction models that rely solely on sequence or structure and ignore the hierarchical nature of EC numbers.
• To improve functional prediction accuracy by integrating both primary amino acid sequences and 3D structural representations using 3Di tokens.
• To enhance model interpretability by identifying key functional residues through attention visualization.
• To develop a multi-scale, multi-modality framework that captures both global protein context and local functional motifs.
• To establish a new paradigm for enzyme function prediction by modeling EC number prediction as a sequential, autoregressive task.

Proposed method

• MAPred uses ProstT5 to generate 3Di tokens from protein sequences, enabling structured representation of 3D protein geometry.
• It employs a dual-pathway architecture: a global feature extractor using interlaced sequence-3Di cross-attention and a local feature extractor using convolutional neural networks (CNNs).
• The model applies an autoregressive prediction head that sequentially predicts each of the four digits of the EC number, modeling their hierarchical dependencies.
• Multi-modality inputs (sequence and 3Di) are fused early in the network to enable joint representation learning.
• Attention mechanisms are used to interpret model predictions, with saliency maps visualized on 3D protein structures to identify functional regions.
• Ablation studies are conducted by disabling individual components (global path, local path, autoregressive head) to evaluate their contribution.

Experimental results

Research questions

• RQ1 Can a multi-modal, multi-scale deep learning model that fuses sequence and 3Di-structured representations improve EC number prediction accuracy?
• RQ2 Does modeling EC number prediction as an autoregressive sequence task enhance performance by leveraging hierarchical label structure?
• RQ3 Can the model learn to attend to biologically relevant functional sites, such as catalytic centers and substrate-binding regions?
• RQ4 How do global and local feature extraction pathways contribute independently to prediction performance?
• RQ5 To what extent does multi-modality input (sequence + 3Di) outperform single-modality inputs?

Key findings

• MAPred achieves an F1 score of 0.610 on the New-392 dataset, outperforming existing models.
• On the Price dataset, MAPred achieves an F1 score of 0.493, demonstrating strong performance on challenging, diverse enzyme sets.
• On the New-815 benchmark, MAPred attains an F1 score of 0.680, indicating high generalization and precision in function prediction.
• Ablation studies show that removing the global feature extraction pathway causes the largest performance drop (F1: 0.651 → 0.354 on New-392), highlighting its critical role.
• The local feature extraction pathway also contributes significantly, though less than the global path, indicating importance of local structural motifs.
• Disabling the autoregressive prediction head reduces performance, confirming that modeling EC number hierarchy improves prediction accuracy.