[Paper Review] Enhancing Protein Predictive Models via Proteins Data Augmentation: A Benchmark and New Directions

Rui Sun, Lirong Wu+3 more|arXiv (Cornell University)|Mar 1, 2024

Gene expression and cancer classificationBiochemistry, Genetics and Molecular Biology3 citations

TL;DR

This paper proposes Automated Protein Augmentation (APA), a framework that adaptively selects optimal data augmentation strategies for protein predictive models. By extending image and text augmentation techniques to proteins and introducing two semantic-level methods—Integrated Gradients Substitution and Back Translation Substitution—APA boosts performance across five protein tasks by an average of 10.55% compared to vanilla models without augmentation.

ABSTRACT

Augmentation is an effective alternative to utilize the small amount of labeled protein data. However, most of the existing work focuses on design-ing new architectures or pre-training tasks, and relatively little work has studied data augmentation for proteins. This paper extends data augmentation techniques previously used for images and texts to proteins and then benchmarks these techniques on a variety of protein-related tasks, providing the first comprehensive evaluation of protein augmentation. Furthermore, we propose two novel semantic-level protein augmentation methods, namely Integrated Gradients Substitution and Back Translation Substitution, which enable protein semantic-aware augmentation through saliency detection and biological knowledge. Finally, we integrate extended and proposed augmentations into an augmentation pool and propose a simple but effective framework, namely Automated Protein Augmentation (APA), which can adaptively select the most suitable augmentation combinations for different tasks. Extensive experiments have shown that APA enhances the performance of five protein related tasks by an average of 10.55% across three architectures compared to vanilla implementations without augmentation, highlighting its potential to make a great impact on the field.

Motivation & Objective

• Address the underexplored area of data augmentation in protein sequence modeling, where labeled data is scarce.
• Benchmark existing image and text augmentation techniques adapted for protein sequences to evaluate their effectiveness.
• Propose novel semantic-level augmentation methods that preserve biological function while generating meaningful variations.
• Develop an automated framework, APA, to dynamically select the most effective augmentation combinations for diverse tasks and architectures.
• Demonstrate that adaptive, semantic-aware augmentation significantly improves model generalization and performance across multiple protein-related tasks.

Proposed method

• Adapted image and text augmentation techniques—such as random masking, token shuffling, and back-translation—into protein sequence-specific methods.
• Introduced Integrated Gradients Substitution, which identifies salient regions in protein sequences via gradient attribution and substitutes them with biologically plausible alternatives.
• Proposed Back Translation Substitution, leveraging reverse translation from mRNA sequences to protein sequences to generate biologically meaningful variants.
• Constructed a comprehensive protein augmentation pool integrating token-level, sequence-level, and semantic-level methods.
• Designed APA, a meta-learning framework that uses validation accuracy to automatically select and combine the most effective augmentation strategies per task and architecture.
• Integrated batch normalization and ablation analysis to validate component contributions and ensure training stability.

Experimental results

Research questions

• RQ1 How effective are existing image and text augmentation techniques when directly applied to protein sequences?
• RQ2 Can semantic-level augmentation methods that leverage saliency maps and biological knowledge improve protein model performance?
• RQ3 To what extent does automated selection of augmentation combinations enhance model generalization across diverse protein tasks and architectures?
• RQ4 What is the contribution of individual components—such as Integrated Gradients Substitution and batch normalization—to the overall performance gain?
• RQ5 How does APA affect training dynamics, including convergence speed and final accuracy?

Key findings

• APA improved the average performance of five protein-related tasks by 10.55% across three deep learning architectures compared to vanilla models without augmentation.
• The ablation study showed that removing Integrated Gradients Substitution led to a significant performance drop, with a 2.29% decrease in subcellular localization accuracy (from 88.26% to 83.16%).
• Heatmap visualization revealed that saliency regions shift dynamically during training, confirming that Integrated Gradients Substitution enables adaptive, context-aware augmentation.
• APA accelerated model convergence and achieved consistently higher test accuracy than the vanilla LSTM across 50 training epochs on the subcellular localization task.
• The removal of batch normalization led to a 1.82% drop in subcellular localization accuracy (from 88.26% to 84.99%), indicating its role in training stability.
• Back Translation Substitution demonstrated strong semantic preservation, generating biologically plausible variants that maintained functional relevance while enhancing model robustness.