[Paper Review] Advances of Deep Learning in Protein Science: A Comprehensive Survey

Bozhen Hu, Cheng Tan+8 more|arXiv (Cornell University)|Mar 8, 2024

Genetics, Bioinformatics, and Biomedical Research5 citations

TL;DR

This comprehensive survey reviews deep learning advances in protein science, focusing on protein representation learning, model architectures, pretraining paradigms, and key applications like structure and function prediction, plus challenges and future directions.

ABSTRACT

Protein representation learning plays a crucial role in understanding the structure and function of proteins, which are essential biomolecules involved in various biological processes. In recent years, deep learning has emerged as a powerful tool for protein modeling due to its ability to learn complex patterns and representations from large-scale protein data. This comprehensive survey aims to provide an overview of the recent advances in deep learning techniques applied to protein science. The survey begins by introducing the developments of deep learning based protein models and emphasizes the importance of protein representation learning in drug discovery, protein engineering, and function annotation. It then delves into the fundamentals of deep learning, including convolutional neural networks, recurrent neural networks, attention models, and graph neural networks in modeling protein sequences, structures, and functions, and explores how these techniques can be used to extract meaningful features and capture intricate relationships within protein data. Next, the survey presents various applications of deep learning in the field of proteins, including protein structure prediction, protein-protein interaction prediction, protein function prediction, etc. Furthermore, it highlights the challenges and limitations of these deep learning techniques and also discusses potential solutions and future directions for overcoming these challenges. This comprehensive survey provides a valuable resource for researchers and practitioners in the field of proteins who are interested in harnessing the power of deep learning techniques. By consolidating the latest advancements and discussing potential avenues for improvement, this review contributes to the ongoing progress in protein research and paves the way for future breakthroughs in the field.

Motivation & Objective

• Highlight the role of protein representation learning in drug discovery, protein engineering, and function annotation.
• Summarize foundational deep learning architectures and their adaptation to protein sequences, structures, and functions.
• Discuss pretraining and fine-tuning paradigms, including self-supervised learning and large protein models.
• Review applications in protein structure prediction, protein–protein interaction prediction, and protein property prediction.
• Identify challenges, limitations, and potential future research directions in deep learning for proteins.

Proposed method

• Survey the developments of deep learning based protein models and protein representation learning.
• Explain fundamental architectures (CNNs, RNNs, attention models, GNNs) and their use in sequences, structures, and functions.
• Describe transformer-based LMs (BERT, GPT) and their role in protein modeling.
• Discuss graph-based representations and message-passing in protein graphs for structure and interaction tasks.
• Compare pre-trained protein models (e.g., ProtTrans, ESM, GearNet) and the pretrain–finetune paradigm.
• Provide resources and datasets for deep protein methods and outline limitations and future directions.

Experimental results

Research questions

• RQ1 What are the major deep learning architectures and representations used for proteins?
• RQ2 How have pre-training and fine-tuning paradigms been applied to protein modeling and what benefits do they provide?
• RQ3 What are the key applications of deep learning in PSP, PPI, and function prediction, and what challenges exist?
• RQ4 How can multi-level protein structure information be leveraged in pre-training and downstream tasks?
• RQ5 What are the limitations and future directions for deep learning methods in protein science?

Key findings

• Protein representation learning is central to tasks in drug discovery, protein engineering, and function annotation.
• Pre-trained protein encoders like ProtTrans, ESM, and GearNet demonstrate effectiveness across various protein tasks.
• Architectures such as CNNs, RNNs/LSTMs, Transformers, and Graph Neural Networks are adapted to model sequences, structures, and functions of proteins.
• Large-scale pre-trained language models and transfer learning (pre-training and fine-tuning) have become a standard in protein modeling.
• The survey highlights challenges including data scarcity, multimodal and long-tail protein data, and tokenization issues for proteins, and discusses potential future directions.