[论文解读] Advances of Deep Learning in Protein Science: A Comprehensive Survey

Bozhen Hu, Cheng Tan+8 more|arXiv (Cornell University)|Mar 8, 2024

Genetics, Bioinformatics, and Biomedical Research被引用 5

一句话总结

本综述全面回顾蛋白质科学中深度学习的进展，聚焦蛋白质表示学习、模型架构、预训练范式，以及结构与功能预测等关键应用，并讨论挑战与未来方向。

ABSTRACT

Protein representation learning plays a crucial role in understanding the structure and function of proteins, which are essential biomolecules involved in various biological processes. In recent years, deep learning has emerged as a powerful tool for protein modeling due to its ability to learn complex patterns and representations from large-scale protein data. This comprehensive survey aims to provide an overview of the recent advances in deep learning techniques applied to protein science. The survey begins by introducing the developments of deep learning based protein models and emphasizes the importance of protein representation learning in drug discovery, protein engineering, and function annotation. It then delves into the fundamentals of deep learning, including convolutional neural networks, recurrent neural networks, attention models, and graph neural networks in modeling protein sequences, structures, and functions, and explores how these techniques can be used to extract meaningful features and capture intricate relationships within protein data. Next, the survey presents various applications of deep learning in the field of proteins, including protein structure prediction, protein-protein interaction prediction, protein function prediction, etc. Furthermore, it highlights the challenges and limitations of these deep learning techniques and also discusses potential solutions and future directions for overcoming these challenges. This comprehensive survey provides a valuable resource for researchers and practitioners in the field of proteins who are interested in harnessing the power of deep learning techniques. By consolidating the latest advancements and discussing potential avenues for improvement, this review contributes to the ongoing progress in protein research and paves the way for future breakthroughs in the field.

研究动机与目标

• 突出蛋白质表示学习在药物发现、蛋白质工程和功能注解中的作用。
• 总结基础深度学习架构及其在蛋白质序列、结构和功能中的适应。
• 讨论预训练与微调范式，包括自监督学习和大型蛋白质模型。
• 回顾在蛋白质结构预测、蛋白质-蛋白质相互作用预测和蛋白质性质预测中的应用。
• 识别蛋白质深度学习领域的挑战、局限性及潜在未来研究方向。

提出的方法

• 综述基于深度学习的蛋白质模型与蛋白质表示学习的发展。
• 解释基本架构（CNNs、RNNs、注意力模型、GNNs）及其在序列、结构和功能中的应用。
• 描述基于 transformer 的语言模型（BERT、GPT）及它们在蛋白质建模中的作用。
• 讨论基于图的表示与蛋白质图中的消息传递在结构和相互作用任务中的应用。
• 比较预训练蛋白质模型（如 ProtTrans、ESM、GearNet）以及 pretrain–finetune 范式。
• 提供深度蛋白质方法的资源与数据集，并概述局限性与未来方向。

实验结果

研究问题

• RQ1 用于蛋白质的主要深度学习架构和表示有哪些？
• RQ2 预训练和微调范式在蛋白质建模中是如何应用的，它们带来了哪些好处？
• RQ3 深度学习在 PSP、PPI 和功能预测中的关键应用有哪些，以及存在哪些挑战？
• RQ4 如何在预训练和下游任务中利用多层次的蛋白质结构信息？
• RQ5 蛋白质科学中深度学习方法的局限性与未来方向是什么？

主要发现

• 蛋白质表示学习是药物发现、蛋白质工程和功能注解等任务的核心。
• Pre-trained protein encoders like ProtTrans、ESM、and GearNet demonstrate effectiveness across various protein tasks.
• Architectures such as CNNs, RNNs/LSTMs, Transformers, and Graph Neural Networks are adapted to model sequences, structures, and functions of proteins.
• Large-scale pre-trained language models and transfer learning (pre-training and fine-tuning) have become a standard in protein modeling.
• The survey highlights challenges including data scarcity, multimodal and long-tail protein data, and tokenization issues for proteins, and discusses potential future directions.