[Paper Review] Computational Protein Science in the Era of Large Language Models (LLMs)

Wenqi Fan, Yi Zhou+6 more|ArXiv.org|Jan 17, 2025

Genetics, Bioinformatics, and Biomedical Research6 citations

TL;DR

The paper surveys protein language models (pLMs) and their use in structure prediction, function prediction, and design, framing a sequence-structure-function language and categorizing pLMs by learned knowledge.

ABSTRACT

Considering the significance of proteins, computational protein science has always been a critical scientific field, dedicated to revealing knowledge and developing applications within the protein sequence-structure-function paradigm. In the last few decades, Artificial Intelligence (AI) has made significant impacts in computational protein science, leading to notable successes in specific protein modeling tasks. However, those previous AI models still meet limitations, such as the difficulty in comprehending the semantics of protein sequences, and the inability to generalize across a wide range of protein modeling tasks. Recently, LLMs have emerged as a milestone in AI due to their unprecedented language processing & generalization capability. They can promote comprehensive progress in fields rather than solving individual tasks. As a result, researchers have actively introduced LLM techniques in computational protein science, developing protein Language Models (pLMs) that skillfully grasp the foundational knowledge of proteins and can be effectively generalized to solve a diversity of sequence-structure-function reasoning problems. While witnessing prosperous developments, it's necessary to present a systematic overview of computational protein science empowered by LLM techniques. First, we summarize existing pLMs into categories based on their mastered protein knowledge, i.e., underlying sequence patterns, explicit structural and functional information, and external scientific languages. Second, we introduce the utilization and adaptation of pLMs, highlighting their remarkable achievements in promoting protein structure prediction, protein function prediction, and protein design studies. Then, we describe the practical application of pLMs in antibody design, enzyme design, and drug discovery. Finally, we specifically discuss the promising future directions in this fast-growing field.

Motivation & Objective

• Explain the sequence-structure-function language of proteins and the role of AI in computational protein science.
• Categorize existing protein language models by the knowledge they master: sequence patterns, explicit structure/function information, and external language.
• Summarize how pLMs are used and adapted for structure prediction, function prediction, and protein design.
• Discuss practical biomedical applications of pLMs and outline future directions.

Proposed method

• Classify pLMs into sequence-based, structure-and-function-enhanced, and multimodal categories.
• Describe pre-training objectives and architectures of representative pLMs and how they relate to protein knowledge.
• Explain how pLM representations are used for structure, function, and design tasks through encoder-decoder or integrated architectures.
• Discuss optimization strategies for LLMs in biology including fine-tuning, prompting, and PEFT mechanisms.
• Review downstream applications such as antibody design, enzyme design, and drug discovery.

Experimental results

Research questions

• RQ1 How do pLMs capture and utilize knowledge about protein sequences, structures, and functions?
• RQ2 What categories of pLMs exist and what knowledge do they master?
• RQ3 How are pLMs adapted to improve protein structure prediction, function prediction, and design tasks?
• RQ4 What are current biomedical applications of pLMs and what future directions are anticipated?

Key findings

• pLMs can infer structural and functional information from protein sequences even without explicit evolutionary data in some cases.
• Single-sequence pLMs scale with parameter size and improve knowledge about protein structure at atomic resolution.
• pLMs are effectively used in structure prediction, function prediction, and protein design through various encoder, decoder, and prompting strategies.
• Multi-task and question-answering frameworks enable unified handling of sequence-structure-function reasoning tasks.
• There exist distinct categories of pLMs with different data inputs, training objectives, and architectural designs influencing their suitability for specific protein tasks.