[Paper Review] Leveraging Biomolecule and Natural Language through Multi-Modal Learning: A Survey

Qizhi Pei, Zhou, Zhimeng+7 more|arXiv (Cornell University)|Mar 3, 2024

Machine Learning in Bioinformatics5 citations

TL;DR

A comprehensive survey of cross biomolecule-language modeling, detailing representations, learning frameworks, tasks, datasets, and future directions.

ABSTRACT

The integration of biomolecular modeling with natural language (BL) has emerged as a promising interdisciplinary area at the intersection of artificial intelligence, chemistry and biology. This approach leverages the rich, multifaceted descriptions of biomolecules contained within textual data sources to enhance our fundamental understanding and enable downstream computational tasks such as biomolecule property prediction. The fusion of the nuanced narratives expressed through natural language with the structural and functional specifics of biomolecules described via various molecular modeling techniques opens new avenues for comprehensively representing and analyzing biomolecules. By incorporating the contextual language data that surrounds biomolecules into their modeling, BL aims to capture a holistic view encompassing both the symbolic qualities conveyed through language as well as quantitative structural characteristics. In this review, we provide an extensive analysis of recent advancements achieved through cross modeling of biomolecules and natural language. (1) We begin by outlining the technical representations of biomolecules employed, including sequences, 2D graphs, and 3D structures. (2) We then examine in depth the rationale and key objectives underlying effective multi-modal integration of language and molecular data sources. (3) We subsequently survey the practical applications enabled to date in this developing research area. (4) We also compile and summarize the available resources and datasets to facilitate future work. (5) Looking ahead, we identify several promising research directions worthy of further exploration and investment to continue advancing the field. The related resources and contents are updating in https://github.com/QizhiPei/Awesome-Biomolecule-Language-Cross-Modeling.

Motivation & Objective

• Survey biomolecule representations (1D sequences, 2D graphs, 3D structures) and their roles in BL modeling.
• Examine rationale, goals, and core learning frameworks for integrating language with biomolecules.
• Catalog current applications in property prediction, generation, and retrieval.
• Summarize available resources, datasets, and benchmarks to accelerate future work.
• Identify open challenges and promising directions to advance BL research.

Proposed method

• Classify and analyze biomolecular representations including sequences, graphs, and structures.
• Survey machine learning frameworks such as GPT-based pre-training and multi-stream architectures.
• Discuss representation learning strategies, training tasks, and learning objectives for BL.
• Review practical applications in prediction, generation, and information retrieval.
• Compile datasets, models, and benchmarks and outline future research directions.

Experimental results

Research questions

• RQ1 What are the prevalent biomolecule representations used in cross-modal biomolecule-language (BL) modeling?
• RQ2 What learning frameworks and representation strategies effectively integrate language with biomolecular data?
• RQ3 What applications have been demonstrated via BL models, and what are their performance trends?
• RQ4 What resources, datasets, and benchmarks currently support BL research?
• RQ5 What are the main challenges and promising directions for future BL work?

Key findings

• Cross-modal BL modeling combines textual, molecular, and protein data to create richer representations for downstream tasks.
• Foundational models like MolT5 and BioT5 demonstrate strong retrieval and generation capabilities between molecules and text.
• Architectures span encoder-only, decoder-only, encoder-decoder, and dual/multi-stream designs, including PaLM-E style frameworks.
• There exists a growing set of datasets, models, and benchmarking resources to accelerate BL research (e.g., publicly available resources and contents updating in the cited GitHub repository).
• Instruction following and agent/assistant paradigms enable zero-shot tasks and interactive biomolecule knowledge retrieval with large language models.
• The survey highlights open challenges such as interpretability and generalization, and outlines future directions for BL research.