[Paper Review] Unifying Sequences, Structures, and Descriptions for Any-to-Any Protein Generation with the Large Multimodal Model HelixProtX

Zhiyuan Chen, Tianhao Chen+5 more|ArXiv.org|Jul 12, 2024

Genetics, Bioinformatics, and Biomedical Research4 citations

TL;DR

HelixProtX is a large multimodal language model that enables any-to-any protein generation across sequences, structures, and textual descriptions, leveraging unified training to outperform state-of-the-art models in sequence design, structure prediction, and description generation. It achieves superior accuracy and high self-consistency across diverse protein modalities, demonstrating the potential of unified multimodal modeling in protein research.

ABSTRACT

Proteins are fundamental components of biological systems and can be represented through various modalities, including sequences, structures, and textual descriptions. Despite the advances in deep learning and scientific large language models (LLMs) for protein research, current methodologies predominantly focus on limited specialized tasks -- often predicting one protein modality from another. These approaches restrict the understanding and generation of multimodal protein data. In contrast, large multimodal models have demonstrated potential capabilities in generating any-to-any content like text, images, and videos, thus enriching user interactions across various domains. Integrating these multimodal model technologies into protein research offers significant promise by potentially transforming how proteins are studied. To this end, we introduce HelixProtX, a system built upon the large multimodal model, aiming to offer a comprehensive solution to protein research by supporting any-to-any protein modality generation. Unlike existing methods, it allows for the transformation of any input protein modality into any desired protein modality. The experimental results affirm the advanced capabilities of HelixProtX, not only in generating functional descriptions from amino acid sequences but also in executing critical tasks such as designing protein sequences and structures from textual descriptions. Preliminary findings indicate that HelixProtX consistently achieves superior accuracy across a range of protein-related tasks, outperforming existing state-of-the-art models. By integrating multimodal large models into protein research, HelixProtX opens new avenues for understanding protein biology, thereby promising to accelerate scientific discovery.

Motivation & Objective

• To address the fragmented nature of current protein generation models that specialize in single tasks, such as sequence-to-structure or description-to-sequence.
• To unify multiple protein modalities—amino acid sequences, 3D structures, and textual descriptions—into a single generative framework for end-to-end, flexible protein modeling.
• To reduce research complexity by replacing multiple specialized models with one system capable of handling diverse input-output modality combinations.
• To improve model reliability and consistency through self-consistency evaluation across modalities.
• To accelerate scientific discovery in protein biology by enabling holistic, multimodal protein generation and design.

Proposed method

• HelixProtX is built on a large multimodal language model architecture that jointly encodes and generates protein sequences, 3D structures (backbone only), and natural language descriptions.
• The model is trained end-to-end on a diverse, curated dataset of paired protein modalities, enabling zero-shot and few-shot generalization across all 9 possible input-output modality combinations.
• It employs a unified tokenization scheme that maps sequences, structural coordinates (Cα atoms), and textual descriptions into a shared latent space for cross-modal alignment.
• The model uses a masked autoencoding objective for sequence and structure reconstruction and a conditional language modeling objective for description generation.
• Self-consistency evaluation is performed via X-Y-X and Y-X cycles, where outputs from one modality are used as inputs to reconstruct the original modality, measuring fidelity and consistency.
• Joint training of multiple protein generation tasks is used to improve generalization and reduce error propagation compared to independent training.

Experimental results

Research questions

• RQ1 Can a single unified multimodal model achieve high-quality protein generation across all 9 possible input-output modality combinations?
• RQ2 How does the self-consistency of HelixProtX compare across different modality transitions, especially in sequence and structure reconstruction?
• RQ3 Does joint training of multiple protein generation tasks lead to better performance than training each task independently?
• RQ4 How does HelixProtX perform in challenging tasks such as structure-to-sequence design and description-to-structure generation compared to state-of-the-art models?
• RQ5 To what extent does the model preserve functional and structural fidelity when generating proteins from textual descriptions?

Key findings

• HelixProtX achieves a sequence identity of 0.564 and TM-score of 0.792 in structure-to-sequence design, outperforming ProteinMPNN (0.401 identity, 0.787 TM-score).
• In description-to-sequence generation, HelixProtX achieves 0.486 sequence identity and 0.770 TM-score, surpassing Chroma (0.075 identity, 0.284 TM-score).
• For sequence-to-structure prediction, HelixProtX achieves 2.723 RMSD and 0.416 TM-score, outperforming HelixFold-Single (1.497 RMSD, 0.770 TM-score) in structure quality despite higher RMSD.
• In description-to-structure generation, HelixProtX achieves 2.724 RMSD and 0.407 TM-score, significantly outperforming Chroma (3.558 RMSD, 0.308 TM-score).
• The unified training approach improves BLEU-4 (0.624 vs. 0.541) and ROUGE-F1 (0.829 vs. 0.752) in structure-to-description generation compared to independent training.
• Self-consistency evaluation shows strong alignment between X-Y-X and Y-X cycles, indicating high reliability and consistency in cross-modal generation.