[Paper Review] A Text-guided Protein Design Framework

Shengchao Liu, Li, Yanjing+11 more|arXiv (Cornell University)|Feb 9, 2023

Machine Learning in BioinformaticsBiochemistry, Genetics and Molecular Biology23 citations

TL;DR

ProteinDT is a multi-modal framework that aligns text and protein sequences to enable text-guided protein generation, zero-shot editing, and property prediction, using SwissProtCLAP for training. It achieves over 90% retrieval accuracy in text-to-protein generation and best hit ratios on multiple zero-shot editing tasks.

ABSTRACT

Current AI-assisted protein design mainly utilizes protein sequential and structural information. Meanwhile, there exists tremendous knowledge curated by humans in the text format describing proteins' high-level functionalities. Yet, whether the incorporation of such text data can help protein design tasks has not been explored. To bridge this gap, we propose ProteinDT, a multi-modal framework that leverages textual descriptions for protein design. ProteinDT consists of three subsequent steps: ProteinCLAP which aligns the representation of two modalities, a facilitator that generates the protein representation from the text modality, and a decoder that creates the protein sequences from the representation. To train ProteinDT, we construct a large dataset, SwissProtCLAP, with 441K text and protein pairs. We quantitatively verify the effectiveness of ProteinDT on three challenging tasks: (1) over 90% accuracy for text-guided protein generation; (2) best hit ratio on 12 zero-shot text-guided protein editing tasks; (3) superior performance on four out of six protein property prediction benchmarks.

Motivation & Objective

• Bridge text descriptions and protein sequences to enable design tasks beyond sequence/structure alone.
• Learn a joint text-protein representation to support zero-shot text-guided generation and editing.
• Develop a robust decoder-based generative pathway conditioned on text-derived representations.
• Evaluate the framework on text-to-protein generation, zero-shot editing, and protein property prediction.
• Quantify benefits over baselines and analyze dataset and evaluation limitations.

Proposed method

• Construct SwissProtCLAP with 441K text–protein pairs to train cross-modal representations.
• Train ProteinCLAP to align text and protein sequence representations via contrastive learning (EBM-NCE/InfoNCE).
• Introduce ProteinFacilitator to map text prompts to protein representations before decoding.
• Employ three decoders (one autoregressive Transformer and two diffusion models) to generate protein sequences conditionally.
• Use two downstream tasks: text-to-protein generation and zero-shot text-guided editing, plus protein property prediction for robustness.

Experimental results

Research questions

• RQ1 Can textual descriptions be effectively aligned with protein sequences to guide design tasks?
• RQ2 How does a text-conditioned encoder-decoder pipeline perform on generation, editing, and property prediction compared to baselines?
• RQ3 What is the impact of the ProteinFacilitator component on text-to-protein generation accuracy?
• RQ4 How well do diffusion-based decoders compare to autoregressive models in this setting?
• RQ5 What are the limitations and evaluation challenges for text-guided protein design?

Key findings

• ProteinDT achieves high retrieval accuracy in text-to-protein generation (over 90% in the reported setup).
• ProteinDT attains best hit ratios across 10 zero-shot text-guided protein editing tasks.
• AR-Transformer with ProteinFacilitator generally outperforms diffusion variants in generation tasks.
• ProteinCLAP-based representations yield superior performance on four of six protein property prediction benchmarks.
• Qualitative results show plausible text-driven edits affecting structure, stability, and peptide binding.