[论文解读] ProteinGPT: Multimodal LLM for Protein Property Prediction and Structure Understanding

Yijia Xiao, Edward W. Sun+3 more|arXiv (Cornell University)|Aug 21, 2024

Machine Learning in Bioinformatics被引用 6

一句话总结

ProteinGPT 是一个多模态蛋白质对话系统，将蛋白质序列和结构融入到一个大型语言模型（LLM）中，以实现交互式提问和简明的问答，采用在大型 ProteinQA 数据集上由 GPT-4o 指导的指令微调方法进行训练。

ABSTRACT

Understanding biological processes, drug development, and biotechnological advancements requires a detailed analysis of protein structures and functions, a task that is inherently complex and time-consuming in traditional protein research. To streamline this process, we introduce ProteinGPT, a state-of-the-art multimodal large language model for proteins that enables users to upload protein sequences and/or structures for comprehensive analysis and responsive inquiries. ProteinGPT integrates protein sequence and structure encoders with linear projection layers to ensure precise representation adaptation and leverages a large language model (LLM) to generate accurate, contextually relevant responses. To train ProteinGPT, we constructed a large-scale dataset of 132,092 proteins, each annotated with 20-30 property tags and 5-10 QA pairs per protein, and optimized the instruction-tuning process using GPT-4o. Experiments demonstrate that ProteinGPT effectively generates informative responses to protein-related questions, achieving high performance on both semantic and lexical metrics and significantly outperforming baseline models and general-purpose LLMs in understanding and responding to protein-related queries. Our code and data are available at https://github.com/ProteinGPT/ProteinGPT.

研究动机与目标

• 通过将序列和结构数据与自然语言交互结合，推动更快速且更准确的蛋白质分析。
• 开发一个多模态管线，将蛋白质序列与结构嵌入映射到对 LLM 友好的表示。
• 创建一个大规模、丰富注释的指令微调数据集（ProteinQA），以教会模型回答简明且相关的蛋白质问题。

提出的方法

• 两个冻结的编码器（序列：esm2_t36_3B_UR50D；结构：esm_if1_gvp4_t16_142M_UR50）生成嵌入。
• 一个线性投影层将两者嵌入与 LLM 嵌入对齐，以形成软提示。
• 两阶段训练：（1）通过投影层进行模态对齐；（2）使用 GPT-4o 指导的问答数据集进行指令微调。
• ProteinQA 提供 132,092 个蛋白质样本，包含摘要、20–30 个性质标签，以及每个样本 5–10 条问答对。
• 用 GPT-4o 对问答进行扩充，创建用于指令微调的显式 Question-Answer 对。
• 评估包括对多种基础 LLM 的语义指标（BERTScore、PubMedBERT-Score、GPT Score）和词汇指标（ROUGE）。

Figure 1: ProteinGPT Modality Fusion & Alignment Stage: we freeze the encoder blocks and train the linear project layer to learn how to align protein structure and protein sequence representations with text. In the alignment stage, the input to the training is only the projected protein representati

实验结果

研究问题

• RQ1 多模态融合蛋白质序列与结构是否能提升基于 LLM 的蛋白质理解与问答准确性？
• RQ2 模态对齐后再进行指令微调对蛋白质问题的语义和词汇问答性能有何影响？
• RQ3 ProteinGPT 是否在蛋白质相关的查询上优于普通 LLM 和基于 GPT 的基线模型？

主要发现

• ProteinGPT 在评估中获得较高的语义相似度分数，但由于输出为句子形式，词汇重叠较低。
• 在测试的模型中，Llama-3 和 Mistral 主干架构对 ProteinGPT 的性能贡献最好。
• ProteinGPT 在所测试任务的语义评估上始终优于普通 LLM 基线和 OpenAI GPT 模型。
• 某些主干下，闭合式问答准确率约为 70–80%，表明对单字正确答案的事实性回应较为可靠。
• 模态对齐相比于普通 LLM 提高了性能，且指令微调在仅模态对齐基础上带来额外收益。

Figure 2: ProteinGPT Instruction Tuning Stage: we utilize the QA pairs and property tags in ProteinQA to tune the LLM to follow instructions and give concise responses. For instruction alignment, explicit prompts ( Questions on the protein) are included at the beginning of the prompt.