[Paper Review] ProLLM: Protein Chain-of-Thoughts Enhanced LLM for Protein-Protein Interaction Prediction

Mingyu Jin, Haochen Xue+6 more|arXiv (Cornell University)|Mar 30, 2024

Bioinformatics and Genomic Networks4 citations

TL;DR

This paper proposes ProLLM, a large language model framework enhanced with Protein Chain-of-Thought (ProCoT) prompting to predict protein-protein interactions (PPIs) by modeling biological signaling pathways as natural language reasoning chains. ProCoT enables the model to capture indirect, multi-step protein interactions beyond direct physical binding, significantly improving prediction accuracy and generalizability over existing methods on benchmark datasets.

ABSTRACT

The prediction of protein-protein interactions (PPIs) is crucial for understanding biological functions and diseases. Previous machine learning approaches to PPI prediction mainly focus on direct physical interactions, ignoring the broader context of nonphysical connections through intermediate proteins, thus limiting their effectiveness. The emergence of Large Language Models (LLMs) provides a new opportunity for addressing this complex biological challenge. By transforming structured data into natural language prompts, we can map the relationships between proteins into texts. This approach allows LLMs to identify indirect connections between proteins, tracing the path from upstream to downstream. Therefore, we propose a novel framework ProLLM that employs an LLM tailored for PPI for the first time. Specifically, we propose Protein Chain of Thought (ProCoT), which replicates the biological mechanism of signaling pathways as natural language prompts. ProCoT considers a signaling pathway as a protein reasoning process, which starts from upstream proteins and passes through several intermediate proteins to transmit biological signals to downstream proteins. Thus, we can use ProCoT to predict the interaction between upstream proteins and downstream proteins. The training of ProLLM employs the ProCoT format, which enhances the model's understanding of complex biological problems. In addition to ProCoT, this paper also contributes to the exploration of embedding replacement of protein sites in natural language prompts, and instruction fine-tuning in protein knowledge datasets. We demonstrate the efficacy of ProLLM through rigorous validation against benchmark datasets, showing significant improvement over existing methods in terms of prediction accuracy and generalizability. The code is available at: https://github.com/MingyuJ666/ProLLM.

Motivation & Objective

• To address the limitation of existing PPI prediction methods that focus only on direct physical interactions and ignore indirect, pathway-based connections.
• To develop a novel framework that leverages large language models (LLMs) to model complex, sequential protein signaling pathways as natural language reasoning chains.
• To improve PPI prediction performance by integrating protein-specific embeddings and instruction fine-tuning on biological knowledge datasets.
• To evaluate the effectiveness of ProCoT in capturing cumulative, multi-step protein interactions that traditional models miss.
• To demonstrate that LLMs can outperform graph-based and CNN-based models in PPI prediction when properly prompted with biologically meaningful reasoning structures.

Proposed method

• ProCoT (Protein Chain of Thought) is introduced as a natural language prompting format that represents signaling pathways as step-by-step reasoning sequences from upstream to downstream proteins.
• The framework transforms protein interaction data into structured natural language prompts that encode the sequential logic of biological signal transduction.
• Protein-specific embeddings from ProtTrans are integrated into the prompts to preserve sequence-level biological information.
• The model is fine-tuned using instruction tuning on a curated protein knowledge dataset to enhance its ability to reason about PPIs.
• Multiple LLM backbones (Flan-T5-base, Flan-T5-large, Flan-T5-XL, LLaMA-v1-7b) are evaluated to assess performance and stability.
• Ablation studies isolate the contributions of ProCoT, embedding replacement, and instruction fine-tuning to quantify their impact on prediction performance.

Experimental results

Research questions

• RQ1 Can modeling protein signaling pathways as natural language reasoning chains (ProCoT) improve PPI prediction beyond direct interaction detection?
• RQ2 How does the integration of protein-specific embeddings and instruction fine-tuning affect the model’s ability to generalize across diverse PPI datasets?
• RQ3 What is the relative contribution of ProCoT compared to other components like embedding replacement and instruction tuning in enhancing model performance?
• RQ4 Does the use of LLMs with chain-of-thought prompting outperform traditional GNN and CNN-based methods in PPI prediction tasks?
• RQ5 How stable and generalizable is the ProLLM framework across multiple benchmark datasets with varying sizes and biological contexts?

Key findings

• ProLLM with the Flan-T5-large backbone achieved the highest micro-F1 score of 87.12 ± 1.68 on the STRING dataset, outperforming all other models and baselines.
• The ProCoT prompting format alone led to a significant performance boost, with micro-F1 increasing from 78.32 ± 2.65 (without ProCoT) to 91.03 ± 1.63 when all components were used.
• The ablation study confirmed that ProCoT has the most substantial impact on performance, with a 12.7-point improvement in micro-F1 on the Human dataset compared to the baseline without ProCoT.
• Flan-T5-large achieved the best overall performance, outperforming larger models like LLaMA-v1-7b, which showed higher instability and lower accuracy.
• Instruction fine-tuning and embedding replacement contributed positively but had a smaller impact compared to ProCoT, indicating that reasoning structure is the primary driver of performance gains.
• The model demonstrated strong generalizability, achieving high micro-F1 scores across all four benchmark datasets: Human, SHS27K, SHS148K, and STRING.