[Paper Review] ProtSolM: Protein Solubility Prediction with Multi-modal Features

Yang Tan, Zheng Jia+2 more|arXiv (Cornell University)|Jun 28, 2024

Protein purification and stability5 citations

TL;DR

ProtSolM is a deep learning framework that integrates protein sequence, structure, and physicochemical properties, pre-trains on large multi-modal data, and fine-tunes on the PDBSol dataset to predict protein solubility with state-of-the-art performance.

ABSTRACT

Understanding protein solubility is essential for their functional applications. Computational methods for predicting protein solubility are crucial for reducing experimental costs and enhancing the efficiency and success rates of protein engineering. Existing methods either construct a supervised learning scheme on small-scale datasets with manually processed physicochemical properties, or blindly apply pre-trained protein language models to extract amino acid interaction information. The scale and quality of available training datasets leave significant room for improvement in terms of accuracy and generalization. To address these research gaps, we propose \sol, a novel deep learning method that combines pre-training and fine-tuning schemes for protein solubility prediction. ProtSolM integrates information from multiple dimensions, including physicochemical properties, amino acid sequences, and protein backbone structures. Our model is trained using \data, the largest solubility dataset that we have constructed. PDBSol includes over $60,000$ protein sequences and structures. We provide a comprehensive leaderboard of existing statistical learning and deep learning methods on independent datasets with computational and experimental labels. ProtSolM achieved state-of-the-art performance across various evaluation metrics, demonstrating its potential to significantly advance the accuracy of protein solubility prediction.

Motivation & Objective

• Motivate the need for accurate protein solubility prediction to aid protein engineering and reduce experimental costs.
• Introduce ProtSolM, a multi-modal deep learning model combining sequence, structure, and physicochemical features.
• Create and utilize PDBSol, a large solubility dataset, for training and evaluation.
• Demonstrate state-of-the-art performance against existing methods on independent datasets.

Proposed method

• Two-module architecture: an AA-level encoding module pre-trained with self-supervised learning to capture sequence and structure embeddings using ESM2 and roto-translation equivariant GNNs.
• Fine-tuning module that combines AA-level representations with protein-level hand-crafted physicochemical features, using attention pooling and a weighted residual connection with a pLDDT penalty.
• Protein-level representation is formed by attention pooling over AA embeddings, concatenated with 42 hand-crafted features, and fed through fully connected read-out layers for binary solubility prediction.
• Pre-training task: denoising AA types with multinomial noise to learn structure-aware evolutionary embeddings.
• Fine-tuning data: PDBSol, the largest solubility dataset to date (over 60k proteins with sequences, structures, and labels).
• Model variants tested include ProtSolM with different backbone configurations (k10/k20/k30 and hidden size h512).

Experimental results

Research questions

• RQ1 Can integrating sequence, structure, and physicochemical properties improve protein solubility prediction beyond sequence-only or structure-only models?
• RQ2 Does pre-training on large multi-modal protein data improve generalization to independent solubility benchmarks?
• RQ3 What is the contribution of each component (pLDDT penalty, attention pooling, hand-crafted features) to prediction performance?
• RQ4 How does ProtSolM perform relative to existing supervised and fine-tuned pre-trained models on standard and external solubility benchmarks?

Key findings

• ProtSolM achieves state-of-the-art performance on standard and external solubility benchmarks across multiple metrics.
• ProtSolM variants (k10/k20/k30 with h512) significantly outperform baseline models on accuracy, precision, recall, AUC, and MCC on standard test data.
• ProtSolM maintains strong performance on external test datasets, though gains are smaller than on the standard test set.
• Ablation studies show that the attention pooling and the pLDDT penalty, along with hand-crafted protein-level features, contribute meaningfully to performance; removing any degrades results.
• t-SNE visualization indicates separability of soluble vs insoluble samples in ProtSolM’s learned representation, supporting discriminative embedding.
• ProtSolM outperforms both traditional supervised models and fine-tuned self-supervised language models on the standard test set.