[Paper Review] Peptide-GPT: Generative Design of Peptides using Generative Pre-trained Transformers and Bio-informatic Supervision

Aayush Shah, Chakradhar Guntuboina+1 more|arXiv (Cornell University)|Oct 25, 2024

Machine Learning in Bioinformatics4 citations

TL;DR

Peptide-GPT introduces a fine-tuned ProtGPT2 model for de novo generation of peptides with specific biological properties—hemolytic, non-hemolytic, non-fouling, and soluble—using bio-informatic filtering and deep learning classifiers. It achieves 76.26% accuracy in hemolytic, 72.46% in non-hemolytic, 78.84% in non-fouling, and 68.06% in solubility prediction, demonstrating the effectiveness of NLP-based protein design with structural and functional validation.

ABSTRACT

In recent years, natural language processing (NLP) models have demonstrated remarkable capabilities in various domains beyond traditional text generation. In this work, we introduce PeptideGPT, a protein language model tailored to generate protein sequences with distinct properties: hemolytic activity, solubility, and non-fouling characteristics. To facilitate a rigorous evaluation of these generated sequences, we established a comprehensive evaluation pipeline consisting of ideas from bioinformatics to retain valid proteins with ordered structures. First, we rank the generated sequences based on their perplexity scores, then we filter out those lying outside the permissible convex hull of proteins. Finally, we predict the structure using ESMFold and select the proteins with pLDDT values greater than 70 to ensure ordered structure. The properties of generated sequences are evaluated using task-specific classifiers - PeptideBERT and HAPPENN. We achieved an accuracy of 76.26% in hemolytic, 72.46% in non-hemolytic, 78.84% in non-fouling, and 68.06% in solubility protein generation. Our experimental results demonstrate the effectiveness of PeptideGPT in de novo protein design and underscore the potential of leveraging NLP-based approaches for paving the way for future innovations and breakthroughs in synthetic biology and bioinformatics. Codes, models, and data used in this study are freely available at: https://github.com/aayush-shah14/PeptideGPT.

Motivation & Objective

• To develop a generative AI model for designing peptides with specific functional properties such as hemolytic activity, solubility, and non-fouling behavior.
• To integrate bio-informatic validation steps—perplexity ranking, convex hull filtering, and pLDDT-based structure prediction—to ensure structural viability of generated sequences.
• To evaluate the performance of generated peptides using task-specific classifiers (PeptideBERT and HAPPENN) to assess functional accuracy.
• To enable the discovery of novel peptide sequences that lie on the periphery of known protein space, particularly for non-fouling proteins.
• To provide open-source access to models, code, and data for reproducibility and future development in synthetic biology and bioengineering.

Proposed method

• Fine-tune ProtGPT2 on four distinct datasets of experimentally verified peptides: hemolytic, non-hemolytic, non-fouling, and soluble proteins.
• Apply perplexity scoring to rank generated sequences by likelihood and coherence.
• Filter sequences using the convex hull of known valid protein sequences to exclude non-biological or unstable sequences.
• Predict 3D structure using ESMFold and retain only those with pLDDT > 70 to ensure ordered, stable folding.
• Classify final sequences using PeptideBERT and HAPPENN to evaluate functional properties with task-specific accuracy.
• Use sampling parameters (max length, top-k, repetition penalty) to generate 5,000 sequences per property across five random seeds.

Experimental results

Research questions

• RQ1 Can a fine-tuned generative pre-trained transformer model effectively generate novel peptide sequences with desired functional properties such as hemolytic activity or solubility?
• RQ2 How effective is the integration of bio-informatic filtering (perplexity, convex hull, pLDDT) in ensuring structural validity and biological plausibility of generated peptides?
• RQ3 To what extent do generated peptides align with the distribution of known functional proteins in latent space, particularly for non-fouling sequences?
• RQ4 Can the model generate peptides that lie on the periphery of known protein space, enabling discovery of novel functional motifs?
• RQ5 What is the functional accuracy of the generated peptides when evaluated by specialized classifiers like PeptideBERT and HAPPENN?

Key findings

• Peptide-GPT achieved 76.26% accuracy in identifying hemolytic peptides, demonstrating strong performance in generating sequences with targeted lytic activity.
• The model achieved 72.46% accuracy in generating non-hemolytic peptides, indicating effective suppression of hemolytic properties.
• Non-fouling peptide generation reached the highest accuracy at 78.84%, suggesting strong capture of surface properties that resist non-specific binding.
• Soluble peptide generation achieved 68.06% accuracy, reflecting the model’s capability in generating sequences with favorable solubility characteristics.
• The non-hemolytic task had the highest structural stability, with 44.46% of valid sequences achieving pLDDT > 70, indicating high confidence in folded structure.
• t-SNE visualization showed that generated non-fouling sequences lie on the boundary of known protein space, suggesting exploration of novel, functional regions.