[Paper Review] PepTune: De Novo Generation of Therapeutic Peptides with Multi-Objective-Guided Discrete Diffusion

Shaoqing Tang, Yinuo Zhang+1 more|arXiv (Cornell University)|Dec 23, 2024

Protein purification and stability6 citations

TL;DR

PepTune introduces a multi-objective, discrete diffusion framework with Monte Carlo Tree Search guidance to generate peptide SMILES that are valid, diverse, and optimized for multiple therapeutic properties including binding, permeability, and solubility.

ABSTRACT

We present PepTune, a multi-objective discrete diffusion model for simultaneous generation and optimization of therapeutic peptide SMILES. Built on the Masked Discrete Language Model (MDLM) framework, PepTune ensures valid peptide structures with a novel bond-dependent masking schedule and invalid loss function. To guide the diffusion process, we introduce Monte Carlo Tree Guidance (MCTG), an inference-time multi-objective guidance algorithm that balances exploration and exploitation to iteratively refine Pareto-optimal sequences. MCTG integrates classifier-based rewards with search-tree expansion, overcoming gradient estimation challenges and data sparsity. Using PepTune, we generate diverse, chemically-modified peptides simultaneously optimized for multiple therapeutic properties, including target binding affinity, membrane permeability, solubility, hemolysis, and non-fouling for various disease-relevant targets. In total, our results demonstrate that MCTG for masked discrete diffusion is a powerful and modular approach for multi-objective sequence design in discrete state spaces.

Motivation & Objective

• Motivate the challenge of designing peptides that satisfy multiple conflicting therapeutic objectives.
• Develop a discrete diffusion-based generative model for peptide SMILES that can include non-natural amino acids and cyclic structures.
• Introduce a multi-objective guidance framework based on Monte Carlo Tree Search to optimize for multiple properties.
• Provide property prediction models for peptide SMILES to support objective guidance.
• Demonstrate case studies showing improved binding and permeability for target proteins.

Proposed method

• Build PepMDLM, a masked diffusion language model using a RoFormer backbone to generate peptide SMILES with bond-dependent masking to enforce peptide bonds.
• Introduce a bond-dependent masking schedule that prioritizes unmasking peptide bonds earlier in generation.
• Add a global invalid SMILES loss to penalize invalid peptide structures and scale penalties by token probability.
• Use Monte Carlo Tree Search with Gumbel-based unmasking to explore and expand sequences, producing Pareto-optimal peptides across multiple objectives.
• Train regression and classification models on peptide SMILES to predict binding affinity and membrane permeability, and classifiers for solubility, hemolysis, and non-fouling.
• Conduct case studies on targets including TfR, GLP-1R, GFAP, NCAM1, and AMHR2 to demonstrate multi-objective peptide design and docking validation.

Experimental results

Research questions

• RQ1 Can a discrete diffusion model generate valid, chemically modified peptide SMILES that include non-natural amino acids and cyclic structures?
• RQ2 Can multi-objective guidance via MCTS produce Pareto-optimal peptide sequences balancing binding affinity, permeability, solubility, non-fouling, and hemolysis?
• RQ3 Are peptide SMILES-based property predictors accurate enough to guide generation for therapeutic targets?
• RQ4 Do PepTune-generated peptides demonstrate competitive docking and binding to clinically relevant targets compared with known binders?
• RQ5 Is the approach generalizable across diverse targets including membrane-bound receptors and intracellular proteins?

Key findings

• PepMDLM achieves a valid peptide generation rate of 45% at length ~15 amino acids and 36% at length ~30 amino acids after filtering.
• PepTune reaches 100% validity in MCTS-guided generation within 20 iterations and maintains diversity and uniqueness comparable to the unconditional model.
• PepTune-conditioned peptides show higher predicted binding affinity distributions than unconditional samples and dock to targets with competitive scores (e.g., TfR, GLP-1R).
• For TfR, PepTune yielded docked binders with scores below -6.0 kcal/mol, with top binders at -8.4 kcal/mol, rivaling known binder T7.
• GLP-1R-conditioned peptides achieved docking scores of -7.4 and -7.0 kcal/mol, surpassing some existing GLP-1R agonists in docking context.
• GFAP intracellular targets were addressed with binders showing docking scores below -7 kcal/mol, supporting potential intracellular engagement.