[Paper Review] Protein structure generation via folding diffusion

Kevin Wu, Kevin Yang+4 more|arXiv (Cornell University)|Sep 30, 2022

Protein Structure and DynamicsBiochemistry, Genetics and Molecular Biology34 citations

TL;DR

FoldingDiff is a diffusion-based model that generates protein backbone structures by denoising six internal angles per residue, avoiding Cartesian coordinates and equivariant networks, yielding realistic and designable backbones.

ABSTRACT

The ability to computationally generate novel yet physically foldable protein structures could lead to new biological discoveries and new treatments targeting yet incurable diseases. Despite recent advances in protein structure prediction, directly generating diverse, novel protein structures from neural networks remains difficult. In this work, we present a new diffusion-based generative model that designs protein backbone structures via a procedure that mirrors the native folding process. We describe protein backbone structure as a series of consecutive angles capturing the relative orientation of the constituent amino acid residues, and generate new structures by denoising from a random, unfolded state towards a stable folded structure. Not only does this mirror how proteins biologically twist into energetically favorable conformations, the inherent shift and rotational invariance of this representation crucially alleviates the need for complex equivariant networks. We train a denoising diffusion probabilistic model with a simple transformer backbone and demonstrate that our resulting model unconditionally generates highly realistic protein structures with complexity and structural patterns akin to those of naturally-occurring proteins. As a useful resource, we release the first open-source codebase and trained models for protein structure diffusion.

Motivation & Objective

• Motivate computational generation of novel, foldable protein backbones.
• Introduce a biologically inspired angular representation to simplify modeling.
• Demonstrate unconditional generation of realistic backbone structures.
• Showcase designability of generated structures using inverse folding and structure prediction tools.
• Release open-source code and trained models for protein folding diffusion.

Proposed method

• Represent each residue with six consecutive backbone angles (phi, psi, omega, theta1, theta2, theta3) to specify the next residue in a chain.
• Use a denoising diffusion probabilistic model (diffusion model) with a vanilla transformer to model pξ(xt−1|xt) over angular data.
• Forward process uses a wrapped Gaussian noise for angular data; reverse process denoises from noise to structured angles.
• Wrap angular outputs to [-π, π) and apply a wrapped smooth L1 loss to handle periodicity.
• Train on the CATH de-duplicated backbone dataset with an 80/10/10 split and 128-residue windowing; use cosine variance schedule with T=1000 and AdamW optimizer.
• Provide a sampling procedure (FoldingDiff) that iteratively refines angles from unfolded to folded states.

Experimental results

Research questions

• RQ1 Can unconditional diffusion generate realistic distributions of protein backbone angles?
• RQ2 Do generated angle configurations correspond to natural secondary structure motifs (e.g., helices, sheets) in Ramachandran space?
• RQ3 Are the generated backbones designable, i.e., can they be realized by sequences via inverse folding and structure prediction?
• RQ4 What is the impact of using an internal-angle representation on chirality and equivariance requirements in generation?

Key findings

• Generated angle distributions closely match the test-set distributions across all six angles.
• Ramachandran analysis shows generated backbones recapitulate all three major regions (right-handed helices, left-handed helices, beta sheets) and respect chirality.
• 177 of 780 generated backbones are designable (scTM ≥ 0.5) using ProteinMPNN and OmegaFold (22.7%).
• Designability varies with length; shorter structures (≈70 residues) tend to have higher designability.
• Structures more similar to training data tend to be more designable (higher TM-score to training set).
• The generated backbone diversity is high and not limited to near-memorized exemplars, suggesting broad structural variety.