[Paper Review] Diffusion probabilistic modeling of protein backbones in 3D for the motif-scaffolding problem

Brian L. Trippe, Jason Yim+5 more|arXiv (Cornell University)|Jun 8, 2022

Protein Structure and Dynamics96 citations

TL;DR

The paper introduces ProtDiff, a 3D diffusion model for protein backbones, and SMCDiff, a sequential Monte Carlo method for conditional sampling to scaffold motifs. It can generate diverse, longer scaffolds (up to 80 residues) that align with AlphaFold2 predictions.

ABSTRACT

Construction of a scaffold structure that supports a desired motif, conferring protein function, shows promise for the design of vaccines and enzymes. But a general solution to this motif-scaffolding problem remains open. Current machine-learning techniques for scaffold design are either limited to unrealistically small scaffolds (up to length 20) or struggle to produce multiple diverse scaffolds. We propose to learn a distribution over diverse and longer protein backbone structures via an E(3)-equivariant graph neural network. We develop SMCDiff to efficiently sample scaffolds from this distribution conditioned on a given motif; our algorithm is the first to theoretically guarantee conditional samples from a diffusion model in the large-compute limit. We evaluate our designed backbones by how well they align with AlphaFold2-predicted structures. We show that our method can (1) sample scaffolds up to 80 residues and (2) achieve structurally diverse scaffolds for a fixed motif.

Motivation & Objective

• Motivate motif-scaffolding as a scalable, diverse scaffold design problem.
• Develop a 3D diffusion model (ProtDiff) for protein backbones that respects geometric invariances.
• Create a conditional sampling method (SMCDiff) to scaffold motifs by inpainting unconditionally trained diffusion models.
• Demonstrate that scaffolds up to 80 residues can be generated and are consistent with AlphaFold2 predictions.

Proposed method

• ProtDiff: an E(3)-equivariant graph neural network diffusion model over 3D protein backbones.
• Fully connected graph representation with sequence-ordered nodes and sinusoidal positional encodings.
• Noise prediction via epsilon_theta, implemented as an EGNN with translation/rotation equivariance.
• SMCDiff: a sequential Monte Carlo method for exact conditional sampling from an unconditional diffusion model, enabling motif inpainting.
• Theoretical guarantee: if the diffusion model matches the data, SMCDiff provides exact conditional samples in the large-compute limit.

Experimental results

Research questions

• RQ1 Can a diffusion model learn a distribution over realistic 3D protein backbones extendable to longer scaffolds around a given motif?
• RQ2 Can SMCDiff produce diverse and accurate motif-scaffolding scaffolds conditioned on a motif, with provable correctness in the large-compute limit?
• RQ3 How well do generated backbones align with AlphaFold2-predicted structures and support designable motifs?
• RQ4 What are the trade-offs between scaffold length, designability, and computational cost for conditional motif scaffolding?

Key findings

• ProtDiff can sample backbone structures up to 80 residues around a motif.
• SMCDiff enables diverse scaffold generation around a fixed motif and attains motif RMSD below 1 Å for 80-residue scaffolds in at least the 5trv case.
• Backbone samples achieve designable alignment with AlphaFold2 predictions, with scTM > 0.5 indicating designability in tested cases.
• Unconditionally sampled backbones show diversity and some designability (11.8% with scTM > 0.5 across 50–128 residues), but left-handed helices are common (45% have a left-handed helix).
• SMCDiff provides asymptotically exact conditional samples given an accurate diffusion model and sufficient compute (number of particles).
• Conditional scaffolding with 64 particles takes about 2 minutes per sample, competitive with other inpainting approaches.