[Paper Review] Generating Novel, Designable, and Diverse Protein Structures by Equivariantly Diffusing Oriented Residue Clouds

Yeqing Lin, Mohammed AlQuraishi|arXiv (Cornell University)|Jan 29, 2023

Protein Structure and Dynamics29 citations

TL;DR

Genie uses a denoising diffusion probabilistic model with SE(3)-equivariant reasoning over oriented residue frames to generate designable, novel, and diverse protein backbones.

ABSTRACT

Proteins power a vast array of functional processes in living cells. The capability to create new proteins with designed structures and functions would thus enable the engineering of cellular behavior and development of protein-based therapeutics and materials. Structure-based protein design aims to find structures that are designable (can be realized by a protein sequence), novel (have dissimilar geometry from natural proteins), and diverse (span a wide range of geometries). While advances in protein structure prediction have made it possible to predict structures of novel protein sequences, the combinatorially large space of sequences and structures limits the practicality of search-based methods. Generative models provide a compelling alternative, by implicitly learning the low-dimensional structure of complex data distributions. Here, we leverage recent advances in denoising diffusion probabilistic models and equivariant neural networks to develop Genie, a generative model of protein structures that performs discrete-time diffusion using a cloud of oriented reference frames in 3D space. Through in silico evaluations, we demonstrate that Genie generates protein backbones that are more designable, novel, and diverse than existing models. This indicates that Genie is capturing key aspects of the distribution of protein structure space and facilitates protein design with high success rates. Code for generating new proteins and training new versions of Genie is available at https://github.com/aqlaboratory/genie.

Motivation & Objective

• Motivate de novo protein design beyond folds explored by evolution and known natural proteins.
• Develop a generative model that produces designable, novel, and diverse protein backbones.
• Leverage equivariant neural networks to preserve geometric and chiral constraints during generation.

Proposed method

• Perform discrete-time diffusion in Cartesian space for Cα coordinates with a cosine variance schedule over T=1000 steps.
• Use an SE(3)-equivariant denoiser that reasons over a cloud of reference frames (F frames) built from discrete Frenet-Serret frames to predict noise εθ.
• Encode residues with an SE(3)-invariant encoder and decode with an SE(3)-equivariant decoder that uses IPA and Backbone Update Network to update frames.
• Represent residues as Cα coordinate clouds in the forward process but as a cloud of reference frames in the reverse process to retain angular information and chirality.
• Train by predicting noise at each diffusion step with a loss that minimizes the L2 error between true and predicted noise (based on Ho et al. 2020).
• Sample proteins by starting from random white-noise coordinates xT and reversing the diffusion to x0.

Figure 1: Architecture of SE(3)-equivariant denoiser, including SE(3)-invariant encoder (bottom left) and SE(3)-equivariant decoder (bottom right). Notation: $r$ : number of residues, $c_{s}$ : dimensionality of single representation, $c_{p}$ : dimensionality of pair representation.

Experimental results

Research questions

• RQ1 Can a DDPM operating on a cloud of oriented residue frames generate protein backbones that are more designable, novel, and diverse than existing models?
• RQ2 How does SE(3)-equivariant reasoning with oriented residue clouds impact designability, diversity, and novelty of generated structures?
• RQ3 How do Genie’s short-model (SCOPe-based) and long-model (SwissProt-based) performances compare to other DDPM-based methods in designability, diversity, and novelty?

Key findings

• Genie achieves higher designability, diversity, and novelty than ProtDiff and FoldingDiff on short proteins (128 residues max).
• Among confidently designable structures, Genie shows higher pLDDT scores and greater chirality correctness than competing models.
• Genie exhibits more diverse secondary structure content and lower maximum TM scores to other designed structures, indicating broader fold coverage.
• For long proteins (up to 256 residues), Genie outperforms FrameDiff on sampling quality but trails behind RFDiffusion, with competitive designability and diversity metrics.
• Genie-generated designs include a meaningful fraction of novel folds (TM<0.5 to training set) and demonstrate a wide design space visualization via MDS.
• Across datasets, Genie maintains designability while achieving substantial novelty and broad structural diversity, with 4.1M parameter efficiency and competitive sampling times.

Figure 2: Analysis of structures generated by Genie and short models. (A) Heatmap of the relative frequencies of generated domains with specific combinations of highest scTM and pLDDT values achieved by ProtDiff, FoldingDiff, and Genie. (B) Heatmap of relative frequencies of confidently designable d