[Paper Review] DiffPack: A Torsional Diffusion Model for Autoregressive Protein Side-Chain Packing

Yangtian Zhan, Zuobai Zhang+3 more|arXiv (Cornell University)|Jun 1, 2023

Protein purification and stability16 citations

TL;DR

DiffPack introduces an autoregressive torsional diffusion model to predict protein side-chain conformations by modeling the joint distribution of the four side-chain torsion angles, achieving state-of-the-art angle accuracy with far fewer parameters and improving AlphaFold2 side-chain predictions.

ABSTRACT

Proteins play a critical role in carrying out biological functions, and their 3D structures are essential in determining their functions. Accurately predicting the conformation of protein side-chains given their backbones is important for applications in protein structure prediction, design and protein-protein interactions. Traditional methods are computationally intensive and have limited accuracy, while existing machine learning methods treat the problem as a regression task and overlook the restrictions imposed by the constant covalent bond lengths and angles. In this work, we present DiffPack, a torsional diffusion model that learns the joint distribution of side-chain torsional angles, the only degrees of freedom in side-chain packing, by diffusing and denoising on the torsional space. To avoid issues arising from simultaneous perturbation of all four torsional angles, we propose autoregressively generating the four torsional angles from $χ_1$ to $χ_4$ and training diffusion models for each torsional angle. We evaluate the method on several benchmarks for protein side-chain packing and show that our method achieves improvements of $11.9\%$ and $13.5\%$ in angle accuracy on CASP13 and CASP14, respectively, with a significantly smaller model size ($60 imes$ fewer parameters). Additionally, we show the effectiveness of our method in enhancing side-chain predictions in the AlphaFold2 model. Code is available at https://github.com/DeepGraphLearning/DiffPack.

Motivation & Objective

• Motivate and address the challenge of predicting side-chain conformations given fixed backbone structures.
• Model the exact degrees of freedom in side-chain packing as torsional angles rather than Cartesian coordinates.
• Propose an autoregressive diffusion framework to generate side-chain torsions χ1–χ4 with SE(3)-invariant guidance.
• Evaluate on CASP13/CASP14 benchmarks and non-native backbones to demonstrate accuracy and efficiency.

Proposed method

• Model side-chain conformation in torsion space, reducing variables to the four torsional angles χ1–χ4 (with residue-specific counts).
• Apply a forward Variance-Exploding SDE in torsion space and learn score functions to denoise in the torsion domain.
• Use an autoregressive factorization p(χ1,χ2,χ3,χ4)=p(χ1)·p(χ2|χ1)·p(χ3|χ1,χ2)·p(χ4|χ1,χ2,χ3) with separate diffusion models for each χi.
• Employ an SE(3)-invariant score network based on GearNet-Edge to predict torsion angles from backbone and partially generated side-chains.
• Adopt teacher-forcing during training to isolate perturbations to the target torsion angles and avoid cumulative distortions.
• Enhance sampling with multi-round denoising, annealed temperature sampling, and a confidence model to select high-quality conformations.
• Handle π-rotation symmetry by adjusting the diffusion kernel to account for equivalent torsion states.

Figure 1 : Overview of DiffPack. Given a protein sequence and backbone structure, we aim to model the conditional distribution of side-chain conformation. (A) Distribution of side-chain conformation is modeled through diffusion process in torsion space $\mathbb{T}^{m}$ . An SE(3)-invariant network i

Experimental results

Research questions

• RQ1 Can modeling side-chain packing in torsion space better respect covalent bond constraints than Cartesian-coordinate approaches?
• RQ2 Does autoregressively diffusing χ1–χ4 improve generation stability and accuracy over joint diffusion on all four angles?
• RQ3 How does DiffPack perform on CASP13/14 and on non-native backbones (e.g., AlphaFold2) compared to state-of-the-art methods?
• RQ4 Is the DiffPack model size significantly smaller while preserving or improving accuracy?
• RQ5 Do sampling enhancements (multi-round, annealing, confidence model) improve inference quality?

Key findings

• DiffPack achieves 11.9% and 13.5% improvements in angle accuracy on CASP13 and CASP14, respectively.
• The model uses ~60x fewer parameters than the prior state-of-the-art AttnPacker.
• DiffPack attains the best Angle MAE and Angle Accuracy across χ1–χ4 and all/respective residue categories on CASP13 and CASP14.
• On AlphaFold2 non-native backbones, DiffPack consistently outperforms AlphaFold2 and other baselines in most metrics.
• Autoregressive diffusion (χ1→χ4) outperforms joint diffusion and random diffusion baselines in training stability and final quality.
• The approach preserves chemical validity (bond lengths/angles) and can reveal interactions like π–π stacking without extra priors.

Figure 2 : Illustration of four torsional angles.