[Paper Review] Generative Diffusion Models on Graphs: Methods and Applications

Cheng-Yi Liu, Wenqi Fan+6 more|arXiv (Cornell University)|Feb 6, 2023

Bioinformatics and Genomic Networks8 citations

TL;DR

A comprehensive survey of generative diffusion models on graphs, detailing SMLD, DDPM, and SGM paradigms, their graph adaptations, and applications to molecule and protein modeling.

ABSTRACT

Diffusion models, as a novel generative paradigm, have achieved remarkable success in various image generation tasks such as image inpainting, image-to-text translation, and video generation. Graph generation is a crucial computational task on graphs with numerous real-world applications. It aims to learn the distribution of given graphs and then generate new graphs. Given the great success of diffusion models in image generation, increasing efforts have been made to leverage these techniques to advance graph generation in recent years. In this paper, we first provide a comprehensive overview of generative diffusion models on graphs, In particular, we review representative algorithms for three variants of graph diffusion models, i.e., Score Matching with Langevin Dynamics (SMLD), Denoising Diffusion Probabilistic Model (DDPM), and Score-based Generative Model (SGM). Then, we summarize the major applications of generative diffusion models on graphs with a specific focus on molecule and protein modeling. Finally, we discuss promising directions in generative diffusion models on graph-structured data. For this survey, we also created a GitHub project website by collecting the supporting resources for generative diffusion models on graphs, at the link: https://github.com/ChengyiLIU-cs/Generative-Diffusion-Models-on-Graphs

Motivation & Objective

• Provide a comprehensive overview of generative diffusion methods on graphs (SMLD, DDPM, SGM).
• Summarize representative graph diffusion algorithms and their key techniques.
• Highlight major applications in molecule and protein modeling and discuss future directions.

Proposed method

• Present three diffusion paradigms (SMLD, DDPM, SGM) and their graph adaptations.
• Describe graph-specific forward and reverse processes and score/marginal estimation approaches.
• Discuss discrete vs continuous diffusion, and equivariant/graph-based architectures used for graphs.
• Summarize representative methods like EDP-GNN, ConfGF, Haefeli et al., DiGress, EDMs, GDSS, GSDM, GraphGDP, and SGGM extensions for graphs.

Experimental results

Research questions

• RQ1 What are the main diffusion paradigms used for graphs and how are they adapted to graph structure?
• RQ2 What are the representative graph diffusion algorithms and their core techniques?
• RQ3 What are the key applications of generative diffusion models on graphs in molecules and proteins?
• RQ4 What future directions are promising for diffusion models on graph-structured data?

Key findings

• The paper categorizes graph diffusion methods into SMLD, DDPM, and SGM and summarizes representative algorithms for each.
• Graph diffusion methods address discrete structures, complex dependencies, and permutation invariance through specialized forward/reverse processes and graph neural architectures.
• Applications are centered on molecule modeling (conformation generation, docking) and protein modeling, with numerous architecture variants (equivariant, graph transformers, energy guidance).
• There exists a GitHub resource compiling supporting materials for diffusion models on graphs.
• Several advances integrate domain knowledge (energy functions, interatomic forces, equivariance) and autoregressive versus one-shot generation strategies.