[Paper Review] Permutation Invariant Graph Generation via Score-Based Generative Modeling

Chenhao Niu, Yang Song+4 more|arXiv (Cornell University)|Mar 1, 2020

Advanced Graph Neural Networks30 citations

TL;DR

The paper presents a permutation-invariant graph generation approach based on score-based generative modeling, using a permutation-equivariant EDP-GNN to model score functions and sampling via annealed Langevin dynamics; it achieves competitive generation quality and excels in edgewise predictions.

ABSTRACT

Learning generative models for graph-structured data is challenging because graphs are discrete, combinatorial, and the underlying data distribution is invariant to the ordering of nodes. However, most of the existing generative models for graphs are not invariant to the chosen ordering, which might lead to an undesirable bias in the learned distribution. To address this difficulty, we propose a permutation invariant approach to modeling graphs, using the recent framework of score-based generative modeling. In particular, we design a permutation equivariant, multi-channel graph neural network to model the gradient of the data distribution at the input graph (a.k.a., the score function). This permutation equivariant model of gradients implicitly defines a permutation invariant distribution for graphs. We train this graph neural network with score matching and sample from it with annealed Langevin dynamics. In our experiments, we first demonstrate the capacity of this new architecture in learning discrete graph algorithms. For graph generation, we find that our learning approach achieves better or comparable results to existing models on benchmark datasets.

Motivation & Objective

• Address permutation invariance in graph generative modeling.
• Introduce a permutation-equivariant score network for graphs.
• Leverage annealed Langevin dynamics to sample graphs from the learned score.
• Propose the EDP-GNN architecture with multi-channel adjacency matrices.
• Demonstrate competitive generation quality on benchmark datasets and strong edgewise prediction results.

Proposed method

• Model the gradient of the data distribution (the score) with a permutation-equivariant graph neural network (EDP-GNN).
• Train the score model using score matching with Gaussian perturbations of adjacency matrices.
• Sample graphs via annealed Langevin dynamics with noise level conditioning.
• Handle discrete graph data by perturbing adjacency matrices and quantizing the final samples.
• Ensure permutation invariance of the learned graph distribution through the equivariant score and its line integral representation.

Experimental results

Research questions

• RQ1 Can a permutation-equivariant score function lead to a permutation-invariant distribution over graphs?
• RQ2 How effective is the proposed EDP-GNN at learning edgewise features and graph distributions compared to existing graph generative models?
• RQ3 Does annealed Langevin sampling of a learned score yield graph samples of competitive quality on benchmark datasets?

Key findings

• EDP-GNN with multi-channel adjacency matrices improves edgewise prediction over vanilla GNNs, especially on weighted graphs.
• The score-based generative approach yields graph samples with generation quality comparable to GraphRNN and GraphNVP on benchmark metrics.
• The model can learn to map perturbed graphs to denoised structures, revealing interpretable intermediate channel representations.
• On graph generation tasks, the method achieves competitive MMD metrics across multiple statistics and datasets.
• Permutation equivariance of the score network induces permutation invariance in the implied graph distribution.