[Paper Review] GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Models

Jiaxuan You, Rex Ying+3 more|arXiv (Cornell University)|Feb 24, 2018

Graph Theory and AlgorithmsComputer Science318 citations

TL;DR

GraphRNN trains autoregressive graph generators that build graphs node-by-node and edge-by-edge, using BFS-ordered sequences to model complex dependencies and achieve state-of-the-art graph generation performance with scalability to much larger graphs.

ABSTRACT

Modeling and generating graphs is fundamental for studying networks in biology, engineering, and social sciences. However, modeling complex distributions over graphs and then efficiently sampling from these distributions is challenging due to the non-unique, high-dimensional nature of graphs and the complex, non-local dependencies that exist between edges in a given graph. Here we propose GraphRNN, a deep autoregressive model that addresses the above challenges and approximates any distribution of graphs with minimal assumptions about their structure. GraphRNN learns to generate graphs by training on a representative set of graphs and decomposes the graph generation process into a sequence of node and edge formations, conditioned on the graph structure generated so far. In order to quantitatively evaluate the performance of GraphRNN, we introduce a benchmark suite of datasets, baselines and novel evaluation metrics based on Maximum Mean Discrepancy, which measure distances between sets of graphs. Our experiments show that GraphRNN significantly outperforms all baselines, learning to generate diverse graphs that match the structural characteristics of a target set, while also scaling to graphs 50 times larger than previous deep models.

Motivation & Objective

• Learn a generative model for graphs directly from observed graph data without strong hand-crafted assumptions.
• Decompose graph generation into a node-sequence and an edge-sequence process using shared RNNs.
• Introduce a BFS-based ordering to improve tractability and scalability for varying graph sizes.

Proposed method

• Represent graphs as sequences of adjacency vectors S^π generated per node under a BFS ordering.
• Use a graph-level RNN (GRU) to generate nodes and an edge-level RNN to generate the node's adjacency to previous nodes.
• Provide two GraphRNN variants: GraphRNN-S (Multivariate Bernoulli) and the full Dependent Bernoulli sequence with hierarchical RNNs.
• Adopt BFS-based ordering to bound the number of edge predictions per node and achieve near-quadratic time in practice.
• Evaluate using a Maximum Mean Discrepancy (MMD) framework on distributions of degree, clustering, and motif counts.

Experimental results

Research questions

• RQ1 Can a deep autoregressive model learn to generate graphs from data with varying sizes and without fixed node sets?
• RQ2 Does BFS ordering reduce computational complexity and improve scalability and performance for graph generation?
• RQ3 How close are generated graphs to real graphs in higher-order statistics such as degree distributions, clustering, and motifs?
• RQ4 Do the proposed models generalize across diverse graph types (grid-like, community, protein, ego networks)?

Key findings

• GraphRNN substantially outperforms traditional and recent deep graph generative baselines across datasets.
• GraphRNN achieves about 80-90% average reduction in MMD compared with baselines.
• The approach scales to graphs roughly 50x larger than previous deep models.
• GraphRNN-S performs strongly on protein-like nearest-neighbor graphs, where edge dependencies are simpler.
• The model achieves low training-test NLL gaps, indicating good generalization.
• GraphRNN demonstrates robustness when interpolating between Barabási-Albert and Erdős-Rényi graph structures.