[Paper Review] Diffusion Improves Graph Learning

Johannes Gasteiger, Stefan Weißenberger+1 more|arXiv (Cornell University)|Oct 28, 2019

Advanced Graph Neural NetworksComputer Science175 citations

TL;DR

Graph diffusion convolution (GDC) introduces a sparsified generalized graph diffusion to create a richer, localized neighborhood for learning, yielding consistent improvements across various GNNs and graph models. It bridges spatial and spectral approaches and is plug-and-play with minimal overhead.

ABSTRACT

Graph convolution is the core of most Graph Neural Networks (GNNs) and usually approximated by message passing between direct (one-hop) neighbors. In this work, we remove the restriction of using only the direct neighbors by introducing a powerful, yet spatially localized graph convolution: Graph diffusion convolution (GDC). GDC leverages generalized graph diffusion, examples of which are the heat kernel and personalized PageRank. It alleviates the problem of noisy and often arbitrarily defined edges in real graphs. We show that GDC is closely related to spectral-based models and thus combines the strengths of both spatial (message passing) and spectral methods. We demonstrate that replacing message passing with graph diffusion convolution consistently leads to significant performance improvements across a wide range of models on both supervised and unsupervised tasks and a variety of datasets. Furthermore, GDC is not limited to GNNs but can trivially be combined with any graph-based model or algorithm (e.g. spectral clustering) without requiring any changes to the latter or affecting its computational complexity. Our implementation is available online.

Motivation & Objective

• Reconcile the strengths of spatial (message passing) and spectral graph methods by extending aggregation beyond 1-hop neighborhoods.
• Introduce and formalize Graph Diffusion Convolution (GDC) as a sparsified diffusion-based graph transformation.
• Demonstrate that GDC consistently improves performance across supervised and unsupervised tasks on diverse datasets.
• Analyze the spectral properties of GDC and its relation to polynomial filters and traditional diffusion processes.
• Show that GDC is plug-and-play and compatible with any graph-based model or algorithm without changing computational complexity.

Proposed method

• Define generalized graph diffusion as S = sum_{k=0}^{∞} θ_k T^k with constraints ensuring convergence.
• Use transition matrices T (e.g., random walk, symmetric with self-loops) to model diffusion.
• Sparsify the resulting diffusion matrix to obtain a sparse graph Ŝ (top-k or ε-threshold) and compute the corresponding transition matrix on Ŝ.
• Apply standard graph-based models on the sparsified diffusion graph Ŝ (or its symmetric version) to obtain new embeddings or predictions.
• Relate GDC to spectral filters by showing the equivalence between diffusion-based and polynomial filters via ξ_j coefficients (Eq. 4) and demonstrate special cases like PPR and heat kernel (Eq. 5).
• Demonstrate that the diffusion step preserves locality, enables linear-time approximations, and is compatible with non-GNN models (e.g., spectral clustering).

Experimental results

Research questions

• RQ1 Does graph diffusion convolution (GDC) consistently improve performance across a wide range of models and datasets?
• RQ2 How does sparsification of the diffusion matrix affect spectral properties and learning outcomes?
• RQ3 What choices of diffusion coefficients θ_k and transition matrices T yield robust, dataset-agnostic gains?
• RQ4 Can GDC be effectively integrated with non-GNN graph-based methods without changing their complexity?
• RQ5 How do diffusion-based smoothing and increased neighborhood range influence label efficiency and generalization?

Key findings

• GDC consistently improves accuracy across multiple models (GCN, GAT, JK, GIN, ARMA) and datasets (Cora, Citeseer, PubMed, Coauthor CS, Amazon Computers, Amazon Photo).
• Sparsification of the diffusion matrix not only reduces computation but often improves prediction accuracy (top-k and ε-threshold methods).
• PPR and heat kernel-based θ_k coefficients yield strong, robust performance across datasets and models, with simple hyperparameters in narrow stable ranges.
• GDC expands the effective receptive field, allowing distant nodes to influence learning and improving performance even with low label rates.
• GDC can fix models that would otherwise degrade on certain datasets (e.g., GAT), and performance gains are observed in both supervised and unsupervised settings, including clustering.