[Paper Review] Convergence and Stability of Graph Convolutional Networks on Large Random Graphs

Nicolas Keriven, Alberto Bietti+1 more|arXiv (Cornell University)|Jun 2, 2020

Opportunistic and Delay-Tolerant Networks18 citations

TL;DR

This paper establishes non-asymptotic convergence and stability guarantees for Graph Convolutional Networks (GCNs) on large random graphs using a continuous limit framework. By modeling graphs via latent variables and similarity kernels, it proves GCNs converge to their continuous counterparts under relatively sparse conditions (average degree ∼ log n), and introduces deformation-based stability metrics that generalize Euclidean CNN stability to graphs.

ABSTRACT

We study properties of Graph Convolutional Networks (GCNs) by analyzing their behavior on standard models of random graphs, where nodes are represented by random latent variables and edges are drawn according to a similarity kernel. This allows us to overcome the difficulties of dealing with discrete notions such as isomorphisms on very large graphs, by considering instead more natural geometric aspects. We first study the convergence of GCNs to their continuous counterpart as the number of nodes grows. Our results are fully non-asymptotic and are valid for relatively sparse graphs with an average degree that grows logarithmically with the number of nodes. We then analyze the stability of GCNs to small deformations of the random graph model. In contrast to previous studies of stability in discrete settings, our continuous setup allows us to provide more intuitive deformation-based metrics for understanding stability, which have proven useful for explaining the success of convolutional representations on Euclidean domains.

Motivation & Objective

• Address the lack of theoretical understanding for GCN behavior on large, non-isomorphic graphs where discrete isomorphism notions become irrelevant.
• Overcome limitations of prior stability analyses that rely on discrete, unintuitive graph distance metrics.
• Provide non-asymptotic convergence bounds for GCNs on relatively sparse random graphs with average degree growing logarithmically with node count.
• Establish a continuous framework to define intuitive, geometric deformation metrics for graph structures, analogous to those used in Euclidean signal processing.
• Relate discrete graph signal distances to Wasserstein-type metrics in the continuous limit, enabling new theoretical stability insights.

Proposed method

• Model large random graphs using latent space representations where nodes are i.i.d. draws from a distribution and edges are generated via a similarity kernel.
• Define a continuous counterpart to discrete GCNs by replacing the graph Laplacian with an integral operator acting on functions over the latent space.
• Use Schur’s test to bound the operator norms of GCN layers, ensuring stability under perturbations.
• Apply generalized Hoeffding’s inequality and spectral concentration results to control deviations between empirical and expected graph operators.
• Introduce a Wasserstein-type metric between continuous graph signals to quantify stability under small deformations of the underlying graph model.
• Establish recursive Lipschitz bounds on GCN layers by analyzing the propagation of differences through layers using the graph Laplacian operator.

Experimental results

Research questions

• RQ1 How do GCNs converge to a continuous limit as the number of nodes in a random graph grows, especially under sparse conditions?
• RQ2 Can the stability of GCNs to structural perturbations be formalized using continuous, geometric deformation metrics rather than discrete graph distances?
• RQ3 What are the non-asymptotic convergence bounds for GCNs on relatively sparse random graphs with average degree ∼ log n?
• RQ4 How does the continuous limit framework relate discrete graph signal distances to metrics in the continuous representation space?
• RQ5 To what extent can the theoretical stability of GCNs be generalized from Euclidean domains to non-Euclidean graph structures?

Key findings

• The paper establishes non-asymptotic convergence of GCNs to their continuous counterpart under relatively sparse random graph models with average degree growing logarithmically with the number of nodes.
• The convergence results hold without requiring the similarity kernel to be smooth or bounded away from zero, broadening applicability to models like SBMs and ε-graphs.
• Stability to small graph deformations is characterized via a Wasserstein-type metric between continuous graph signal representations, providing a geometric interpretation analogous to CNNs.
• For permutation-equivariant GCNs, the paper relates discrete graph signal distances to Wasserstein distances in the continuous limit, a novel connection not previously observed in the literature.
• Spectral concentration bounds for the normalized Laplacian are used to control the deviation between empirical and expected graph operators, with high probability under mild assumptions on edge probabilities.
• The analysis yields explicit, finite-sample bounds on GCN generalization error and stability, valid even when the graph is not dense, overcoming limitations of prior asymptotic or dense-graph-only analyses.