[Paper Review] Expander Graph Propagation

Andreea Deac, Marc Lackenby+1 more|arXiv (Cornell University)|Oct 6, 2022

Advanced Graph Neural Networks4 citations

TL;DR

This paper proposes Expander Graph Propagation (EGP), a novel GNN method that leverages efficiently constructible expander graphs—specifically Cayley graphs of SL(2,ℤₙ)—to enable global, bottleneck-free message passing with linear time and space complexity. Despite using negatively curved edges, EGP avoids oversquashing and achieves state-of-the-art performance gains on OGB benchmarks without architectural modifications or input preprocessing.

ABSTRACT

Deploying graph neural networks (GNNs) on whole-graph classification or regression tasks is known to be challenging: it often requires computing node features that are mindful of both local interactions in their neighbourhood and the global context of the graph structure. GNN architectures that navigate this space need to avoid pathological behaviours, such as bottlenecks and oversquashing, while ideally having linear time and space complexity requirements. In this work, we propose an elegant approach based on propagating information over expander graphs. We leverage an efficient method for constructing expander graphs of a given size, and use this insight to propose the EGP model. We show that EGP is able to address all of the above concerns, while requiring minimal effort to set up, and provide evidence of its empirical utility on relevant graph classification datasets and baselines in the Open Graph Benchmark. Importantly, using expander graphs as a template for message passing necessarily gives rise to negative curvature. While this appears to be counterintuitive in light of recent related work on oversquashing, we theoretically demonstrate that negatively curved edges are likely to be required to obtain scalable message passing without bottlenecks. To the best of our knowledge, this is a previously unstudied result in the context of graph representation learning, and we believe our analysis paves the way to a novel class of scalable methods to counter oversquashing in GNNs.

Motivation & Objective

• Address the challenge of global information propagation in graph neural networks (GNNs) for whole-graph classification tasks.
• Overcome pathological behaviors such as oversquashing and bottlenecks in deep GNNs while maintaining linear time and space complexity.
• Develop a method that requires no input-specific preprocessing and is easily integrated into existing GNN architectures.
• Theoretically and empirically investigate the role of negative curvature in message passing and its relationship to oversquashing.
• Demonstrate that negatively curved edges in expander graphs do not trigger oversquashing, contrary to prior assumptions, and may be necessary for scalable communication.

Proposed method

• Construct sparse, 4-regular expander graphs using Cayley graphs of the special linear group SL(2,ℤₙ), which have logarithmic diameter and are efficiently precomputable.
• Use the precomputed expander graph as a fixed message-passing template, over which GNN layers propagate node features to achieve global context aggregation.
• Interleave EGP layers into standard GNN architectures (e.g., GIN) without modifying the model’s parameter count or latent dimensionality.
• Leverage theoretical results on the special linear group to ensure the expander graphs are sparse, highly connected, and have low diameter.
• Apply curvature analysis to show that while edges in the expander graphs are negatively curved, they do not induce oversquashing due to bounded curvature magnitude.
• Use the Open Graph Benchmark (OGB) to evaluate EGP on diverse datasets including molecular property prediction, protein interaction networks, and code summarization tasks.

Experimental results

Research questions

• RQ1 Can expander graphs be used as a scalable, fixed message-passing template to enable global information flow in GNNs without quadratic complexity?
• RQ2 Why do negatively curved edges in expander graphs not lead to oversquashing, despite prior work linking negative curvature to this issue?
• RQ3 Can EGP be integrated into standard GNNs without architectural changes while still improving performance on graph classification tasks?
• RQ4 Is the use of expander graphs as a communication backbone theoretically justified in terms of avoiding bottlenecks and maintaining linear complexity?
• RQ5 Does the empirical performance of EGP exceed that of standard GNNs and other oversquashing mitigation methods on real-world benchmarks?

Key findings

• EGP achieves significant performance improvements on all four OGB datasets—ogbg-molhiv (+3.76%), ogbg-molpcba (+2.78%), ogbg-ppa (+1.96%), and ogbg-code2 (+0.13%)—without changing model parameters or latent dimensions.
• The EGP layer consistently outperforms the baseline GIN model across diverse graph types, including small molecules, large syntax trees, and protein-protein interaction networks.
• Theoretical analysis shows that while edges in the constructed expander graphs are negatively curved, the curvature magnitude is insufficient to trigger the oversquashing conditions identified in prior work.
• The Cayley graphs of SL(2,ℤₙ) are sparse (|E| = O(|V|)), 4-regular, and have logarithmic diameter, enabling efficient global communication in O(log |V|) hops.
• EGP requires no input-specific preprocessing and can be seamlessly integrated into existing GNN architectures via simple layer insertion.
• Empirical results demonstrate that EGP effectively mitigates bottlenecks and oversquashing, even in deep GNNs, by enabling scalable, long-range information flow through expander topology.