[Paper Review] Understanding over-squashing and bottlenecks on graphs via curvature

Jake Topping, Francesco Di Giovanni+3 more|arXiv (Cornell University)|Nov 29, 2021

Topological and Geometric Data Analysis57 references34 citations

TL;DR

The paper introduces a new edge-based Balanced Forman curvature to quantify bottlenecks in graphs, proves negative curvature edges cause over-squashing in GNNs, and proposes curvature-guided SDRF graph rewiring to alleviate it.

ABSTRACT

Most graph neural networks (GNNs) use the message passing paradigm, in which node features are propagated on the input graph. Recent works pointed to the distortion of information flowing from distant nodes as a factor limiting the efficiency of message passing for tasks relying on long-distance interactions. This phenomenon, referred to as 'over-squashing', has been heuristically attributed to graph bottlenecks where the number of $k$-hop neighbors grows rapidly with $k$. We provide a precise description of the over-squashing phenomenon in GNNs and analyze how it arises from bottlenecks in the graph. For this purpose, we introduce a new edge-based combinatorial curvature and prove that negatively curved edges are responsible for the over-squashing issue. We also propose and experimentally test a curvature-based graph rewiring method to alleviate the over-squashing.

Motivation & Objective

• Provide a precise geometric description of over-squashing in GNNs and link it to graph bottlenecks.
• Introduce a tractable, edge-based curvature (Balanced Forman curvature) that bounds and explains message-passing bottlenecks.
• Show that negatively curved edges drive bottlenecks and over-squashing.
• Propose and evaluate a curvature-driven graph rewiring (Stochastic Discrete Ricci Flow) to alleviate bottlenecks.
• Compare curvature-based rewiring with diffusion-based approaches and demonstrate improved topology preservation.

Proposed method

• Define the Jacobian of node representations to quantify over-squashing and relate it to powers of the augmented normalized adjacency matrix.
• Introduce Balanced Forman curvature Ric(i,j) as a local, computable edge curvature with Ric(i,j) > -2 and prove Ric(i,j) ≤ κ(i,j) (Ollivier curvature).
• Prove that edges with negative curvature induce bottlenecks that hinder cross-node information propagation (Theorem 4).
• Relate curvature to bottlenecks via Cheeger constant h_G and spectral gap λ1 (Proposition 5).
• Propose Stochastic Discrete Ricci Flow (SDRF) to surgically modify negatively curved edges and (optionally) remove positively curved edges, within a graph-edit distance bound.
• Compare SDRF to diffusion-based rewiring (DIGL/PPR) and analyze effects on topology and homophily-dependent performance.

Experimental results

Research questions

• RQ1 What graph topological features drive over-squashing in message-passing neural networks?
• RQ2 Can a discrete curvature notion on edges (Balanced Forman curvature) detect bottlenecks responsible for over-squashing?
• RQ3 Does negatively curved edge structure causally induce bottlenecks that impede long-range information flow in GNNs?
• RQ4 Can curvature-guided rewiring (SDRF) reduce bottlenecks more effectively than diffusion-based methods without heavy graph modification?
• RQ5 How do curvature-based rewiring methods influence graph topology preservation and performance across datasets with varying homophily?

Key findings

• Negatively curved edges are identified as primary contributors to bottlenecks and over-squashing in GNNs (Theorem 4).
• Balanced Forman curvature provides a computable lower bound to Ollivier curvature and characterizes local edge geometry; Ric(i,j) > -2 generally.
• There is a theoretical link between curvature and the Cheeger constant, implying positive curvature bounds can control the spectral gap and bottleneck severity (Proposition 5).
• SDRF (Stochastic Discrete Ricci Flow) surgically targets negatively curved edges to alleviate bottlenecks while preserving graph topology better than diffusion-based rewiring.
• Experimentally, SDRF improves node classification accuracy across nine datasets, especially on low-homophily graphs, and preserves degree distributions closer to the original than DIGL.
• Curvature-based rewiring shows superior topology preservation (smaller graph-edit impact) compared to random-walk diffusion-based rewiring while achieving competitive or better performance.