[Paper Review] Geom-GCN: Geometric Graph Convolutional Networks

Hongbin Pei, Bingzhe Wei+3 more|arXiv (Cornell University)|Feb 13, 2020

Advanced Graph Neural Networks30 references117 citations

TL;DR

Geom-GCN introduces a geometric aggregation scheme that maps graphs to a latent space to build structural neighborhoods and capture long-range dependencies, achieving state-of-the-art results on multiple graph benchmarks.

ABSTRACT

Message-passing neural networks (MPNNs) have been successfully applied to representation learning on graphs in a variety of real-world applications. However, two fundamental weaknesses of MPNNs' aggregators limit their ability to represent graph-structured data: losing the structural information of nodes in neighborhoods and lacking the ability to capture long-range dependencies in disassortative graphs. Few studies have noticed the weaknesses from different perspectives. From the observations on classical neural network and network geometry, we propose a novel geometric aggregation scheme for graph neural networks to overcome the two weaknesses. The behind basic idea is the aggregation on a graph can benefit from a continuous space underlying the graph. The proposed aggregation scheme is permutation-invariant and consists of three modules, node embedding, structural neighborhood, and bi-level aggregation. We also present an implementation of the scheme in graph convolutional networks, termed Geom-GCN (Geometric Graph Convolutional Networks), to perform transductive learning on graphs. Experimental results show the proposed Geom-GCN achieved state-of-the-art performance on a wide range of open datasets of graphs. Code is available at https://github.com/graphdml-uiuc-jlu/geom-gcn.

Motivation & Objective

• Address two core weaknesses of traditional MPNNs: loss of structural information in node neighborhoods and limited ability to capture long-range dependencies in disassortative graphs.
• Propose a geometric aggregation scheme operating on both graph and latent space.
• Provide a concrete implementation (Geom-GCN) and validate across diverse graph datasets.
• Analyze ablations to understand contributions of graph-space and latent-space neighborhoods.
• Explore embedding choices (Isomap, Poincare, struc2vec) to tailor Geom-GCN to different graph topologies.

Proposed method

• Introduce a three-module geometric aggregation scheme: node embedding to map nodes into a latent space, structural neighborhood defined in graph and latent space, and a bi-level aggregation to update node features.
• Define N_g(v) as graph neighbors and N_s(v) as latent-space neighbors within radius ρ, enabling long-range similarity capture.
• Use a relational operator τ over latent-space positions to assign geometric relationships r to neighbor pairs; τ partitions space into a finite set R.
• Perform bi-level aggregation with low-level aggregation p over sub-neighborhoods (i,r) and high-level aggregation q over virtual nodes (i,r) to produce h_v^{l+1}, ensuring permutation invariance.
• Instantiate Geom-GCN by choosing embedding methods (Isomap, Poincare, struc2vec) yielding Geom-GCN-I, Geom-GCN-P, Geom-GCN-S; implement τ with a 2D Euclidean or hyperbolic space; set p as a permutation-invariant sum and q as concatenation (then final mean at last layer).
• Demonstrate the approach on transductive node classification with 2-layer GCN-style architecture and compare against GCN and GAT.

Experimental results

Research questions

• RQ1 How can latent-space geometry be leveraged to preserve structural information in neighborhoods during graph convolutions?
• RQ2 Can latent-space neighborhoods capture long-range dependencies in disassortative graphs more effectively than purely graph-based neighborhoods?
• RQ3 What is the impact of different embedding spaces (Isomap, Poincare, struc2vec) on Geom-GCN performance across various graph topologies?
• RQ4 Does incorporating both graph-space and latent-space neighborhoods yield consistent improvements over single-neighborhood variants?
• RQ5 What is the trade-off between model complexity and performance when using geometric aggregation for large graphs?

Key findings

• Geom-GCN achieves state-of-the-art performance on a wide range of open graph datasets compared to GCN and GAT.
• The Geom-GCN-P variant (Poincare embedding) often yields strong results, especially on graphs with hierarchical structure; embedding choice significantly affects performance.
• Isomap-based Geom-GCN-I already improves performance by preserving distance patterns; combining latent-space neighborhoods enhances results on several datasets.
• Ablation studies show that both graph and latent-space neighborhoods can contribute to gains, but in some cases single-neighborhood variants outperform two-neighborhood variants, suggesting potential benefits from attention over neighborhoods in future work.
• Experiments demonstrate that the latent-space neighborhood helps capture long-range dependencies, particularly in disassortative graphs, by aggregating messages from structurally similar but distant nodes.