[Paper Review] Nested Graph Neural Networks

Muhan Zhang, Pan Li|arXiv (Cornell University)|Oct 25, 2021

Advanced Graph Neural NetworksComputer Science63 references33 citations

TL;DR

NGNN introduces nested rooted subgraphs to learn node representations, proving strictly more powerful than 1-WL and standard GNNs while maintaining linear-time complexity, and demonstrating consistent performance gains across multiple base GNNs and datasets.

ABSTRACT

Graph neural network (GNN)'s success in graph classification is closely related to the Weisfeiler-Lehman (1-WL) algorithm. By iteratively aggregating neighboring node features to a center node, both 1-WL and GNN obtain a node representation that encodes a rooted subtree around the center node. These rooted subtree representations are then pooled into a single representation to represent the whole graph. However, rooted subtrees are of limited expressiveness to represent a non-tree graph. To address it, we propose Nested Graph Neural Networks (NGNNs). NGNN represents a graph with rooted subgraphs instead of rooted subtrees, so that two graphs sharing many identical subgraphs (rather than subtrees) tend to have similar representations. The key is to make each node representation encode a subgraph around it more than a subtree. To achieve this, NGNN extracts a local subgraph around each node and applies a base GNN to each subgraph to learn a subgraph representation. The whole-graph representation is then obtained by pooling these subgraph representations. We provide a rigorous theoretical analysis showing that NGNN is strictly more powerful than 1-WL. In particular, we proved that NGNN can discriminate almost all r-regular graphs, where 1-WL always fails. Moreover, unlike other more powerful GNNs, NGNN only introduces a constant-factor higher time complexity than standard GNNs. NGNN is a plug-and-play framework that can be combined with various base GNNs. We test NGNN with different base GNNs on several benchmark datasets. NGNN uniformly improves their performance and shows highly competitive performance on all datasets.

Motivation & Objective

• Motivate the need to move beyond rooted subtrees to capture richer local graph structures for graph classification.
• Propose NGNN to encode rooted subgraphs around each node via a two-level GNN framework.
• Prove NGNN is strictly more powerful than 1-WL and standard message passing GNNs under defined conditions.
• Demonstrate NGNN's scalability and compatibility as a drop-in enhancement for existing base GNNs.

Proposed method

• Define rooted subgraphs and the NGNN architecture with inner base GNNs operating on rooted subgraphs.
• Apply a subgraph pooling layer to obtain a final root-node representation from each rooted subgraph.
• Use an outer GNN (or graph pooling) to aggregate root-node representations into a whole-graph representation.
• Show that base GNNs can be any GNN; all inner GNNs share parameters and operate on copied subgraphs.
• Prove NGNN’s enhanced expressive power theoretically (discriminating almost all r-regular graphs) and discuss complexity (linear in graph size).
• Optionally augment node features with subgraph-specific features (e.g., distance encoding) to improve representations.

Experimental results

Research questions

• RQ1 Can NGNN distinguish graphs that 1-WL cannot distinguish?
• RQ2 How much performance improvement does NGNN provide when combined with various base GNNs on benchmark datasets?
• RQ3 Is NGNN scalable to large graphs while maintaining comparative efficiency to standard GNNs?
• RQ4 How does NGNN compare with higher-order WL-based GNNs in practice?
• RQ5 What is the impact of subgraph height and base GNN choice on NGNN performance?

Key findings

• NGNN can discriminate almost all pairs of n-sized r-regular graphs where 1-WL fails.
• NGNN consistently improves performance across multiple base GNNs on QM9, TU, and OGB benchmarks.
• NGNN maintains linear time and space complexity with respect to graph size, given small rooted subgraphs.
• A single or few NGNN layers suffice due to subgraph pooling enabling richer representations without deep architectures.
• Experimental results show NGNN’s competitive or superior performance to strong baselines on open benchmarks.