[Paper Review] How Powerful are Graph Neural Networks?

Keyulu Xu, Weihua Hu+2 more|arXiv (Cornell University)|Oct 1, 2018

Advanced Graph Neural Networks40 references386 citations

TL;DR

The paper analyzes the expressive power of GNNs, showing they are at most as powerful as the Weisfeiler-Lehman test, and introduces Graph Isomorphism Network (GIN) which achieves WL-level expressiveness; it validates theory with graph classification benchmarks.

ABSTRACT

Graph Neural Networks (GNNs) are an effective framework for representation learning of graphs. GNNs follow a neighborhood aggregation scheme, where the representation vector of a node is computed by recursively aggregating and transforming representation vectors of its neighboring nodes. Many GNN variants have been proposed and have achieved state-of-the-art results on both node and graph classification tasks. However, despite GNNs revolutionizing graph representation learning, there is limited understanding of their representational properties and limitations. Here, we present a theoretical framework for analyzing the expressive power of GNNs to capture different graph structures. Our results characterize the discriminative power of popular GNN variants, such as Graph Convolutional Networks and GraphSAGE, and show that they cannot learn to distinguish certain simple graph structures. We then develop a simple architecture that is provably the most expressive among the class of GNNs and is as powerful as the Weisfeiler-Lehman graph isomorphism test. We empirically validate our theoretical findings on a number of graph classification benchmarks, and demonstrate that our model achieves state-of-the-art performance.

Motivation & Objective

• Motivate and formalize the need to understand the representational capacity of GNNs for graphs.
• Characterize the discriminative power of common GNN variants in distinguishing graph structures.
• Propose a maximally powerful GNN (GIN) that matches WL-test power and validate empirically on benchmarks.
• Compare less powerful aggregators (mean, max) and 1-layer perceptrons to illustrate limitations.
• Demonstrate state-of-the-art or competitive performance on graph classification tasks while highlighting the role of expressiveness.

Proposed method

• Model GNN aggregation as multiset functions over neighbor features and analyze injectiveness of these aggregations.
• Show that any aggregation-based GNN is at most as powerful as the WL test (WL distinguishability).
• Prove that injective neighbor aggregation and injective graph readout yield WL-level power.
• Develop Graph Isomorphism Network (GIN) using sum-based injective multiset functions via deep multisets (Lemma 5) and MLPs (Corollary 6).
• Propose a graph-level readout that concatenates READOUT across all layers to capture information from all depths (Jumping Knowledge style).
• Compare GIN with variants using mean/max pooling and 1-layer perceptrons to illustrate expressiveness differences.

Experimental results

Research questions

• RQ1 Can GNNs distinguish graph structures as effectively as the Weisfeiler-Lehman test?
• RQ2 What aggregation/readout conditions are required for a GNN to be as powerful as WL?
• RQ3 How do common GNN variants (GCN, GraphSAGE with mean/max pooling) fare in distinguishing simple graph structures?
• RQ4 Can we design a simple architecture (GIN) that achieves WL-level expressiveness and tractable learning?
• RQ5 Do more powerful GNNs translate to better performance on graph classification benchmarks?

Key findings

• GNNs are at most as powerful as the WL test in distinguishing graph structures.
• If neighbor aggregation and graph readout are injective, GNNs can be as powerful as the WL test.
• Popular variants like GCN and GraphSAGE with mean or max pooling cannot distinguish certain simple graph structures.
• GIN achieves WL-level expressiveness by using injective multiset functions parameterized by MLPs (deep multisets).
• Graph-level readout across multiple depths preserves WL-like discriminative power and improves performance on benchmarks.
• Empirical results show GIN matches or exceeds baselines in many graph classification datasets.