[Paper Review] A Survey on The Expressive Power of Graph Neural Networks

Ryoma Sato|arXiv (Cornell University)|Mar 9, 2020

Advanced Graph Neural NetworksComputer Science91 references84 citations

TL;DR

This survey analyzes the expressive power of graph neural networks (GNNs), their limitations revealed by WL tests, and provably powerful variants including GINs, k-GNNs, and higher-order invariant/equivariant GNNs.

ABSTRACT

Graph neural networks (GNNs) are effective machine learning models for various graph learning problems. Despite their empirical successes, the theoretical limitations of GNNs have been revealed recently. Consequently, many GNN models have been proposed to overcome these limitations. In this survey, we provide a comprehensive overview of the expressive power of GNNs and provably powerful variants of GNNs.

Motivation & Objective

• Assess what graphs GNNs can distinguish in node and graph classification tasks.
• Explain the connection between GNNs and the Weisfeiler–Lehman (WL) algorithm.
• Review provably powerful GNN variants that overcome standard GNN limitations.
• Highlight the XS correspondence among GNNs, WL, and distributed local algorithms.

Proposed method

• Describe the standard message passing formulation of GNNs and its limitations in distinguishing graphs.
• Introduce the 1-WL and k-WL algorithms and their relation to GNNs.
• Present Graph Isomorphism Networks (GINs) as injective aggregators to achieve 1-WL power.
• Discuss higher-order GNNs (k-GNNs) and their connection to set k-WL and k-FWL.
• Outline higher order invariant/equivariant networks and their universality properties.
• Explain how higher order structures enable universality with respect to permutation groups.

Experimental results

Research questions

• RQ1 What graphs can vanilla GNNs distinguish, and what are the inherent limitations?
• RQ2 How do GNNs relate to the WL algorithm in terms of expressive power?
• RQ3 What architectures or variants can make GNNs as powerful as WL (and beyond) for graph isomorphism tasks?
• RQ4 What are the trade-offs (e.g., memory) for higher-order or invariant/equivariant GNNs and their practicality?

Key findings

• Vanilla message passing GNNs are no more powerful than the 1-WL algorithm.
• GINs can achieve the expressive power of 1-WL by using injective aggregation.
• k-GNNs (based on set k-WL) can match the set k-WL, with higher power at the cost of memory.
• Higher order invariant/equivariant networks provide universality results, enabling modeling of invariant/equivariant functions independent of node count.
• There is a hierarchy: 1-WL ≈ 2-WL; k-WL extends WL power, and k-GNNs realize set k-WL power with practical trade-offs.