[Paper Review] How Powerful are K-hop Message Passing Graph Neural Networks

Jiarui Feng, Yixin Chen+3 more|arXiv (Cornell University)|May 26, 2022

Advanced Graph Neural Networks37 citations

TL;DR

The paper theoretically analyzes the expressive power of K-hop message passing GNNs, distinguishes two kernels, proves they outperform 1-WL but are bounded by 3-WL, and introduces KP-GNN with peripheral subgraphs to significantly boost expressiveness, backed by experiments.

ABSTRACT

The most popular design paradigm for Graph Neural Networks (GNNs) is 1-hop message passing -- aggregating information from 1-hop neighbors repeatedly. However, the expressive power of 1-hop message passing is bounded by the Weisfeiler-Lehman (1-WL) test. Recently, researchers extended 1-hop message passing to K-hop message passing by aggregating information from K-hop neighbors of nodes simultaneously. However, there is no work on analyzing the expressive power of K-hop message passing. In this work, we theoretically characterize the expressive power of K-hop message passing. Specifically, we first formally differentiate two different kernels of K-hop message passing which are often misused in previous works. We then characterize the expressive power of K-hop message passing by showing that it is more powerful than 1-WL and can distinguish almost all regular graphs. Despite the higher expressive power, we show that K-hop message passing still cannot distinguish some simple regular graphs and its expressive power is bounded by 3-WL. To further enhance its expressive power, we introduce a KP-GNN framework, which improves K-hop message passing by leveraging the peripheral subgraph information in each hop. We show that KP-GNN can distinguish many distance regular graphs which could not be distinguished by previous distance encoding or 3-WL methods. Experimental results verify the expressive power and effectiveness of KP-GNN. KP-GNN achieves competitive results across all benchmark datasets.

Motivation & Objective

• Differentiate two distinct K-hop neighbor kernels (shortest-path distance vs graph diffusion) and clarify their impact on expressiveness.
• Characterize the expressive power of K-hop message passing and compare it to 1-WL and 3-WL bounds.
• Propose KP-GNN to augment K-hop MPNNs with peripheral subgraph information.
• Provide theoretical guarantees and empirical evidence across simulated and real-world datasets.

Proposed method

• Define two K-hop neighbor kernels: shortest-path-distance (spd) and graph-diffusion (gd).
• Present the K-hop message passing framework with per-hop MES/UPD and a combine step (Equation (3)).
• Show that proper K-hop GNNs are strictly more powerful than 1-hop GNNs for K>1 (Proposition 1).
• Prove that the expressive power of any proper K-hop GNN is bounded by 3-WL (Theorem 2).
• Introduce KP-GNN by augmenting per-hop messages with peripheral subgraph information (Equations (4)-(5)).
• Demonstrate augmented expressiveness on distance-regular graphs and provide empirical results (Tables and experiments).

Experimental results

Research questions

• RQ1 How does K-hop message passing compare to 1-WL in terms of graph distinguishability?
• RQ2 Do K-hop GNNs distinguish almost all regular graphs, and what are their theoretical limits?
• RQ3 Can incorporating peripheral subgraph information (KP-GNN) overcome the 3-WL bound for standard K-hop GNNs?
• RQ4 How do KP-GNN variants perform on synthetic and real-world benchmarks compared to existing GNNs?

Key findings

• K-hop GNNs with K>1 are strictly more powerful than 1-hop GNNs and can distinguish many regular graphs.
• With one layer and modest K, 2-hop GNNs can distinguish almost all pairs of n-sized r-regular graphs under certain conditions.
• The expressive power of proper K-hop GNNs is bounded by 3-WL for any kernel.
• KP-GNN, which includes peripheral subgraph information, distinguishes distance-regular graphs that standard K-hop GNNs cannot.
• Empirical results show KP-GNN achieving competitive or state-of-the-art performance across multiple datasets and tasks.