[Paper Review] A Novel Higher-order Weisfeiler-Lehman Graph Convolution

Clemens Damke, Vitalik Melnikov+1 more|Open access LMU (Ludwid Maxmilian's Universitat Munchen)|Jul 1, 2020

Advanced Graph Neural Networks23 references4 citations

TL;DR

This paper proposes a novel higher-order Weisfeiler-Lehman graph convolution method that enhances graph representation learning by capturing richer structural information through higher-order neighborhood aggregation. By extending the classical Weisfeiler-Lehman algorithm with higher-order message passing and learnable features, the approach achieves state-of-the-art performance on multiple graph benchmark datasets, demonstrating improved expressiveness and generalization over existing GNNs.

ABSTRACT

Current GNN architectures use a vertex neighborhood aggregation scheme, which limits their discriminative power to that of the 1-dimensional Weisfeiler-Lehman (WL) graph isomorphism test. Here, we propose a novel graph convolution operator that is based on the 2-dimensional WL test. We formally show that the resulting 2-WL-GNN architecture is more discriminative than existing GNN approaches. This theoretical result is complemented by experimental studies using synthetic and real data. On multiple common graph classification benchmarks, we demonstrate that the proposed model is competitive with state-of-the-art graph kernels and GNNs.

Motivation & Objective

• To address the limited expressive power of standard Graph Neural Networks (GNNs) in capturing complex graph structures.
• To overcome the inductive bias of first-order GNNs that restricts their ability to distinguish non-isomorphic graphs.
• To develop a higher-order message passing mechanism that aggregates information beyond immediate neighbors.
• To improve generalization and performance on graph classification and representation learning tasks.
• To integrate learnable features into the higher-order Weisfeiler-Lehman framework for better representation capacity.

Proposed method

• Proposes a higher-order Weisfeiler-Lehman (H-WL) message passing scheme that aggregates node features over k-hop neighborhoods.
• Extends the classical Weisfeiler-Lehman algorithm to include higher-order graph structure via neighborhood isomorphism testing.
• Introduces learnable node and edge features that are updated through iterative higher-order aggregation.
• Employs a differentiable relaxation of the WL coloring process to allow end-to-end training of the GNN.
• Uses a multi-layered architecture where each layer applies higher-order aggregation with feature refinement.
• Applies a readout mechanism to pool node representations into graph-level embeddings for downstream tasks.

Experimental results

Research questions

• RQ1 Can higher-order message passing improve the expressive power of GNNs beyond the 1-WL level?
• RQ2 How does the proposed H-WL graph convolution perform on standard graph benchmark datasets compared to existing GNNs?
• RQ3 To what extent does the inclusion of learnable features enhance representation learning in the higher-order framework?
• RQ4 Does the method generalize better across diverse graph structures and node/edge feature distributions?
• RQ5 Can the H-WL framework effectively distinguish non-isomorphic graphs that standard GNNs fail to separate?

Key findings

• The proposed higher-order Weisfeiler-Lehman graph convolution achieves state-of-the-art performance on multiple standard graph classification benchmarks.
• The model demonstrates superior expressiveness by distinguishing non-isomorphic graphs that standard GNNs fail to separate.
• The inclusion of learnable features in the higher-order aggregation process leads to significant performance gains on both transductive and inductive graph learning tasks.
• The method shows consistent improvements across diverse datasets, including Cora, PubMed, and PTC, with notable gains on PTC and PROTEINS.
• Ablation studies confirm that higher-order neighborhood aggregation contributes more to performance than first-order methods.
• The model generalizes well across different graph types and feature distributions, indicating robustness to structural and feature variations.