[Paper Review] Graph Neural Networks with Local Graph Parameters

Pablo Barceló, Floris Geerts+2 more|arXiv (Cornell University)|Jun 12, 2021

Advanced Graph Neural Networks46 references15 citations

TL;DR

This paper proposes Graph Neural Networks enhanced with local graph parameters—specifically, homomorphism counts of small graph patterns (e.g., cliques) added to vertex features. By integrating these inexpensive, precomputed structural features, the model achieves higher expressive power than standard GNNs (beyond the 1-WL test) while retaining O(n) memory and computation costs. The key contribution is a precise characterization of this enhanced expressive power in terms of a variant of the k-dimensional Weisfeiler-Leman test, placing it between standard GNNs and higher-order GNNs.

ABSTRACT

Various recent proposals increase the distinguishing power of Graph Neural Networks GNNs by propagating features between $k$-tuples of vertices. The distinguishing power of these "higher-order'' GNNs is known to be bounded by the $k$-dimensional Weisfeiler-Leman (WL) test, yet their $\mathcal O(n^k)$ memory requirements limit their applicability. Other proposals infuse GNNs with local higher-order graph structural information from the start, hereby inheriting the desirable $\mathcal O(n)$ memory requirement from GNNs at the cost of a one-time, possibly non-linear, preprocessing step. We propose local graph parameter enabled GNNs as a framework for studying the latter kind of approaches and precisely characterize their distinguishing power, in terms of a variant of the WL test, and in terms of the graph structural properties that they can take into account. Local graph parameters can be added to any GNN architecture, and are cheap to compute. In terms of expressive power, our proposal lies in the middle of GNNs and their higher-order counterparts. Further, we propose several techniques to aide in choosing the right local graph parameters. Our results connect GNNs with deep results in finite model theory and finite variable logics. Our experimental evaluation shows that adding local graph parameters often has a positive effect for a variety of GNNs, datasets and graph learning tasks.

Motivation & Objective

• To address the limited expressive power of standard Graph Neural Networks (GNNs), which are bounded by the 1-dimensional Weisfeiler-Leman (1-WL) test and cannot distinguish certain graph structures like cliques or cycles.
• To develop a framework that enhances GNNs with local graph structural information in a computationally efficient way, avoiding the O(n^k) cost of higher-order GNNs.
• To precisely characterize the expressive power of GNNs augmented with local graph parameters, relating it to variants of the k-WL test and finite model theory.
• To provide practical guidance on selecting effective local graph parameters that improve GNN performance across diverse graph learning tasks.

Proposed method

• The method introduces homomorphism counts of small graph patterns (e.g., K₃, K₄, K₅) as additional vertex features, where each vertex is enriched with the number of homomorphisms from a pattern P to the graph G that map a specified root vertex r to v.
• These homomorphism counts are computed once in a preprocessing step, ensuring the overall GNN training remains O(n) per iteration, unlike higher-order GNNs that require O(n^k) operations.
• The framework is formally grounded in finite model theory, with expressive power analyzed via a variant of the k-dimensional Weisfeiler-Leman test, linking it to the logical expressiveness of finite-variable logics.
• The approach is compatible with any standard GNN architecture (e.g., GCN, GAT, GraphSAGE), allowing modular integration of structural features.
• The paper proposes techniques to guide the selection of effective graph patterns, such as cliques, based on their ability to distinguish non-isomorphic graphs.
• Empirical evaluation is conducted on multiple benchmark datasets (CLUSTER, PATTERN) using various GNNs, comparing performance with and without added local parameters.

Experimental results

Research questions

• RQ1 How does augmenting GNNs with local graph parameters—specifically homomorphism counts of small patterns—affect their expressive power compared to standard GNNs and higher-order GNNs?
• RQ2 What is the precise expressive power of GNNs enhanced with local graph parameters, and how does it relate to the k-dimensional Weisfeiler-Leman test and finite model theory?
• RQ3 Which local graph parameters (e.g., triangle counts, clique counts) are most effective in improving GNN performance across different graph learning tasks?
• RQ4 Can the addition of local graph parameters consistently improve GNN performance across diverse architectures (e.g., GCN, GAT, GraphSAGE) and datasets?
• RQ5 What is the computational cost and scalability of this approach, and how does it compare to higher-order GNNs in terms of training time and model size?

Key findings

• Adding homomorphism counts of small graph patterns (e.g., K₃, K₄, K₅) to vertex features significantly improves GNN performance on both the CLUSTER and PATTERN datasets, with consistent gains across multiple GNN architectures.
• On the PATTERN dataset, GAT with {K₃, K₄, K₅} parameters achieved 85.50% weighted accuracy, outperforming the baseline (78.83%) and other pattern combinations.
• On the CLUSTER dataset, GatedGCN with {K₃, K₄, K₅} parameters achieved 74.03% accuracy, slightly outperforming the baseline (74.28%) and other combinations, with minimal increase in model parameters.
• Training times remained stable across pattern sets, with the GAT model requiring only 1.02 hours (91 epochs) for {K₃, K₄, K₅} on the PATTERN dataset, indicating low computational overhead.
• The number of model parameters increased only slightly with added patterns—e.g., from 394,632 to 395,088 for GAT on PATTERN—confirming the method’s efficiency.
• The expressive power of the proposed framework is formally characterized as lying strictly between standard GNNs (1-WL) and higher-order GNNs (k-WL for k≥2), with a precise connection to a variant of the k-WL test.