[Paper Review] The Expressive Power of Graph Neural Networks: A Survey

Bingxu Zhang, Changjun Fan+5 more|arXiv (Cornell University)|Aug 16, 2023

Advanced Graph Neural Networks4 citations

TL;DR

This survey provides a unified theoretical framework for understanding the expressive power of Graph Neural Networks (GNNs), categorizing methods to enhance it into three pillars: graph feature enhancement, graph topology enhancement, and GNN architecture enhancement. It identifies limitations in current benchmarks and evaluation metrics, advocates for physics-informed GNN design and graph neural architecture search, and offers a standardized path for future research on GNN expressivity.

ABSTRACT

Graph neural networks (GNNs) are effective machine learning models for many graph-related applications. Despite their empirical success, many research efforts focus on the theoretical limitations of GNNs, i.e., the GNNs expressive power. Early works in this domain mainly focus on studying the graph isomorphism recognition ability of GNNs, and recent works try to leverage the properties such as subgraph counting and connectivity learning to characterize the expressive power of GNNs, which are more practical and closer to real-world. However, no survey papers and open-source repositories comprehensively summarize and discuss models in this important direction. To fill the gap, we conduct a first survey for models for enhancing expressive power under different forms of definition. Concretely, the models are reviewed based on three categories, i.e., Graph feature enhancement, Graph topology enhancement, and GNNs architecture enhancement.

Motivation & Objective

• To address the lack of a comprehensive survey on GNN expressive power, especially regarding subgraph counting, connectivity learning, and logical abilities beyond the Weisfeiler-Lehman (WL) test.
• To unify the definition of GNN expressive power by distinguishing between feature embedding ability and topology representation capability.
• To identify key factors limiting GNN expressive power, including deficiencies in capturing global graph structure and limitations of permutation-invariant aggregation.
• To critique existing benchmarks and evaluation metrics, highlighting issues like synthetic datasets, inconsistent baselines, and overreliance on overall model performance for assessing expressivity.
• To propose future research directions, including physics-informed GNN design and graph neural architecture search (GraphNAS) tailored for expressive power enhancement.

Proposed method

• Proposes a unified theoretical framework that defines GNN expressive power through two components: feature embedding ability and topology representation capability.
• Classifies existing methods to enhance expressive power into three categories: graph feature enhancement, graph topology enhancement, and GNN architecture enhancement.
• Analyzes the limitations of the k-WL hierarchy as a measure of expressive power, noting its inaccuracy for k > 3 and the need for more refined evaluation hierarchies.
• Reviews physics-inspired GNNs that depart from the message-passing paradigm, using differential geometry and algebraic topology to achieve higher expressive power.
• Proposes graph neural architecture search (GraphNAS) as a low-cost, automated method to design more expressive GNNs, emphasizing the need for search spaces that prioritize expressiveness over generalization.
• Critiques current evaluation practices, advocating for standardized benchmarks with real-world datasets and dedicated metrics for expressive ability, independent of downstream task performance.

Experimental results

Research questions

• RQ1 What are the key components that define the expressive power of GNNs beyond the classical WL test?
• RQ2 How do graph feature enhancement, topology enhancement, and architecture enhancement contribute differently to improving GNN expressivity?
• RQ3 Why are current evaluation benchmarks and metrics for GNN expressive power considered unreliable or insufficient?
• RQ4 What are the underlying factors limiting the expressive power of GNNs, and how do they relate to permutation invariance and global structure capture?
• RQ5 How can physics-informed models and graph neural architecture search (GraphNAS) be leveraged to design more expressive GNNs?

Key findings

• The expressive power of GNNs is constrained by their inability to effectively capture global graph topology and maintain permutation invariance, which limits their ability to distinguish non-isomorphic graphs.
• Existing evaluation practices are flawed due to reliance on synthetic datasets without node features, inconsistent baseline selection, and the use of overall model performance as a proxy for expressivity, which confounds multiple influencing factors.
• The k-WL hierarchy is too coarse for k > 3 and fails to provide reliable expressive power assessment, necessitating the development of more refined evaluation hierarchies.
• Physics-informed GNNs that use differential geometry and algebraic topology can achieve higher expressive power by breaking away from the traditional message-passing paradigm.
• Graph neural architecture search (GraphNAS) offers a promising, automated path to enhance expressive power, but current search spaces prioritize generalization over expressiveness, creating a critical research gap.
• A unified theoretical framework is proposed to standardize the analysis of GNN expressive power, integrating feature and topology representation, and enabling systematic evaluation and comparison of future models.