[Paper Review] Generalization and Representational Limits of Graph Neural Networks

Vikas Garg, Stefanie Jegelka+1 more|arXiv (Cornell University)|Feb 14, 2020

Advanced Graph Neural NetworksComputer Science52 references54 citations

TL;DR

The paper shows that standard local-information GNNs cannot compute key graph properties, and provides data-dependent generalization bounds for message-passing GNNs accounting for local permutation invariance, plus a more expressive GNN variant.

ABSTRACT

We address two fundamental questions about graph neural networks (GNNs). First, we prove that several important graph properties cannot be computed by GNNs that rely entirely on local information. Such GNNs include the standard message passing models, and more powerful spatial variants that exploit local graph structure (e.g., via relative orientation of messages, or local port ordering) to distinguish neighbors of each node. Our treatment includes a novel graph-theoretic formalism. Second, we provide the first data dependent generalization bounds for message passing GNNs. This analysis explicitly accounts for the local permutation invariance of GNNs. Our bounds are much tighter than existing VC-dimension based guarantees for GNNs, and are comparable to Rademacher bounds for recurrent neural networks.

Motivation & Objective

• Demonstrate that several important graph properties are not computable by LU-GNNs and related local-structure GNN variants.
• Develop a graph-theoretic formalism to analyze CPNGNNs and related models.
• Provide data-dependent generalization bounds for message-passing GNNs that account for local permutation invariance.
• Introduce a more expressive GNN variant (H-DCPN) that leverages additional geometric and port-information to overcome identified limitations.

Proposed method

• Define Locally Unordered GNNs (LU-GNNs) and variants like CPNGNN and DimeNet.
• Construct graphs that LU-GNNs cannot distinguish, thus cannot compute properties such as girth, circumference, diameter, radius, conjoint cycles, total cycles, and k-cliques.
• Introduce port-covers and port-locally isomorphic graph concepts to formalize limits of CPNGNNs.
• Propose H-DCPN as a more expressive variant combining directional messaging, port numbering, and geometric plane features.
• Derive data-dependent generalization bounds for mean-field GNNs by unrolling local computation trees and bounding Rademacher complexity with permutation-invariant aggregations.
• research_questions([

Experimental results

Research questions

• RQ1 Can LU-GNNs distinguish graphs that differ on key properties like girth, diameter, or clique structure?
• RQ2 Do port-numbering based GNNs (CPNGNNs) overcome the limitations of LU-GNNs across these properties?
• RQ3 What are the data-dependent generalization guarantees for message-passing GNNs that account for local permutation invariance?

Key findings

• LU-GNNs cannot compute several important graph properties using only local information.
• CPNGNNs can distinguish some graphs that LU-GNNs cannot, but their ability depends on the chosen port numbering.
• There exist consistent port orderings where CPNGNNs fail to decide several graph properties, and DimeNet can still be insufficient in others.
• DimeNet with permutation-invariant readout cannot decide girth, circumference, diameter, radius, or total number of cycles; port-numbering alone is also insufficient.
• A more expressive variant, H-DCPN, leveraging plane-angle features, can distinguish graphs that fail for both CPNGNN and DimeNet.
• The paper derives data-dependent generalization bounds for GNNs that match RNN-type bounds in dependence on depth, width, and sample size, with explicit handling of local permutation invariance.