[Paper Review] On the equivalence between graph isomorphism testing and function approximation with GNNs

Zhengdao Chen, Soledad Villar+2 more|arXiv (Cornell University)|May 29, 2019

Advanced Graph Neural NetworksComputer Science98 citations

TL;DR

This paper proves an equivalence between a GNN’s ability to distinguish non-isomorphic graphs and its capacity to universally approximate permutation-invariant functions on graphs, introduces a sigma-algebra framework to compare GNN expressiveness, and proposes Ring-GNN, an extension of 2-IGN that distinguishes certain regular graphs that 2-IGN cannot.

ABSTRACT

Graph Neural Networks (GNNs) have achieved much success on graph-structured data. In light of this, there have been increasing interests in studying their expressive power. One line of work studies the capability of GNNs to approximate permutation-invariant functions on graphs, and another focuses on the their power as tests for graph isomorphism. Our work connects these two perspectives and proves their equivalence. We further develop a framework of the expressive power of GNNs that incorporates both of these viewpoints using the language of sigma-algebra, through which we compare the expressive power of different types of GNNs together with other graph isomorphism tests. In particular, we prove that the second-order Invariant Graph Network fails to distinguish non-isomorphic regular graphs with the same degree. Then, we extend it to a new architecture, Ring-GNN, which succeeds in distinguishing these graphs and achieves good performances on real-world datasets.

Motivation & Objective

• Bridge the two perspectives on GNN expressive power: graph isomorphism testing and invariant function approximation.
• Formalize expressiveness via sigma-algebras to compare GNN variants and tests.
• Show limitations of 2-IGN on non-isomorphic regular graphs with the same degree.
• Propose Ring-GNN as a tractable extension that enhances expressiveness over 2-IGN.

Proposed method

• Define GIso-discriminating and universally-approximating function families on graph spaces.
• Show that universal approximation implies GIso-discrimination and, conversely, that GIso-discrimination with augmentation yields universal approximation (finite and continuous feature spaces).
• Characterize expressiveness via sigma-algebras generated by function families and relate to graph isomorphism classes.
• Demonstrate a limitation of 2-IGN (cannot distinguish certain non-isomorphic regular graphs) and introduce Ring-GNN to overcome this by leveraging a ring of invariant matrix operators.
• Provide a practical Ring-GNN architecture with layers combining permutation-equivariant linear maps and nonlinearities to capture higher-order interactions without high-order tensors.

Experimental results

Research questions

• RQ1 Can a class of permutation-invariant graph functions distinguish all non-isomorphic graphs if it can universally approximate invariant functions?
• RQ2 How can sigma-algebras generated by function classes quantify and compare the expressive power of different GNN architectures?
• RQ3 Does extending 2-IGN with a ring of invariant operators (Ring-GNN) surpass its limitations on regular graphs?
• RQ4 What are the practical computational trade-offs when increasing expressive power with Ring-GNN compared to higher-order GNNs?
• RQ5 How do GNNs perform on synthetic (CSL) and real-world datasets when evaluated as isomorphism tests vs. function approximators?

Key findings

• Universality of a function class implies GIso-discrimination; conversely, GIso-discrimination with augmentation yields universal approximation in finite spaces.
• The sigma-algebra framework equates the expressiveness of function families to the fineness of the generated sigma-algebras, enabling formal comparisons among GNN variants.
• 2-IGN cannot distinguish non-isomorphic regular graphs with the same degree, illustrating a limitation in expressiveness.
• Ring-GNN, an extension based on a ring of invariant matrices, is able to distinguish CSL graph pairs that 2-IGN cannot, demonstrating increased expressive power.
• Ring-GNN achieves competitive or superior performance on CSL classification and multiple real-world datasets (IMDB, COLLAB, MUTAG, PTC, PROTEINS) compared to 2-IGN; in some cases it matches or exceeds GIN.
• The Ring-GNN architecture maintains tractable complexity (roughly O(n^2.38)) versus higher-order tensor GNNs, while enabling higher-order interactions.