[Paper Review] Relational Pooling for Graph Representations

Ryan L. Murphy, Balasubramaniam Srinivasan+2 more|arXiv (Cornell University)|Mar 6, 2019

Advanced Graph Neural NetworksComputer Science63 references38 citations

TL;DR

Relational Pooling (RP) provides a permutation-invariant framework that can be combined with various neural architectures to achieve maximal expressiveness for graphs, even surpassing WL-GNNs in some cases, and enables RP-GNNs with enhanced power.

ABSTRACT

This work generalizes graph neural networks (GNNs) beyond those based on the Weisfeiler-Lehman (WL) algorithm, graph Laplacians, and diffusions. Our approach, denoted Relational Pooling (RP), draws from the theory of finite partial exchangeability to provide a framework with maximal representation power for graphs. RP can work with existing graph representation models and, somewhat counterintuitively, can make them even more powerful than the original WL isomorphism test. Additionally, RP allows architectures like Recurrent Neural Networks and Convolutional Neural Networks to be used in a theoretically sound approach for graph classification. We demonstrate improved performance of RP-based graph representations over state-of-the-art methods on a number of tasks.

Motivation & Objective

• Motivate a most-powerful graph representation framework beyond WL-based GNNs for graphs with vertex/edge features.
• Propose Relational Pooling (RP) as a permutation-invariant aggregation over graph representations.
• Show that RP can be combined with diverse neural architectures (RNNs, CNNs, MLPs, GNNs) and theoretically achieve maximal graph expressiveness.
• Introduce tractable approximations to RP to enable scalable graph classification and demonstrate empirical improvements.

Proposed method

• Define RP as a joint permutation-invariant function that averages a permutation-sensitive function over all vertex relabelings (Equation 1).
• Extend RP to bipartite graphs via separate RP (Equation 2).
• Show RP achieves maximal expressiveness for finite graphs when the inner function is sufficiently expressive (Theorem 2.1).
• Enhance WL-GNNs by embedding permutation-sensitive identifiers to distinguish symmetric neighbor structures and form RP-GNN (Equation 5; Theorem 2.2).
• Describe neural architectures for the inner function including RNNs, CNNs, and GNNs, and how they integrate with RP.
• Present tractability strategies: canonical orientations, pi-SGD, and k-ary RP to reduce computation (Sections 2.3.1–2.3.3).

Experimental results

Research questions

• RQ1 Can Relational Pooling provide a most-powerful graph representation beyond WL-based GNNs for distinguishing nonisomorphic graphs?
• RQ2 How can RP be integrated with existing neural architectures to enhance graph classification while remaining tractable?
• RQ3 Do approximations (canonical orientations, pi-SGD, k-ary RP) preserve enough expressiveness to outperform WL-GNNs in practice?
• RQ4 What is the theoretical gain in expressiveness when combining RP with WL-GNNs (RP-GNN) relative to standard WL tests?
• RQ5 How does RP generalize to joint and separate invariance for non-bipartite and bipartite graphs respectively?

Key findings

• RP defines a maximally expressive permutation-invariant representation for graphs with finite vertex/edge attribute sets (Theorem 2.1).
• RP-GNN is strictly more expressive than a standard WL-GNN, and can be more powerful than the WL test when using a GIN as the inner function (Theorem 2.2).
• Permutation-sensitive inner functions can be leveraged within RP to create more powerful graph representations.
• Approximate RP methods (canonical orientations, pi-SGD, k-ary RP) provide tractable yet expressive approximations suitable for graph classification.
• RP can be implemented with diverse architectures (RNNs, CNNs, MLPs, GNNs) to build flexible, powerful graph representations.