[Paper Review] Rethinking pooling in graph neural networks

Diego Mesquita, Amauri H. Souza+1 more|arXiv (Cornell University)|Oct 22, 2020

Advanced Graph Neural NetworksComputer Science52 references40 citations

TL;DR

The paper shows that local pooling is not essential for GNNs' performance; non-local pooling variants perform comparably, and convolutions mainly drive learned representations.

ABSTRACT

Graph pooling is a central component of a myriad of graph neural network (GNN) architectures. As an inheritance from traditional CNNs, most approaches formulate graph pooling as a cluster assignment problem, extending the idea of local patches in regular grids to graphs. Despite the wide adherence to this design choice, no work has rigorously evaluated its influence on the success of GNNs. In this paper, we build upon representative GNNs and introduce variants that challenge the need for locality-preserving representations, either using randomization or clustering on the complement graph. Strikingly, our experiments demonstrate that using these variants does not result in any decrease in performance. To understand this phenomenon, we study the interplay between convolutional layers and the subsequent pooling ones. We show that the convolutions play a leading role in the learned representations. In contrast to the common belief, local pooling is not responsible for the success of GNNs on relevant and widely-used benchmarks.

Motivation & Objective

• Question the assumed necessity of locality-preserving pooling in GNNs.
• Assess performance of non-local pooling variants on standard graph classification benchmarks.
• Analyze how convolutional layers interact with pooling to shape representations.
• Provide simple baselines to sanity-check pooling design choices in GNNs.

Proposed method

• Replicate representative pooling-heavy models (Graclus, DiffPool, GMN) and simplify pooling by randomization or using the complement graph.
• Evaluate on four graph-classification datasets (ZINC, NCI1, IMDB-B, SMNIST) and additional datasets.
• Replace learned or clustering-based pooling with non-local or randomized pooling variants.
• Examine learned embeddings to understand pooling impact and the role of early smoothing in convolutions.
• Compare with global pooling baselines to assess pooling necessity.

Experimental results

Research questions

• RQ1 Does locality-preserving pooling critically affect GNN performance on standard benchmarks?
• RQ2 Can random or non-local pooling match or exceed the performance of traditional pooling schemes?
• RQ3 How do convolutional layers influence the effectiveness of pooling in GNNs?
• RQ4 Are clustering-regularization losses essential for pooling quality in diffusion-based approaches?
• RQ5 What is the performance impact of using non-local pooling across diverse datasets?

Key findings

• Non-local pooling variants (Complement, Uniform/Normal/Bernoulli random) match or outperform their local pooling counterparts across tasks.
• Graclus and Complement achieve similar performance on NCI1, IMDB-B, SMNIST, and ZINC, with Complement within close range of Graclus.
• DiffPool with random poolings often yields higher average accuracy than DiffPool in the reported tasks.
• GMN with distance or random variants performs on par with GMN, often with faster training times.
• Using a single convolution before pooling significantly degrades performance compared to multiple convolutions, highlighting the importance of early smoothing.
• Global mean pooling on top of multiple convolutions yields performance comparable to local pooling methods.