[Paper Review] Towards Sparse Hierarchical Graph Classifiers

Cătălina Cangea, Petar Veličković+3 more|arXiv (Cornell University)|Nov 3, 2018

Advanced Graph Neural Networks19 references198 citations

TL;DR

The paper introduces a scalable, sparsity-preserving hierarchical graph classifier that uses a learnable pooling mechanism to downsample graphs without quadratic memory, achieving competitive results to DiffPool on standard benchmarks.

ABSTRACT

Recent advances in representation learning on graphs, mainly leveraging graph convolutional networks, have brought a substantial improvement on many graph-based benchmark tasks. While novel approaches to learning node embeddings are highly suitable for node classification and link prediction, their application to graph classification (predicting a single label for the entire graph) remains mostly rudimentary, typically using a single global pooling step to aggregate node features or a hand-designed, fixed heuristic for hierarchical coarsening of the graph structure. An important step towards ameliorating this is differentiable graph coarsening---the ability to reduce the size of the graph in an adaptive, data-dependent manner within a graph neural network pipeline, analogous to image downsampling within CNNs. However, the previous prominent approach to pooling has quadratic memory requirements during training and is therefore not scalable to large graphs. Here we combine several recent advances in graph neural network design to demonstrate that competitive hierarchical graph classification results are possible without sacrificing sparsity. Our results are verified on several established graph classification benchmarks, and highlight an important direction for future research in graph-based neural networks.

Motivation & Objective

• Motivate graph classification and its limitations with fixed pooling or global pooling.
• Develop a differentiable, sparse pooling layer that downsamples graphs without quadratic memory.
• Integrate convolution, pooling, and readout to build an end-to-end graph classification model.
• Demonstrate scalability and competitive performance on standard graph classification benchmarks.

Proposed method

• Graph convolution using an inductive mean-pooling propagation rule with self-loops: MP(X,A)=sigma(D^(-1) Â X Θ + X Θ' ).
• Pooling layer that drops nodes to a fixed ratio k, using a projection vector p and top-k selection to form a sparser graph: y=Xp/||p||, i=top-k(y,k), X'=X⊙tanh(y) at indices i, A'=A_i,i.
• Readout via per-layer summaries s^(l) = (1/N^(l)) sum_i x_i^(l) || max_i x_i^(l), with final graph representation s = sum_l s^(l) and an MLP for prediction.
• Training uses Adam with dataset-specific learning rates and three conv-pool blocks to preserve 80% of nodes after pooling.
• Compared to DiffPool, the method maintains O(V+E) storage rather than quadratic, enabling scalability.

Experimental results

Research questions

• RQ1 Can differentiable pooling be implemented in graph CNNs without incurring quadratic memory costs?
• RQ2 Does sparsity-aware hierarchical pooling retain competitive performance on standard graph classification benchmarks?
• RQ3 How does the proposed method compare to DiffPool and sparse aggregation baselines across datasets?
• RQ4 Is multi-scale readout via Jumping Knowledge-style aggregation beneficial for classification performance?

Key findings

• The proposed sparse pooling method achieves competitive accuracy, closely matching DiffPool variants while avoiding quadratic memory usage.
• On Enzymes, D&D, Collab, and Proteins, the model outperforms the sparse GraphSAGE baseline and is within roughly 1 percentage point of DiffPool variants on most datasets.
• The method demonstrates favorable GPU memory behavior and scales to larger graphs where DiffPool incurs higher memory demands.
• In experiments, the Ours model often attains the best or near-best scores among tested approaches, validating sparsity-friendly hierarchical pooling.