[Paper Review] Spectral Clustering with Graph Neural Networks for Graph Pooling

Filippo Maria Bianchi, Daniele Grattarola+1 more|arXiv (Cornell University)|Jun 30, 2019

Complex Network Analysis Techniques36 references76 citations

TL;DR

The paper introduces MinCutPool, a differentiable GNN-based pooling operator that learns cluster assignments by continuous relaxation of minCUT, avoiding eigen-decomposition and enabling out-of-sample clustering and end-to-end training. It shows superior performance on unsupervised node clustering and supervised graph classification across multiple benchmarks.

ABSTRACT

Spectral clustering (SC) is a popular clustering technique to find strongly connected communities on a graph. SC can be used in Graph Neural Networks (GNNs) to implement pooling operations that aggregate nodes belonging to the same cluster. However, the eigendecomposition of the Laplacian is expensive and, since clustering results are graph-specific, pooling methods based on SC must perform a new optimization for each new sample. In this paper, we propose a graph clustering approach that addresses these limitations of SC. We formulate a continuous relaxation of the normalized minCUT problem and train a GNN to compute cluster assignments that minimize this objective. Our GNN-based implementation is differentiable, does not require to compute the spectral decomposition, and learns a clustering function that can be quickly evaluated on out-of-sample graphs. From the proposed clustering method, we design a graph pooling operator that overcomes some important limitations of state-of-the-art graph pooling techniques and achieves the best performance in several supervised and unsupervised tasks.

Motivation & Objective

• Motivate pooling in GNNs with a principled, scalable alternative to spectral clustering that incorporates node features.
• Develop a differentiable GNN-based clustering method that approximates the normalized minCUT objective.
• Derive a graph pooling operator (MinCutPool) that coarsens graphs while enabling end-to-end training.
• Enable out-of-sample clustering by learning a transferable clustering function.
• Demonstrate improvements over state-of-the-art pooling methods on unsupervised and supervised tasks.

Proposed method

• Formulate a continuous relaxation of the normalized minCUT objective and train a GNN to output soft cluster assignments S.
• Compute node representations via message-passing on the input graph and pass them through an MLP with softmax to obtain S.
• Define an unsupervised loss Lu = Lc + Lo that approximates minCUT (Lc) and enforces near-orthogonal, balanced clusters (Lo).
• Derive MinCutPool: A pooling layer that uses S to produce Xpool = S^T X and Apool = S^T Ã S, with a post-processing normalization to produce Âpool for subsequent MP layers.
• Train end-to-end with a task-specific loss to balance clustering quality and downstream performance.
• Provide differentiable pooling that can be stacked hierarchically for deep GNN architectures.

Experimental results

Research questions

• RQ1 Can a GNN learn a clustering function that approximates the spectral minCUT objective without computing Laplacian eigenvectors?
• RQ2 Does a differentiable MinCutPool improve clustering quality and downstream graph task performance compared to model-free and other model-based pooling methods?
• RQ3 Can the learned pooling generalize to out-of-sample graphs and enable end-to-end training for supervised tasks?
• RQ4 How does jointly optimizing the minCUT objective with a task loss affect avoiding degenerate clustering solutions?
• RQ5 What are the empirical gains of MinCutPool on unsupervised node clustering and supervised graph classification across benchmarks?

Key findings

• MinCutPool achieves superior or competitive performance compared to state-of-the-art pooling methods on several tasks.
• The proposed unsupervised loss and orthogonality regularization help avoid degenerate clustering solutions in practice.
• MinCutPool provides better reconstruction and information retention in autoencoder evaluations than competing pooling methods.
• In supervised graph classification, MinCutPool consistently attains equal or better accuracy than alternative pooling strategies across multiple datasets.
• The approach enables end-to-end training and deeper hierarchical graph representations without requiring spectral decomposition.