[Paper Review] Can Hybrid Geometric Scattering Networks Help Solve the Maximum Clique Problem?

Yimeng Min, Frederik Wenkel+2 more|arXiv (Cornell University)|Jun 3, 2022

Advanced Graph Neural Networks4 citations

TL;DR

This paper proposes a hybrid geometric scattering-based graph neural network (GNN) to efficiently approximate solutions to the NP-hard maximum clique (MC) problem. By combining band-pass wavelet filters with GCN-type convolutions, the model mitigates the oversmoothing problem, enabling accurate node representation learning; it achieves state-of-the-art accuracy and inference speed, with 0.04–0.07% of the parameters of prior GNNs while outperforming Gurobi and heuristic solvers on large, complex graphs.

ABSTRACT

We propose a geometric scattering-based graph neural network (GNN) for approximating solutions of the NP-hard maximum clique (MC) problem. We construct a loss function with two terms, one which encourages the network to find highly connected nodes and the other which acts as a surrogate for the constraint that the nodes form a clique. We then use this loss to train an efficient GNN architecture that outputs a vector representing the probability for each node to be part of the MC and apply a rule-based decoder to make our final prediction. The incorporation of the scattering transform alleviates the so-called oversmoothing problem that is often encountered in GNNs and would degrade the performance of our proposed setup. Our empirical results demonstrate that our method outperforms representative GNN baselines in terms of solution accuracy and inference speed as well as conventional solvers like Gurobi with limited time budgets. Furthermore, our scattering model is very parameter efficient with only $\sim$ 0.1\% of the number of parameters compared to previous GNN baseline models.

Motivation & Objective

• To address the challenge of solving the NP-hard maximum clique (MC) problem efficiently using deep learning.
• To overcome the oversmoothing problem in standard GNNs that hinders node discrimination in clique-finding tasks.
• To design a parameter-efficient, fast inference model that approximates the MC with high accuracy on large and complex graphs.
• To leverage geometric scattering transforms to enhance node representation discriminability for clique detection.
• To develop a two-stage framework combining unsupervised GNN learning with a constraint-preserving decoder for clique extraction.

Proposed method

• The model uses a hybrid architecture combining graph convolutional networks (GCN) with geometric scattering filters to extract multi-scale, band-pass features from graph structures.
• A two-term loss function is employed: one term promotes high connectivity of predicted nodes, and the other acts as a surrogate for clique constraints.
• The scattering transform incorporates wavelet-based filters that preserve structural boundaries, reducing oversmoothing and improving node representation differentiation.
• Node representations are learned as probability vectors in [0,1]^n, indicating likelihood of belonging to the maximum clique.
• A rule-based greedy decoder extracts the final clique by selecting nodes with the highest probabilities, enforcing clique constraints post-prediction.
• The model is trained end-to-end with a differentiable loss, enabling efficient optimization despite the NP-hard nature of the target problem.

Experimental results

Research questions

• RQ1 Can a geometric scattering-based GNN architecture effectively mitigate the oversmoothing problem in graph neural networks for combinatorial optimization?
• RQ2 Does incorporating band-pass wavelet filters improve node representation quality for identifying maximum cliques in complex graphs?
• RQ3 Can a hybrid GNN with minimal parameters achieve superior inference speed and accuracy compared to state-of-the-art GNNs and conventional solvers like Gurobi?
• RQ4 How does the performance of the proposed model scale across different graph complexity levels (easy, medium, hard)?
• RQ5 To what extent does the use of a constraint-preserving decoder enhance solution quality compared to standard decoding strategies?

Key findings

• On the LARGE-hard dataset, the model achieved an approximation score of 0.975 ± 0.050, outperforming all other neural network baselines.
• The model inferred solutions 12 times faster than RUN-CSP (accurate), the closest competing neural network, on the LARGE-hard dataset.
• For the same dataset, Gurobi and the heuristic (Xu et al., 2005) achieved higher accuracy but required 131 and 49 times longer inference times, respectively.
• The model used only 1,001 parameters on the COLLAB dataset, representing just 0.05% of the 1,880,709 parameters in the Erdös’ GNN baseline.
• On the IMDB dataset, the model achieved 0.972 ± 0.259 approximation score with 705 parameters, significantly outperforming GNN baselines in both speed and accuracy.
• The hybrid scattering model maintained high performance across all sub-datasets (easy, medium, hard), with the largest performance gap appearing on medium and hard instances.