[Paper Review] Learning fine-grained search space pruning and heuristics for combinatorial optimization

Juho Lauri, Sourav Dutta+2 more|arXiv (Cornell University)|Jan 5, 2020

Advanced Graph Neural Networks4 citations

TL;DR

This paper proposes a novel, interpretable machine learning framework that prunes irrelevant nodes and edges in combinatorial optimization problems—specifically the maximum clique problem—by learning local, feature-based rules to reduce search space. The method achieves up to 99% pruning in sparse graphs and delivers up to 5× speedups over state-of-the-art solvers, while also revealing that neighborhood degree chi-squared values are highly predictive of clique membership, enabling a new, superior heuristic.

ABSTRACT

Combinatorial optimization problems arise in a wide range of applications from diverse domains. Many of these problems are NP-hard and designing efficient heuristics for them requires considerable time and experimentation. On the other hand, the number of optimization problems in the industry continues to grow. In recent years, machine learning techniques have been explored to address this gap. We propose a framework for leveraging machine learning techniques to scale-up exact combinatorial optimization algorithms. In contrast to the existing approaches based on deep-learning, reinforcement learning and restricted Boltzmann machines that attempt to directly learn the output of the optimization problem from its input (with limited success), our framework learns the relatively simpler task of pruning the elements in order to reduce the size of the problem instances. In addition, our framework uses only interpretable learning models based on intuitive features and thus the learning process provides deeper insights into the optimization problem and the instance class, that can be used for designing better heuristics. For the classical maximum clique enumeration problem, we show that our framework can prune a large fraction of the input graph (around 99 % of nodes in case of sparse graphs) and still detect almost all of the maximum cliques. This results in several fold speedups of state-of-the-art algorithms. Furthermore, the model used in our framework highlights that the chi-squared value of neighborhood degree has a statistically significant correlation with the presence of a node in a maximum clique, particularly in dense graphs which constitute a significant challenge for modern solvers. We leverage this insight to design a novel heuristic for this problem outperforming the state-of-the-art. Our heuristic is also of independent interest for maximum clique detection and enumeration.

Motivation & Objective

• To address the scalability challenge of NP-hard combinatorial optimization problems by reducing the search space efficiently.
• To design a machine learning framework that learns interpretable pruning rules rather than directly predicting optimal solutions.
• To improve the performance of exact solvers like FMC(H) through data-driven, feature-based pruning strategies.
• To uncover meaningful structural insights about instance classes (e.g., maximum cliques) via interpretable models.
• To develop a new heuristic for maximum clique enumeration that outperforms existing state-of-the-art methods.

Proposed method

• The framework uses a multi-stage pruning pipeline based on local, interpretable features—such as neighborhood degree and chi-squared values—computed per node.
• A gradient-boosted decision tree model is trained to classify nodes as likely to be in a maximum clique, enabling fine-grained pruning of non-promising elements.
• The pruning process is applied iteratively, reducing the graph size before running a base exact solver (e.g., FMC(H)).
• The method integrates with existing solvers by preprocessing the input graph, thus acting as a plug-in acceleration layer.
• Feature importance analysis from the model reveals that the chi-squared value of neighborhood degree is a statistically significant predictor of clique membership, especially in dense graphs.
• The framework is extended with MoMC (Multi-Objective Model Combination) to balance pruning aggressiveness and solution accuracy.

Experimental results

Research questions

• RQ1 Can interpretable, local features be used to effectively prune search space in combinatorial optimization without relying on complex black-box models?
• RQ2 To what extent can learning-based pruning reduce the size of NP-hard problem instances while preserving optimal solutions?
• RQ3 What structural features are most predictive of node inclusion in maximum cliques, and can they inform better heuristics?
• RQ4 How does the proposed pruning strategy compare to state-of-the-art solvers in terms of speedup and solution quality on real-world and synthetic graphs?
• RQ5 Can the insights from the learned model be leveraged to design a new, superior heuristic for maximum clique enumeration?

Key findings

• The framework achieves an average of 99% node pruning in sparse graphs while preserving nearly all maximum cliques, significantly reducing the search space.
• On real-world graphs, the method delivers up to 5× speedup over FMC(H), with ALTHEA+MoMC achieving optimal solutions on previously unsolved instances.
• The chi-squared value of neighborhood degree is identified as a statistically significant predictor of clique membership, particularly in dense graphs where solvers typically struggle.
• ALTHEA outperforms FMC(H) in accuracy by up to 6× on dense ER random graphs (n=256, p=0.75), achieving over 70% accuracy compared to FMC(H)’s ~50%.
• The framework maintains high pruning rates (up to 50%) even in dense graphs (p=0.5), with run-time speedups of up to 1.5× over FMC(H) on G(64,p) graphs.
• The new heuristic derived from the model’s insights—ALTHEA—outperforms RMC in both speed and solution quality, despite RMC’s perfect accuracy and high computational cost.