[论文解读] Solving Mixed Integer Programs Using Neural Networks

Vinod Nair, Sergey Bartunov+17 more|arXiv (Cornell University)|Dec 23, 2020

Machine Learning and Algorithms参考文献 69被引用 42

一句话总结

论文提出 Neural Diving 和 Neural Branching 来增强 MIP 求解器，在大规模真实世界数据集和 MIPLIB 数据集上通过学习原始可行性和分支策略的图神经网络，实现对 SCIP 的大幅改进。

ABSTRACT

Mixed Integer Programming (MIP) solvers rely on an array of sophisticated heuristics developed with decades of research to solve large-scale MIP instances encountered in practice. Machine learning offers to automatically construct better heuristics from data by exploiting shared structure among instances in the data. This paper applies learning to the two key sub-tasks of a MIP solver, generating a high-quality joint variable assignment, and bounding the gap in objective value between that assignment and an optimal one. Our approach constructs two corresponding neural network-based components, Neural Diving and Neural Branching, to use in a base MIP solver such as SCIP. Neural Diving learns a deep neural network to generate multiple partial assignments for its integer variables, and the resulting smaller MIPs for un-assigned variables are solved with SCIP to construct high quality joint assignments. Neural Branching learns a deep neural network to make variable selection decisions in branch-and-bound to bound the objective value gap with a small tree. This is done by imitating a new variant of Full Strong Branching we propose that scales to large instances using GPUs. We evaluate our approach on six diverse real-world datasets, including two Google production datasets and MIPLIB, by training separate neural networks on each. Most instances in all the datasets combined have $10^3-10^6$ variables and constraints after presolve, which is significantly larger than previous learning approaches. Comparing solvers with respect to primal-dual gap averaged over a held-out set of instances, the learning-augmented SCIP is 2x to 10x better on all datasets except one on which it is $10^5$x better, at large time limits. To the best of our knowledge, ours is the first learning approach to demonstrate such large improvements over SCIP on both large-scale real-world application datasets and MIPLIB.

研究动机与目标

• Motivate the use of machine learning to automatically construct heuristics for solving large-scale MIPs.
• Develop two neural components, Neural Diving and Neural Branching, to improve primal feasibility and branching decisions.
• Integrate the learned heuristics into SCIP to form a Neural Solver tailored to a dataset.
• Evaluate on large real-world and MIPLIB datasets to demonstrate performance gains over Tuned SCIP.

提出的方法

• Represent MIPs as bipartite graphs with nodes for variables and constraints and edges weighted by constraint coefficients.
• Use a graph convolutional network (GCN) to produce node embeddings that feed neural predictors for both tasks.
• Neural Diving learns to generate partial variable assignments that define smaller sub-MIPs solved by SCIP to obtain high-quality joint assignments.
• Neural Branching learns a scalable imitation policy for variable selection in branch-and-bound by approximating Full Strong Branching using a GPU-enabled ADMM-based approach.
• Train Neural Diving and Neural Branching on dataset-specific MIPs and integrate them into SCIP to form a Neural Solver.
• Evaluate using primal-dual gap and survival plots on held-out instances across multiple large MIP datasets.

实验结果

研究问题

• RQ1 Can learning-based components improve the quality and speed of MIP solving when integrated into a state-of-the-art solver like SCIP?
• RQ2 Do two neural components—one for primal heuristic generation and one for branching policy—lead to cumulative improvements over a tuned baseline?
• RQ3 How well do these neural models scale to large real-world MIPs and heterogeneous datasets such as MIPLIB?
• RQ4 Does GPU-enabled imitation of strong branching via ADMM provide scalable and effective training data for branching decisions?
• RQ5 Is the combined Neural Solver competitive with or superior to standard SCIP across diverse applications?

主要发现

• Neural Solver often achieves significantly smaller primal-dual gaps than Tuned SCIP on four of five datasets with the largest MIPs.
• Neural Branching yields 2-20x better average dual gap on held-out instances at large time limits on four datasets.
• Neural Diving reaches close to 1% average primal gap on several datasets and can be faster than Tuned SCIP by factors of 3–10 on large instances.
• On two datasets, Neural Diving alone reaches the 1% primal gap faster than Gurobi in head-to-head tests.
• On MIPLIB open instances, the approach finds three new best-known assignments, outperforming commercial solvers.
• The combined Neural Solver matches SCIP on the smallest or most challenging fifth dataset, demonstrating competitive performance across diverse cases.