[论文解读] Learning Cut Selection for Mixed-Integer Linear Programming via Hierarchical Sequence Model

Zhihai Wang, Xijun Li+6 more|arXiv (Cornell University)|Feb 1, 2023

Formal Methods in Verification被引用 14

一句话总结

论文提出了一种分层序列模型（HEM），用于在 MILP 求解器中学习裁剪选择，联合优化选择多少裁剪以及要添加的有序子集，从而提高求解器效率并对更大规模实例具有良好泛化能力。

ABSTRACT

Cutting planes (cuts) are important for solving mixed-integer linear programs (MILPs), which formulate a wide range of important real-world applications. Cut selection -- which aims to select a proper subset of the candidate cuts to improve the efficiency of solving MILPs -- heavily depends on (P1) which cuts should be preferred, and (P2) how many cuts should be selected. Although many modern MILP solvers tackle (P1)-(P2) by manually designed heuristics, machine learning offers a promising approach to learn more effective heuristics from MILPs collected from specific applications. However, many existing learning-based methods focus on learning which cuts should be preferred, neglecting the importance of learning the number of cuts that should be selected. Moreover, we observe from extensive empirical results that (P3) what order of selected cuts should be preferred has a significant impact on the efficiency of solving MILPs as well. To address this challenge, we propose a novel hierarchical sequence model (HEM) to learn cut selection policies via reinforcement learning. Specifically, HEM consists of a two-level model: (1) a higher-level model to learn the number of cuts that should be selected, (2) and a lower-level model -- that formulates the cut selection task as a sequence to sequence learning problem -- to learn policies selecting an ordered subset with the size determined by the higher-level model. To the best of our knowledge, HEM is the first method that can tackle (P1)-(P3) in cut selection simultaneously from a data-driven perspective. Experiments show that HEM significantly improves the efficiency of solving MILPs compared to human-designed and learning-based baselines on both synthetic and large-scale real-world MILPs, including MIPLIB 2017. Moreover, experiments demonstrate that HEM well generalizes to MILPs that are significantly larger than those seen during training.

研究动机与目标

• 强调裁剪选择在求解 MILP 中的关键作用，并指出现有基于学习的方法在裁剪数量和排序方面的不足。
• 提出一种分层强化学习方法（HEM），以联合学习裁剪的数量和排序。
• 在合成和真实世界的 MILP 上（包括 MIPLIB 2017），相对于基线显示经验性提升，并具备对更大实例的良好泛化能力。

提出的方法

• 将裁剪选择形式化为一个两层强化学习问题：上层策略选择裁剪的比率（数量），下层序列模型选择该大小的有序裁剪子集。
• 将状态表示为（LP 松弛、候选裁剪、LP 最优解）并对每个候选裁剪使用13个设计特征。
• 将动作空间定义为候选裁剪的所有有序子集；推导出分层策略，其中最终动作先从上层策略中采样比率 k，然后再通过基于指针网络的序列模型采样大小为 floor(N*k) 的有序子集。
• 用 tanh-Gaussian 分布对上层策略进行参数化，并用 LSTM+MLP 对状态进行嵌入以预测其均值/方差。
• 下层策略是一个序列模型（指针网络），产生一个长度为 k 的有序裁剪索引序列；训练以捕捉裁剪之间的相互作用和顺序效应。
• 使用分层策略梯度进行训练，推导对高层和低层参数的梯度方程，并采用并行、类似 A3C 的训练方案。

实验结果

研究问题

• RQ1 两层级结构是否能有效同时学习所选裁剪的数量和排序，以提高 MILP 求解效率？
• RQ2 将裁剪选择建模为序列问题（裁剪之间存在交互）是否优于独立评分方法？
• RQ3 所提出的方法对训练中未见的更大规模 MILP 的泛化能力如何？
• RQ4 在不同基准测试中，添加裁剪的顺序对求解器性能的影响如何？
• RQ5 该方法能否处理现实世界的、如 MIPLIB 2017 实例这类大规模 MILP？

主要发现

• HEM 在合成和真实世界 MILP（包括 MIPLIB 2017 基准）上，在求解效率方面显著优于人工设计和基于学习的基线。
• 所选裁剪的顺序对求解器效率有显著影响，所选裁剪的比率也具有影响力。
• HEM 在简单、中等、困难数据集上均实现显著改进，并对比训练使用的 MILP 更大规模的实例具有泛化能力。
• 与 SBP 及其他基线相比，HEM 在多个基准上显示出原始对偶间隙积分（PD integral）和求解时间的显著减少。
• 在大规模真实世界生产计划问题上的实验证明了在现代 MILP 求解器中部署 HEM 的实际好处。