[论文解读] Melding the Data-Decisions Pipeline: Decision-Focused Learning for Combinatorial Optimization

Bryan Wilder, Bistra Dilkina+1 more|arXiv (Cornell University)|Sep 14, 2018

Machine Learning and Data Classification参考文献 32被引用 8

一句话总结

本文提出决策聚焦学习（decision-focused learning），一种通过反向传播离散问题的连续松弛，将机器学习模型端到端地与组合优化相结合的框架。通过直接将模型训练与决策质量对齐，而非预测准确性，该方法显著提升了优化性能——例如在二分图匹配中实现70%更多的匹配，尽管其标准准确率指标较低。

ABSTRACT

Creating impact in real-world settings requires artificial intelligence techniques to span the full pipeline from data, to predictive models, to decisions. These components are typically approached separately: a machine learning model is first trained via a measure of predictive accuracy, and then its predictions are used as input into an optimization algorithm which produces a decision. However, the loss function used to train the model may easily be misaligned with the end goal, which is to make the best decisions possible. Hand-tuning the loss function to align with optimization is a difficult and error-prone process (which is often skipped entirely). We focus on combinatorial optimization problems and introduce a general framework for decision-focused learning, where the machine learning model is directly trained in conjunction with the optimization algorithm to produce high-quality decisions. Technically, our contribution is a means of integrating common classes of discrete optimization problems into deep learning or other predictive models, which are typically trained via gradient descent. The main idea is to use a continuous relaxation of the discrete problem to propagate gradients through the optimization procedure. We instantiate this framework for two broad classes of combinatorial problems: linear programs and submodular maximization. Experimental results across a variety of domains show that decision-focused learning often leads to improved optimization performance compared to traditional methods. We find that standard measures of accuracy are not a reliable proxy for a predictive model's utility in optimization, and our method's ability to specify the true goal as the model's training objective yields substantial dividends across a range of decision problems.

研究动机与目标

• 解决人工智能流水线中预测模型准确率与最终决策质量之间的错位问题。
• 克服两阶段训练（先预测后优化）的局限性，其中模型损失与优化目标解耦。
• 开发一种通用框架，将离散组合优化集成到可微分机器学习流水线中。
• 通过使用离散优化问题的连续松弛，实现基于梯度的预测模型训练。
• 证明决策聚焦学习在真实决策质量上优于传统两阶段方法，即使预测准确率较低。

提出的方法

• 使用离散组合问题的连续松弛（如线性规划、子模最大化）以实现通过优化过程的梯度反向传播。
• 对松弛问题的最优解进行关于模型预测的解析微分，从而支持端到端训练。
• 使用模型所诱导决策的目标值作为损失函数，而非标准准确率指标，训练预测模型（如神经网络、随机森林）。
• 在推理阶段，将松弛问题的连续解四舍五入为离散可行决策。
• 将该框架应用于两类主要的组合问题：线性规划与子模最大化，并采用定制化的松弛方法。
• 使用反映决策质量的代理目标，确保模型学习对优化至关重要的特征，即使这会降低标准准确率。

实验结果

研究问题

• RQ1 与标准两阶段训练相比，直接基于决策质量训练预测模型是否能显著改善优化结果？
• RQ2 在组合优化中，标准预测准确率指标（如MSE、AUC）与实际决策质量的相关性如何？
• RQ3 决策聚焦模型的预测与两阶段模型有何不同？它们优先关注哪些特征？
• RQ4 离散问题的连续松弛是否能有效支持组合优化场景下的基于梯度的训练？
• RQ5 在低信号、真实世界领域中，尽管预测准确率较低，决策聚焦学习是否能持续优于传统方法？

主要发现

• 决策聚焦学习在多个领域显著提升了决策质量：在二分图匹配中，NN2-Decision方法实现的匹配数比次优方法高出近70%。
• 尽管均方误差（MSE）和交叉熵损失更差，决策聚焦模型如NN1-Decision在多样化推荐领域中实现的目标值比其两阶段对应模型高出2.7倍。
• 通过决策聚焦学习训练的预测模型与真实通道输出权重之间表现出强相关性（R² = 0.94）——这是最优决策的关键指标，即使预测结果在视觉上与真实值差异较大。
• 标准准确率指标（如MSE和AUC）与决策质量相关性极低；例如，MSE表现最佳的模型（NN2-2Stage）在二分图匹配中的解质量最差。
• 当通过两阶段方法训练时，如随机森林等高表达能力模型在决策聚焦设置中表现不佳，表明仅靠模型容量不足以实现良好性能，必须配合正确的目标对齐。
• 该方法的成功源于聚焦于决策关键特征（如高输出权重通道），即使以整体预测准确率为代价，证明通过任务特定训练目标可更优地优化决策质量。