[Paper Review] Benders Cut Classification via Support Vector Machines for Solving Two-stage Stochastic Programs

Huiwen Jia, Siqian Shen|arXiv (Cornell University)|Jun 14, 2019

Supply Chain and Inventory Management45 references4 citations

TL;DR

This paper proposes LearnBD, a learning-enhanced Benders decomposition algorithm that uses support vector machines (SVM) to classify and selectively generate only the most valuable Benders cuts—those significantly improving bounds or binding at optimality. By filtering out less useful cuts, LearnBD reduces the size and solution time of the relaxed master problem, achieving faster convergence across capacitated facility location and multi-commodity network design instances compared to traditional Benders decomposition.

ABSTRACT

We consider Benders decomposition for solving two-stage stochastic programs with complete recourse based on finite samples of the uncertain parameters. We define the Benders cuts binding at the final optimal solution or the ones significantly improving bounds over iterations as valuable cuts. We propose a learning-enhanced Benders decomposition (LearnBD) algorithm, which adds a cut classification step in each iteration to selectively generate cuts that are more likely to be valuable cuts. The LearnBD algorithm includes two phases: (i) sampling cuts and collecting information from training problems and (ii) solving testing problems with a support vector machine (SVM) cut classifier. We run the LearnBD algorithm on instances of capacitated facility location and multi-commodity network design under uncertain demand. Our results show that SVM cut classifier works effectively for identifying valuable cuts, and the LearnBD algorithm reduces the total solving time of all instances for different problems with various sizes and complexities.

Motivation & Objective

• To address the computational bottleneck in Benders decomposition caused by the rapid growth of relaxed master problem (RMP) size due to accumulating cuts.
• To improve solution efficiency in two-stage stochastic programs with complete recourse by identifying and prioritizing only the most valuable Benders cuts.
• To develop a transferable machine learning framework that enables reusing trained cut classifiers across different problem instances with varying sizes and structures.
• To evaluate the effectiveness of SVM-based cut classification in reducing total solving time while maintaining optimality gap control.

Proposed method

• The LearnBD algorithm introduces a two-phase process: (i) collecting training data from solved instances to extract cut characteristics such as cut violation and number of cuts per scenario, and (ii) training an SVM classifier on this data to predict valuable cuts.
• Cut characteristics are normalized using problem-specific scaling factors: relative total transportation cost for cut violation and network size (|W|×|F|) for cut frequency to ensure transferability across instances.
• In each iteration, the trained SVM classifier predicts which cuts are likely to be valuable, and only those cuts are added to the RMP, reducing its size and solving time.
• The algorithm uses sample average approximation (SAA) to handle continuous uncertainty by generating finite scenarios via Monte Carlo sampling.
• The method integrates Benders decomposition with supervised learning, where the SVM is trained to distinguish between cuts that significantly improve the bound or are binding at optimality.
• The framework supports transfer learning: a classifier trained on one instance (e.g., cap62) can be applied to another (e.g., cap42) with proper feature scaling, improving efficiency without retraining.

Experimental results

Research questions

• RQ1 Can an SVM-based classifier effectively distinguish between valuable and non-valuable Benders cuts in two-stage stochastic programs?
• RQ2 Does filtering cuts using a learned classifier reduce the size and solution time of the relaxed master problem compared to traditional Benders decomposition?
• RQ3 Can a trained cut classifier be successfully transferred from one problem instance to another with different sizes or structural parameters?
• RQ4 How does the performance of the LearnBD algorithm scale across diverse instances of capacitated facility location and multi-commodity network design problems?

Key findings

• LearnBD reduced the total solving time by 18.5% on average across tested instances compared to traditional Benders decomposition, with a 29.5% reduction in RMP solution time for cap42 when using a transferred classifier from cap62.
• The SVM cut classifier achieved a 0.01% optimality gap on cap42 using only 4,304 cuts when trained on the same instance, compared to 6,000 cuts in the standard Benders approach.
• When transferring the classifier trained on cap62 to solve cap42, the algorithm still achieved a 12.5% reduction in total time (97.96s vs. 111.96s) and reduced the number of RMPs solved from 30 to 29.
• The use of normalized cut features—scaled by relative transportation cost and network size—enabled effective transfer learning across instances with different numbers of facilities and warehouses.
• The number of cuts added per iteration was significantly reduced (from 6,000 to 4,304) when using the SVM classifier, directly contributing to faster RMP solution times.
• The method maintained optimality gaps below 0.01% across all tested instances, demonstrating that performance gains were not achieved at the cost of solution quality.