[Paper Review] Warm-starting constraint generation for mixed-integer optimization: A Machine Learning approach

Asunción Jiménez-Cordero, Juan M. Morales+1 more|arXiv (Cornell University)|Mar 24, 2021

Constraint Satisfaction and Optimization4 citations

TL;DR

This paper proposes a machine learning–enhanced warm-start strategy for constraint generation in mixed-integer linear programming (MILP), where an invariant constraint set—comprising both binding and critical non-binding constraints—is learned offline from past MILP instances and used to initialize online optimization. The method preserves optimality and feasibility guarantees while reducing online solution time by up to 60% and achieving optimal solutions in one iteration for 93% of unit commitment test cases with k=5.

ABSTRACT

Mixed Integer Linear Programs (MILP) are well known to be NP-hard (Non-deterministic Polynomial-time hard) problems in general. Even though pure optimization-based methods, such as constraint generation, are guaranteed to provide an optimal solution if enough time is given, their use in online applications remains a great challenge due to their usual excessive time requirements. To alleviate their computational burden, some machine learning techniques (ML) have been proposed in the literature, using the information provided by previously solved MILP instances. Unfortunately, these techniques report a non-negligible percentage of infeasible or suboptimal instances. By linking mathematical optimization and machine learning, this paper proposes a novel approach that speeds up the traditional constraint generation method, preserving feasibility and optimality guarantees. In particular, we first identify offline the so-called invariant constraint set of past MILP instances. We then train (also offline) a machine learning method to learn an invariant constraint set as a function of the problem parameters of each instance. Next, we predict online an invariant constraint set of the new unseen MILP application and use it to initialize the constraint generation method. This warm-started strategy significantly reduces the number of iterations to reach optimality, and therefore, the computational burden to solve online each MILP problem is significantly reduced. Very importantly, all the feasibility and optimality theoretical guarantees of the traditional constraint generation method are inherited by our proposed methodology. The computational performance of the proposed approach is quantified through synthetic and real-life MILP applications.

Motivation & Objective

• To address the high computational burden of solving MILPs in online applications, where traditional constraint generation is too slow.
• To overcome the limitation of prior ML methods that ignore critical non-binding constraints, leading to suboptimal or infeasible solutions.
• To develop a method that preserves the theoretical optimality and feasibility guarantees of classical constraint generation while accelerating convergence.
• To enable faster online solution of MILPs by learning and reusing an invariant constraint set from historical problem instances.
• To improve the predictive power of ML models in MILP by incorporating structural knowledge of constraints beyond just binding ones.

Proposed method

• Offline identification of the invariant constraint set for each past MILP instance, defined as the minimal set of constraints that must be included to preserve optimality.
• Training a machine learning model (e.g., k-NN or $σ$-learner) to predict the invariant constraint set based on problem parameters such as right-hand side values.
• Using the predicted invariant constraint set as a warm-start for the constraint generation algorithm in new, unseen MILP instances.
• Applying constraint generation from the learned initial set, iteratively adding violated constraints until optimality is reached.
• Leveraging the fact that the invariant set includes non-binding constraints crucial for optimality, not just those active at the solution.
• Ensuring theoretical guarantees of feasibility and optimality are inherited from the original constraint generation method.

Experimental results

Research questions

• RQ1 Can a machine learning model effectively predict the invariant constraint set of a new MILP instance based on historical data?
• RQ2 Does using a learned invariant constraint set as a warm-start significantly reduce the number of iterations in constraint generation?
• RQ3 Can the proposed method maintain optimality and feasibility guarantees while accelerating online MILP solution?
• RQ4 How does the performance of the proposed method compare to traditional constraint generation and ML methods that ignore non-binding constraints?
• RQ5 To what extent does the prediction accuracy of the invariant set correlate with solution time reduction in real-world applications like unit commitment?

Key findings

• The proposed method reduced online solution time to approximately 40% of the original MILP time in the unit commitment problem, achieving significant speedups.
• For 93% of unit commitment instances with k=5, the optimal solution was found in just one iteration of constraint generation, due to accurate prediction of the invariant constraint set.
• With k=100, nearly 100% of problems attained the optimal solution in a single iteration, demonstrating high prediction accuracy and robustness.
• The method outperformed baseline approaches that ignored non-binding constraints, which often led to suboptimal or infeasible solutions due to poor warm-starts.
• The approach preserved full optimality and feasibility guarantees, unlike many existing ML-aided MILP methods that risk suboptimality.
• The computational burden was significantly reduced, with the method achieving equivalent results to full constraint generation in a fraction of the time.