[Paper Review] Optimization with Constraint Learning: A Framework and Survey

Adejuyigbe O. Fajemisin, Donato Maragno+1 more|arXiv (Cornell University)|Oct 5, 2021

Constraint Satisfaction and Optimization4 citations

TL;DR

This paper proposes a structured five-step framework for Optimization with Constraint Learning (OCL), enabling practitioners to systematically learn constraints from data when explicit formulations are unavailable. It reviews recent OCL literature, identifies trends in model selection and embedding, and highlights key challenges and opportunities in uncertainty integration, efficient solver compatibility, and model verification for robust optimization outcomes.

ABSTRACT

Many real-life optimization problems frequently contain one or more constraints or objectives for which there are no explicit formulas. If data is however available, these data can be used to learn the constraints. The benefits of this approach are clearly seen, however there is a need for this process to be carried out in a structured manner. This paper therefore provides a framework for Optimization with Constraint Learning (OCL) which we believe will help to formalize and direct the process of learning constraints from data. This framework includes the following steps: (i) setup of the conceptual optimization model, (ii) data gathering and preprocessing, (iii) selection and training of predictive models, (iv) resolution of the optimization model, and (v) verification and improvement of the optimization model. We then review the recent OCL literature in light of this framework, and highlight current trends, as well as areas for future research.

Motivation & Objective

• To formalize the process of learning constraints from data in optimization problems where explicit formulations are infeasible.
• To provide a comprehensive, step-by-step framework for OCL to guide practitioners and reduce methodological pitfalls.
• To review and categorize recent literature on constraint learning using the proposed framework, identifying common approaches and research gaps.
• To highlight underexplored opportunities in uncertainty-aware modeling, efficient solver integration, and formal verification of learned constraints.
• To support the broader operations research community by offering a unified reference for constraint learning in optimization.

Proposed method

• The framework comprises five stages: (i) conceptual model setup, (ii) data gathering and preprocessing, (iii) selection and training of predictive models, (iv) resolution of the optimization model with embedded ML constraints, and (v) verification and iterative improvement.
• Predictive models such as regression trees, neural networks, and symbolic regression are used to learn constraints from feasible/infeasible data, avoiding explicit formula derivation.
• The framework emphasizes embedding ML models as constraints in mathematical optimization, distinguishing it from surrogate modeling or parameter prediction.
• It excludes methods focused on learning entire problem structures (e.g., CP model learning), SMT, or uncertainty set learning, focusing only on constraint-specific learning.
• The authors advocate for future work on uncertainty-aware optimization, efficient solver integration (e.g., hiding non-essential network components), and formal verification of learned models.
• Active learning and adaptive sampling are proposed as strategies to improve model quality with minimal data, balancing exploration and exploitation.

Experimental results

Research questions

• RQ1 How can constraint learning be systematically structured to avoid ad hoc approaches in real-world optimization?
• RQ2 Which predictive modeling techniques are most effective and embeddable in optimization frameworks for constraint learning?
• RQ3 How can uncertainty in predictive models be meaningfully incorporated into optimization to improve robustness?
• RQ4 What are the computational challenges in embedding complex models like deep neural networks into optimization solvers, and how can they be mitigated?
• RQ5 How can learned constraint models be formally verified and iteratively improved for reliability and performance?

Key findings

• The proposed OCL framework provides a structured, repeatable process for learning constraints from data, reducing risks of inconsistent or suboptimal modeling.
• Neural networks, decision trees, and symbolic regression are among the most commonly used methods for learning constraints, though less explored techniques may offer advantages.
• Incorporating uncertainty measures (e.g., Softmax scores, misclassification counts) into optimization via robust or stochastic programming remains underexplored but promising.
• Efficient embedding of ML models into solvers—such as hiding non-essential activation functions—can significantly improve computational performance.
• Formal verification frameworks for learned constraints, similar to those for neural networks, are needed to ensure reliability and solution robustness.
• Ensemble methods are underutilized in constraint learning, despite their superior predictive performance in other ML applications.