[Paper Review] Ecole: A Gym-like Library for Machine Learning in Combinatorial Optimization Solvers

Antoine Prouvost, Justin Dumouchelle+4 more|arXiv (Cornell University)|Nov 11, 2020

Constraint Satisfaction and Optimization38 references19 citations

TL;DR

Ecole is an open-source, Gym-like library that exposes key decision tasks in combinatorial optimization solvers—such as variable and node selection—as Markov decision processes (MDPs), enabling reinforcement learning with a standardized, extensible interface. It reduces the barrier to entry for researchers by providing a modular, scalable API compatible with modern ML frameworks, and demonstrates a 5–10% reduction in branch-and-bound tree size using RL for variable selection.

ABSTRACT

We present Ecole, a new library to simplify machine learning research for combinatorial optimization. Ecole exposes several key decision tasks arising in general-purpose combinatorial optimization solvers as control problems over Markov decision processes. Its interface mimics the popular OpenAI Gym library and is both extensible and intuitive to use. We aim at making this library a standardized platform that will lower the bar of entry and accelerate innovation in the field. Documentation and code can be found at https://www.ecole.ai.

Motivation & Objective

• To address reproducibility issues in machine learning for combinatorial optimization by standardizing benchmarks, features, and evaluation metrics.
• To lower the barrier to entry for researchers in operations research and machine learning by providing a unified, intuitive API for integrating ML with general-purpose solvers.
• To accelerate innovation at the intersection of ML and combinatorial optimization by enabling plug-and-play integration of state-of-the-art ML algorithms with industrial-strength solvers.
• To support the adoption of modern ML techniques in traditional optimization solvers by offering a modular, extensible platform compatible with existing ML workflows.

Proposed method

• Ecole exposes core decision tasks in branch-and-cut solvers—such as variable selection and hyperparameter tuning—as partially observable MDPs (PO-MDPs) with a Gym-like interface.
• It provides standardized environment classes (e.g., ecole.environment.Branching) that encapsulate solver interactions, state observation, and reward shaping.
• The library supports multiple observation functions, including graph-based representations (e.g., NodeBipartite) and aggregated feature vectors, to represent solver states.
• Reward functions are modular and configurable, with built-in support for metrics like number of nodes and LP iterations, enabling task-specific learning objectives.
• It integrates with popular ML frameworks via a clean Python API, allowing seamless training of policies using RL algorithms such as REINFORCE.
• The library is designed for extensibility, with support for new environments, observation functions, and reward functions, and is distributed under a BSD-3 license for broad adoption.

Experimental results

Research questions

• RQ1 Can a standardized, Gym-like interface improve reproducibility and comparability in machine learning research for combinatorial optimization solvers?
• RQ2 How can the integration of machine learning with existing general-purpose solvers be simplified for researchers from both ML and operations research communities?
• RQ3 To what extent can reinforcement learning improve solver performance on key decision tasks like variable selection in branch-and-bound?
• RQ4 Can a modular, extensible library accelerate the adoption of modern ML techniques in traditional optimization solver components?
• RQ5 How does the performance of learned policies compare to expert heuristics when applied to real-world problem instances?

Key findings

• Ecole enables researchers to define complex optimization learning environments with just a few lines of code, significantly reducing implementation complexity compared to direct solver APIs.
• A reinforcement learning policy trained on variable selection using the ecole.environment.Branching environment achieved a 5–10% reduction in branch-and-bound tree size on combinatorial auction instances.
• The library’s modular design allows for easy integration of new observation functions, reward functions, and solver environments, supporting future expansion to tasks like node selection and cut generation.
• The use of imitation learning to pre-train policies (e.g., mimicking strong branching) followed by policy optimization via REINFORCE led to effective and stable learning curves.
• Ecole’s compatibility with popular ML tools and package managers like conda enhances accessibility and adoption across research communities.
• The library supports reproducible research by standardizing problem generators, features, and evaluation metrics, reducing fragmentation in the field.