[Paper Review] Solving the Optimal Experiment Design Problem with Mixed-Integer Convex Methods

Deborah Hendrych, Mathieu Besançon+1 more|arXiv (Cornell University)|Dec 18, 2023

Machine Learning and Algorithms25 references2 citations

TL;DR

This paper proposes Boscia.jl, a novel mixed-integer convex optimization framework for solving the Optimal Experiment Design Problem (OEDP) using a nonlinear branch-and-bound algorithm with Frank-Wolfe-based node relaxations. It outperforms generic and specialized MINLP solvers, especially on large-scale OEDP instances, by preserving the original problem structure and efficiently exploiting the polytope of constraints.

ABSTRACT

We tackle the Optimal Experiment Design Problem, which consists of choosing experiments to run or observations to select from a finite set to estimate the parameters of a system. The objective is to maximize some measure of information gained about the system from the observations, leading to a convex integer optimization problem. We leverage Boscia.jl, a recent algorithmic framework, which is based on a nonlinear branch-and-bound algorithm with node relaxations solved to approximate optimality using Frank-Wolfe algorithms. One particular advantage of the method is its efficient utilization of the polytope formed by the original constraints which is preserved by the method, unlike alternative methods relying on epigraph-based formulations. We assess the method against both generic and specialized convex mixed-integer approaches. Computational results highlight the performance of the proposed method, especially on large and challenging instances.

Motivation & Objective

• To address the computational challenge of solving large-scale Optimal Experiment Design Problems (OEDP) with mixed-integer nonlinear programming (MINLP).
• To develop a flexible and efficient framework that preserves the original problem structure without reformulation into epigraph or conic forms.
• To evaluate and compare the performance of Boscia.jl against generic and specialized MINLP solvers on diverse OEDP formulations.
• To establish convergence guarantees for Frank-Wolfe algorithms on continuous relaxations of OEDP problems under smoothness and self-concordance assumptions.

Proposed method

• Employs a nonlinear branch-and-bound framework with continuous relaxations solved via the Frank-Wolfe algorithm.
• Uses a problem formulation that optimizes a nonlinear information function over a truncated, scaled probability simplex intersected with integer constraints.
• Preserves the original problem structure by avoiding epigraph or conic reformulations, maintaining the geometric properties of the constraint polytope.
• Leverages L-smoothness and generalized self-concordance of information functions to ensure convergence of the Frank-Wolfe subroutines.
• Integrates a combinatorial solver at each node to efficiently generate integer-feasible solutions.
• Supports a wide range of information criteria (A, D, G, V, and general p-norms) without requiring problem-specific reformulations.

Experimental results

Research questions

• RQ1 Can a nonlinear branch-and-bound approach with Frank-Wolfe-based relaxations outperform generic and specialized MINLP solvers on large-scale OEDP instances?
• RQ2 Does preserving the original problem structure—particularly the constraint polytope—lead to superior performance compared to epigraph-based reformulations?
• RQ3 To what extent does the Boscia.jl framework maintain efficiency and convergence when applied to diverse information criteria beyond A- and D-optimality?
• RQ4 How do different objective formulations (e.g., log-transformed or non-log) affect the performance and gap convergence of the solver on large instances?
• RQ5 What is the scalability and robustness of Boscia.jl under correlated and independent data settings in fusion and optimal design problems?

Key findings

• Boscia.jl solved 90% of the GTI-Fusion problems with independent data and p=0.25, achieving a geometric mean solve time of 6.57 seconds, significantly outperforming Co-BnB and SCIP.
• On large-scale D-Fusion problems with m=500, Boscia.jl achieved a relative gap of 0.0169 and absolute gap of 0.01 under independent data, while Co-BnB had an infinite gap on unsolved instances.
• For the A-Fusion problem with n=25 and correlated data, Boscia.jl reduced the relative gap to 1.0491 and absolute gap to 2.6105, outperforming Co-BnB’s 0.9264 and 1.0258 on the same metric.
• Boscia.jl with log-Tr objective achieved 100% success rate on GTI-Fusion with p=0.25 and independent data, solving all 30 instances with a geometric mean time of 1.71 seconds.
• The framework demonstrated superior convergence behavior, with the absolute gap decreasing steadily over time in all tested instances, as shown in Figure 7.
• Boscia.jl solved 3 instances of the D-Fusion problem with independent data that were not solved by Co-BnB within the time limit, highlighting its robustness on difficult instances.