[Paper Review] An interior point method for nonlinear constrained derivative-free optimization

Andrea Brilli, Giampaolo Liuzzi+1 more|arXiv (Cornell University)|Aug 11, 2021

Advanced Optimization Algorithms Research4 citations

TL;DR

This paper proposes a derivative-free interior point method for nonlinearly constrained optimization where objective and constraint functions are black-box and non-relaxable. It combines a log-barrier approach for inequality constraints and a quadratic penalty for equality constraints within a merit function, proving convergence to KKT points under mild assumptions and demonstrating efficiency on standard test problems compared to state-of-the-art solvers.

ABSTRACT

In this paper we consider constrained optimization problems where both the objective and constraint functions are of the black-box type. Furthermore, we assume that the nonlinear inequality constraints are non-relaxable, i.e. their values and that of the objective function cannot be computed outside of the feasible region. This situation happens frequently in practice especially in the black-box setting where function values are typically computed by means of complex simulation programs which may fail to execute if the considered point is outside of the feasible region. For such problems, we propose a new derivative-free optimization method which is based on the use of a merit function that handles inequality constraints by means of a log-barrier approach and equality constraints by means of a quadratic penalty approach. We prove convergence of the proposed method to KKT stationary points of the problem under quite mild assumptions. Furthermore, we also carry out a preliminary numerical experience on standard test problems and comparison with a state-of-the-art solver which shows efficiency of the proposed method.

Motivation & Objective

• To address nonlinearly constrained derivative-free optimization problems where objective and constraint functions are black-box and non-relaxable, i.e., evaluations fail outside the feasible region.
• To develop a method that handles non-relaxable constraints without discontinuities by using interior penalization instead of extreme penalty approaches.
• To ensure convergence to KKT stationary points under mild assumptions, even when derivatives are unavailable.
• To design a practical algorithm that integrates log-barrier and penalty terms effectively in a derivative-free setting.

Proposed method

• Uses a merit function combining log-barrier terms for inequality constraints $ g(x) \leq 0 $ and quadratic penalty terms for equality constraints $ h(x) = 0 $, defined as $ P(x;\epsilon) = f(x) - \epsilon \sum \log[-g_j(x)] + \frac{1}{\epsilon} \sum |h_j(x)|^\nu $ with $ \nu > 1 $.
• Imposes the condition that $ P(x;\epsilon) = +\infty $ when $ g(x) \not< 0 $, ensuring unrelaxable constraints are never violated.
• Solves the subproblem $ \min P(x;\epsilon) $ subject to $ x \in \stackrel{\circ}{\mathcal{S}} \cap X $, where $ \stackrel{\circ}{\mathcal{S}} $ is the interior of the inequality constraint set.
• Employs a pattern search-based algorithm to minimize the merit function without requiring gradient information.
• Uses a line search strategy with a filter-type acceptance rule to ensure sufficient decrease in the merit function.
• Applies a trust-region-like framework with adaptive penalty parameter $ \epsilon_k $, decreasing it as the iterates approach optimality.

Experimental results

Research questions

• RQ1 Can a derivative-free interior point method effectively handle non-relaxable black-box constraints without introducing discontinuities or ill-conditioning?
• RQ2 Does combining log-barrier and quadratic penalty terms in a merit function lead to convergence to KKT points under mild assumptions when derivatives are unavailable?
• RQ3 How does the proposed method compare in practice to state-of-the-art derivative-free solvers for constrained black-box problems?
• RQ4 Can the algorithm maintain global convergence and avoid convergence to infeasible or suboptimal points in the presence of complex simulation-based constraints?

Key findings

• The proposed method converges to KKT stationary points of the problem under mild assumptions, even when derivatives are unavailable or untrustworthy.
• The sequences of Lagrange multipliers for inequality and equality constraints remain bounded along the sequence of iterates, ensuring stability.
• The convergence proof relies on the Mangasarian-Fromovitz constraint qualification (MFCQ) holding at the limit point, which is assumed to hold in the analysis.
• Numerical experiments on standard test problems show the method is competitive with state-of-the-art derivative-free solvers in terms of convergence quality and robustness.
• The method effectively avoids the ill-conditioning and discontinuities introduced by extreme penalty approaches, leading to more reliable convergence.
• The use of a log-barrier for non-relaxable constraints ensures that the merit function remains finite and smooth in the interior of the feasible region, enabling stable optimization.