[Paper Review] Nonlinear Programming Solvers for Unconstrained and Constrained Optimization Problems: a Benchmark Analysis

Giovanni Lavezzi, Kidus Guye+1 more|arXiv (Cornell University)|Apr 11, 2022

Advanced Optimization Algorithms Research39 references4 citations

TL;DR

This paper benchmarks 23 nonlinear programming (NLP) solvers across 60 unconstrained and constrained benchmark problems using MATLAB, evaluating accuracy, convergence rate, and speed. Key findings show BARON delivers the highest accuracy and convergence rate but is slowest, while PENLAB offers the best balance of performance; FMINCON, KNITRO, SNOPT, and MIDACO provide strong trade-offs for practical applications.

ABSTRACT

In this paper we propose a set of guidelines to select a solver for the solution of nonlinear programming problems. With this in mind, we present a comparison of the convergence performances of commonly used solvers for both unconstrained and constrained nonlinear programming problems. The comparison involves accuracy, convergence rate, and convergence speed. Because of its popularity among research teams in academia and industry, MATLAB is used as common implementation platform for the solvers. Our study includes solvers which are either freely available, or require a license, or are fully described in literature. In addition, we differentiate solvers if they allow the selection of different optimal search methods. As result, we examine the performances of 23 algorithms to solve 60 benchmark problems. To enrich our analysis, we will describe how, and to what extent, convergence speed and accuracy can be improved by changing the inner settings of each solver.

Motivation & Objective

• To provide a systematic comparison of NLP solvers for both unconstrained and constrained optimization problems.
• To guide researchers and practitioners in selecting optimal solvers based on accuracy, convergence rate, and computational speed.
• To evaluate the impact of solver settings on performance, particularly for high-accuracy and fast-solution configurations.
• To establish a standardized benchmark using standard NLP test problems with up to 30 variables and 9 constraints.
• To support solver selection in real-time and high-performance applications by analyzing trade-offs across popular MATLAB-based solvers.

Proposed method

• The study evaluates 23 solvers, including MATLAB’s built-in solvers (e.g., fmincon, fminunc), gradient-based methods, and metaheuristics (e.g., APSO), all implemented in MATLAB.
• Benchmark problems are drawn from standard collections (Hedar, Schittkowski, Floudas & Pardalos) with up to 30 variables and 9 constraints.
• Performance is measured using four metrics: mean error (E), standard deviation of error (σ), convergence rate (γ), and average CPU time (CPU).
• Two solver configurations are tested: high-accuracy (HA) and quick-solution (QS) settings, to assess sensitivity to internal parameter tuning.
• Solvers are ranked based on mean error (for HA) and CPU time (for QS), with statistical comparison across 60 test problems.
• Results are presented in detailed tables comparing solver performance across all 60 benchmark problems under both settings.

Experimental results

Research questions

• RQ1 Which NLP solver achieves the highest accuracy and fastest convergence rate across unconstrained and constrained benchmark problems?
• RQ2 How do different solver configurations (e.g., BARON’s various modes, fmincon’s algorithms) affect convergence speed and accuracy?
• RQ3 What is the trade-off between solution accuracy and computational speed among widely used MATLAB-based solvers?
• RQ4 Which solvers offer the best balance of accuracy, convergence rate, and speed for real-time or high-performance optimization tasks?
• RQ5 To what extent can convergence performance be improved by tuning solver-specific internal settings?

Key findings

• BARON consistently achieved the highest accuracy (mean error < 2.1e-3) and best convergence rate (92.0% success rate) across all constrained problems, particularly in high-accuracy (HA) mode.
• PENLAB delivered the best overall balance of accuracy (mean error 1.127e-4), convergence rate (31.0%), and speed (0.0379s average CPU time), outperforming others in the HA setting.
• FMINCON, KNITRO, SNOPT, and MIDACO provided strong trade-offs between accuracy, convergence rate, and speed, with SNOPT being the fastest (0.0027s CPU time) in the QS setting.
• In the high-accuracy setting, BARON (auto) achieved a mean error of 2.054e-3 with near-zero standard deviation (4.556e-17), indicating near-perfect consistency.
• The particle swarm optimizer (APSO) showed poor accuracy (mean error 1.173e+0) and was significantly slower than gradient-based solvers, especially in HA mode.
• Tuning solver settings significantly improved performance: for example, BARON’s performance remained excellent across all modes, while FMINCON’s SQP method achieved faster convergence (0.0071s) than its active-set variant (0.0165s) in the QS setting.