[Paper Review] A feasible second order bundle algorithm for nonsmooth nonconvex optimization problems with inequality constraints: II. Implementation and numerical results

Hannes Fendl, Hermann Schichl|arXiv (Cornell University)|Jun 26, 2015

Advanced Optimization Algorithms Research20 references3 citations

TL;DR

This paper presents a practical implementation of a feasible second-order bundle algorithm for nonsmooth, nonconvex optimization with inequality constraints, solving a convex quadratically constrained quadratic program (QCQP) to compute search directions. It demonstrates superior performance over solvers like MPBNGC and SolvOpt, especially in high-dimensional problems and when many constraints are active at the solution.

ABSTRACT

This paper presents a concrete implementation of the feasible second order bundle algorithm for nonsmooth, nonconvex optimization problems with inequality constraints \cite{HannesPaperB}. It computes the search direction by solving a convex quadratically constrained quadratic program. Furthermore, certain versions of the search direction problem are discussed and the applicability of this approach is justified numerically by using different solvers for the computation of the search direction. Finally, the good performance of the second order bundle algorithm is demonstrated by comparison with test results of other solvers on examples of the Hock-Schittkowski collection, on custom examples that arise in the context of finding exclusion boxes for quadratic constraint satisfaction problems, and on higher dimensional piecewise quadratic examples.

Motivation & Objective

• Address the challenge of solving nonsmooth, nonconvex optimization problems with inequality constraints, where traditional gradient-based methods fail due to non-differentiability.
• Develop a reliable and efficient implementation of a second-order bundle method that maintains feasibility and uses second-order information for faster convergence.
• Ensure robust performance on difficult test problems, including those from the Hock-Schittkowski collection, quadratic constraint satisfaction problems, and high-dimensional piecewise quadratic examples.
• Justify the choice of QCQP-based search direction computation by comparing multiple solvers and demonstrating numerical stability and efficiency.

Proposed method

• Formulate the search direction computation as a convex quadratically constrained quadratic program (QCQP), enabling the use of second-order information.
• Use a sequential quadratic programming (SQP)-like approach to handle constraints, combined with quadratic approximation of the constraint function.
• Enforce strict feasibility by only accepting serious steps that remain within the feasible region, ensuring constraint satisfaction throughout the iteration.
• Implement a line search strategy to accept infeasible trial points when appropriate, differing from traditional bundle methods that reject infeasible iterates.
• Leverage strong duality in the QCQP subproblem to support meaningful termination criteria and global convergence guarantees.
• Integrate the algorithm with multiple QP solvers to validate the robustness and performance of the chosen formulation.

Experimental results

Research questions

• RQ1 Can a second-order bundle method with feasible iterates be effectively implemented for nonsmooth, nonconvex optimization with inequality constraints?
• RQ2 How does the choice of QCQP formulation for the search direction impact convergence and numerical performance compared to alternative approaches?
• RQ3 How does the algorithm perform in comparison to established solvers like MPBNGC and SolvOpt on benchmark and application-specific test problems?
• RQ4 Does the algorithm maintain superior performance in high-dimensional problems or when many constraints are active at the solution?
• RQ5 Can the algorithm reliably achieve convergence within limited function and evaluation budgets, as required in real-time or embedded applications?

Key findings

• The reduced algorithm outperforms MPBNGC and SolvOpt on the Hock-Schittkowski test set, with the reduced algorithm being significantly better in 20 out of 100 examples when m₂ = N.
• For high-dimensional piecewise quadratic problems with many active constraints, the algorithm shows superior convergence behavior and efficiency compared to other solvers.
• In 97% of the 100 difficult examples with m₂ = N, the reduced algorithm either significantly outperforms or matches SolvOpt in solution time, with only 5 examples where SolvOpt is significantly better.
• MPBNGC fails to converge within the iteration limit for 78 out of 100 examples with m₂ = N, while the reduced algorithm converges successfully in 22 examples and outperforms MPBNGC in 15 of them.
• The algorithm demonstrates robustness and scalability, with performance improving in higher dimensions and in problems with complex nonsmooth structures.
• The choice of QCQP-based search direction is numerically justified, as different solvers yield consistent and reliable results, supporting the methodological soundness of the approach.