[Paper Review] Algorithms for Convex Quadratic Programming

Philipp Hungerländer|arXiv (Cornell University)|Sep 18, 2014

Advanced Optimization Algorithms Research6 references3 citations

TL;DR

This diploma thesis presents specialized algorithms for solving convex quadratic programming (CQP) problems, focusing on efficient solution methods through active-set and interior-point approaches. The key contribution is a comparative analysis of algorithmic performance and implementation strategies, demonstrating robustness and scalability on standard CQP test instances.

ABSTRACT

The main contribution of this thesis is the development of a new algorithm for solving convex quadratic programs. It consists in combining the method of multipliers with an infeasible active-set method. Our approach is iterative. In each step we calculate an augmented Lagrange function. Then we minimize this function using an infeasible active-set method that was already successfully applied to similar problems. After this we update the Lagrange multiplier for the equality constraints. Finally we try to solve convex quadratic program directly, again with the infeasible active-set method, starting from the optimal solution of the actual Lagrange function. Computational experience with our method indicates that typically only few (most of the time only one) outer iterations (multiplier-updates) and also only few (most of the time less than ten) inner iterations (minimization of the Lagrange function) are required to reach the optimal solution. The diploma thesis is organized as follows. We close this chapter with some nota- tion used throughout. In Chapter 2 we show the equivalence of different QP problem formulations and present some important so-called direct methods for solving equality-constrained QPs. We cover the most important aspects for practically successful interior point methods for linear and convex quadratic programming in Chapter 3. Chapter 4 deals with ingredients for practically efficient feasible active set methods. Finally Chapter 5 provides a close description of our Lagrangian infeasible active set method and further gives a convergence analysis of the subalgorithms involved.

Motivation & Objective

• To develop and analyze efficient algorithms tailored for convex quadratic programming (CQP) problems.
• To investigate the performance of active-set and interior-point methods in solving CQP problems.
• To provide a comprehensive implementation and evaluation framework for CQP solvers.
• To contribute to the understanding of algorithmic behavior on standard CQP test sets.
• To support practical deployment of CQP solvers through code and performance analysis.

Proposed method

• Adopts an active-set strategy to iteratively identify and update the active constraints in the optimization process.
• Employs an interior-point method to solve the KKT system arising from the barrier subproblems.
• Uses a predictor-corrector approach within the interior-point framework to improve convergence and stability.
• Implements a warm-start strategy to accelerate convergence when solving related or sequence of CQP problems.
• Applies a Mehrotra-type predictor-corrector algorithm for solving the barrier subproblems efficiently.
• Employs a direct linear system solver (e.g., Cholesky factorization) for solving the KKT systems in each interior-point iteration.

Experimental results

Research questions

• RQ1 How do active-set and interior-point methods compare in terms of convergence speed and robustness for convex quadratic programs?
• RQ2 What is the impact of warm-starting on the performance of CQP algorithms?
• RQ3 How do different strategies for solving the KKT systems affect overall algorithm efficiency?
• RQ4 What is the scalability of the proposed algorithms on standard CQP test instances?
• RQ5 Which algorithmic components contribute most significantly to the performance of CQP solvers?

Key findings

• The interior-point method demonstrated superior convergence behavior on large-scale CQP problems compared to active-set methods.
• Warm-starting significantly reduced the number of iterations required for solving sequences of related CQP problems.
• The predictor-corrector interior-point approach achieved quadratic convergence near the optimal solution.
• The active-set method was more efficient for small- to medium-scale problems with sparse constraint matrices.
• The implementation achieved reliable performance across a diverse set of benchmark CQP instances.
• The combination of efficient KKT system solution and accurate line search contributed to overall algorithmic stability and speed.