[Paper Review] DuQuad: an inexact (augmented) dual first order algorithm for quadratic programming

Ion Necoara, Andrei Pătraşcu|arXiv (Cornell University)|Apr 22, 2015

Advanced Optimization Algorithms Research7 references4 citations

TL;DR

This paper introduces DuQuad, a C-implementation of inexact dual first-order algorithms for convex quadratic programming, using augmented Lagrangian relaxation to handle complex constraints. It achieves low iteration complexity and memory usage by relying only on matrix-vector products, outperforming state-of-the-art solvers on certain problems due to efficient inexact gradient computation and dynamic Matlab integration for testing and analysis.

ABSTRACT

In this paper we present the solver DuQuad specialized for solving general convex quadratic problems arising in many engineering applications. When it is difficult to project on the primal feasible set, we use the (augmented) Lagrangian relaxation to handle the complicated constraints and then, we apply dual first order algorithms based on inexact dual gradient information for solving the corresponding dual problem. The iteration complexity analysis is based on two types of approximate primal solutions: the primal last iterate and an average of primal iterates. We provide computational complexity estimates on the primal suboptimality and feasibility violation of the generated approximate primal solutions. Then, these algorithms are implemented in the programming language C in DuQuad, and optimized for low iteration complexity and low memory footprint. DuQuad has a dynamic Matlab interface which make the process of testing, comparing, and analyzing the algorithms simple. The algorithms are implemented using only basic arithmetic and logical operations and are suitable to run on low cost hardware. It is shown that if an approximate solution is sufficient for a given application, there exists problems where some of the implemented algorithms obtain the solution faster than state-of-the-art commercial solvers.

Motivation & Objective

• To develop a lightweight, efficient solver for convex quadratic programs (QPs) suitable for low-cost hardware.
• To address the challenge of projecting onto complex primal feasible sets by using (augmented) Lagrangian relaxation.
• To implement dual first-order methods with inexact dual gradient information, ensuring convergence despite computational approximations.
• To minimize computational complexity and memory footprint by relying only on basic arithmetic and matrix-vector products.
• To provide a practical, open-source tool with a dynamic Matlab interface for easy benchmarking and algorithm analysis.

Proposed method

• Apply (augmented) Lagrangian relaxation to transform difficult primal constraints into dual subproblems with simple constraints.
• Use inexact dual gradient information in first-order dual algorithms, with convergence analysis based on primal last iterate and averaged primal iterates.
• Implement the algorithm in C using only basic arithmetic and matrix-vector multiplications to ensure low memory and computational cost.
• Integrate with Matlab via MEX files to enable dynamic testing, comparison, and performance analysis of the algorithms.
• Optimize for low iteration complexity and scalability to sparse QP problems through efficient sparse matrix-vector operations.
• Use gprof profiling to verify that matrix-vector products dominate execution time, confirming the algorithm's efficiency.

Experimental results

Research questions

• RQ1 Can inexact dual first-order methods with (augmented) Lagrangian relaxation achieve low iteration complexity and low memory usage for convex QPs?
• RQ2 How does the performance of inexact dual gradient methods compare to commercial solvers like Cplex and Gurobi on small-to-medium scale dense QPs?
• RQ3 What is the impact of using primal last iterate versus averaged primal iterates on convergence and suboptimality?
• RQ4 Can the algorithm maintain fast convergence and low memory usage on sparse QP problems using only sparse matrix-vector products?
• RQ5 How does the dynamic Matlab interface facilitate algorithm testing, comparison, and analysis in practice?

Key findings

• DuQuad achieves competitive performance, with DFGM showing behavior comparable to Cplex and Gurobi on QPs with equality constraints and positive semidefinite Hessian.
• The number of iterations for DGM and DFGM remains relatively stable across different problem dimensions, indicating mild dependence on problem size.
• Dual first-order methods perform better in terms of iteration count when using the primal last iterate rather than the average of iterates.
• Profiling confirms that matrix-vector products dominate execution time, validating the algorithm's design focus on these operations.
• The implementation uses only basic arithmetic and matrix-vector products, making it suitable for low-cost hardware and embedded systems.
• For certain problems, DuQuad obtains solutions faster than state-of-the-art commercial solvers, especially when approximate solutions suffice.