[Paper Review] QSDPNAL: A two-phase augmented Lagrangian method for convex quadratic semidefinite programming

Xudong Li, Defeng Sun+1 more|arXiv (Cornell University)|Dec 30, 2015

Advanced Optimization Algorithms Research30 references21 citations

TL;DR

This paper proposes QSDPNAL, a two-phase augmented Lagrangian method for large-scale convex quadratic semidefinite programming (QSDP). Phase I uses a first-order method with inexact Schur complement decomposition to achieve moderate accuracy or generate a good initial point; Phase II employs an inexact semismooth Newton method within an augmented Lagrangian framework, achieving R-superlinear convergence of the KKT residual and demonstrating high efficiency and robustness on large-scale problems.

ABSTRACT

In this paper, we present a two-phase augmented Lagrangian method, called QSDPNAL, for solving convex quadratic semidefinite programming (QSDP) problems with constraints consisting of a large number of linear equality, inequality constraints, a simple convex polyhedral set constraint, and a positive semidefinite cone constraint. A first order algorithm which relies on the inexact Schur complement based decomposition technique is developed in QSDPNAL-Phase I with the aim of solving a QSDP problem to moderate accuracy or using it to generate a reasonably good initial point for the second phase. In QSDPNAL-Phase II, we design an augmented Lagrangian method (ALM) where the inner subproblem in each iteration is solved via inexact semismooth Newton based algorithms. Simple and implementable stopping criteria are designed for the ALM. Moreover, under mild conditions, we are able to establish the rate of convergence of the proposed algorithm and prove the R-(super)linear convergence of the KKT residual. In the implementation of QSDPNAL, we also develop efficient techniques for solving large scale linear systems of equations under certain subspace constraints. More specifically, simpler and yet better conditioned linear systems are carefully designed to replace the original linear systems and novel shadow sequences are constructed to alleviate the numerical difficulties brought about by the crucial subspace constraints. Extensive numerical results for various large scale QSDPs show that our two-phase algorithm is highly efficient and robust in obtaining accurate solutions.

Motivation & Objective

• Address the lack of efficient solvers for large-scale convex quadratic semidefinite programming (QSDP) problems with high-dimensional constraints.
• Overcome the numerical and computational limitations of existing interior-point methods, which suffer from ill-conditioning and high per-iteration cost.
• Develop a scalable and robust algorithm capable of handling large-scale QSDP problems where storing the operator Q explicitly is infeasible.
• Design a two-phase framework that combines first-order and second-order methods to balance efficiency and accuracy.
• Establish global convergence and local R-superlinear convergence of the KKT residual under mild conditions, with implementable stopping criteria.

Proposed method

• Propose a two-phase augmented Lagrangian method: QSDPNAL-Phase I uses a first-order algorithm based on inexact Schur complement decomposition to solve the dual problem to moderate accuracy.
• In QSDPNAL-Phase II, apply an augmented Lagrangian method where inner subproblems are solved via inexact semismooth Newton methods, leveraging second-order information for fast local convergence.
• Design novel linear systems with improved conditioning by replacing original systems under subspace constraints, enhancing numerical stability.
• Construct novel shadow sequences to mitigate numerical difficulties arising from subspace constraints in the linear systems.
• Implement efficient iterative solvers for large-scale linear systems under subspace constraints, avoiding explicit matrix storage of the operator Q.
• Integrate simple, implementable stopping criteria for the augmented Lagrangian scheme, ensuring practical convergence monitoring.

Experimental results

Research questions

• RQ1 Can a two-phase augmented Lagrangian method effectively solve large-scale convex QSDP problems where standard interior-point methods fail due to ill-conditioning?
• RQ2 How can first-order and second-order methods be combined in a single framework to achieve both global convergence and fast local convergence?
• RQ3 What numerical techniques can be developed to efficiently solve large-scale linear systems under subspace constraints without explicitly forming the full operator Q?
• RQ4 Can implementable stopping criteria be designed for the augmented Lagrangian method that ensure convergence without excessive computational cost?
• RQ5 What is the convergence rate of the KKT residual in the proposed algorithm, and under what conditions can superlinear convergence be achieved?

Key findings

• QSDPNAL achieves R-superlinear convergence of the KKT residual under mild conditions, demonstrating fast local convergence in Phase II.
• The algorithm is highly efficient and robust, solving large-scale QSDP problems with up to 2003 variables and over 50,000 constraints in under 17 hours.
• For well-posed problems, QSDPNAL-Phase I alone often suffices, achieving moderate accuracy in under 30 minutes even for large instances.
• Phase II significantly outperforms Phase I on ill-conditioned or difficult problems, with solution times of up to 16.5 hours for high-accuracy solutions.
• The use of smarter, better-conditioned linear systems and shadow sequences reduces numerical errors and improves solver stability.
• Numerical results show that QSDPNAL consistently achieves feasibility and optimality gaps below 1e-6 on benchmark problems, including sensor network localization and SDP relaxations.