[Paper Review] Interior-point algorithms for convex optimization based on primal-dual metrics

Tor G. J. Myklebust, Levent Tunçel|arXiv (Cornell University)|Nov 8, 2014

Advanced Optimization Algorithms Research39 references6 citations

TL;DR

This paper introduces a new primal-dual interior-point algorithm for convex optimization using a variable metric based on a line integral of the Hessian of a logarithmically homogeneous self-concordant barrier. It achieves the best-known iteration complexity of $ O( heta^{1/2} an^{-1}(1/ heta)) $ for general convex cones, matching the complexity of symmetric cone programming, and establishes connections to Riemannian geometry and Gaussian quadrature for efficient computation.

ABSTRACT

We propose and analyse primal-dual interior-point algorithms for convex optimization problems in conic form. The families of algorithms we analyse are so-called short-step algorithms and they match the current best iteration complexity bounds for primal-dual symmetric interior-point algorithm of Nesterov and Todd, for symmetric cone programming problems with given self-scaled barriers. Our results apply to any self-concordant barrier for any convex cone. We also prove that certain specializations of our algorithms to hyperbolic cone programming problems (which lie strictly between symmetric cone programming and general convex optimization problems in terms of generality) can take advantage of the favourable special structure of hyperbolic barriers. We make new connections to Riemannian geometry, integrals over operator spaces, Gaussian quadrature, and strengthen the connection of our algorithms to quasi-Newton updates and hence first-order methods in general.

Motivation & Objective

• To develop primal-dual interior-point algorithms for general convex optimization problems with arbitrary convex cones, extending results from symmetric cone programming.
• To introduce a new primal-dual scaling map based on a line integral of the Hessian of a logarithmically homogeneous self-concordant barrier (LHSCB), enabling a variable metric framework.
• To prove that the proposed algorithm achieves the best-known worst-case iteration complexity of $ O( heta^{1/2} an^{-1}(1/ heta)) $, matching the state-of-the-art for symmetric cone programming.
• To establish new mathematical connections between interior-point methods, Riemannian geometry, operator integrals, and Gaussian quadrature rules.
• To design efficient approximation schemes for the new scaling map using Gaussian quadrature, particularly for hyperbolic barriers, with bounded error.

Proposed method

• The algorithm uses a predictor-corrector scheme based on Newton directions derived from a primal-dual variable metric defined via the Hessian of a logarithmically homogeneous self-concordant barrier (LHSCB).
• The primal-dual scaling map is defined as a line integral of the Hessian along a path from the current iterate to the central path, forming a Riemannian metric that adapts to local curvature.
• The method employs low-rank updates to approximate the integral scaling map, with error bounds derived from self-concordance and operator norm control.
• For hyperbolic barriers, the paper uses Gaussian quadrature rules (e.g., Gauss-Legendre) to approximate the integral scaling map, with error bounds depending on the order of the quadrature and the norm of the search direction.
• The algorithm is analyzed under the assumption of a self-concordant barrier, leveraging properties such as the Dikin ellipsoid containment and Hessian Lipschitz continuity.
• The analysis connects the new metric to quasi-Newton updates and first-order methods, strengthening the link between interior-point and quasi-Newton frameworks.

Experimental results

Research questions

• RQ1 Can primal-dual interior-point methods for general convex cones achieve the same iteration complexity as those for symmetric cones, using only a self-concordant barrier?
• RQ2 How can a variable metric based on the Hessian of a barrier function be constructed and efficiently approximated for use in interior-point algorithms?
• RQ3 What is the approximation error of using Gaussian quadrature to compute the integral-based primal-dual scaling map, and how does it affect convergence?
• RQ4 How do the new scaling maps and metrics relate to Riemannian geometry and operator theory in the context of convex optimization?
• RQ5 Can low-rank updates to the integral scaling map preserve convergence properties and iteration complexity when the initial approximation is close to the central path?

Key findings

• The proposed predictor-corrector algorithm achieves an iteration complexity of $ O( heta^{1/2} an^{-1}(1/ heta)) $, matching the best-known bound for symmetric cone programming, where $ heta $ is the barrier parameter.
• The new primal-dual scaling map, defined as a line integral of the Hessian, provides a variable metric that generalizes the Dikin ellipsoid and enriches the connection to Riemannian geometry.
• For hyperbolic barriers, the integral scaling map can be approximated using Gauss-Legendre quadrature with an error bound of the form $ rac{1}{(1 - orm{ abla_P}_{x})^2} orm{ abla_P}_{x}^{2k} $, where $ k $ is the quadrature order.
• Low-rank updates to the integral scaling map maintain small norm when near the central path, provided the initial approximation is sufficiently accurate.
• The paper establishes that the conjugate barrier can be computed via an oracle for the primal barrier, without degrading the worst-case iteration complexity in infeasible-start or self-dual embedding methods.
• The analysis reveals a deep connection between the new metric and quasi-Newton updates, suggesting that interior-point methods can be interpreted as adaptive quasi-Newton methods with curvature-based steps.