[Paper Review] Operator Splitting for Conic Optimization via Homogeneous Self-Dual Embedding

Brendan O’Donoghue, Eric Chu+2 more|arXiv (Cornell University)|Dec 11, 2013

Advanced Optimization Algorithms Research30 citations

TL;DR

This paper proposes a first-order operator splitting method based on the homogeneous self-dual embedding to solve large-scale convex cone programs. By reformulating the problem as a feasibility problem and applying the alternating directions method of multipliers, the method simultaneously computes primal and dual solutions or certificates of infeasibility/unboundedness, is parameter-free, and scales efficiently to very large problems with lower per-iteration cost than traditional first-order methods.

ABSTRACT

We introduce a first order method for solving very large convex cone programs. The method uses an operator splitting method, the alternating directions method of multipliers, to solve the homogeneous self-dual embedding, an equivalent feasibility problem involving finding a nonzero point in the intersection of a subspace and a cone. This approach has several favorable properties. Compared to interior-point methods, first-order methods scale to very large problems, at the cost of requiring more time to reach very high accuracy. Compared to other first-order methods for cone programs, our approach finds both primal and dual solutions when available or a certificate of infeasibility or unboundedness otherwise, is parameter-free, and the per-iteration cost of the method is the same as applying a splitting method to the primal or dual alone. We discuss efficient implementation of the method in detail, including direct and indirect methods for computing projection onto the subspace, scaling the original problem data, and stopping criteria. We describe an open-source implementation, which handles the usual (symmetric) non-negative, second-order, and semidefinite cones as well as the (non-self-dual) exponential and power cones and their duals. We report numerical results that show speedups over interior-point cone solvers for large problems, and scaling to very large general cone programs.

Motivation & Objective

• To address the scalability limitations of interior-point methods for large-scale convex cone programs.
• To develop a first-order method that simultaneously provides primal and dual solutions or infeasibility/unboundedness certificates.
• To ensure the method is parameter-free and maintains low per-iteration computational cost.
• To enable efficient solution of problems involving non-negative, second-order, semidefinite, exponential, and power cones.

Proposed method

• The method reformulates the cone program as a homogeneous self-dual embedding, transforming it into a feasibility problem over the intersection of a subspace and a cone.
• It applies the alternating directions method of multipliers (ADMM) to solve the embedded feasibility problem.
• The algorithm computes projections onto the subspace using direct or iterative methods, depending on problem structure.
• Problem data are scaled to improve condition numbers and accelerate convergence.
• Stopping criteria are based on primal and dual infeasibility measures derived from the homogeneous self-dual solution.
• An open-source implementation supports symmetric cones (e.g., second-order, semidefinite) and non-self-dual cones (e.g., exponential, power) and their duals.

Experimental results

Research questions

• RQ1 Can a first-order method be designed to solve large-scale cone programs efficiently while maintaining accuracy and robustness?
• RQ2 How can a single algorithm simultaneously recover both primal and dual solutions or infeasibility/unboundedness certificates?
• RQ3 What is the impact of data scaling and projection computation on the convergence and performance of the method?
• RQ4 How does the method compare in practice to interior-point solvers on large-scale problems?
• RQ5 Can the method be efficiently implemented for non-self-dual cones such as exponential and power cones?

Key findings

• The method achieves significant speedups over interior-point solvers on large-scale cone programs, demonstrating superior scalability.
• It successfully computes both primal and dual solutions when feasible, or provides infeasibility/unboundedness certificates when applicable.
• The per-iteration cost is equivalent to applying a splitting method to the primal or dual alone, making it computationally efficient.
• The method is parameter-free, eliminating the need for tuning parameters such as step-sizes.
• Numerical results confirm robust performance across various cone types, including non-self-dual cones like exponential and power cones.
• The open-source implementation supports a broad range of cones and scales effectively to very large problems.