[Paper Review] Differentiable Convex Optimization Layers

Akshay Agrawal, Brandon Amos+4 more|arXiv (Cornell University)|Oct 28, 2019

Stochastic Gradient Optimization TechniquesComputer Science132 citations

TL;DR

This paper introduces disciplined parametrized programming (DPP) and affine-solver-affine (ASA) form to enable analytic differentiation through disciplined convex programs, implemented in CVXPY 1.1 with differentiable layers for PyTorch and TensorFlow 2.0, and demonstrates applications in learning and control.

ABSTRACT

Recent work has shown how to embed differentiable optimization problems (that is, problems whose solutions can be backpropagated through) as layers within deep learning architectures. This method provides a useful inductive bias for certain problems, but existing software for differentiable optimization layers is rigid and difficult to apply to new settings. In this paper, we propose an approach to differentiating through disciplined convex programs, a subclass of convex optimization problems used by domain-specific languages (DSLs) for convex optimization. We introduce disciplined parametrized programming, a subset of disciplined convex programming, and we show that every disciplined parametrized program can be represented as the composition of an affine map from parameters to problem data, a solver, and an affine map from the solver's solution to a solution of the original problem (a new form we refer to as affine-solver-affine form). We then demonstrate how to efficiently differentiate through each of these components, allowing for end-to-end analytical differentiation through the entire convex program. We implement our methodology in version 1.1 of CVXPY, a popular Python-embedded DSL for convex optimization, and additionally implement differentiable layers for disciplined convex programs in PyTorch and TensorFlow 2.0. Our implementation significantly lowers the barrier to using convex optimization problems in differentiable programs. We present applications in linear machine learning models and in stochastic control, and we show that our layer is competitive (in execution time) compared to specialized differentiable solvers from past work.

Motivation & Objective

• Provide a framework to differentiate through disciplined convex programs (DCP-compliant) in a way that is friendly to DSLs for convex optimization.
• Introduce disciplined parametrized programming (DPP) and ASA form to ensure parameter-to-problem data mapping is affine.
• Implement DPP/ASA in CVXPY 1.1 and create differentiable layers for PyTorch and TensorFlow 2.0.
• Demonstrate applications in sensitivity analysis for linear models and learning control-Lyapunov policies in stochastic control.
• Show competitive runtime compared to specialized differentiable solvers for quadratic programs.

Proposed method

• Define disciplined parametrized programming (DPP) as a grammar for parametrized disciplined convex programs with parameter-affine structure.
• Canonicalize DPP to affine-solver-affine (ASA) form, where problem data mapping is affine and differentiable.
• Express the derivative of the solution map via D^T S(θ) = D^T C(θ) D^T s(A,b,c) D^T R( x̃* ), using a conic solver for the ASA cone program.
• Represent the canonicalizer C as a sparse matrix Q and a sparse tensor R to enable efficient forward/backward passes.
• Implement the approach in CVXPY 1.1, and provide differentiable convex optimization layers in PyTorch and TensorFlow 2.0, leveraging diffcp for cone-program derivatives.
• Provide examples and compare runtime with specialized solvers (e.g., qpth) on dense and sparse QPs.

Experimental results

Research questions

• RQ1 How can disciplined convex programs be differentiated end-to-end when parameterized?
• RQ2 Can DPP and ASA form allow efficient, automatic differentiation through convex optimization layers without backpropagating through canonicalization?
• RQ3 What is the practical performance of differentiable convex layers in mainstream ML frameworks compared to specialized solvers?
• RQ4 Can this approach enable new end-to-end learning applications in sensitivity analysis and stochastic control?

Key findings

• The paper introduces DPP and ASA form, enabling affine mappings from parameters to problem data and from solver output to the original problem solution.
• An implementation in CVXPY 1.1 plus differentiable layers for PyTorch and TensorFlow 2.0 is provided, with batched support.
• Canonicalization is accelerated by representing the canonicalizer as sparse matrices, enabling efficient gradient computation.
• Experiments demonstrate applications in data poisoning sensitivity analysis and convex approximate dynamic programming for stochastic control.
• For quadratic programs, the proposed layer is competitive with specialized solvers like qpth in dense settings and faster in sparse settings.
• The approach significantly reduces canonicalization time compared to previous methods.
• The framework is applicable to a wide range of convex problems solvable by standard DSLs like CVX, CVXPY, Convex.jl, and CVXR.