[Paper Review] Resolvent Splitting for Sums of Monotone Operators with Minimal Lifting

Yura Malitsky, Matthew K. Tam|arXiv (Cornell University)|Aug 6, 2021

Optimization and Variational Analysis18 references4 citations

TL;DR

This paper resolves the open problem of minimal lifting for frugal resolvent splitting algorithms solving the sum of $n \geq 2$ maximally monotone operators. It proves that the minimal lifting dimension is $d^*(n) = n-1$, establishes a new family of such algorithms with $n-1$-fold lifting, and applies them to decentralized optimization and multi-block ADMM, demonstrating convergence and practical utility.

ABSTRACT

In this work, we study fixed point algorithms for finding a zero in the sum of $n\geq 2$ maximally monotone operators by using their resolvents. More precisely, we consider the class of such algorithms where each resolvent is evaluated only once per iteration. For any algorithm from this class, we show that the underlying fixed point operator is necessarily defined on a $d$-fold Cartesian product space with $d\geq n-1$. Further, we show that this bound is unimprovable by providing a family of examples for which $d=n-1$ is attained. This family includes the Douglas-Rachford algorithm as the special case when $n=2$. Applications of the new family of algorithms in distributed decentralised optimisation and multi-block extensions of the alternation direction method of multipliers (ADMM) are discussed.

Motivation & Objective

• To resolve the open problem of determining the minimal lifting dimension $d^*(n)$ for frugal resolvent splitting algorithms solving the sum of $n \geq 2$ maximally monotone operators.
• To establish a theoretical lower bound of $d^*(n) \geq n-1$ for any such algorithm using techniques inspired by Ryu (2022).
• To construct a new family of frugal resolvent splittings that achieve $d^*(n) = n-1$, proving the bound is tight and unimprovable.
• To apply the new algorithmic framework to distributed decentralized optimization and multi-block ADMM, enabling new schemes without a central coordinator.
• To provide convergence analysis and numerical validation for the proposed methods in structured optimization settings.

Proposed method

• Derives a theoretical lower bound on lifting dimension using fixed point operator analysis and properties of monotone operators, showing $d^*(n) \geq n-1$ for $n \geq 2$.
• Constructs a new family of frugal resolvent splittings that operate on the $(n-1)$-fold Cartesian product space $\mathcal{H}^{n-1}$, evaluating each resolvent exactly once per iteration.
• Designs a novel decentralized optimization scheme that avoids a central coordinator by using the new splitting framework, relying only on local resolvent evaluations.
• Develops a multi-block extension of ADMM based on the new splitting, with updates structured to allow distributed computation across $n$ blocks.
• Employs a fixed-point iteration framework where the operator is defined on $\mathcal{H}^{n-1}$, with updates involving vector additions, scalar multiplications, and resolvents $J_{A_i}$.
• Proves convergence of the new algorithms under standard assumptions, including maximal monotonicity and feasibility of the problem.

Experimental results

Research questions

• RQ1 What is the minimal lifting dimension $d^*(n)$ for frugal resolvent splitting algorithms solving the sum of $n \geq 2$ maximally monotone operators?
• RQ2 Can the theoretical lower bound $d^*(n) \geq n-1$ be achieved, and if so, does there exist a constructive algorithm achieving it?
• RQ3 Can the new family of $n$-operator resolvent splittings be applied to decentralized optimization without a central coordinator?
• RQ4 Can the framework be extended to a multi-block ADMM variant that avoids the product space reformulation and supports distributed computation?
• RQ5 How do the new algorithms compare in convergence behavior and iteration complexity to existing methods like PDHG and two-block ADMM?

Key findings

• The minimal lifting dimension for frugal resolvent splitting of $n \geq 2$ maximally monotone operators is proven to be $d^*(n) = n-1$, resolving an open problem.
• The theoretical lower bound $d^*(n) \geq n-1$ is established using fixed point operator analysis and operator monotonicity properties.
• A new family of frugal resolvent splittings is constructed that achieves $n-1$-fold lifting, proving the bound is tight and unimprovable.
• The new algorithms enable a decentralized optimization scheme that does not require a central coordinator, relying only on local resolvent evaluations.
• A multi-block ADMM extension is proposed that generalizes the two-block method and avoids the standard product space reformulation, with convergence proven.
• Numerical experiments on matrix decomposition problems show that the new multi-block ADMM performs comparably to existing methods like ASALM in terms of solution quality, though with potentially slower convergence per iteration.