[Paper Review] Optimal Decentralized Distributed Algorithms for Stochastic Convex Optimization

Eduard Gorbunov, Darina Dvinskikh+1 more|arXiv (Cornell University)|Nov 17, 2019

Stochastic Gradient Optimization Techniques93 references34 citations

TL;DR

This paper proposes novel decentralized distributed algorithms for stochastic convex optimization with affine constraints, leveraging primal and dual approaches using inexact proximal steps and gradient sliding. It establishes optimal convergence rates in terms of communication rounds and oracle calls, achieving O(√L/μ log(1/ε)) iterations and eO(max{√L/μ, σ²/ε}) oracle calls for ε-accuracy, with tight bounds under biased and unbiased stochastic oracles.

ABSTRACT

We consider stochastic convex optimization problems with affine constraints and develop several methods using either primal or dual approach to solve it. In the primal case we use special penalization technique to make the initial problem more convenient for using optimization methods. We propose algorithms to solve it based on Similar Triangles Method with Inexact Proximal Step for the convex smooth and strongly convex smooth objective functions and methods based on Gradient Sliding algorithm to solve the same problems in the non-smooth case. We prove the convergence guarantees in smooth convex case with deterministic first-order oracle. We propose and analyze three novel methods to handle stochastic convex optimization problems with affine constraints: SPDSTM, R-RRMA-AC-SA and SSTM_sc. All methods use stochastic dual oracle. SPDSTM is the stochastic primal-dual modification of STM and it is applied for the dual problem when the primal functional is strongly convex and Lipschitz continuous on some ball. R-RRMA-AC-SA is an accelerated stochastic method based on the restarts of RRMA-AC-SA and SSTM_sc is just stochastic STM for strongly convex problems. Both methods are applied to the dual problem when the primal functional is strongly convex, smooth and Lipschitz continuous on some ball and use stochastic dual first-order oracle. We develop convergence analysis for these methods for the unbiased and biased oracles respectively. Finally, we apply all aforementioned results and approaches to solve decentralized distributed optimization problem and discuss optimality of the obtained results in terms of communication rounds and number of oracle calls per node.

Motivation & Objective

• To develop optimal decentralized distributed algorithms for stochastic convex optimization under affine constraints.
• To bridge the gap between theoretical convergence rates and practical communication efficiency in decentralized settings.
• To analyze convergence under both biased and unbiased stochastic oracles, particularly in decentralized networks.
• To establish tight bounds on communication rounds and oracle calls for smooth and strongly convex problems.
• To extend accelerated methods to primal-dual frameworks with inexact oracles and biased stochastic gradients.

Proposed method

• Proposes a primal approach using a penalization technique to reformulate the problem for efficient optimization with Similar Triangles Method (STM) and inexact proximal steps.
• Introduces gradient sliding algorithms for non-smooth cases, enabling efficient handling of stochastic subgradients.
• Develops three novel methods: SPDSTM, SSTM sc, and R-RRMA-AC-SA2, tailored for biased and unbiased stochastic oracles.
• Applies dual approach with strongly convex dual functions and restart techniques to accelerate convergence.
• Uses direct acceleration for dual problems and proves convergence via Lyapunov functions and stochastic error bounds.
• Employs variance reduction and inexact oracle analysis to derive tight bounds on iteration and oracle complexity.

Experimental results

Research questions

• RQ1 What is the optimal communication complexity for decentralized stochastic convex optimization with affine constraints?
• RQ2 How can accelerated primal-dual methods be designed for biased stochastic oracles in decentralized networks?
• RQ3 What is the trade-off between oracle calls and communication rounds in decentralized stochastic optimization?
• RQ4 Can inexact proximal steps and gradient sliding be combined to achieve optimal convergence in non-smooth settings?
• RQ5 How do biased and unbiased stochastic oracles affect convergence rates in decentralized dual methods?

Key findings

• The proposed R-RRMA-AC-SA2 achieves optimal convergence with O(√L/μ log(1/ε)) communication rounds for ε-accuracy in the unbiased case.
• For biased oracles (SPDSTM, SSTM sc), the method attains O(√L/μ log(1/ε)) convergence with eO(max{√L/μ, σ²/ε}) oracle calls.
• The algorithmic framework ensures that f(˜xN) − f(x*) ≤ (2 + √(2C/λmax(A⊤A) + G1))ε/Ry with high probability.
• The method achieves ∥A˜xN∥₂ ≤ (1 + √(2C) + G1/√λmax(A⊤A))ε/Ry with probability at least 1 − 4β.
• The analysis proves that the total number of oracle calls is eO(max{√L/μ, σ²/ε}) for ε-accuracy, matching known lower bounds.
• The convergence rate is optimal in terms of communication rounds and oracle complexity, with tight bounds derived via stochastic Lyapunov functions and inexact oracle analysis.