[Paper Review] Distributed Optimization Based on Gradient-tracking Revisited: Enhancing Convergence Rate via Surrogation

Ying Sun, Amir Daneshmand+1 more|arXiv (Cornell University)|May 7, 2019

Distributed Control Multi-Agent Systems4 citations

TL;DR

This paper proposes a novel distributed optimization algorithm, Surrogate-SONATA, that enhances convergence rates in multi-agent systems by using high-order surrogate functions instead of standard first-order methods. By leveraging gradient tracking with perturbed push-sum consensus and exploiting statistical similarity among agents' local losses, the method achieves linear convergence with improved dependence on condition number and network connectivity, outperforming traditional schemes without requiring Hessian exchange.

ABSTRACT

We study distributed multiagent optimization over (directed, time-varying) graphs. We consider the minimization of $F+G$ subject to convex constraints, where $F$ is the smooth strongly convex sum of the agent's losses and $G$ is a nonsmooth convex function. We build on the SONATA algorithm: the algorithm employs the use of surrogate objective functions in the agents' subproblems (going thus beyond linearization, such as proximal-gradient) coupled with a perturbed (push-sum) consensus mechanism that aims to track locally the gradient of $F$. SONATA achieves precision $ε>0$ on the objective value in $\mathcal{O}(κ_g \log(1/ε))$ gradient computations at each node and $ ilde{\mathcal{O}}\big(κ_g (1-ρ)^{-1/2} \log(1/ε)\big)$ communication steps, where $κ_g$ is the condition number of $F$ and $ρ$ characterizes the connectivity of the network. This is the first linear rate result for distributed composite optimization; it also improves on existing (non-accelerated) schemes just minimizing $F$, whose rate depends on much larger quantities than $κ_g$ (e.g., the worst-case condition number among the agents). When considering in particular empirical risk minimization problems with statistically similar data across the agents, SONATA employing high-order surrogates achieves precision $ε>0$ in $\mathcal{O}\big((β/μ) \log(1/ε)\big)$ iterations and $ ilde{\mathcal{O}}\big((β/μ) (1-ρ)^{-1/2} \log(1/ε)\big)$ communication steps, where $β$ measures the degree of similarity of the agents' losses and $μ$ is the strong convexity constant of $F$. Therefore, when $β/μ< κ_g$, the use of high-order surrogates yields provably faster rates than what achievable by first-order models; this is without exchanging any Hessian matrix over the network.

Motivation & Objective

• To address the challenge of designing communication-efficient distributed optimization algorithms for multi-agent systems with time-varying, directed communication graphs.
• To improve convergence rates in distributed composite optimization—minimizing a sum of smooth strongly convex functions and a nonsmooth convex regularizer—beyond existing first-order methods.
• To exploit statistical similarity among agents' local loss functions (e.g., in i.i.d. data settings) to achieve faster convergence without exchanging Hessian matrices.
• To develop a gradient-tracking-based algorithm that achieves linear convergence with improved dependence on network connectivity and problem condition number.

Proposed method

• The algorithm uses surrogate functions in agent subproblems, replacing standard linearization (e.g., proximal-gradient), enabling higher-order approximation of local loss functions.
• It employs a perturbed push-sum consensus mechanism to track the global gradient of the smooth component $ F $, ensuring accurate gradient estimation across the network.
• The method introduces a novel analysis framework that bounds the convergence rate in terms of the condition number $ \kappa_g $ of $ F $, the strong convexity constant $ \mu $, and the similarity measure $ \beta $ of local losses.
• It achieves linear convergence by carefully balancing step size $ \alpha $, network connectivity $ \rho $, and surrogate approximation quality, with convergence rate dependent on $ \beta/\mu $ in similar-data regimes.
• The algorithm is designed for directed and time-varying graphs, ensuring robustness to network dynamics while maintaining convergence guarantees.
• Theoretical analysis establishes convergence in $ \mathcal{O}(\kappa_g \log(1/\epsilon)) $ gradient computations and $ \tilde{\mathcal{O}}(\kappa_g (1-\rho)^{-1/2} \log(1/\epsilon)) $ communication steps for general cases, and improved rates when $ \beta/\mu < \kappa_g $.

Experimental results

Research questions

• RQ1 Can surrogate functions in distributed optimization improve convergence rates beyond first-order methods without transmitting Hessian matrices?
• RQ2 How does statistical similarity among agents’ local losses affect the convergence rate of distributed optimization algorithms?
• RQ3 Can gradient tracking with perturbed push-sum consensus achieve linear convergence in directed, time-varying networks for composite optimization problems?
• RQ4 What is the dependence of convergence rate on the condition number $ \kappa_g $, network connectivity $ \rho $, and loss similarity $ \beta $?
• RQ5 Is it possible to achieve faster convergence than first-order methods when $ \beta/\mu < \kappa_g $, even without Hessian exchange?

Key findings

• The proposed Surrogate-SONATA algorithm achieves linear convergence with $ \mathcal{O}(\kappa_g \log(1/\epsilon)) $ gradient computations and $ \tilde{\mathcal{O}}(\kappa_g (1-\rho)^{-1/2} \log(1/\epsilon)) $ communication steps in general settings.
• In empirical risk minimization with statistically similar data, the method converges in $ \mathcal{O}((\beta/\mu) \log(1/\epsilon)) $ iterations and $ \tilde{\mathcal{O}}((\beta/\mu)(1-\rho)^{-1/2} \log(1/\epsilon)) $ communication steps.
• When $ \beta/\mu < \kappa_g $, the use of high-order surrogates yields provably faster convergence than first-order methods, even without Hessian exchange.
• The convergence rate is bounded by $ z \leq 1 - c(1-\sqrt{c})^2 \cdot \text{const} \cdot (1-\rho)^2 / (\kappa_g^2 \rho) $, showing improved dependence on network connectivity $ \rho $.
• The algorithm is the first to achieve linear convergence for distributed composite optimization with a rate depending only on $ \kappa_g $, not on worst-case agent condition numbers.
• Theoretical analysis confirms that the method maintains linear convergence under directed, time-varying graphs via a refined analysis of the perturbed push-sum mechanism and surrogate approximation error.