[Paper Review] D-SPIDER-SFO: A Decentralized Optimization Algorithm with Faster Convergence Rate for Nonconvex Problems

Taoxing Pan, Jun Liu+1 more|arXiv (Cornell University)|Nov 28, 2019

Stochastic Gradient Optimization Techniques31 references4 citations

TL;DR

This paper proposes D-SPIDER-SFO, a decentralized optimization algorithm for nonconvex problems that achieves a gradient computation cost of 𝒪(𝜖⁻³) to find an 𝜖-approximate first-order stationary point—matching the state-of-the-art performance of its centralized counterpart, SPIDER-SFO. The method leverages variance reduction and decentralized consensus to enable faster convergence with balanced communication in peer-to-peer networks.

ABSTRACT

Decentralized optimization algorithms have attracted intensive interests recently, as it has a balanced communication pattern, especially when solving large-scale machine learning problems. Stochastic Path Integrated Differential Estimator Stochastic First-Order method (SPIDER-SFO) nearly achieves the algorithmic lower bound in certain regimes for nonconvex problems. However, whether we can find a decentralized algorithm which achieves a similar convergence rate to SPIDER-SFO is still unclear. To tackle this problem, we propose a decentralized variant of SPIDER-SFO, called decentralized SPIDER-SFO (D-SPIDER-SFO). We show that D-SPIDER-SFO achieves a similar gradient computation cost---that is, $\mathcal{O}(ε^{-3})$ for finding an $ε$-approximate first-order stationary point---to its centralized counterpart. To the best of our knowledge, D-SPIDER-SFO achieves the state-of-the-art performance for solving nonconvex optimization problems on decentralized networks in terms of the computational cost. Experiments on different network configurations demonstrate the efficiency of the proposed method.

Motivation & Objective

• To develop a decentralized variant of SPIDER-SFO that achieves the same convergence rate as its centralized counterpart for nonconvex optimization.
• To address the gap in fast convergence rates for decentralized nonconvex optimization, particularly in scenarios where communication load is a bottleneck.
• To enable faster convergence with reduced computational cost in decentralized networks by leveraging variance-reduction techniques.
• To demonstrate that decentralized algorithms can achieve near-optimal gradient complexity in nonconvex settings, matching theoretical lower bounds.

Proposed method

• Proposes D-SPIDER-SFO, a decentralized extension of the SPIDER-SFO algorithm, using local stochastic gradient estimators with variance reduction.
• Employs a consensus-based update rule where each node communicates only with its neighbors to maintain balanced communication load.
• Introduces a hybrid sampling strategy with three hyperparameters: S₁, S₂, and q, controlling the number of samples per iteration for gradient estimation.
• Uses a diminishing step size and adaptive sampling to balance convergence speed and variance reduction across iterations.
• Applies a local update mechanism that combines gradient information from multiple samples to reduce variance while maintaining decentralization.
• Theoretical analysis relies on bounding the expected norm of the gradient at the average of local iterates, ensuring convergence to an 𝜖-approximate stationary point.

Experimental results

Research questions

• RQ1 Can a decentralized optimization algorithm achieve the same 𝒪(𝜖⁻³) gradient computation cost as SPIDER-SFO for nonconvex problems?
• RQ2 Does D-SPIDER-SFO maintain fast convergence under decentralized network constraints with balanced communication?
• RQ3 How does the communication cost of D-SPIDER-SFO compare to centralized SPIDER-SFO and other decentralized baselines like D-PSGD or D²?
• RQ4 What is the theoretical convergence guarantee of D-SPIDER-SFO in terms of gradient norm and expected function value decrease?

Key findings

• D-SPIDER-SFO achieves a gradient computation cost of 𝒪(𝜖⁻³) to find an 𝜖-approximate first-order stationary point, matching the theoretical lower bound for nonconvex problems.
• The algorithm matches the convergence rate of centralized SPIDER-SFO, making it the state-of-the-art in decentralized nonconvex optimization.
• The method reduces communication burden on the most active node compared to centralized SPIDER-SFO, improving scalability in high-latency networks.
• Experiments on a CPU cluster with 8 nodes show D-SPIDER-SFO outperforms D-PSGD and D² in convergence speed and stability.
• The theoretical analysis confirms that the expected gradient norm at the average iterate converges to 𝒪(𝜖²) within K iterations, with a total gradient cost bounded by 𝒪(𝜖⁻³).