[Paper Review] Improving the Sample and Communication Complexity for Decentralized Non-Convex Optimization: A Joint Gradient Estimation and Tracking Approach

Haoran Sun, Songtao Lu+1 more|arXiv (Cornell University)|Oct 13, 2019

Stochastic Gradient Optimization Techniques64 references12 citations

TL;DR

This paper proposes D-GET, a decentralized stochastic optimization algorithm that jointly performs local gradient estimation and global gradient tracking to reduce sample and communication complexity in non-convex problems. It achieves optimal $Ø(mn^{1/2}\epsilon^{-1})$ sample complexity and $Ø(\epsilon^{-1})$ communication complexity for finite-sum problems, and $Ø(m\epsilon^{-3/2})$ sample complexity for online problems—significantly improving over prior bounds.

ABSTRACT

Many modern large-scale machine learning problems benefit from decentralized and stochastic optimization. Recent works have shown that utilizing both decentralized computing and local stochastic gradient estimates can outperform state-of-the-art centralized algorithms, in applications involving highly non-convex problems, such as training deep neural networks. In this work, we propose a decentralized stochastic algorithm to deal with certain smooth non-convex problems where there are $m$ nodes in the system, and each node has a large number of samples (denoted as $n$). Differently from the majority of the existing decentralized learning algorithms for either stochastic or finite-sum problems, our focus is given to both reducing the total communication rounds among the nodes, while accessing the minimum number of local data samples. In particular, we propose an algorithm named D-GET (decentralized gradient estimation and tracking), which jointly performs decentralized gradient estimation (which estimates the local gradient using a subset of local samples) and gradient tracking (which tracks the global full gradient using local estimates). We show that, to achieve certain $ε$ stationary solution of the deterministic finite sum problem, the proposed algorithm achieves an $\mathcal{O}(mn^{1/2}ε^{-1})$ sample complexity and an $\mathcal{O}(ε^{-1})$ communication complexity. These bounds significantly improve upon the best existing bounds of $\mathcal{O}(mnε^{-1})$ and $\mathcal{O}(ε^{-1})$, respectively. Similarly, for online problems, the proposed method achieves an $\mathcal{O}(m ε^{-3/2})$ sample complexity and an $\mathcal{O}(ε^{-1})$ communication complexity, while the best existing bounds are $\mathcal{O}(mε^{-2})$ and $\mathcal{O}(ε^{-2})$, respectively.

Motivation & Objective

• Address the high sample and communication complexity in decentralized non-convex optimization for large-scale machine learning.
• Overcome limitations of existing decentralized algorithms that either require full-batch gradient evaluations or suffer from high communication rounds.
• Develop a method that simultaneously minimizes local data access and inter-node communication while maintaining convergence to an $ \epsilon$-stationary solution.
• Achieve improved theoretical complexity bounds for both finite-sum and online settings in decentralized non-convex optimization.

Proposed method

• Propose D-GET, a decentralized algorithm that combines local stochastic gradient estimation with global gradient tracking using a consensus-based network structure.
• Use a two-level sampling strategy: large batches for full gradient estimation every $q$-th iteration and smaller batches for variance-reduced gradient tracking.
• Introduce a gradient tracking mechanism that maintains an estimate of the global gradient using local estimates and network averaging via the mixing matrix $\mathbf{W}$.
• Leverage variance reduction techniques to bound estimation error in stochastic gradients, ensuring convergence despite sampling noise.
• Design a hybrid update rule that alternates between local updates and global gradient tracking, with periodic full-batch gradient re-estimation.
• Formulate the problem as a consensus optimization problem over a network, where each node maintains a local variable and communicates only with neighbors.

Experimental results

Research questions

• RQ1 Can joint gradient estimation and tracking reduce the total number of local data samples required to reach an $ \epsilon$-stationary solution in decentralized non-convex optimization?
• RQ2 How does the communication complexity scale with the desired accuracy $\epsilon$ in decentralized stochastic optimization?
• RQ3 Can the proposed method achieve better sample complexity than existing decentralized algorithms in both finite-sum and online settings?
• RQ4 What is the theoretical trade-off between local computation (sample complexity) and inter-node communication (round complexity) in decentralized non-convex optimization?
• RQ5 Does the use of variance-reduced gradient tracking lead to improved convergence rates compared to standard decentralized stochastic gradient methods?

Key findings

• For the finite-sum problem, D-GET achieves an $Ø(mn^{1/2}\epsilon^{-1})$ sample complexity, which improves upon the best existing bound of $Ø(mn\epsilon^{-1})$.
• The communication complexity of D-GET is $Ø(\epsilon^{-1})$, matching the best existing bound but achieved with significantly lower sample usage.
• For online problems, D-GET achieves an $Ø(m\epsilon^{-3/2})$ sample complexity, improving over the prior best of $Ø(m\epsilon^{-2})$.
• The communication complexity remains $Ø(\epsilon^{-1})$ for online problems, outperforming existing methods that require $Ø(\epsilon^{-2})$ rounds.
• Theoretical analysis confirms that the algorithm converges to an $ \epsilon$-stationary solution with high probability under standard smoothness and bounded variance assumptions.
• The method achieves optimal trade-offs between local computation and communication, making it suitable for large-scale decentralized learning with privacy and scalability constraints.