[Paper Review] Stochastic Distributed Learning with Gradient Quantization and Variance Reduction

Samuel Horváth, Dmitry Kovalev+3 more|arXiv (Cornell University)|Apr 10, 2019

Stochastic Gradient Optimization Techniques56 references81 citations

TL;DR

The paper develops DIANA-type distributed optimization algorithms capable of arbitrary unbiased gradient quantization, and introduces variance-reduced variants that achieve linear convergence to arbitrary accuracy even with quantized communications. It provides theoretical guarantees across strongly convex, convex, and non-convex regimes and validates improvements via experiments.

ABSTRACT

We consider distributed optimization where the objective function is spread among different devices, each sending incremental model updates to a central server. To alleviate the communication bottleneck, recent work proposed various schemes to compress (e.g.\ quantize or sparsify) the gradients, thereby introducing additional variance $ω\geq 1$ that might slow down convergence. For strongly convex functions with condition number $κ$ distributed among $n$ machines, we (i) give a scheme that converges in $\mathcal{O}((κ+ κ\fracω{n} + ω)$ $\log (1/ε))$ steps to a neighborhood of the optimal solution. For objective functions with a finite-sum structure, each worker having less than $m$ components, we (ii) present novel variance reduced schemes that converge in $\mathcal{O}((κ+ κ\fracω{n} + ω+ m)\log(1/ε))$ steps to arbitrary accuracy $ε> 0$. These are the first methods that achieve linear convergence for arbitrary quantized updates. We also (iii) give analysis for the weakly convex and non-convex cases and (iv) verify in experiments that our novel variance reduced schemes are more efficient than the baselines.

Motivation & Objective

• Motivate and address communication bottlenecks in distributed optimization by compressing gradient updates without sacrificing convergence.
• Extend gradient quantization to a general, unbiased setting with provable convergence bounds.
• Introduce variance-reduced, quantized-distributed algorithms that converge to the optimum (not just a neighborhood) in convex and strongly convex settings.
• Provide theoretical analysis across strongly convex, convex, and non-convex regimes.
• Empirically compare proposed methods against baselines to demonstrate communication efficiency and performance.

Proposed method

• Adopt a DIANA framework with arbitrary omega-quantization operators to compress gradient updates while maintaining unbiasedness.
• Use local accumulators h_i^k on workers and update them with quantized gradient differences to reduce variance.
• Prove linear convergence under omega-quantization with suitable step sizes and Lyapunov function, and derive iteration bounds.
• Extend to variance-reduced schemes (VR-DIANA) based on SVRG, SAGA, and L-SVRG, using quantized updates and per-worker gradient tables or epoch gradients.
• Provide three variants (SVRG-based, SAGA-based, L-SVRG-based) that achieve linear convergence in strongly convex cases and O(1/k) in convex/non-convex cases, with quantified communication savings.
• Discuss and model different quantization operators (random dithering, sparsification, block quantization) and their impact on omega and convergence.

Experimental results

Research questions

• RQ1 Can arbitrary unbiased gradient quantization be integrated into distributed optimization without sacrificing convergence guarantees?
• RQ2 Do variance-reduced, quantized distributed methods achieve linear convergence to the optimum across strongly convex, convex, and non-convex problems?
• RQ3 How does the quantization parameter omega affect convergence rates and communication costs in practice?
• RQ4 What are the comparative trade-offs between iteration complexity and communication savings for VR-DIANA variants under finite-sum and non-sum objectives?
• RQ5 Do empirical results support theoretical benefits of quantization and variance reduction over baselines in realistic datasets?

Key findings

• DIANA with arbitrary omega-quantization achieves linear convergence to the optimum for strongly convex problems under appropriate stepsizes.
• Variance-reduced quantized schemes (VR-DIANA) converge to arbitrary accuracy with rates depending on kappa, omega, n, and m, and can match non-quantized rates when omega is well-controlled.
• In finite-sum settings, VR-DIANA variants attain linear convergence (strongly convex) and O(1/k) (convex/non-convex) with communication costs reduced by quantization.
• The auxiliary variables h_i^k and gradient-difference quantization enable effective variance reduction even when gradients are compressed.
• Experiments show that VR-DIANA variants match baselines in iterations while significantly reducing total communication, especially when communication is the bottleneck.
• The work provides concrete guidance on selecting quantization schemes (e.g., dithering, sparsification, block quantization) to balance convergence and communication.