[Paper Review] Linear convergence in optimization over directed graphs with row-stochastic matrices

Chenguang Xi, Van Sy+3 more|arXiv (Cornell University)|Nov 18, 2016

Distributed Control Multi-Agent Systems37 references3 citations

TL;DR

This paper proposes a fully distributed optimization algorithm for directed graphs that achieves linear convergence, $O(\mu^k)$, without requiring agents to know their neighbors' out-degrees—a key limitation in prior methods. By leveraging row-stochastic matrices and a novel gradient tracking mechanism, the algorithm ensures fast, scalable convergence in strongly-convex, smooth problems with minimal communication overhead.

ABSTRACT

This paper considers a distributed optimization problem over a multi-agent network, in which the objective function is a sum of individual cost functions at the agents. We focus on the case when communication between the agents is described by a \emph{directed} graph. Existing distributed optimization algorithms for directed graphs require at least the knowledge of the neighbors' out-degree at each agent (due to the requirement of column-stochastic matrices). In contrast, our algorithm requires no such knowledge. Moreover, the proposed algorithm achieves the best known rate of convergence for this class of problems, $O(μ^k)$ for $0

Motivation & Objective

• Address the challenge of distributed optimization over directed graphs where existing methods require agents to know their neighbors' out-degrees.
• Overcome the impracticality of column-stochastic matrix construction in broadcast-based or asymmetric communication networks.
• Develop a fully distributed algorithm that maintains fast convergence without relying on dual variables or diminishing step-sizes.
• Achieve the best-known linear convergence rate, $O(\mu^k)$, for strongly-convex, smooth objective functions in directed networks.
• Ensure practical deployability by requiring only local knowledge of unique agent identifiers and row-stochastic weights.

Proposed method

• Formulate the distributed optimization problem as minimizing the sum of local, strongly-convex, and Lipschitz-continuous gradient functions over a directed graph.
• Use a gradient tracking mechanism to estimate the global gradient using local information and row-stochastic weight matrices.
• Design a novel update rule that combines local gradient descent with consensus on gradient estimates using row-stochastic matrices.
• Introduce a constant step-size strategy to ensure linear convergence, avoiding the slow sublinear rates of diminishing step-sizes.
• Leverage unique agent identifiers to enable the algorithm to converge without requiring out-degree information.
• Prove convergence via a Lyapunov-like analysis that bounds the error in both primal iterates and gradient estimates.

Experimental results

Research questions

• RQ1 Can a distributed optimization algorithm achieve linear convergence over directed graphs without requiring agents to know their neighbors' out-degrees?
• RQ2 Is it possible to achieve $O(\mu^k)$ convergence using only row-stochastic matrices instead of column-stochastic or doubly-stochastic ones?
• RQ3 How does the proposed algorithm compare in convergence speed and communication efficiency to existing methods like DEXTRA, ADD-OPT, and Subgradient-Push?
• RQ4 What are the theoretical conditions under which row-stochastic matrices can ensure convergence in distributed optimization over directed graphs?
• RQ5 Can the algorithm be implemented in a fully distributed manner with minimal global coordination or knowledge of network topology?

Key findings

• The proposed algorithm achieves linear convergence, $O(\mu^k)$, for strongly-convex functions with Lipschitz-continuous gradients, matching the best-known rate in the literature.
• The algorithm does not require knowledge of agents' out-degrees, a significant practical advantage over methods like DEXTRA, ADD-OPT, and Push-DIGing.
• Numerical experiments on a distributed logistic regression problem show that the algorithm converges faster than sublinear methods such as Subgradient-Push and Directed-Distributed Subgradient Descent.
• The algorithm maintains fast convergence even with a constant step-size, unlike subgradient-based methods that require diminishing step-sizes.
• The convergence is robust to network topology, as demonstrated on a strongly-connected directed graph with 10 agents and 3-dimensional parameters.
• The theoretical analysis confirms that the error in primal iterates and gradient estimates decays exponentially, bounded by $\|\mathbf{x}_k - \mathbf{x}^*\|_2 \leq \gamma_1^k \|\nabla \mathbf{f}_k\|_2$ for some $\gamma_1 < 1$.