[Paper Review] Random Coordinate Descent for Resource Allocation in Open Multi-Agent Systems

Charles Monnoyer de Galland, Renato Vizuete+3 more|arXiv (Cornell University)|May 20, 2022

Distributed Control Multi-Agent Systems4 citations

TL;DR

This paper proposes a distributed Random Coordinate Descent (RCD) algorithm for resource allocation in open multi-agent systems where agents can join or leave during execution. It establishes linear convergence in expectation to a steady-state error bound, derived from the interplay between algorithm convergence in closed systems and the impact of agent replacements on the optimal solution trajectory.

ABSTRACT

We propose a method for analyzing the distributed random coordinate descent algorithm for solving separable resource allocation problems in the context of an open multiagent system, where agents can be replaced during the process. In particular, we characterize the evolution of the distance to the minimizer in expectation by following a time-varying optimization approach which builds on two components. First, we establish the linear convergence of the algorithm in closed systems, in terms of the estimate towards the minimizer, for general graphs and appropriate step-size. Second, we estimate the change of the optimal solution after a replacement, in order to evaluate its effect on the distance between the current estimate and the minimizer. From these two elements, we derive stability conditions in open systems and establish the linear convergence of the algorithm towards a steady-state expected error. Our results enable to characterize the trade-off between speed of convergence and robustness to agent replacements, under the assumptions that local functions are smooth, strongly convex, and have their minimizers located in a given ball. The approach proposed in this paper can moreover be extended to other algorithms guaranteeing linear convergence in closed system.

Motivation & Objective

• Address the challenge of resource allocation in open multi-agent systems where agent composition changes dynamically during optimization.
• Overcome limitations of fixed-system optimization methods that assume stable agent sets and cannot handle real-time agent arrivals or departures.
• Develop a distributed, low-complexity algorithm suitable for large-scale systems with sparse or dynamic communication topologies.
• Characterize the trade-off between convergence speed and robustness to agent replacements in terms of expected error.
• Establish theoretical stability and convergence guarantees for the RCD algorithm under time-varying optimal solutions due to replacements.

Proposed method

• Formulate the resource allocation problem as a separable, constrained optimization problem with local convex cost functions and a global equality constraint on weighted agent states.
• Apply the Random Coordinate Descent (RCD) algorithm, which updates only one agent’s state per iteration based on a randomly selected edge, minimizing computational cost.
• Analyze convergence in closed systems using a time-varying optimization framework, proving linear convergence in an appropriate norm for general graphs and suitable step-sizes.
• Model agent replacements as perturbations to the optimal solution, estimating the change in the minimizer using the local function curvature and replacement frequency.
• Derive an upper bound on the expected distance to the minimizer by combining the closed-system convergence rate and the variation in the optimal solution due to replacements.
• Use the generalized inverse of the Laplacian matrix and a weighted norm to analyze convergence and stability in the presence of heterogeneous agents and network topologies.

Experimental results

Research questions

• RQ1 How does the RCD algorithm behave in open multi-agent systems where agents can be replaced during execution?
• RQ2 What is the impact of agent replacements on the convergence of the RCD algorithm, and how can the resulting error be bounded in expectation?
• RQ3 Can linear convergence be guaranteed in open systems, and under what conditions on the step-size and system parameters?
• RQ4 How do network topology (e.g., complete vs. ring graphs) and agent heterogeneity (e.g., different weights) affect the convergence and stability of the algorithm?
• RQ5 What trade-off exists between convergence speed and robustness to agent replacements in terms of expected error?

Key findings

• The RCD algorithm achieves linear convergence in expectation to a steady-state error in open systems, provided the step-size is chosen appropriately relative to the system's spectral properties.
• The expected error is bounded by a term proportional to the variation of the optimal solution due to replacements, scaled by the inverse of the second smallest eigenvalue of the weighted Laplacian.
• For a complete graph with homogeneous agents, the bound is tightest when the condition number κ is small, and the error converges to a small neighborhood of the true minimizer.
• In sparse graphs (e.g., ring topology) or with heterogeneous agents, the range of valid step-sizes shrinks due to reduced spectral gap, affecting convergence speed and stability.
• The theoretical upper bound on the expected error (Corollary 3) shows conservatism, especially in dense or well-conditioned systems, but remains qualitatively accurate across different topologies and replacement models.
• Numerical simulations confirm linear convergence in both random and adversarial replacement scenarios, with the error trajectory closely following the derived upper bound in various configurations.