[Paper Review] Gradient-Tracking over Directed Graphs for solving Leaderless Multi-Cluster Games

Jan Zimmermann, Tatiana Tatarenko+2 more|arXiv (Cornell University)|Feb 18, 2021

Distributed Control Multi-Agent Systems15 references6 citations

TL;DR

This paper proposes a distributed, discrete-time gradient-tracking algorithm for solving leaderless multi-cluster games over directed communication graphs. By enabling agents to collaboratively track gradients within clusters and exchange decisions across clusters, the method achieves linear convergence to a Nash equilibrium that also satisfies local social welfare optimality, generalizing prior leader-follower and undirected graph approaches.

ABSTRACT

We are concerned with finding Nash Equilibria in agent-based multi-cluster games, where agents are separated into distinct clusters. While the agents inside each cluster collaborate to achieve a common goal, the clusters are considered to be virtual players that compete against each other in a non-cooperative game with respect to a coupled cost function. In such scenarios, the inner-cluster problem and the game between the clusters need to be solved simultaneously. Therefore, the resulting inter-cluster Nash Equilibrium should also be a minimizer of the social welfare problem inside the clusters. In this work, this setup is cast as a distributed optimization problem with sparse state information. Hence, critical information, such as the agent's cost functions, remain private. We present a distributed algorithm that converges with a linear rate to the optimal solution. Furthermore, we apply our algorithm to an extended cournot game to verify our theoretical results.

Motivation & Objective

• To solve multi-cluster games where clusters compete non-cooperatively while agents within clusters cooperate to minimize joint cost.
• To design a distributed algorithm that preserves privacy of individual cost functions and operates with sparse, local communication.
• To extend existing gradient-based methods to directed communication graphs, moving beyond undirected or leader-follower architectures.
• To achieve linear convergence rate in discrete time, improving on slower or continuous-time alternatives.
• To verify theoretical results through simulation of an extended Cournot game with multiple factories and companies.

Proposed method

• Agents maintain two variables: one for local decision estimation and one for gradient-tracking within their cluster.
• The algorithm uses constant step-sizes, enabling faster convergence compared to diminishing step-size methods.
• Inter-cluster communication is modeled via a directed graph, allowing multiple agents per cluster to communicate externally.
• The method approximates the update dynamics as a linear time-invariant state-space system to prove convergence.
• Gradient-tracking enables agents to estimate the aggregate gradient of the coupled cost function across clusters.
• The algorithm ensures that consensus on decisions coincides with the Nash equilibrium and local social welfare optimality.

Experimental results

Research questions

• RQ1 Can a distributed algorithm achieve linear convergence to a Nash equilibrium in a leaderless, multi-cluster game setting?
• RQ2 How can gradient-tracking be adapted to work over directed communication graphs in a multi-cluster game?
• RQ3 Can the algorithm simultaneously satisfy both inter-cluster Nash equilibrium and intra-cluster social welfare optimality?
• RQ4 How does the performance compare to leader-follower architectures in terms of convergence speed and communication structure?
• RQ5 What is the impact of directed communication on convergence and algorithm robustness?

Key findings

• The algorithm converges linearly to the optimal solution under standard assumptions, including strong monotonicity of the game mapping.
• Consensus on agent decisions is achieved within 300 iterations, with absolute differences between agents' estimates below 1.90×10⁻³ in all dimensions.
• The normed error between the agent’s estimate and the true Nash equilibrium state is ǫ = ||x₁ − x*||₂ = 0.005 after 300 iterations.
• The algorithm achieves comparable accuracy to leader-follower methods in fewer iterations (300 vs. 500) with the same step-size, indicating faster convergence.
• The proposed method generalizes existing leader-follower and undirected graph approaches by allowing arbitrary directed communication topologies.
• The simulation of an extended Cournot game confirms that the solution satisfies both inter-cluster Nash equilibrium and intra-cluster social welfare optimality.