[Paper Review] Mean Field Games for Multi-agent Systems with Multiplicative Noises

Bing‐Chang Wang, Yuan‐Hua Ni+1 more|arXiv (Cornell University)|Jun 7, 2019

Distributed Control Multi-Agent Systems31 references4 citations

TL;DR

This paper develops mean field game strategies for multi-agent systems with control-dependent multiplicative noises, using a limiting optimal control problem and consistent mean field approximations to derive decentralized ε-Nash equilibria. For integrator systems, it proves that under mild conditions, agents achieve mean-square consensus via convexity-based strategies, with explicit stability conditions derived from Riccati-type equations.

ABSTRACT

This paper studies mean field games for multi-agent systems with control-dependent multiplicative noises. For the general systems with nonuniform agents, we obtain a set of decentralized strategies by solving an auxiliary limiting optimal control problem subject to consistent mean field approximations. The set of decentralized strategies is further shown to be an $\varepsilon$-Nash equilibrium. For the integrator multiagent systems, we design a set of $\varepsilon$-Nash strategies by exploiting the convexity property of the limiting problem. It is shown that under the mild conditions all the agents achieve mean-square consensus.

Motivation & Objective

• To address decentralized control in large-population multi-agent systems with control-dependent multiplicative noise, where traditional additive noise models are insufficient.
• To develop a mean field game framework that accounts for state- and control-dependent stochastic disturbances in multi-agent dynamics.
• To establish conditions under which decentralized strategies form an ε-Nash equilibrium in general non-uniform multi-agent systems.
• To investigate mean-square consensus in integrator-type multi-agent systems under multiplicative noise using convexity and Riccati-based control design.
• To compare consensus performance under multiplicative versus additive noise, showing fundamental differences in convergence behavior.

Proposed method

• Solves an auxiliary limiting optimal control problem to approximate the mean field effect in large-population systems with multiplicative noise.
• Implements consistent mean field approximations to derive a fixed-point equation linking individual optimal response and population-level behavior.
• Derives decentralized strategies based on the solution of the limiting problem and the mean field effect, ensuring ε-Nash equilibrium properties.
• For integrator systems, exploits convexity of the limiting problem to design explicit ε-Nash strategies using Riccati equations.
• Uses a stochastic linear-quadratic (LQ) framework with diffusion terms depending on both state and control to model multiplicative noise.
• Applies stability analysis on the closed-loop system to derive conditions for mean-square consensus, based on the stability of a matrix pair derived from system parameters.

Experimental results

Research questions

• RQ1 How can decentralized strategies be designed for multi-agent systems with control-dependent multiplicative noise?
• RQ2 Under what conditions does the derived set of strategies form an ε-Nash equilibrium in general non-uniform multi-agent systems?
• RQ3 Can mean-square consensus be achieved in integrator-type multi-agent systems with multiplicative noise, and what are the necessary and sufficient conditions?
• RQ4 How does the consensus behavior differ between systems with multiplicative noise and those with additive noise?
• RQ5 What role does the Riccati equation play in stabilizing the system and ensuring consensus under multiplicative noise?

Key findings

• The proposed decentralized strategies form an ε-Nash equilibrium for general non-uniform multi-agent systems with multiplicative noise, derived via consistent mean field approximations.
• For integrator systems, the mean-field game approach yields a unique bounded solution for the consistency equations: (s(t), x̄(t)) ≡ (−P x̄₀, x̄₀).
• Mean-square consensus is achieved if and only if the matrix pair [−B(R+DᵀPD)⁻¹BᵀP, −D(R+DᵀPD)⁻¹BᵀP] is stable.
• When R > 0 and ρ = 0, the system is guaranteed to achieve mean-square consensus due to the stability of the associated matrix pair.
• Numerical results show that agents under multiplicative noise reach consensus, while the same system under additive noise fails to do so, highlighting the distinct dynamical behavior.
• The framework successfully extends mean field game theory to stochastic systems with multiplicative noise, which are more complex than additive noise cases due to control-dependent diffusion terms.