[Paper Review] Mean field limit of a continuous time finite state game

Diogo A. Gomes, Joana Mohr+1 more|arXiv (Cornell University)|Nov 12, 2010

Stochastic processes and financial applications11 references3 citations

TL;DR

This paper establishes the mean field limit of a continuous-time, finite-state game with N+1 players switching between two states, proving convergence of the N-player symmetric Markov perfect equilibrium to a mean field model as N→∞. The key result is a rigorous O(1/N) rate of convergence for the value functions and strategies, derived via coupled ODEs for the mean field dynamics and terminal-cost value functions under Lipschitz and boundedness assumptions on the cost and transition rates.

ABSTRACT

Mean field games is a recent area of study introduced by Lions and Lasry in a series of seminal papers in 2006. Mean field games model situations of competition between large number of rational agents that play non-cooperative dynamic games under certain symmetry assumptions. They key step is to develop a mean field model, in a similar way that what is done in statistical physics in order to construct a mathematically tractable model. A main question that arises in the study of such mean field problems is the rigorous justification of the mean field models by a limiting procedure. In this paper we consider the mean field limit of two-state Markov decision problem as the number of players $N o \infty$. First we establish the existence and uniqueness of a symmetric partial information Markov perfect equilibrium. Then we derive a mean field model and characterize its main properties. This mean field limit is a system of coupled ordinary differential equations with initial-terminal data. Our main result is the convergence as $N o \infty$ of the $N$ player game to the mean field model and an estimate of the rate of convergence.

Motivation & Objective

• To rigorously justify the mean field approximation in continuous-time, finite-state, symmetric Markov decision games with partial information.
• To establish existence and uniqueness of a symmetric partial information Markov perfect equilibrium for the N+1 player game.
• To derive a mean field model as a system of coupled ODEs with initial and terminal conditions.
• To prove convergence of the N-player game to the mean field model as N→∞ with explicit rate estimation.
• To generalize discrete-time results to continuous time and extend to non-stationary, non-ergodic settings.

Proposed method

• Formulates the N+1 player game as a continuous-time Markov decision process with symmetric, partial-information strategies.
• Characterizes the symmetric Markov perfect equilibrium via a nonlinear ODE system derived from Hamilton-Jacobi-Bellman equations.
• Derives the mean field limit as a coupled system of ODEs: one for the empirical state distribution θ(t) with initial condition, and one for the value function with terminal condition.
• Applies Dynkin's formula and Lipschitz estimates to control the difference between N-player and mean field value functions and strategies.
• Uses Gronwall-type inequalities to bound the error in the value function and strategy distribution, leading to an O(1/N) convergence rate.
• Employs uniform bounds on differences in value functions and transition rates to control the propagation of error across time.

Experimental results

Research questions

• RQ1 Does the symmetric Markov perfect equilibrium exist and remain unique in the continuous-time, finite-state N-player game with partial information?
• RQ2 Can the mean field limit be rigorously derived as N→∞, and what is the resulting system of equations?
• RQ3 How fast does the N-player game converge to the mean field model, and can a quantitative rate be established?
• RQ4 What conditions on the cost and transition functions ensure the stability and convergence of the mean field approximation?
• RQ5 Is the mean field model well-posed, and does it admit a unique solution under the same assumptions as the N-player game?

Key findings

• The N-player game admits a unique symmetric partial information Markov perfect equilibrium, characterized by a nonlinear ODE system.
• The mean field limit is a well-posed initial-terminal value problem consisting of two coupled ODEs: one for the state distribution θ(t) with initial data, and one for the value function with terminal data.
• The value function and strategy distribution of the N-player game converge to those of the mean field model at rate O(1/N) as N→∞.
• The convergence rate is established under the condition that TC < 1, where T is the time horizon and C is a constant depending on Lipschitz constants of the cost and transition functions.
• The proof relies on Gronwall-type estimates applied to the difference between N-player and mean field value functions and strategies, using uniform bounds on transition rate differences.
• The error bounds are derived via Dynkin’s formula and Lipschitz continuity of the Hamiltonian, ensuring stability of the mean field approximation.