[Paper Review] Discrete-time Risk-sensitive Mean-field Games

Naci Saldı, Tamer Başar+1 more|arXiv (Cornell University)|Aug 12, 2018

Stochastic processes and financial applications2 references5 citations

TL;DR

This paper establishes the existence of a mean-field equilibrium in discrete-time risk-sensitive mean-field games under an infinite-horizon discounted-cost criterion, using exponential utility to model risk-sensitivity. It proves that the mean-field equilibrium policy forms an approximate Nash equilibrium for large but finite populations, extending risk-sensitive control to mean-field game theory in discrete time for the first time with general Polish state and action spaces and bounded cost functions.

ABSTRACT

In this paper, we study a class of discrete-time mean-field games under the infinite-horizon risk-sensitive discounted-cost optimality criterion. Risk-sensitivity is introduced for each agent (player) via an exponential utility function. In this game model, each agent is coupled with the rest of the population through the empirical distribution of the states, which affects both the agent's individual cost and its state dynamics. Under mild assumptions, we establish the existence of a mean-field equilibrium in the infinite-population limit as the number of agents ($N$) goes to infinity, and then show that the policy obtained from the mean-field equilibrium constitutes an approximate Nash equilibrium when $N$ is sufficiently large.

Motivation & Objective

• To extend mean-field game theory to discrete-time systems with risk-sensitive cost criteria, where risk-sensitivity is modeled via exponential utility.
• To establish the existence of a mean-field equilibrium in the infinite-population limit under general Polish state and action spaces and bounded one-stage costs.
• To demonstrate that the mean-field equilibrium policy constitutes an approximate Nash equilibrium for games with a large but finite number of agents.
• To bridge the gap between continuous-time risk-sensitive mean-field games and their discrete-time counterparts, which have largely been restricted to risk-neutral settings.

Proposed method

• Formulates a discrete-time mean-field game model where each agent's cost and dynamics depend on the empirical distribution of the population's states.
• Introduces risk-sensitivity via an exponential utility function, leading to a risk-sensitive discounted-cost criterion for each agent.
• Applies the Nash certainty equivalence (NCE) principle to decouple the infinite-population game into a single-agent stochastic control problem with a distributional constraint.
• Uses the Fokker-Planck equation to describe the evolution of the limiting state distribution and the Hamilton-Jacobi-Bellman (HJB) equation to characterize the optimal policy.
• Employs weak convergence of probability measures and uniform equicontinuity arguments to prove convergence of empirical distributions to the limiting measure.
• Establishes the convergence of the empirical distribution of agent states to the limiting distribution under the mean-field equilibrium policy, ensuring consistency.

Experimental results

Research questions

• RQ1 Does a mean-field equilibrium exist in discrete-time risk-sensitive mean-field games under general Polish state and action spaces and bounded cost functions?
• RQ2 Can the policy derived from the mean-field equilibrium serve as an approximate Nash equilibrium for finite-population games with a large number of agents?
• RQ3 How does risk-sensitivity, modeled via exponential utility, affect the structure and existence of equilibria in discrete-time mean-field games?
• RQ4 What conditions ensure the convergence of empirical distributions of agent states to the limiting distribution in the infinite-population limit?
• RQ5 How do the properties of the transition kernel and cost function influence the stability and continuity of the mean-field equilibrium?

Key findings

• The paper proves the existence of a mean-field equilibrium in the infinite-population limit for discrete-time risk-sensitive mean-field games with general Polish state and action spaces and bounded one-stage costs.
• The mean-field equilibrium policy is shown to constitute an approximate Nash equilibrium for games with a sufficiently large but finite number of agents.
• The convergence of the empirical distribution of agent states to the limiting distribution is established via weak convergence and uniform equicontinuity arguments.
• The family of value functions and transition kernels is shown to be uniformly bounded and equicontinuous, ensuring stability in the limit.
• The proof relies on the uniform convergence of the empirical distribution of the first agent's state to the limiting distribution, which is a key step in verifying the NCE principle.
• The result holds under mild assumptions, including boundedness of the cost function and continuity of the transition kernel and cost function in the mean-field term.