[Paper Review] Nonzero-sum stochastic games and mean-field games with impulse controls

Matteo Basei, Haoyang Cao+1 more|arXiv (Cornell University)|Jan 23, 2019

Stochastic processes and financial applications40 references11 citations

TL;DR

This paper develops a verification framework for Nash equilibria in nonzero-sum stochastic games with impulse controls, establishing sufficient conditions for N-player games and proving that the mean-field game (MFG) solution provides an ϵ-Nash equilibrium with ϵ=O(1/√N). It applies the theory to a two-player cash management game, showing competition leads to less frequent but larger interventions, enhancing control efficiency.

ABSTRACT

We consider a general class of nonzero-sum $N$-player stochastic games with impulse controls, where players control the underlying dynamics with discrete interventions. We adopt a verification approach and provide sufficient conditions for the Nash equilibria (NEs) of the game. We then consider the limit situation of $N o \infty$, that is, a suitable mean-field game (MFG) with impulse controls. We show that under appropriate technical conditions, the existence of unique NE solution to the MFG, which is an $ε$-NE approximation to the $N$-player game, with $ε=O\left(\frac{1}{\sqrt{N}} ight)$. As an example, we analyze in details a class of two-player stochastic games which extends the classical cash management problem to the game setting. In particular, we present numerical analysis for the cases of the single player, the two-player game, and the MFG, showing the impact of competition on the player's optimal strategy, with sensitivity analysis of the model parameters.

Motivation & Objective

• To establish sufficient conditions for Nash equilibria in nonzero-sum N-player stochastic games with impulse controls using a verification approach.
• To analyze the limit behavior as N→∞, deriving conditions for the existence of a unique Nash equilibrium in the corresponding mean-field game (MFG).
• To demonstrate that the MFG solution serves as an ϵ-Nash equilibrium approximation to the N-player game with ϵ=O(1/√N).
• To investigate the impact of competition on optimal control strategies through a detailed analysis of a two-player cash management game.
• To perform sensitivity analysis on model parameters and compare optimal policies across single-player, two-player, and mean-field game settings.

Proposed method

• Formulate the N-player impulse control game using stochastic differential equations with jump dynamics driven by Dirac delta functions at intervention times.
• Define the payoff functional to include running costs, individual control costs, and cross-impact costs from other players’ interventions.
• Derive a system of Quasi-Variational Inequalities (QVIs) as the necessary condition for Nash equilibrium, incorporating non-local operators due to impulse control.
• Apply a verification theorem to show that solutions to the QVIs correspond to Nash equilibria under appropriate regularity and boundary conditions.
• Construct the mean-field game limit by considering the empirical distribution of states as N→∞, leading to a McKean–Vlasov type formulation.
• Prove that the solution to the MFG is an ϵ-Nash equilibrium for the N-player game with ϵ=O(1/√N), under suitable technical assumptions on the coefficients and controls.

Experimental results

Research questions

• RQ1 Under what conditions does a Nash equilibrium exist in a nonzero-sum N-player stochastic game with impulse controls?
• RQ2 How does the mean-field game limit of such a game relate to the original N-player game in terms of equilibrium approximation?
• RQ3 What is the rate of convergence of the MFG solution to an ϵ-Nash equilibrium of the N-player game?
• RQ4 How does competition alter the optimal impulse control strategy compared to a single-agent setting?
• RQ5 How do changes in control costs, volatility, and discount rates affect the thresholds and long-run behavior of the state process in the game?

Key findings

• The paper establishes a verification theorem for Nash equilibria in N-player impulse control stochastic games via a system of Quasi-Variational Inequalities (QVIs).
• Under appropriate technical conditions, the solution to the mean-field game (MFG) is an ϵ-Nash equilibrium for the N-player game with ϵ=O(1/√N).
• In the two-player cash management game, competition leads to less frequent interventions but larger jump sizes compared to the single-player case.
• Sensitivity analysis shows that increasing the fixed cost of decreasing the state (K⁻) raises the upper boundary of the non-action region (u), reducing intervention frequency.
• Increasing the fixed cost of increasing the state (K⁺) reduces the lower boundary (d), also decreasing intervention frequency, while both K⁻ and K⁺ increase jump sizes.
• Higher volatility (σ) or discount rate (r) leads to reduced intervention frequency and larger jump sizes, with the mean state level increasing with σ and first increasing then decreasing with r.