[Paper Review] On the convergence of closed-loop Nash equilibria to the mean field game limit

Daniel Lacker|arXiv (Cornell University)|Aug 8, 2018

Stochastic processes and financial applications6 citations

TL;DR

This paper establishes the convergence of closed-loop Nash equilibria in large-population stochastic differential games to weak mean field game equilibria, even when the mean field equilibrium is non-unique. By adapting compactness arguments from the open-loop case, it proves that all limit points of $n$-player equilibria are weak MFG equilibria, which include additional randomness compared to standard (strong) equilibria, thus extending convergence results beyond uniqueness assumptions.

ABSTRACT

This paper continues the study of the mean field game (MFG) convergence problem: In what sense do the Nash equilibria of $n$-player stochastic differential games converge to the mean field game as $n\ ightarrow\\infty$? Previous work on this problem took two forms. First, when the $n$-player equilibria are open-loop, compactness arguments permit a characterization of all limit points of $n$-player equilibria as weak MFG equilibria, which contain additional randomness compared to the standard (strong) equilibrium concept. On the other hand, when the $n$-player equilibria are closed-loop, the convergence to the MFG equilibrium is known only when the MFG equilibrium is unique and the associated "master equation" is solvable and sufficiently smooth. This paper adapts the compactness arguments to the closed-loop case, proving a convergence theorem that holds even when the MFG equilibrium is non-unique. Every limit point of $n$-player equilibria is shown to be the same kind of weak MFG equilibrium as in the open-loop case. Some partial results and examples are discussed for the converse question, regarding which of the weak MFG equilibria can arise as the limit of $n$-player (approximate) equilibria.

Motivation & Objective

• To resolve the convergence problem of $n$-player closed-loop Nash equilibria to mean field game (MFG) equilibria as $n \to \infty$.
• To extend convergence results beyond the restrictive assumption of MFG equilibrium uniqueness.
• To characterize the limiting behavior of $n$-player equilibria using weak MFG equilibria, which incorporate additional randomness.
• To establish a framework that unifies open-loop and closed-loop convergence theories via compactness arguments.
• To explore which weak MFG equilibria can arise as limits of $n$-player approximate equilibria.

Proposed method

• Adapts compactness arguments from open-loop equilibrium analysis to the closed-loop setting.
• Introduces and analyzes relaxed $n$-player games and relaxed mean field equilibria to handle limit points.
• Uses joint measurability and regular conditional laws to construct measurable versions of conditional expectations for random measures.
• Applies the Markovian projection theorem to identify limiting dynamics under weak convergence.
• Employs projection lemmas and tightness arguments to control the convergence of empirical measures and controls.
• Leverages the master equation framework and stability results for SDEs with random coefficients to ensure well-posedness of limiting dynamics.

Experimental results

Research questions

• RQ1 Under what conditions do closed-loop Nash equilibria of $n$-player stochastic differential games converge to a mean field game equilibrium as $n \to \infty$?
• RQ2 Can convergence be established when the mean field game equilibrium is non-unique?
• RQ3 What kind of limiting object arises as the limit of $n$-player closed-loop equilibria?
• RQ4 Which weak mean field game equilibria can be realized as limits of $n$-player (approximate) equilibria?
• RQ5 How do weak MFG equilibria, which include extra randomness, relate to the original closed-loop equilibrium structure?

Key findings

• All limit points of $n$-player closed-loop Nash equilibria are weak mean field game equilibria, even when the MFG equilibrium is non-unique.
• The convergence result holds without requiring the solvability or smoothness of the master equation, unlike prior results.
• The limiting dynamics are identified via a weak form of the master equation, incorporating randomness through conditional expectations.
• The paper constructs measurable versions of conditional means of random measures, enabling rigorous analysis of relaxed controls.
• It establishes that weak MFG equilibria are the natural limiting objects for closed-loop equilibria, analogous to the open-loop case.
• Partial converse results show that certain weak MFG equilibria can arise as limits of $n$-player approximate equilibria, though not all may be attainable.