[Paper Review] Signatured Deep Fictitious Play for Mean Field Games with Common Noise

Ming Min, Ruimeng Hu|arXiv (Cornell University)|Jun 6, 2021

Sports Analytics and Performance50 references4 citations

TL;DR

This paper proposes Signatured Deep Fictitious Play (Sig-DFP), a single-loop deep learning algorithm that leverages rough path theory and fictitious play to solve mean-field games (MFGs) with common noise, eliminating the need for computationally prohibitive nested-loop simulations. The method achieves superior efficiency and accuracy, enabling the first deep learning solution for extended MFGs with common noise and significantly reducing memory and runtime demands compared to existing approaches.

ABSTRACT

Existing deep learning methods for solving mean-field games (MFGs) with common noise fix the sampling common noise paths and then solve the corresponding MFGs. This leads to a nested-loop structure with millions of simulations of common noise paths in order to produce accurate solutions, which results in prohibitive computational cost and limits the applications to a large extent. In this paper, based on the rough path theory, we propose a novel single-loop algorithm, named signatured deep fictitious play, by which we can work with the unfixed common noise setup to avoid the nested-loop structure and reduce the computational complexity significantly. The proposed algorithm can accurately capture the effect of common uncertainty changes on mean-field equilibria without further training of neural networks, as previously needed in the existing machine learning algorithms. The efficiency is supported by three applications, including linear-quadratic MFGs, mean-field portfolio game, and mean-field game of optimal consumption and investment. Overall, we provide a new point of view from the rough path theory to solve MFGs with common noise with significantly improved efficiency and an extensive range of applications. In addition, we report the first deep learning work to deal with extended MFGs (a mean-field interaction via both the states and controls) with common noise.

Motivation & Objective

• To address the high computational cost of existing deep learning methods for mean-field games (MFGs) with common noise, which rely on nested-loop simulations of millions of common noise paths.
• To eliminate the nested-loop structure by enabling unfixed common noise handling through signature-based representation from rough path theory.
• To develop a scalable and efficient algorithm capable of capturing the impact of common uncertainty changes on mean-field equilibria without retraining neural networks.
• To extend deep learning applications to extended MFGs—where mean-field interaction depends on both states and controls—under common noise, a previously unaddressed setting.
• To demonstrate the method’s effectiveness across diverse applications, including linear-quadratic MFGs, portfolio games, and optimal consumption-investment problems.

Proposed method

• The algorithm integrates deep fictitious play with signatures from rough path theory to represent the pathwise dependence of common noise on the solution, enabling a single-loop training process.
• It parameterizes the optimal control policy using deep neural networks and optimizes globally via fictitious play, iteratively improving strategies based on population-level responses.
• Signatures of the common noise paths are used as input features, with truncation at depth M to ensure computational feasibility while preserving path information.
• The method avoids solving complex equations such as the master equation, stochastic Fokker-Planck/Hamilton-Jacobi-Bellman systems, or McKean-Vlasov FBSDEs, which are typically intractable.
• Convergence is proven as both the signature depth M and fictitious play stage n approach infinity, under the universal approximation property of neural networks.
• The algorithm supports heterogeneous agents and extended MFGs with joint state-control interactions under common noise, a novel contribution in deep learning for MFGs.

Experimental results

Research questions

• RQ1 Can a single-loop deep learning algorithm be designed to solve MFGs with common noise more efficiently than existing nested-loop approaches?
• RQ2 Can rough path signatures effectively encode the impact of common noise on mean-field equilibria without fixing noise paths?
• RQ3 Does the integration of fictitious play with signature-based representation enable accurate capture of equilibrium strategies under common noise without retraining?
• RQ4 Can the proposed method handle extended MFGs with joint state and control interactions under common noise, a setting previously unaddressed in deep learning?
• RQ5 What is the computational and convergence trade-off between signature depth M and the number of fictitious play stages n in practice?

Key findings

• Sig-DFP reduces memory usage and running time significantly compared to the nested algorithm, enabling training with up to (2^15, 2^15) individual and common noise paths, whereas the nested method fails beyond (2^8, 2^8) due to memory constraints.
• At (INP, CNP) = (2^15, 2^15), Sig-DFP achieves relative L² errors of 11% and 6% on control and cost, respectively, compared to 79.4% and 16.2% for the nested algorithm.
• The running time of Sig-DFP increases only moderately with signature depth M and dimension n₀, with a 2.6-hour runtime for (n₀=5, M=4), demonstrating scalability to higher dimensions.
• A signature depth of M=2 provides optimal accuracy for one-dimensional problems, as the log of control and state processes are approximately linear or quadratic, making higher depths redundant.
• Sig-DFP achieves convergence to mean-field equilibrium as M and n approach infinity, under the universal approximation of neural networks, with strong empirical support across three benchmark problems.
• This work reports the first deep learning solution for extended MFGs with common noise, where mean-field interaction depends on both states and controls, expanding the scope of deep learning in MFGs.