[Paper Review] Alternating the Population and Control Neural Networks to Solve High-Dimensional Stochastic Mean-Field Games

A. T. Lin, Samy Wu Fung+3 more|arXiv (Cornell University)|Feb 24, 2020

Stochastic processes and financial applications62 references70 citations

TL;DR

APAC-Net is a novel deep learning framework that solves high-dimensional stochastic mean-field games (MFGs) by formulating the problem as a convex-concave saddle point optimization, parameterizing the value and density functions via neural networks. Inspired by generative adversarial networks (GANs), it alternately trains a population network (discriminator-like) and a control network (generator-like), enabling scalable solutions for MFGs up to 100 dimensions—previously intractable with grid-based methods due to the curse of dimensionality.

ABSTRACT

We present APAC-Net, an alternating population and agent control neural network for solving stochastic mean field games (MFGs). Our algorithm is geared toward high-dimensional instances of MFGs that are beyond reach with existing solution methods. We achieve this in two steps. First, we take advantage of the underlying variational primal-dual structure that MFGs exhibit and phrase it as a convex-concave saddle point problem. Second, we parameterize the value and density functions by two neural networks, respectively. By phrasing the problem in this manner, solving the MFG can be interpreted as a special case of training a generative adversarial network (GAN). We show the potential of our method on up to 100-dimensional MFG problems.

Motivation & Objective

• Address the computational intractability of high-dimensional stochastic mean-field games (MFGs) due to the curse of dimensionality in traditional grid-based solvers.
• Develop a scalable, deep learning-based method for solving stochastic MFGs in high-dimensional spaces (up to 100D), where existing numerical methods fail.
• Leverage the variational primal-dual structure of potential MFGs to reframe the problem as a convex-concave saddle point optimization.
• Establish a formal connection between MFGs and generative adversarial networks (GANs), enabling the use of GAN-style training dynamics.
• Provide a differentiable, grid-free framework that parameterizes both the value function and agent density using neural networks, avoiding spatial discretization.

Proposed method

• Reformulate the stochastic MFG system as a convex-concave saddle point problem using the variational primal-dual formulation derived from the Benamou-Brenier framework.
• Parameterize the value function ϕ and the agent density ρ using two deep neural networks: the control network (generator) and the population network (discriminator).
• Frame the solution of the MFG as a minimax optimization problem analogous to Wasserstein GANs, where the discriminator evaluates the consistency between the agent policy and population distribution.
• Use alternating optimization: train the control network to minimize the HJB residual while the population network maximizes the consistency with the Fokker-Planck dynamics.
• Enforce the 1-Lipschitz constraint on the population network (discriminator) via gradient penalty, as in WGAN-GP, to stabilize training and avoid mode collapse.
• Train the networks end-to-end using stochastic gradient descent with adaptive optimizers, minimizing the HJB residual and enforcing the Fokker-Planck equation via loss functions derived from the saddle-point formulation.

Experimental results

Research questions

• RQ1 Can a deep learning framework based on GANs effectively solve high-dimensional stochastic mean-field games beyond the reach of traditional grid-based methods?
• RQ2 Does the variational primal-dual formulation of MFGs enable a stable and scalable training procedure when parameterized with neural networks?
• RQ3 How well does the APAC-Net framework generalize across different dimensions (e.g., 2D, 50D, 100D) and stochasticity levels (ν > 0)?
• RQ4 Can the method handle complex interactions such as congestion and symmetric obstacles in high-dimensional spaces?
• RQ5 To what extent does the GAN-inspired training dynamic prevent mode collapse and ensure accurate density and policy estimation in MFGs?

Key findings

• APAC-Net successfully solves stochastic MFGs up to 100 dimensions, demonstrating scalability where grid-based methods fail due to the curse of dimensionality.
• The method achieves low HJB residual losses—below 100 for 2D, below 500 for 50D, and below 800 for 100D—indicating high accuracy in solving the coupled HJB-FP system.
• In the congestion problem with a bottleneck, the network learns to split the agent density around obstacles, with results consistent across 2D, 50D, and 100D, validating generalization.
• With increased stochasticity (ν = 0.4), the dynamics become dominated by noise, but the network still maintains stable training and meaningful policy learning, showing robustness.
• The analytic solution derived for a quadratic Hamiltonian and logarithmic interaction term (Eq. A.11) was accurately recovered by APAC-Net, confirming the method’s correctness on a known benchmark.
• The use of gradient penalty in the population network (discriminator) effectively prevents mode collapse and stabilizes training, as evidenced by consistent density splitting and low residual losses across dimensions.