[Paper Review] Deep Learning for Mean Field Optimal Transport
This paper proposes three deep learning-based numerical methods for solving mean field optimal transport (MFOT) problems, where agents cooperatively minimize a social cost while reaching a prescribed terminal distribution. The methods leverage neural networks to learn optimal controls, solve forward-backward PDE systems, or use an augmented Lagrangian primal-dual framework, demonstrating accurate solutions on linear-quadratic and congestion-modeling test cases.
Mean field control (MFC) problems have been introduced to study social optima in very large populations of strategic agents. The main idea is to consider an infinite population and to simplify the analysis by using a mean field approximation. These problems can also be viewed as optimal control problems for McKean-Vlasov dynamics. They have found applications in a wide range of fields, from economics and finance to social sciences and engineering. Usually, the goal for the agents is to minimize a total cost which consists in the integral of a running cost plus a terminal cost. In this work, we consider MFC problems in which there is no terminal cost but, instead, the terminal distribution is prescribed. We call such problems mean field optimal transport problems since they can be viewed as a generalization of classical optimal transport problems when mean field interactions occur in the dynamics or the running cost function. We propose three numerical methods based on neural networks. The first one is based on directly learning an optimal control. The second one amounts to solve a forward-backward PDE system characterizing the solution. The third one relies on a primal-dual approach. We illustrate these methods with numerical experiments conducted on two families of examples.
Motivation & Objective
- Address mean field optimal transport problems where the terminal distribution is fixed, rather than minimizing a terminal cost.
- Develop numerical methods based on deep learning to solve MFOT problems that generalize beyond the Schrödinger bridge and standard MFG/MFC setups.
- Overcome limitations of existing methods by enabling solution of problems with complex mean field interactions and non-trivial dynamics.
- Provide scalable, differentiable approaches using neural networks to approximate optimal controls and densities in high-dimensional settings.
- Ensure convergence and accuracy through loss-based training with penalty terms, PDE constraints, and augmented Lagrangian formulations.
Proposed method
- Method 1 uses a deep reinforcement learning-like approach: a neural network learns the optimal control policy via Monte Carlo rollout of the McKean-Vlasov SDE, with a penalty on terminal distribution deviation.
- Method 2 directly approximates the solution to the forward-backward PDE system derived from the optimality conditions of the MFOT problem using a deep neural network.
- Method 3 employs an augmented Lagrangian formulation to enforce the terminal distribution constraint via a primal-dual optimization scheme, with separate networks for value function, density, and Lagrange multiplier.
- All methods use fully connected feedforward neural networks with residual connections, ReLU or sigmoid activations, and are trained via stochastic gradient descent with minibatch sampling.
- Hyperparameters such as penalty weights $ C_W $, $ C_0^{(KFP)} $, $ C_T^{(KFP)} $, $ C^{(KFP)} $, $ C^{(HJB)} $, and $ r $ in the augmented Lagrangian are tuned empirically for stability and convergence.
- For LQ problems, an additional quadratic correction is added to the control network output to improve accuracy and match analytical solutions.
Experimental results
Research questions
- RQ1Can deep learning effectively solve mean field optimal transport problems with a prescribed terminal distribution and no terminal cost?
- RQ2How do different neural network architectures and loss formulations compare in terms of convergence and accuracy for MFOT problems?
- RQ3Can the proposed methods handle non-linear dynamics and complex mean field interactions such as congestion effects?
- RQ4What is the impact of hyperparameters like penalty weights and augmented Lagrangian parameter $ r $ on training stability and performance?
- RQ5To what extent can deep learning-based methods generalize beyond the Schrödinger bridge and linear-quadratic cases?
Key findings
- The three proposed methods achieve high accuracy in matching the analytical solution for the linear-quadratic (LQ) test case, demonstrating convergence and robustness.
- Method 1 successfully learns the optimal control policy by minimizing a penalized cost functional, with the penalty term $ C_W $ adjusted based on state dimension and computational cost.
- Method 2 effectively solves the forward-backward PDE system using a deep neural network, with loss terms enforcing initial, terminal, and PDE constraints.
- Method 3 using the augmented Lagrangian approach shows stable convergence with $ r = 0.1 $, and the use of sigmoid activation on the Lagrange multiplier network ensures bounded density estimates.
- All methods successfully handle non-trivial mean field interactions, including congestion effects, as demonstrated in the congestion test case.
- The numerical experiments confirm that the methods are scalable and effective in high-dimensional settings, with training stabilized through careful hyperparameter tuning and minibatch sampling.
Better researchstarts right now
From reading papers to final review, dramatically reduce your research time.
No credit card · Free plan available
This review was created by AI and reviewed by human editors.