[Paper Review] On the Optimal Rate for the Convergence Problem in Mean Field Control

Samuel Daudin, François Delarue+1 more|arXiv (Cornell University)|May 15, 2023

Bone and Joint DiseasesMedicine3 citations

TL;DR

This paper establishes optimal convergence rates for mean field control systems when the limiting value function lacks full regularity—specifically, when it is not differentiable or unique. By introducing a novel mollification technique using sup-convolution in a Hilbert space of Fourier modes, the authors derive sharp convergence rates: $N^{-1/2}$ under high regularity and $N^{-2/(3d+6)}$ under Lipschitz and semi-concave conditions, matching the optimal rate for uncontrolled systems up to a constant factor.

ABSTRACT

The goal of this work is to obtain optimal rates for the convergence problem in mean field control. Our analysis covers cases where the solutions to the limiting problem may not be unique nor stable. Equivalently the value function of the limiting problem might not be differentiable on the entire space. Our main result is then to derive sharp rates of convergence in two distinct regimes. When the data is sufficiently regular, we obtain rates proportional to $N^{-1/2}$, with $N$ being the number of particles. When the data is merely Lipschitz and semi-concave with respect to the first Wasserstein distance, we obtain rates proportional to $N^{-2/(3d+6)}$. Noticeably, the exponent $2/(3d+6)$ is close to $1/d$, which is the optimal rate of convergence for uncontrolled particle systems driven by data with a similar regularity. The key argument in our approach consists in mollifying the value function of the limiting problem in order to produce functions that are almost classical sub-solutions to the limiting Hamilton-Jacobi equation (which is a PDE set on the space of probability measures). These sub-solutions can be projected onto finite dimensional spaces and then compared with the value functions associated with the particle systems. In the end, this comparison is used to prove the most demanding bound in the estimates. The key challenge therein is thus to exhibit an appropriate form of mollification. We do so by employing sup-convolution within a convenient functional Hilbert space. To make the whole easier, we limit ourselves to the periodic setting. We also provide some examples to show that our results are sharp up to some extent.

Motivation & Objective

• To resolve the convergence problem in mean field control when the limiting value function is not differentiable or unique.
• To derive sharp, optimal convergence rates for particle systems approximating the mean field limit under minimal regularity assumptions.
• To develop a new mollification method capable of regularizing non-classical solutions of the Hamilton-Jacobi-Bellman equation on the Wasserstein space.
• To show that the derived rate $N^{-2/(3d+6)}$ is nearly optimal, matching the known optimal rate for uncontrolled systems under similar regularity.
• To validate the sharpness of the results through explicit examples and comparison with classical estimates.

Proposed method

• The authors employ sup-convolution within a functional Hilbert space of Fourier coefficients to mollify the value function of the limiting mean field control problem.
• This mollification produces functions that are almost classical sub-solutions to the infinite-dimensional Hamilton-Jacobi equation on the space of probability measures.
• The mollified functions are projected onto finite-dimensional subspaces via Fourier truncation, enabling comparison with particle system value functions.
• The comparison is used to derive the most challenging bound in the convergence estimate, relying on the regularity and stability of the mollified sub-solutions.
• The analysis is restricted to the periodic setting on the torus $\mathbb{T}^d$ to simplify the functional analytic framework and ensure well-behaved Fourier representations.
• A key technical tool is the use of a smooth kernel $\rho_n$ with compact support in $\mathcal{B}_n$, ensuring $\Phi^{\eta,n}$ is infinitely differentiable and continuously differentiable in $m$.

Experimental results

Research questions

• RQ1 What is the optimal rate of convergence for mean field control when the limiting value function is not differentiable?
• RQ2 Can a mollification technique be designed to regularize non-smooth solutions of the Hamilton-Jacobi-Bellman equation in the mean field setting?
• RQ3 How does the convergence rate depend on the regularity of the data, particularly when only Lipschitz and semi-concave conditions hold?
• RQ4 Is the derived rate $N^{-2/(3d+6)}$ sharp, and how does it compare to known rates in uncontrolled systems?
• RQ5 Can the mollification method be constructed in a way that preserves the structure of the Hamilton-Jacobi equation and allows for effective comparison with finite-particle systems?

Key findings

• Under high regularity of the data, the convergence rate is $N^{-1/2}$, matching the classical rate for smooth problems.
• When the data is only Lipschitz and semi-concave with respect to the first Wasserstein distance, the convergence rate is $N^{-2/(3d+6)}$.
• The exponent $2/(3d+6)$ is asymptotically close to $1/d$, which is the known optimal rate for uncontrolled particle systems under similar regularity.
• The mollification via sup-convolution in the Fourier-Hilbert space framework ensures that the regularized value functions are smooth and can be compared effectively with finite-dimensional particle system value functions.
• The method establishes a new bound that is the most challenging in the convergence estimate, relying on the construction of sub-solutions that are almost classical.
• The paper provides examples demonstrating that the derived rates are sharp up to a constant factor, confirming the optimality of the results under the given assumptions.