[Paper Review] Mean-field optimal control and optimality conditions in the space of probability measures

Martin Burger, René Pinnau+2 more|arXiv (Cornell University)|Feb 14, 2019

Groundwater flow and contamination studies4 citations

TL;DR

This paper develops a mean-field optimal control framework for systems governed by probability measures, deriving first-order optimality conditions via a Lagrangian approach in the space of probability measures. It establishes a rigorous link between particle-level and mean-field adjoints, proves convergence of optimal controls with explicit rates as particle count increases, and justifies $L^2$-based numerical methods through a direct connection to measure-space adjoints.

ABSTRACT

We derive a framework to compute optimal controls for problems with states in the space of probability measures. Since many optimal control problems constrained by a system of ordinary differential equations (ODE) modelling interacting particles converge to optimal control problems constrained by a partial differential equation (PDE) in the mean-field limit, it is interesting to have a calculus directly on the mesoscopic level of probability measures which allows us to derive the corresponding first-order optimality system. In addition to this new calculus, we provide relations for the resulting system to the first-order optimality system derived on the particle level, and the first-order optimality system based on $L^2$-calculus under additional regularity assumptions. We further justify the use of the $L^2$-adjoint in numerical simulations by establishing a link between the adjoint in the space of probability measures and the adjoint corresponding to $L^2$-calculus. Moreover, we prove a convergence rate for the convergence of the optimal controls corresponding to the particle formulation to the optimal controls of the mean-field problem as the number of particles tends to infinity.

Motivation & Objective

• To develop a first-order optimality system for optimal control problems with states in the space of probability measures, enabling direct numerical implementation.
• To bridge the gap between Hamiltonian-based approaches (e.g., Pontryagin principles) and Lagrangian-based derivations in the mean-field limit.
• To rigorously justify the use of $L^2$-adjoints in numerical simulations by establishing a direct correspondence with adjoints in the space of probability measures.
• To prove a convergence rate for optimal controls derived from $N$-particle systems to their mean-field counterparts as $N \to \infty$.

Proposed method

• Derives first-order optimality conditions using a Lagrangian formulation in the space of probability measures, avoiding explicit use of Lagrangian flows.
• Introduces a momentum equation as the adjoint system, characterizing the dual variable in the measure space.
• Establishes a direct link between the adjoint in the measure space and the classical $L^2$-adjoint under regularity assumptions.
• Uses the $W_2$ Wasserstein distance to quantify the distance between probability measures and derives stability estimates for the state and control variables.
• Applies Gronwall's inequality to the time-derivative of the squared Wasserstein distance between measure trajectories to prove stability.
• Employs Poincaré inequality and energy estimates to derive convergence rates for optimal controls as $N \to \infty$.

Experimental results

Research questions

• RQ1 How can first-order optimality conditions be derived directly in the space of probability measures for mean-field optimal control problems?
• RQ2 What is the precise relationship between the adjoint variable in the measure space and the $L^2$-adjoint used in standard PDE-constrained optimization?
• RQ3 How do optimal controls from finite-particle systems converge to those in the mean-field limit, and what is the convergence rate?
• RQ4 Can the Lagrangian approach in measure space be used to justify and unify existing Hamiltonian and $L^2$-based formulations?
• RQ5 Under what conditions is the minimizer of the mean-field optimal control problem unique?

Key findings

• The paper establishes a first-order optimality system in the space of probability measures using a Lagrangian framework, providing a foundation for numerical implementation.
• A momentum equation is derived as the adjoint system, offering a geometric and analytical characterization of the dual variable in the measure space.
• A direct link is proven between the adjoint in the measure space and the $L^2$-adjoint, justifying the use of $L^2$-calculus in numerical simulations.
• The optimal controls of the $N$-particle system converge to the mean-field optimal control with a rate of $\mathcal{O}(1/\sqrt{N})$ under appropriate regularity and stability conditions.
• Uniqueness of minimizers for both the particle and mean-field problems is established under the same assumptions, via stability estimates and the convergence result.