[Paper Review] A Hybrid Optimal Control Approach to LQG Mean Field Games with Switching and Stopping Strategies.

Dena Firoozi, Ali Pakniyat+1 more|arXiv (Cornell University)|Oct 6, 2018

Stochastic processes and financial applications14 references3 citations

TL;DR

This paper proposes a hybrid optimal control framework integrating Mean Field Game (MFG) and Linear Quadratic Gaussian (LQG) theory to derive an $5c$-Nash equilibrium in a non-cooperative stochastic game with one major agent and two subpopulations of minor agents. The approach models switching dynamics and stopping times via discrete states, yielding optimal switching and stopping strategies alongside best-response controls for all agents under quadratic cost criteria.

ABSTRACT

A novel framework is presented that combines Mean Field Game (MFG) theory and Hybrid Optimal Control (HOC) theory to obtain a unique $\epsilon$-Nash equilibrium for a non-cooperative game with stopping times. We consider the case where there exists one major agent with a significant influence on the system together with a large number of minor agents constituting two subpopulations, each with individually asymptotically negligible effect on the whole system. Each agent has stochastic linear dynamics with quadratic costs, and the agents are coupled in their dynamics by the average state of minor agents (i.e. the empirical mean field). The hybrid feature enters via the indexing by discrete states: (i) the switching of the major agent between alternative dynamics or (ii) the termination of the agents' trajectories in one or both of the subpopulations of minor agents. Optimal switchings and stopping time strategies together with best response control actions for, respectively, the major agent and all minor agents are established with respect to their individual cost criteria by an application of LQG HOC theory.

Motivation & Objective

• To address non-cooperative stochastic games involving a major agent and a large number of minor agents with asymptotically negligible individual influence.
• To model systems where agents experience switching dynamics or termination (stopping times) via discrete state indexing.
• To establish optimal control strategies—both switching and stopping—for the major agent and minor agents under quadratic cost functions.
• To derive a unique $5c$-Nash equilibrium by combining LQG and hybrid optimal control theory.
• To ensure the equilibrium is robust by coupling agent dynamics through the empirical mean field of minor agents.

Proposed method

• Formulates a hybrid optimal control (HOC) structure where discrete states represent switching between dynamics or termination of agent trajectories.
• Applies LQG theory to solve the optimal control problem under linear stochastic dynamics and quadratic cost functions.
• Models the coupling between agents via the empirical mean field, representing the average state of minor agents.
• Derives best-response control actions for all agents using dynamic programming and Hamilton-Jacobi-Bellman equations in the hybrid setting.
• Establishes the existence of an $5c$-Nash equilibrium through the interplay of MFG and HOC frameworks.
• Uses a two-subpopulation structure: minor agents are partitioned into two groups, each with distinct dynamics and stopping behaviors.

Experimental results

Research questions

• RQ1 How can a hybrid optimal control framework be extended to incorporate mean field game dynamics in systems with switching and stopping strategies?
• RQ2 What conditions ensure the existence of a unique $5c$-Nash equilibrium in a non-cooperative game with one major agent and two subpopulations of minor agents?
• RQ3 How do switching dynamics and stopping times affect the optimal control strategies of the major agent and minor agents under quadratic cost criteria?
• RQ4 In what way does the empirical mean field of minor agents influence the equilibrium strategies in the hybrid LQG setting?
• RQ5 Can the combination of LQG and HOC theories yield tractable and optimal control policies for both switching and stopping in large-population stochastic games?

Key findings

• The framework successfully establishes a unique $5c$-Nash equilibrium for the non-cooperative game involving a major agent and two subpopulations of minor agents.
• Optimal switching and stopping time strategies are derived for the major agent and minor agents, respectively, through the integration of LQG and HOC theory.
• The mean field coupling via the empirical average state of minor agents ensures asymptotic negligible individual impact while maintaining system-wide interaction.
• Best-response control actions for all agents are explicitly characterized using dynamic programming and Hamilton-Jacobi-Bellman equations in the hybrid control setting.
• The hybrid structure, incorporating discrete states for switching and termination, enables modeling of complex agent behaviors in large-scale stochastic games.
• The solution framework is scalable and applicable to systems where agents exhibit both switching dynamics and finite-horizon stopping behaviors.