[Paper Review] Linear Quadratic Mean Field Teams: Optimal and Approximately Optimal Decentralized Solutions

Jalal Arabneydi, Aditya Mahajan|arXiv (Cornell University)|Aug 31, 2016

Advanced Optimization Algorithms Research3 references21 citations

TL;DR

This paper establishes that linear control strategies are team-optimal for large-scale decentralized systems with exchangeable agents, where dynamics and costs depend on the mean-field of states and actions. It proves that linear solutions are optimal under full mean-field observation and approximately optimal under partial observation, with approximation error inversely proportional to sub-population size, and solves the problem via K+1 decoupled Riccati equations independent of agent count.

ABSTRACT

We consider team optimal control of decentralized systems with linear dynamics, quadratic costs, and arbitrary disturbance that consist of multiple sub-populations with exchangeable agents (i.e., exchanging two agents within the same sub-population does not affect the dynamics or the cost). Such a system is equivalent to one where the dynamics and costs are coupled across agents through the mean-field (or empirical mean) of the states and actions (even when the primitive random variables are non-exchangeable). Two information structures are investigated. In the first, all agents observe their local state and the mean-field of all sub-populations, in the second, all agents observe their local state but the mean-field of only a subset of the sub-populations. Both information structures are non-classical and not partially nested. Nonetheless, it is shown that linear control strategies are optimal for the first and approximately optimal for the second, the approximation error is inversely proportional to the size of the sub-populations whose mean-fields are not observed. The corresponding gains are determined by the solution of K+1 decoupled standard Riccati equations, where K is the number of sub-populations. The dimensions of the Riccati equations do not depend on the size of the sub-populations, thus the solution complexity is independent of the number of agents. Generalizations to major-minor agents, tracking cost, weighted mean-field, and infinite horizon are provided. The results are illustrated using an example of demand response in smart grids.

Motivation & Objective

• To address team optimal control in large-scale decentralized systems with exchangeable agents and mean-field coupling.
• To analyze systems where agents have limited information—specifically, partial access to mean-fields of sub-populations.
• To establish conditions under which linear control strategies are optimal or approximately optimal despite non-classical information structures.
• To develop a scalable solution method independent of agent count through decoupled Riccati equations.
• To generalize results to major-minor agents, tracking costs, weighted mean-fields, and infinite-horizon settings.

Proposed method

• Model the system as a team decision problem with linear dynamics, quadratic costs, and arbitrary disturbances.
• Use exchangeability of agents within sub-populations to reduce the problem to mean-field coupling, even when primitive variables are non-exchangeable.
• Analyze two information structures: full mean-field observation and partial mean-field observation across sub-populations.
• Prove that linear control strategies are optimal when all mean-fields are observed, leveraging the structure of the team problem.
• Derive approximate optimality for partial observation by bounding the error as inversely proportional to the size of unobserved sub-populations.
• Solve the problem via K+1 decoupled standard Riccati equations, where K is the number of sub-populations, ensuring computational scalability.

Experimental results

Research questions

• RQ1 Under what conditions are linear control strategies team-optimal in large-scale mean-field team problems with decentralized information?
• RQ2 How does partial mean-field observation affect the optimality of linear strategies, and what is the quantifiable approximation error?
• RQ3 Can the solution complexity be decoupled from the number of agents in each sub-population?
• RQ4 How do the results extend to major-minor agent structures and tracking cost formulations?
• RQ5 What is the role of the mean-field in enabling scalable solutions for large-scale decentralized systems?

Key findings

• Linear control strategies are team-optimal when all agents observe the mean-field of all sub-populations.
• For partial mean-field observation, linear strategies are approximately optimal, with approximation error inversely proportional to the size of unobserved sub-populations.
• The solution requires solving K+1 decoupled Riccati equations, where K is the number of sub-populations, and the dimension of each Riccati equation is independent of agent count.
• The computational complexity is thus independent of the total number of agents, enabling scalability to large systems.
• Generalizations to major-minor agents, tracking costs, and weighted mean-fields are provided, maintaining the same structural solution framework.
• An example in smart grid demand response demonstrates the practical applicability of the theoretical framework.