[Paper Review] Dynamic Population Games: A Tractable Intersection of Mean-Field Games and Population Games

Ezzat Elokda, Saverio Bolognani+3 more|arXiv (Cornell University)|Apr 29, 2021

Game Theory and Applications32 references4 citations

TL;DR

This paper introduces dynamic population games, a novel framework that merges mean-field games and population games by endowing agents with time-varying states alongside fixed types. It establishes that stationary equilibria in these dynamic games reduce to standard Nash equilibria in classical population games, enabling tractable analysis and practical application in domains like autonomous mobility and epidemic modeling.

ABSTRACT

In many real-world large-scale decision problems, self-interested agents have individual dynamics and optimize their own long-term payoffs. Important examples include the competitive access to shared resources (e.g., roads, energy, or bandwidth) but also non-engineering domains like epidemic propagation and control. These problems are natural to model as mean-field games. Existing mathematical formulations of mean field games have had limited applicability in practice, since they require solving non-standard initial-terminal-value problems that are tractable only in limited special cases. In this letter, we propose a novel formulation, along with computational tools, for a practically relevant class of Dynamic Population Games (DPGs), which correspond to discrete-time, finite-state-and-action, stationary mean-field games. Our main contribution is a mathematical reduction of Stationary Nash Equilibria (SNE) in DPGs to standard Nash Equilibria (NE) in static population games. This reduction is leveraged to guarantee the existence of a SNE, develop an evolutionary dynamics-based SNE computation algorithm, and derive simple conditions that guarantee stability and uniqueness of the SNE. We provide two examples of applications: fair resource allocation with heterogeneous agents and control of epidemic propagation. Open source software for SNE computation: https://gitlab.ethz.ch/elokdae/dynamic-population-games

Motivation & Objective

• To model large-population strategic interactions where agents have both fixed types and time-varying states that influence and are influenced by actions.
• To address the limitations of stochastic games in large populations by incorporating anonymity and macroscopic state distributions.
• To develop a solution concept—stationary equilibrium—that ensures all agents play best responses and the state distribution remains invariant.
• To establish a reduction of dynamic population games to classical population games, simplifying equilibrium analysis.
• To enable practical modeling of real-world systems such as traffic dynamics and epidemic spread with strategic agent adaptation.

Proposed method

• Model agents with discrete types (fixed characteristics) and discrete time-varying states (e.g., hunger, fatigue) that evolve based on actions and societal state distributions.
• Define individual state transitions via Markovian transition probabilities dependent on the current social state (distribution of actions and states).
• Derive societal-level state dynamics using both synchronous and asynchronous interaction models, with the latter derived via Poisson processes and infinitesimal time steps.
• Formulate rewards based on state-action pairs and social state, enabling utility maximization under long-term discounted rewards.
• Introduce the single-stage deviation principle to simplify dynamic decision-making, reducing infinite-horizon optimization to a one-step best-response problem.
• Prove that stationary equilibria in dynamic population games correspond exactly to standard Nash equilibria in an equivalent classical population game.

Experimental results

Research questions

• RQ1 How can strategic interactions in large populations be modeled when agents have time-varying internal states that affect and are affected by their decisions?
• RQ2 Can the complexity of dynamic, large-population games be reduced to classical, static population game frameworks?
• RQ3 What solution concept ensures stability and optimality in dynamic population games with evolving agent states?
• RQ4 How do asynchronous interaction models affect the evolution of state distributions in large populations?
• RQ5 What conditions guarantee the existence of stationary equilibria in such dynamic settings?

Key findings

• At least one stationary equilibrium exists in every dynamic population game, ensuring the existence of stable strategic configurations.
• Stationary equilibria in dynamic population games are mathematically equivalent to standard Nash equilibria in a suitably defined classical population game.
• The single-stage deviation principle enables tractable equilibrium computation by reducing infinite-horizon optimization to a one-step best-response problem.
• The societal state dynamics are derived as a system of differential equations, with the asynchronous model derived using Poisson interaction processes and infinitesimal time analysis.
• The state distribution evolves according to a linear ODE: $\dot{d}_{\tau}[x] = \delta_{\tau} \left( \sum_{x' \in \mathcal{X}} d_{\tau}[x'] P_{\tau}[x \mid x'] - d_{\tau}[x] \right)$, capturing inflows and outflows.
• The framework enables direct application to real-world systems, such as decentralized traffic management and adaptive epidemic modeling with strategic behavior.