[Paper Review] Distributionally Robust Games: f-Divergence and Learning

Dario Bauso, Jian Gao+1 more|arXiv (Cornell University)|Feb 17, 2017

Agricultural risk and resilience4 citations

TL;DR

This paper introduces distributionally robust games using f-divergence to model distributional uncertainty, enabling robust equilibria under adversarial distribution shifts. It proposes a triality-based dimension reduction and stochastic Bregman learning algorithms that achieve double exponential convergence, significantly outperforming gradient dynamics in convergence speed and robustness—even in non-convex settings with multimodal objectives.

ABSTRACT

In this paper we introduce the novel framework of distributionally robust games. These are multi-player games where each player models the state of nature using a worst-case distribution, also called adversarial distribution. Thus each player's payoff depends on the other players' decisions and on the decision of a virtual player (nature) who selects an adversarial distribution of scenarios. This paper provides three main contributions. Firstly, the distributionally robust game is formulated using the statistical notions of $f$-divergence between two distributions, here represented by the adversarial distribution, and the exact distribution. Secondly, the complexity of the problem is significantly reduced by means of triality theory. Thirdly, stochastic Bregman learning algorithms are proposed to speedup the computation of robust equilibria. Finally, the theoretical findings are illustrated in a convex setting and its limitations are tested with a non-convex non-concave function.

Motivation & Objective

• Address the limitations of Bayesian and distribution-free game models by introducing a new framework that models uncertainty via f-divergence between distributions.
• Overcome the curse of dimensionality in robust game problems through triality theory, enabling efficient computation of robust equilibria.
• Develop a Bregman-based learning algorithm that accelerates convergence to robust equilibria without requiring strong convexity.
• Demonstrate the method's effectiveness in both convex and non-convex settings, including multimodal payoff functions.
• Extend the framework to finite action spaces via mixed strategy convexification and prove existence of robust mixed equilibria.

Proposed method

• Formulate distributionally robust games using f-divergence to define a divergence ball around the true distribution, modeling worst-case adversarial distributions.
• Apply triality theory to transform the infinite-dimensional max-min problem into a finite-dimensional dual problem, reducing computational complexity.
• Propose Bregman dynamics based on Bregman divergence to accelerate convergence, with theoretical guarantees of double exponential decay.
• Implement stochastic Bregman learning using particle swarms to approximate the robust equilibrium in high-dimensional and non-convex settings.
• Use Legendre-Fenchel conjugates of f-divergence generators (e.g., log-sum-exp for exponential family) to derive dual formulations.
• Apply the Bregman flow to both single-agent and multi-agent settings, ensuring convergence to distributionally robust equilibria.

Experimental results

Research questions

• RQ1 How can distributionally robust games be formally defined using f-divergence to model distributional uncertainty in multi-agent systems?
• RQ2 Can triality theory be leveraged to reduce the dimensionality of distributionally robust game problems and ensure existence of equilibria?
• RQ3 How does Bregman-based learning compare to standard gradient dynamics in convergence speed and robustness for convex and non-convex games?
• RQ4 What is the performance of stochastic Bregman learning in non-convex, multimodal payoff functions with multiple local extrema?
• RQ5 Can the framework be extended to finite action spaces via mixed strategy convexification and what conditions ensure existence of robust equilibria?

Key findings

• The stochastic Bregman learning algorithm achieves convergence in approximately 20 times less time than classical gradient dynamics, as shown in the convex case with 1000 samples.
• Double exponential decay is theoretically established for the Bregman dynamics, indicating rapid convergence to the distributionally robust equilibrium.
• In the non-convex setting with a multimodal objective function, the Bregman algorithm successfully converges to the robust Nash equilibrium at (7.9, 7.9) with equilibrium performance around 7.88.
• The algorithm exhibits robustness even when perturbed, as seen in the transition from the robust equilibrium (7.9, 7.9) to a local maximum (7.9, 5.1) under different initializations.
• Existence of distributionally robust equilibria is proven under suitable conditions, including for finite action spaces via mixed strategy convexification.
• The method does not require strong convexity assumptions, distinguishing it from classical gradient-based approaches and enabling broader applicability.