[Paper Review] Poincaré Recurrence, Cycles and Spurious Equilibria in Gradient-Descent-Ascent for Non-Convex Non-Concave Zero-Sum Games

Emmanouil-Vasileios Vlatakis-Gkaragkounis, Lampros Flokas+1 more|arXiv (Cornell University)|Oct 28, 2019

Markov Chains and Monte Carlo Methods28 references19 citations

TL;DR

This paper analyzes gradient-descent-ascent dynamics in non-convex non-concave zero-sum games, showing that such dynamics can exhibit Poincaré recurrence, periodic cycles, and convergence to spurious equilibria—rather than converging to meaningful min-max solutions—due to the indirect competition structure seen in GANs. The study combines optimization, game theory, and dynamical systems to establish these behaviors for a positive measure of initial conditions.

ABSTRACT

We study a wide class of non-convex non-concave min-max games that generalizes over standard bilinear zero-sum games. In this class, players control the inputs of a smooth function whose output is being applied to a bilinear zero-sum game. This class of games is motivated by the indirect nature of the competition in Generative Adversarial Networks, where players control the parameters of a neural network while the actual competition happens between the distributions that the generator and discriminator capture. We establish theoretically, that depending on the specific instance of the problem gradient-descent-ascent dynamics can exhibit a variety of behaviors antithetical to convergence to the game theoretically meaningful min-max solution. Specifically, different forms of recurrent behavior (including periodicity and Poincar\'{e} recurrence) are possible as well as convergence to spurious (non-min-max) equilibria for a positive measure of initial conditions. At the technical level, our analysis combines tools from optimization theory, game theory and dynamical systems.

Motivation & Objective

• To understand the dynamics of gradient-descent-ascent in non-convex non-concave min-max games, particularly in the context of Generative Adversarial Networks (GANs).
• To identify conditions under which convergence to min-max equilibria fails due to recurrent or periodic behaviors.
• To analyze the existence of spurious equilibria that attract trajectories for a positive measure of initial conditions.
• To formally characterize the instability of gradient-descent-ascent in a broad class of games extending standard bilinear zero-sum games.

Proposed method

• Formalizing a class of non-convex non-concave games where players control inputs to a smooth function, whose output feeds into a bilinear zero-sum game.
• Applying tools from dynamical systems theory to analyze long-term behavior of gradient-descent-ascent trajectories.
• Using optimization and game-theoretic frameworks to characterize equilibria and stability properties.
• Establishing the existence of Poincaré recurrence and periodic orbits in the dynamics via topological and analytical arguments.
• Demonstrating that spurious equilibria can be attracting for a positive measure of initial conditions using measure-theoretic arguments.
• Combining techniques from optimization, game theory, and dynamical systems to analyze convergence and recurrence in complex game dynamics.

Experimental results

Research questions

• RQ1 Can gradient-descent-ascent dynamics in non-convex non-concave zero-sum games converge to min-max equilibria, or do recurrent behaviors emerge instead?
• RQ2 Under what conditions do spurious equilibria attract trajectories for a positive measure of initial conditions?
• RQ3 To what extent do Poincaré recurrence and periodicity manifest in the dynamics of such games?
• RQ4 How does the indirect competition structure in GANs give rise to instability in training dynamics?
• RQ5 What theoretical guarantees can be made about convergence or non-convergence in this class of games?

Key findings

• Gradient-descent-ascent dynamics in the studied class of games can exhibit Poincaré recurrence, meaning trajectories return arbitrarily close to their initial states infinitely often.
• Periodic behavior, including stable limit cycles, is possible in the dynamics for certain parameter configurations.
• Spurious (non-min-max) equilibria can be attracting for a positive measure of initial conditions, indicating that convergence to meaningful solutions is not generic.
• Theoretical analysis confirms that convergence to min-max equilibria is not guaranteed, even in smooth, well-structured non-convex non-concave games.
• The dynamics are shown to be fundamentally unstable in the sense of topological dynamics, with recurrent and non-convergent trajectories being prevalent.
• The results highlight inherent limitations of gradient-descent-ascent in GAN training, where such behaviors may explain mode collapse and training instability.