[Paper Review] On Finding Local Nash Equilibria (and Only Local Nash Equilibria) in Zero-Sum Games

Eric Mazumdar, Michael I. Jordan+1 more|arXiv (Cornell University)|Jan 3, 2019

Advanced Bandit Algorithms Research23 references78 citations

TL;DR

The paper introduces Local Symplectic Surgery, a two-timescale method that converges to local Nash equilibria in two-player zero-sum continuous games and avoids non-Nash attracting points that hinder gradient dynamics.

ABSTRACT

We propose local symplectic surgery, a two-timescale procedure for finding local Nash equilibria in two-player zero-sum games. We first show that previous gradient-based algorithms cannot guarantee convergence to local Nash equilibria due to the existence of non-Nash stationary points. By taking advantage of the differential structure of the game, we construct an algorithm for which the local Nash equilibria are the only attracting fixed points. We also show that the algorithm exhibits no oscillatory behaviors in neighborhoods of equilibria and show that it has the same per-iteration complexity as other recently proposed algorithms. We conclude by validating the algorithm on two numerical examples: a toy example with multiple Nash equilibria and a non-Nash equilibrium, and the training of a small generative adversarial network (GAN).

Motivation & Objective

• Identify limitations of gradient-based methods in zero-sum games due to non-Nash attracting points.
• Propose a dynamics-based algorithm whose attracting points correspond exactly to local (differential) Nash equilibria.
• Ensure the method avoids oscillations near equilibria and has comparable per-iteration complexity to existing gradient methods.

Proposed method

• Define a limiting continuous-time flow whose attracting points are exactly local Nash equilibria.
• Introduce a two-timescale discretization (Local Symplectic Surgery) that approximates the limiting dynamics without matrix inversion.
• Use an auxiliary variable to implement a fast-adjustment term that cancels the antisymmetric part of the Jacobian near critical points.
• Prove that the only attracting fixed points of the limiting dynamics are differential Nash equilibria.
• Demonstrate robustness to unbiased stochastic estimates and noise via a two-timescale stochastic approximation framework.
• Provide a small GAN experiment and a toy 2-D example to validate convergence to local Nash equilibria.

Experimental results

Research questions

• RQ1 Can gradient-based dynamics in zero-sum games converge to non-Nash stationary points, and under what conditions?
• RQ2 How can one design dynamics whose attracting fixed points coincide with (differential) local Nash equilibria in zero-sum games?
• RQ3 Does a two-timescale discretization reliably track the desired continuous-time dynamics in the presence of noise?
• RQ4 What is the per-iteration computational cost relative to standard gradient methods, and can matrix inversions be avoided?
• RQ5 Do empirical experiments (toy example and GAN) support convergence to local Nash equilibria and avoidance of non-Nash attractors?

Key findings

• The proposed Local Symplectic Surgery ensures the only attracting fixed points are local Nash equilibria.
• The continuous-time dynamics do not exhibit oscillations near equilibria and have real eigenvalues at critical points, avoiding limit cycles.
• A two-timescale implementation can be computed without matrix inversions and maintains comparable complexity to other gradient-based methods.
• The method eliminates convergence to non-Nash LASEs that ensnare other gradient-adjusted methods such as consensus optimization and symplectic gradient adjustment.
• Numerical experiments show convergence to local Nash equilibria in a toy example and successful training of a small GAN using the proposed approach.