[Paper Review] Solving Combinatorial Games using Products, Projections and Lexicographically Optimal Bases

Swati Gupta, Michel X. Goemans+1 more|arXiv (Cornell University)|Mar 1, 2016

Advanced Bandit Algorithms Research54 references3 citations

TL;DR

This paper presents efficient algorithms for computing symmetric Nash equilibria in two-player zero-sum combinatorial games using lexicographically optimal bases and generalized counting oracles. It introduces a novel primal-style algorithm for Bregman projections on polymatroid base polytopes and a polynomial-time simulation of the multiplicative weights update algorithm via approximate counting oracles, enabling efficient solution of games with exponentially many strategies such as spanning trees and matchings.

ABSTRACT

In order to find Nash-equilibria for two-player zero-sum games where each player plays combinatorial objects like spanning trees, matchings etc, we consider two online learning algorithms: the online mirror descent (OMD) algorithm and the multiplicative weights update (MWU) algorithm. The OMD algorithm requires the computation of a certain Bregman projection, that has closed form solutions for simple convex sets like the Euclidean ball or the simplex. However, for general polyhedra one often needs to exploit the general machinery of convex optimization. We give a novel primal-style algorithm for computing Bregman projections on the base polytopes of polymatroids. Next, in the case of the MWU algorithm, although it scales logarithmically in the number of pure strategies or experts $N$ in terms of regret, the algorithm takes time polynomial in $N$; this especially becomes a problem when learning combinatorial objects. We give a general recipe to simulate the multiplicative weights update algorithm in time polynomial in their natural dimension. This is useful whenever there exists a polynomial time generalized counting oracle (even if approximate) over these objects. Finally, using the combinatorial structure of symmetric Nash-equilibria (SNE) when both players play bases of matroids, we show that these can be found with a single projection or convex minimization (without using online learning).

Motivation & Objective

• To address the computational challenge of finding Nash equilibria in two-player zero-sum games with exponentially many pure strategies, such as spanning trees or matchings.
• To develop efficient online learning algorithms—specifically online mirror descent and multiplicative weights update—for games where strategy polytopes are defined by combinatorial objects.
• To enable polynomial-time computation of symmetric Nash equilibria in matroid-based games by leveraging lexicographically optimal bases.
• To reduce the computational complexity of learning algorithms by exploiting combinatorial structure and generalized approximate counting oracles.
• To establish a connection between symmetric Nash equilibria and lexicographically optimal bases in matroid base polytopes, enabling exact and efficient computation.

Proposed method

• Proposes a novel primal-style algorithm for computing Bregman projections onto base polytopes of polymatroids, enabling efficient online mirror descent in combinatorial games.
• Introduces a general framework to simulate the multiplicative weights update (MWU) algorithm in time polynomial in the natural dimension of the strategy space, provided a generalized approximate counting oracle exists.
• Uses the equivalence between optimization and separation via the ellipsoid method to solve the min-max problem over strategy polytopes in succinct games.
• Applies the concept of lexicographically optimal bases from submodular function theory to characterize and compute symmetric Nash equilibria in matroid games.
• Employs the Inc-fix algorithm and results from Fujishige (1980) to compute lexicographically optimal bases efficiently with respect to weights derived from the loss matrix.
• Establishes that symmetric Nash equilibria in matroid games with diagonal loss matrices correspond exactly to lexicographically optimal bases under inverse loss weights.

Experimental results

Research questions

• RQ1 Can online mirror descent be efficiently implemented for base polytopes of polymatroids without relying on general convex optimization?
• RQ2 Can the multiplicative weights update algorithm be simulated in polynomial time for 0/1 polytopes when the number of pure strategies is exponential?
• RQ3 Under what conditions does a symmetric Nash equilibrium exist and become unique in matroid-based combinatorial games?
• RQ4 How are symmetric Nash equilibria in matroid games related to lexicographically optimal bases of the base polytope?
• RQ5 Can the structure of the loss matrix and the matroid's rank function be used to characterize and compute symmetric equilibria efficiently?

Key findings

• A novel primal-style algorithm for Bregman projections on polymatroid base polytopes enables efficient online mirror descent in combinatorial games.
• The multiplicative weights update algorithm can be simulated in time polynomial in the natural dimension of the strategy space using a generalized approximate counting oracle.
• Symmetric Nash equilibria in matroid games with diagonal loss matrices are equivalent to lexicographically optimal bases under weights $ w(e) = 1/L_{e,e} $, enabling efficient computation.
• For a matroid game with a diagonal loss matrix $ L $, the symmetric Nash equilibrium is unique if $ L $ is positive-definite.
• A symmetric Nash equilibrium exists in a matroid game if and only if every block of the underlying graph is uniformly dense when the loss matrix is the identity.
• Given any positive point $ x $ in the base polytope of a matroid, a matroid game can be constructed such that $ (x,x) $ is a symmetric Nash equilibrium by setting $ L_{e,e} = 1/x_e $.