[Paper Review] Asynchronous Schemes for Stochastic and Misspecified Potential Games and Nonconvex Optimization

Jinlong Lei, Uday V. Shanbhag|arXiv (Cornell University)|Nov 6, 2017

Distributed Control Multi-Agent Systems67 references3 citations

TL;DR

This paper proposes asynchronous inexact proximal best-response schemes for stochastic potential games with parametric misspecification, combining stochastic approximation and projected gradient methods to enable decentralized, low-communication equilibrium computation. It establishes almost sure convergence to a connected subset of Nash equilibria and gap function convergence to zero in mean, even under delayed information and nonconvex potential functions.

ABSTRACT

The distributed computation of equilibria and optima has seen growing interest in a broad collection of networked problems. We consider the computation of equilibria of convex stochastic Nash games characterized by a possibly nonconvex potential function. Our focus is on two classes of stochastic Nash games: (P1): A potential stochastic Nash game, in which each player solves a parameterized stochastic convex program; and (P2): A misspecified generalization, where the player-specific stochastic program is complicated by a parametric misspecification. In both settings, exact proximal BR solutions are generally unavailable in finite time since they necessitate solving parameterized stochastic programs. Consequently, we design two asynchronous inexact proximal BR schemes to solve the problems, where in each iteration a single player is randomly chosen to compute an inexact proximal BR solution with rivals' possibly outdated information. Yet, in the misspecified regime (P2), each player possesses an extra estimate of the misspecified parameter and updates its estimate by a projected stochastic gradient (SG) algorithm. By Since any stationary point of the potential function is a Nash equilibrium of the associated game, we believe this paper is amongst the first ones for stochastic nonconvex (but block convex) optimization problems equipped with almost-sure convergence guarantees. These statements can be extended to allow for accommodating weighted potential games and generalized potential games. Finally, we present preliminary numerics based on applying the proposed schemes to congestion control and Nash-Cournot games.

Motivation & Objective

• To develop decentralized, low-communication algorithms for computing Nash equilibria in stochastic potential games with parametric uncertainty.
• To address the challenge of inexact proximal best-response solutions due to stochastic program intractability in finite time.
• To integrate joint learning of unknown parameters via projected stochastic gradient methods within asynchronous schemes.
• To establish almost sure convergence of iterates to a connected subset of Nash equilibria under general delay and inexactness conditions.
• To extend convergence guarantees to strongly convex player problems and demonstrate applicability to congestion control and Nash-Cournot games.

Proposed method

• Uses asynchronous inexact proximal best-response schemes where one player updates per iteration using delayed rival information and inexact solutions via stochastic approximation.
• Employs a projected stochastic gradient algorithm with increasing batch sizes to estimate unknown parameters in misspecified games.
• Applies block-coordinate descent to the potential function, leveraging equivalence between Nash equilibria and stationary points of the potential.
• Imposes conditions on inexactness sequences to ensure almost sure convergence to a connected subset of Nash equilibria.
• Introduces a gap function that converges to zero in mean, indicating equilibrium quality.
• Extends results to weighted and generalized potential games through appropriate modifications of the scheme.

Experimental results

Research questions

• RQ1 Can asynchronous inexact proximal best-response schemes converge almost surely to a connected subset of Nash equilibria in stochastic potential games with parametric misspecification?
• RQ2 How does the convergence behavior change when player-specific problems are strongly convex versus convex?
• RQ3 What is the trade-off between iteration complexity and communication overhead in asynchronous schemes versus standard stochastic gradient methods?
• RQ4 Can joint learning of unknown parameters via projected stochastic gradient methods be integrated into asynchronous best-response schemes without compromising convergence?
• RQ5 To what extent do the proposed schemes generalize to weighted and generalized potential games?

Key findings

• The proposed asynchronous inexact proximal BR scheme converges almost surely to a connected subset of the set of Nash equilibria under general conditions on inexactness and communication delays.
• The gap function converges to zero in mean, indicating that the iterates approach a Nash equilibrium in terms of game-theoretic optimality.
• In the strongly convex case, an inexact pure BR scheme is shown to converge almost surely, even without proximal regularization.
• The scheme achieves approximately 500 times lower communication overhead than standard asynchronous stochastic gradient methods while requiring about 10 times more gradient steps.
• Numerical results on congestion control and Nash-Cournot games confirm almost sure convergence and demonstrate favorable communication efficiency.