[论文解读] Multiplicative Weights Update with Constant Step-Size in Congestion Games: Convergence, Limit Cycles and Chaos

Gerasimos Palaiopanos, Ioannis Panageas+1 more|arXiv (Cornell University)|Mar 3, 2017

Game Theory and Applications被引用 60

一句话总结

本文证明线性 MWU 在常数学习率下在拥塞游戏中收敛到纳什均衡，通过 Baum-Eagon/EM 解释实现，而指数变体 MWU_e 即便在简单的两代理、两边拥塞设置下也可能表现出极限环和混沌。

ABSTRACT

The Multiplicative Weights Update (MWU) method is a ubiquitous meta-algorithm that works as follows: A distribution is maintained on a certain set, and at each step the probability assigned to element $γ$ is multiplied by $(1 -εC(γ))>0$ where $C(γ)$ is the "cost" of element $γ$ and then rescaled to ensure that the new values form a distribution. We analyze MWU in congestion games where agents use extit{arbitrary admissible constants} as learning rates $ε$ and prove convergence to extit{exact Nash equilibria}. Our proof leverages a novel connection between MWU and the Baum-Welch algorithm, the standard instantiation of the Expectation-Maximization (EM) algorithm for hidden Markov models (HMM). Interestingly, this convergence result does not carry over to the nearly homologous MWU variant where at each step the probability assigned to element $γ$ is multiplied by $(1 -ε)^{C(γ)}$ even for the most innocuous case of two-agent, two-strategy load balancing games, where such dynamics can provably lead to limit cycles or even chaotic behavior.

研究动机与目标

• Motivate study of MWU with constant learning rates in congestion games.
• Prove convergence of MWU_l to fixed points and Nash equilibria under minimal conditions.
• Reveal limitations of MWU_e through explicit counterexamples showing limit cycles and chaos.

提出的方法

• Define MWU dynamics for the linear variant: p_{iγ}(t+1) = p_{iγ}(t) (1 - ε_i c_{iγ}(t)) / (1 - ε_i \, \bar{c}_i(t)).
• Show that the expected potential Ψ(p) decreases along non-equilibrium trajectories.
• Introduce auxiliary function Q(p) with nonnegative-coefficient polynomial structure and apply Baum–Eagon to prove monotone progress.
• Demonstrate that Q(p(t+1)) > Q(p(t)) unless at a fixed point, implying convergence to fixed points.
• Relate the MWU_l dynamics to Baum–Welch/EM as special cases of Baum–Eagon iterations.
• Provide explicit counterexamples showing MWU_e can produce limit cycles and chaotic behavior in simple two-agent, two-edge games.

实验结果

研究问题

• RQ1 Does linear MWU with constant step-size guarantee convergence to fixed points in general congestion games?
• RQ2 Under what conditions does MWU_l converge to Nash equilibria, and can interior initial conditions guarantee convergence?
• RQ3 Can the exponential MWU variant (MWU_e) exhibit non-convergent dynamics such as limit cycles or chaos in simple congestion games?
• RQ4 What structural connection exists between MWU_l convergence and Baum–Eagon/EM procedures?
• RQ5 Do the provided examples fully characterize the limitations of MWU_e in two-edge congestion settings?

主要发现

• MWU_l produces a strictly decreasing expected potential Ψ along any non-equilibrium trajectory, ensuring convergence to fixed points.
• MWU_l converges to fixed points for all initial conditions, and converges to Nash equilibria for interior initial conditions under isolated fixed points.
• The convergence proof hinges on showing MWU_l fits into the Baum–Eagon framework, linking it to Baum–Welch (EM) methods.
• MWU_e can fail to converge even in the simplest two-agent two-edge games, exhibiting limit cycles of length two and, with asymmetric costs, Li–Yorke chaos.
• For certain symmetric setups with large constant ε, MWU_e induces unique limit cycles; with asymmetric costs, periodic or chaotic behavior can arise.