[Paper Review] Tight Lower Bounds for Multiplicative Weights Algorithmic Families

Nick Gravin, Yuval Peres+1 more|arXiv (Cornell University)|Jul 11, 2016

Coding theory and cryptography4 citations

TL;DR

This paper establishes tight regret lower bounds for multiplicative weights algorithmic families in the expert prediction problem, using novel adversarial primitives to prove that the classic Multiplicative Weights Algorithm achieves a regret of exactly $\sqrt{\frac{T\ln k}{2}}$, closing the long-standing gap between upper and lower bounds. It further shows a $\frac{2}{3}$-factor lower bound for broader families with time-varying or randomly distributed learning rates, and characterizes exact regret in the geometric horizon setting.

ABSTRACT

We study the fundamental problem of prediction with expert advice and develop regret lower bounds for a large family of algorithms for this problem. We develop simple adversarial primitives, that lend themselves to various combinations leading to sharp lower bounds for many algorithmic families. We use these primitives to show that the classic Multiplicative Weights Algorithm (MWA) has a regret of $\sqrt{\frac{T \ln k}{2}}$, there by completely closing the gap between upper and lower bounds. We further show a regret lower bound of $\frac{2}{3}\sqrt{\frac{T\ln k}{2}}$ for a much more general family of algorithms than MWA, where the learning rate can be arbitrarily varied over time, or even picked from arbitrary distributions over time. We also use our primitives to construct adversaries in the geometric horizon setting for MWA to precisely characterize the regret at $\frac{0.391}{\sqrtδ}$ for the case of $2$ experts and a lower bound of $\frac{1}{2}\sqrt{\frac{\ln k}{2δ}}$ for the case of arbitrary number of experts $k$.

Motivation & Objective

• To close the gap between known upper and lower bounds on regret for the classic Multiplicative Weights Algorithm (MWA) in the finite horizon expert prediction problem.
• To develop a general framework of adversarial primitives that can be combined to yield sharp regret lower bounds for broad families of learning algorithms.
• To extend the analysis beyond fixed learning rates to include time-varying, decreasing, and stochastically chosen learning rates.
• To precisely characterize regret in the geometric horizon model, where stopping is memoryless with parameter $\delta$.

Proposed method

• Designing adversarial strategies based on structured, alternating patterns of expert advancement (e.g., 'looping' and 'straight-line' phases) to force suboptimal performance.
• Using asymptotic approximations and Taylor expansions to estimate regret expressions under the assumption $e^{\eta(t)} = 1 + \frac{\alpha(t)}{\sqrt{T}}$ with $\alpha(t) = \Theta(1)$.
• Applying variational techniques and integral approximations to bound the sum of regret contributions over time, reducing to integrals of the form $\int_0^1 \sqrt{1-x} \, dx$.
• Constructing adversaries that alternate between symmetric expert advancement and single-expert focusing to isolate and amplify regret contributions.
• Using probabilistic averaging over random parameters (e.g., random starting phase) to derive robust lower bounds across algorithmic families.
• Extending results to the geometric horizon by modeling the problem as a discounted infinite-horizon Markov decision process and deriving optimal adversary structure.

Experimental results

Research questions

• RQ1 What is the exact regret achievable by the classic Multiplicative Weights Algorithm with fixed learning rate?
• RQ2 How does the regret lower bound scale for algorithmic families with time-varying or randomly distributed learning rates?
• RQ3 Can the regret lower bound be improved beyond $\frac{2}{3}\sqrt{\frac{T\ln k}{2}}$ for general families of algorithms?
• RQ4 What is the precise regret in the geometric horizon model for $k$ experts, especially for $k=2$?
• RQ5 How do adversarial strategies based on structured expert advancement patterns affect the lower bound derivation?

Key findings

• The classic Multiplicative Weights Algorithm achieves a regret of exactly $\sqrt{\frac{T\ln k}{2}}$, completely closing the gap between known upper and lower bounds.
• For the broader family $\mathcal{A}_{\text{rand}}$ with learning rates drawn from arbitrary time-dependent distributions, the regret is at least $\frac{2}{3}\sqrt{\frac{T\ln k}{2}}$.
• For odd $k$, the lower bound is $\frac{2}{3}\sqrt{\frac{T\ln k}{2}\left(1 - \frac{1}{k^2}\right)}$, reflecting a small correction due to symmetry breaking.
• In the geometric horizon model with $k=2$ experts, the optimal regret is exactly $\frac{0.391}{\sqrt{\delta}}$ as $\delta \to 0$.
• For general $k$ in the geometric horizon, the regret is at least $\frac{1}{2}\sqrt{\frac{\ln k}{2\delta}}$.
• The assumption $e^{\eta(t)} = 1 + \frac{\alpha(t)}{\sqrt{T}}$ with $\alpha(t) = \Theta(1)$ is without loss of generality for deriving the lower bounds.