[論文レビュー] Operationalizing Stein's Method for Online Linear Optimization: CLT-Based Optimal Tradeoffs

Zhiyu Zhang, Aaditya Ramdas|arXiv (Cornell University)|Feb 6, 2026

Stochastic Gradient Optimization Techniques被引用数 0

ひとこと要約

The paper introduces a Stein's method-based, computationally efficient algorithm for adversarial online linear optimization that achieves additively sharp loss bounds and CLT-inspired optimal tradeoffs.

ABSTRACT

Adversarial online linear optimization (OLO) is essentially about making performance tradeoffs with respect to the unknown difficulty of the adversary. In the setting of one-dimensional fixed-time OLO on a bounded domain, it has been observed since Cover (1966) that achievable tradeoffs are governed by probabilistic inequalities, and these descriptive results can be converted into algorithms via dynamic programming, which, however, is not computationally efficient. We address this limitation by showing that Stein's method, a classical framework underlying the proofs of probabilistic limit theorems, can be operationalized as computationally efficient OLO algorithms. The associated regret and total loss upper bounds are "additively sharp", meaning that they surpass the conventional big-O optimality and match normal-approximation-based lower bounds by additive lower order terms. Our construction is inspired by the remarkably clean proof of a Wasserstein martingale central limit theorem (CLT) due to Röllin (2018). Several concrete benefits can be obtained from this general technique. First, with the same computational complexity, the proposed algorithm improves upon the total loss upper bounds of online gradient descent (OGD) and multiplicative weight update (MWU). Second, our algorithm can realize a continuum of optimal two-point tradeoffs between the total loss and the maximum regret over comparators, improving upon prior works in parameter-free online learning. Third, by allowing the adversary to randomize on an unbounded support, we achieve sharp in-expectation performance guarantees for OLO with noisy feedback.

研究の動機と目的

• Motivate and formalize performance tradeoffs in one-dimensional fixed-time online linear optimization on a bounded domain.
• Develop a computationally efficient algorithm that achieves sharp loss bounds via Stein’s method and CLT insights.
• Provide a framework to realize a continuum of optimal two-point tradeoffs between total loss and regret over comparators.
• Extend guarantees to noisy feedback by allowing adversaries with unbounded support, achieving sharp in-expectation performance.

提案手法

• Introduce Stein’s equation and its solution for convex 1-Lipschitz h as a tool to bound losses.
• Define Algorithm 1: output x_t as an expectation involving f_{s_{t-1},ρ_{t-1},h} and a Gaussian Z, enabling O(1) time per round.
• Relate x_t to a tamed discretization of a backward heat equation, connecting to the continuous-time potential method and FTRL.
• Provide a master bound for Loss_T that splits into a main term -ψ̄_T^*(−∑g_t) and an additive error term err_T.
• Show that by appropriate choices of ρ_t and h, the algorithm dominates standard baselines like OGD and MWU in Regret bounds.
• Establish a lower bound showing optimality up to an additive O(log T) term in simple bounded-adversary settings.

実験結果

リサーチクエスチョン

• RQ1 What are the conditions on the surrogate loss function ψ_T^* to achieve the desired Loss_T bound against adversaries?
• RQ2 Can Stein’s method yield a computationally efficient OLO algorithm with additively sharp loss bounds that approach CLT-type limits?
• RQ3 How can one realize a continuum of optimal two-point tradeoffs between total loss and uniform regret over comparators?
• RQ4 Do these guarantees extend to adversaries with unbounded/noisy feedback, maintaining sharp in-expectation performance?

主な発見

• There exists an O(1) time-per-round algorithm (Algorithm 1) guaranteeing Loss_T ≤ −ψ_T^*(−∑g_t) + O(log T).
• The bound is additively sharp: the O(log T) gap is lower order and does not prevent near-CNT optimality when ψ_T^* grows as Θ(√T).
• For any α > 0, the algorithm achieves Regret_T(u) ≤ γ_Huber(u, α)√T + O(log T) with γ_Huber(u, α) strictly smaller than the OGD bound, and as α → ∞ this prefactor tends to √(2/π).
• The algorithm dominates MWU as well, with a comparable loss bound and improved regret guarantees.
• In the two-point tradeoff setting, the algorithm guarantees both Loss_T ≤ ε√T + O(log T) and Regret_unif_T ≤ γ(ε)√T + O(log T) for ε ∈ (0, √(π/2)].
• With noisy feedback (unbounded adversaries), the method yields sharp in-expectation guarantees corresponding to a nonasymptotic Wasserstein martingale CLT.