[Paper Review] A Local Regret in Nonconvex Online Learning

Sergül Aydöre, Lee H. Dicker+1 more|arXiv (Cornell University)|Nov 13, 2018

Advanced Bandit Algorithms ResearchDecision Sciences5 references3 citations

TL;DR

This paper introduces a novel local regret definition for nonconvex online learning that emphasizes calibration and recent gradients, differing from Hazan's approach by focusing on immediate predictive performance. It proves a logarithmic regret bound under specific conditions, offering a more interpretable and theoretically grounded metric for forecasting models with nonconvex loss functions.

ABSTRACT

We consider an online learning process to forecast a sequence of outcomes for nonconvex models. A typical measure to evaluate online learning algorithms is regret but such standard definition of regret is intractable for nonconvex models even in offline settings. Hence, gradient based definition of regrets are common for both offline and online nonconvex problems. Recently, a notion of local gradient based regret was introduced. Inspired by the concept of calibration and a local gradient based regret, we introduce another definition of regret and we discuss why our definition is more interpretable for forecasting problems. We also provide bound analysis for our regret under certain assumptions.

Motivation & Objective

• To address the intractability of global regret in nonconvex online learning by proposing a more interpretable local regret measure.
• To integrate the concept of calibration into regret analysis for nonconvex forecasting problems.
• To provide theoretical bounds on the proposed regret under standard assumptions in online learning.
• To motivate the development of efficient algorithms that minimize the new regret definition.

Proposed method

• Proposes a w-local regret based on the inner product between gradient directions and directional perturbations, capturing local predictive stability.
• Defines the regret as the squared norm of the average of directional gradient products over a sliding window of size w.
• Uses Taylor approximation and projection operators to model perturbations in parameter space, ensuring feasibility within the convex domain K.
• Applies gradient descent with diminishing learning rate ηt = η/√t to ensure convergence and derive regret bounds.
• Derives theoretical bounds using inequalities involving gradient norms, step size, and Lipschitz constants.
• Analyzes three scenarios: constant learning rate with fixed window, decreasing learning rate with full history window, and decreasing rate with fixed window.

Experimental results

Research questions

• RQ1 How can a more interpretable regret measure be defined for nonconvex online learning, particularly in forecasting contexts?
• RQ2 In what way does the proposed regret improve upon Hazan’s gradient-based local regret in terms of interpretability and focus on current performance?
• RQ3 Under what conditions does the proposed regret achieve sublinear or logarithmic growth with respect to time T?
• RQ4 How does the inclusion of calibration concepts affect the theoretical properties of the regret measure?

Key findings

• The proposed regret achieves a logarithmic bound of O(log T) when using a decreasing learning rate (ηt = η/√t) and a full history window (w = t).
• For a constant window size w and decreasing learning rate, the regret grows quadratically in T but can be made sublinear by increasing w.
• With a constant learning rate and fixed window, the regret is bounded by O(T / w²), which becomes sublinear when w grows with T.
• Theoretical analysis shows that the inner product term ∑⟨Du(xs), ∇fs(xs)⟩ is bounded below by a negative expression involving M, G, and η, enabling the regret bound derivation.
• The bound derivation relies on selecting an optimal direction u aligned with the cumulative gradient, leading to tight control of the regret expression.
• The paper conjectures that algorithms minimizing this new regret can be more efficient and better suited for nonconvex online forecasting.