[Paper Review] New Projection-free Algorithms for Online Convex Optimization with Adaptive Regret Guarantees

Dan Garber, Ben Kretzu|arXiv (Cornell University)|Feb 9, 2022

Advanced Bandit Algorithms Research4 citations

TL;DR

This paper introduces novel projection-free online convex optimization algorithms based on online gradient descent with a new infeasible projection technique, achieving the first adaptive regret guarantees—$O(T^{3/4})$ adaptive regret in the full-information and $O(T^{3/4})$ adaptive expected regret in the bandit setting—using only $O(T)$ calls to a linear optimization oracle (LOO). The approach avoids orthogonal projections, instead relying on efficient oracles, and extends to separation oracles with $O(\sqrt{T})$ adaptive regret in the full-information case.

ABSTRACT

We present new efficient extit{projection-free} algorithms for online convex optimization (OCO), where by projection-free we refer to algorithms that avoid computing orthogonal projections onto the feasible set, and instead relay on different and potentially much more efficient oracles. While most state-of-the-art projection-free algorithms are based on the extit{follow-the-leader} framework, our algorithms are fundamentally different and are based on the extit{online gradient descent} algorithm with a novel and efficient approach to computing so-called extit{infeasible projections}. As a consequence, we obtain the first projection-free algorithms which naturally yield extit{adaptive regret} guarantees, i.e., regret bounds that hold w.r.t. any sub-interval of the sequence. Concretely, when assuming the availability of a linear optimization oracle (LOO) for the feasible set, on a sequence of length $T$, our algorithms guarantee $O(T^{3/4})$ adaptive regret and $O(T^{3/4})$ adaptive expected regret, for the full-information and bandit settings, respectively, using only $O(T)$ calls to the LOO. These bounds match the current state-of-the-art regret bounds for LOO-based projection-free OCO, which are extit{not adaptive}. We also consider a new natural setting in which the feasible set is accessible through a separation oracle. We present algorithms which, using overall $O(T)$ calls to the separation oracle, guarantee $O(\sqrt{T})$ adaptive regret and $O(T^{3/4})$ adaptive expected regret for the full-information and bandit settings, respectively.

Motivation & Objective

• To develop projection-free online convex optimization algorithms that avoid computationally expensive orthogonal projections onto the feasible set.
• To achieve adaptive regret guarantees—regret bounds that hold over any sub-interval of the sequence—without relying on the Follow-The-Leader framework.
• To design algorithms that use only $O(T)$ calls to a linear optimization oracle (LOO) or separation oracle while maintaining state-of-the-art regret bounds.
• To extend the framework to settings where the feasible set is accessed via a separation oracle, achieving improved adaptive regret in the full-information setting.
• To correct and improve upon a prior COLT 2022 version by fixing a flaw in the convergence proof for strongly convex losses, albeit with a logarithmic factor degradation in the regret bound.

Proposed method

• The algorithm is based on online gradient descent with a novel infeasible projection mechanism that computes approximate projections without requiring orthogonal projections onto the feasible set.
• It uses a linear optimization oracle (LOO) to efficiently access the feasible set, enabling computation of descent steps without explicit projection.
• For the bandit setting, the method employs a randomized smoothing technique to estimate gradients, allowing the use of the LOO in a stochastic setting.
• A separation oracle-based variant uses a squeezing procedure to maintain iterates near the feasible set, with a recursive distance reduction strategy to ensure convergence.
• The algorithm dynamically adjusts step sizes and smoothing parameters to balance regret and oracle call efficiency.
• Theoretical analysis leverages distance decay in the infeasible projection process and bounds on gradient estimation error to derive adaptive regret guarantees.

Experimental results

Research questions

• RQ1 Can projection-free online convex optimization algorithms achieve adaptive regret guarantees—i.e., regret bounds over any sub-interval—without relying on the Follow-The-Leader framework?
• RQ2 Is it possible to design such algorithms using only $O(T)$ calls to a linear optimization oracle (LOO), while matching the best-known non-adaptive regret bounds?
• RQ3 How can the framework be extended to settings where the feasible set is accessed via a separation oracle rather than an LOO?
• RQ4 What is the trade-off between regret performance and the number of oracle calls when using infeasible projections instead of orthogonal projections?
• RQ5 Can the algorithm maintain adaptive regret guarantees in the bandit setting with only $O(T)$ calls to the LOO or separation oracle?

Key findings

• The proposed algorithm achieves $O(T^{3/4})$ adaptive regret in the full-information setting and $O(T^{3/4})$ adaptive expected regret in the bandit setting, using only $O(T)$ calls to the linear optimization oracle (LOO).
• For the separation oracle setting, the algorithm guarantees $O(\sqrt{T})$ adaptive regret in the full-information case and $O(T^{3/4})$ adaptive expected regret in the bandit case, again with $O(T)$ total calls.
• The method introduces a new infeasible projection technique that avoids orthogonal projections, enabling efficient implementation via LOO or separation oracles.
• The algorithm is the first projection-free OCO method to naturally yield adaptive regret guarantees, overcoming inherent limitations of Follow-The-Leader-based approaches.
• The paper corrects a flaw in the original COLT 2022 version regarding convergence for strongly convex losses, though the new regret bound is worse by a logarithmic factor.
• Theoretical analysis confirms that the number of separation oracle calls remains $O(T)$, with a distance decay mechanism ensuring convergence despite infeasible iterates.