[Paper Review] Improved Dynamic Regret for Online Frank-Wolfe

Yuanyu Wan, Lijun Zhang+1 more|arXiv (Cornell University)|Feb 11, 2023

Advanced Bandit Algorithms Research4 citations

TL;DR

This paper improves the dynamic regret bounds of Online Frank-Wolfe (OFW) by introducing a line search-based step size rule, achieving tighter bounds for smooth, strongly convex, and interior-minimizer settings. It establishes $O(ig(1+V_Tig)^{2/3}T^{1/3})$ for smooth and strongly convex functions on strongly convex sets, and $O(ig(1+V_Tig))$ for interior minimizers, with further refinement to $O(ig( ext{min}igracevert P_T^*, S_T^*, V_Tigracevert + 1ig)$ using constant FW iterations per round.

ABSTRACT

To deal with non-stationary online problems with complex constraints, we investigate the dynamic regret of online Frank-Wolfe (OFW), which is an efficient projection-free algorithm for online convex optimization. It is well-known that in the setting of offline optimization, the smoothness of functions and the strong convexity of functions accompanying specific properties of constraint sets can be utilized to achieve fast convergence rates for the Frank-Wolfe (FW) algorithm. However, for OFW, previous studies only establish a dynamic regret bound of $O(\sqrt{T}(V_T+\sqrt{D_T}+1))$ by utilizing the convexity of problems, where $T$ is the number of rounds, $V_T$ is the function variation, and $D_T$ is the gradient variation. In this paper, we derive improved dynamic regret bounds for OFW by extending the fast convergence rates of FW from offline optimization to online optimization. The key technique for this extension is to set the step size of OFW with a line search rule. In this way, we first show that the dynamic regret bound of OFW can be improved to $O(\sqrt{T(V_T+1)})$ for smooth functions. Second, we achieve a better dynamic regret bound of $O(T^{1/3}(V_T+1)^{2/3})$ when functions are smooth and strongly convex, and the constraint set is strongly convex. Finally, for smooth and strongly convex functions with minimizers in the interior of the constraint set, we demonstrate that the dynamic regret of OFW reduces to $O(V_T+1)$, and can be further strengthened to $O(\min\{P_T^\ast,S_T^\ast,V_T\}+1)$ by performing a constant number of FW iterations per round, where $P_T^\ast$ and $S_T^\ast$ denote the path length and squared path length of minimizers, respectively.

Motivation & Objective

• Address the limitation of existing OFW algorithms in non-stationary environments by improving dynamic regret bounds.
• Extend fast convergence rates from offline Frank-Wolfe to online settings using smoothness and strong convexity.
• Achieve tighter dynamic regret bounds by leveraging constraint set geometry and minimizer location.
• Develop a line search-based step size rule to enhance convergence and regret performance in online optimization.

Proposed method

• Introduce a line search rule to adaptively set the step size in OFW, improving convergence and regret performance.
• Utilize the Frank-Wolfe gap and descent lemma to bound the progress per iteration under smoothness and strong convexity.
• Derive regret bounds by decomposing the dynamic regret into function variation $V_T$, gradient variation $D_T$, and optimality gaps.
• Apply the line search rule to achieve $O(ig( ext{min}igracevert P_T^*, S_T^*, V_Tigracevert + 1ig)$ regret when minimizers lie in the interior of the constraint set.
• Use the strong convexity of the constraint set and function to accelerate convergence and tighten the $O((1+V_T)^{2/3}T^{1/3})$ bound.
• Establish a recursive regret decomposition using the optimality gap and function variation, enabling tighter analysis via parameter tuning.

Experimental results

Research questions

• RQ1 Can the fast convergence rates of offline Frank-Wolfe be extended to the online setting to improve dynamic regret?
• RQ2 How does the choice of step size affect the dynamic regret of OFW in non-stationary environments?
• RQ3 What dynamic regret bounds can be achieved when functions are smooth and strongly convex with minimizers in the interior of the constraint set?
• RQ4 Can the dynamic regret be further improved by using a constant number of Frank-Wolfe iterations per round?
• RQ5 How do function variation $V_T$, path length $P_T^*$, and squared path length $S_T^*$ influence the dynamic regret in OFW?

Key findings

• The dynamic regret of OFW is improved to $O(ig( ext{smooth functions: } ig( ext{smooth and strongly convex functions on strongly convex sets: } O((1+V_T)^{2/3}T^{1/3})$.
• For smooth and strongly convex functions with minimizers in the interior of the constraint set, the dynamic regret reduces to $O(1+V_T)$.
• By performing a constant number of Frank-Wolfe iterations per round, the dynamic regret can be further tightened to $O( ext{min}igracevert P_T^*, S_T^*, V_Tigracevert + 1)$.
• The line search step size rule enables tighter regret bounds by adaptively balancing progress and stability across rounds.
• The analysis shows that smoothness and strong convexity significantly improve regret performance, even in non-stationary settings.
• The improved bounds close the gap between offline and online Frank-Wolfe convergence rates, demonstrating that offline gains can be transferred to online optimization.