[Paper Review] A Linearly Convergent Conditional Gradient Algorithm with Applications to Online and Stochastic Optimization

Dan Garber, Elad Hazan|arXiv (Cornell University)|Jan 20, 2013

Advanced Bandit Algorithms Research34 references63 citations

TL;DR

This paper presents a novel conditional gradient algorithm for smooth and strongly convex optimization over polyhedral sets that achieves linear convergence with only one linear optimization oracle call per iteration, representing an exponential improvement over prior methods. The algorithm enables optimal regret bounds in online convex optimization with a single linear oracle call per round, resolving an open question in the field.

ABSTRACT

Linear optimization is many times algorithmically simpler than non-linear convex optimization. Linear optimization over matroid polytopes, matching polytopes and path polytopes are example of problems for which we have simple and efficient combinatorial algorithms, but whose non-linear convex counterpart is harder and admits significantly less efficient algorithms. This motivates the computational model of convex optimization, including the offline, online and stochastic settings, using a linear optimization oracle. In this computational model we give several new results that improve over the previous state-of-the-art. Our main result is a novel conditional gradient algorithm for smooth and strongly convex optimization over polyhedral sets that performs only a single linear optimization step over the domain on each iteration and enjoys a linear convergence rate. This gives an exponential improvement in convergence rate over previous results. Based on this new conditional gradient algorithm we give the first algorithms for online convex optimization over polyhedral sets that perform only a single linear optimization step over the domain while having optimal regret guarantees, answering an open question of Kalai and Vempala, and Hazan and Kale. Our online algorithms also imply conditional gradient algorithms for non-smooth and stochastic convex optimization with the same convergence rates as projected (sub)gradient methods.

Motivation & Objective

• To develop a conditional gradient algorithm for smooth and strongly convex optimization over polyhedral sets that achieves linear convergence with minimal linear optimization oracle calls.
• To address the open problem of achieving optimal regret in online convex optimization over polytopes using only a single linear optimization step per round.
• To extend the new algorithm to non-smooth and stochastic convex optimization settings, matching the convergence rates of projected (sub)gradient methods while replacing projections with linear oracle calls.
• To establish theoretical lower bounds on the oracle complexity for conditional gradient-like methods, showing that the proposed algorithm's performance is nearly tight up to logarithmic factors.

Proposed method

• Proposes a new conditional gradient algorithm that performs only one linear optimization oracle call per iteration and achieves linear convergence for smooth and strongly convex objectives over polyhedral sets.
• Introduces a novel line search and update rule that ensures sufficient decrease in the objective function while maintaining feasibility and convergence guarantees.
• Leverages the structure of polyhedral sets to ensure that the algorithm's iterates remain within the feasible region using convex combinations of vertices obtained via linear optimization.
• Applies the new algorithm to online convex optimization by constructing a randomized online variant that maintains regret bounds matching the optimal $ ilde{O}( ext{poly}(D, G) ho ext{poly}(T))$ order.
• Extends the framework to stochastic and non-smooth settings by combining the linearly convergent algorithm with smoothing and sampling techniques, preserving convergence rates.
• Employs a lower bound argument based on the probabilistic simplex to show that the number of linear oracle calls required for $rac{1}{4n}$-accuracy is $ ilde{ heta}(n)$, proving near-optimality of the proposed method.

Experimental results

Research questions

• RQ1 Can a conditional gradient algorithm achieve linear convergence for smooth and strongly convex optimization over polytopes with only one linear optimization oracle call per iteration?
• RQ2 Is it possible to design an online conditional gradient algorithm for polyhedral domains that achieves optimal regret with a single linear oracle call per round?
• RQ3 Can the proposed algorithm be extended to non-smooth and stochastic convex optimization settings while matching the convergence rates of projected (sub)gradient methods?
• RQ4 How tight is the oracle complexity of conditional gradient-like algorithms for smooth and strongly convex problems on polytopes?

Key findings

• The proposed algorithm achieves linear convergence $e^{- ilde{ heta}(t)}$ for smooth and strongly convex optimization over polytopes, an exponential improvement over the previous $t^{-1}$ rate.
• For online convex optimization with convex losses, the algorithm achieves optimal regret of order $ ilde{O}( ho ext{poly}(T))$, matching the $ ilde{O}( ho ext{poly}(T))$ bound of projected gradient methods.
• For online strongly convex losses, the algorithm achieves regret of order $ ilde{O}( ext{poly}( ho) ext{poly}( ext{log} T))$, matching the best known bounds.
• The algorithm requires $O(n ext{log}(1/ heta))$ linear oracle calls to achieve $ heta$-accuracy on the simplex, matching the $ ilde{ heta}(n)$ lower bound up to logarithmic factors.
• The method achieves the same convergence rates as projected (sub)gradient methods in non-smooth and stochastic settings, but replaces expensive projections with efficient linear optimization steps.