[Paper Review] A Continuous-Time Nesterov Accelerated Gradient Method for Centralized and Distributed Online Convex Optimization

Chao Sun, Guoqiang Hu|arXiv (Cornell University)|Sep 26, 2020

Advanced Bandit Algorithms Research31 references4 citations

TL;DR

This paper proposes a continuous-time Nesterov accelerated gradient method (OCT-NAG) for online convex optimization, leveraging a Bregman Lagrangian framework to achieve a constant static regret of $\frac{c}{\sigma}$, independent of time horizon $T$, under boundedness assumptions. The method is extended to distributed settings (DOCT-NAG), achieving $O(\sqrt{T})$ static regret without requiring gradient boundedness or compact constraint sets.

ABSTRACT

This paper studies the online convex optimization problem by using an Online Continuous-Time Nesterov Accelerated Gradient method (OCT-NAG). We show that the continuous-time dynamics generated by the online version of the Bregman Lagrangian achieves a constant static regret $\frac{c}σ$ independent of $T$, provided that some boundedness assumptions on the objective functions and optimal solutions hold. To the best of the authors' knowledge, this is the lowest static regret in the literature (lower than $O( ext{log}(T))$). We further show that under the same assumptions, the dynamic regret of the algorithm is $O(T)$, which is comparable with the existing methods. Simulation results validate the effectiveness and efficiency of the method. Furthermore, the simulation shows that the algorithm performs well in terms of the dynamic regret for some specific scaling conditions. In addition, we consider the application of the proposed online optimization method in distributed online optimization problems, and show that the proposed algorithm achieves an $O(\sqrt{T})$ static regret, which is comparable with the existing distributed online optimization methods. Different from these methods, the proposed method requires neither the gradient boundedness assumption nor the compact constraint set assumption, which allows different objective functions and different optimization problems with those in the literature. A comparable dynamic regret is obtained. Simulation results show the effectiveness and efficiency of the distributed algorithm.

Motivation & Objective

• To develop a continuous-time online optimization method with provably low regret for time-varying convex objectives.
• To extend the Nesterov accelerated gradient method to online and distributed settings using continuous-time dynamics.
• To relax standard assumptions such as gradient boundedness and compact constraint sets in distributed online optimization.
• To analyze and compare static and dynamic regret performance under different scaling conditions.
• To validate the method through simulations in centralized and distributed scenarios.

Proposed method

• The method employs a Bregman Lagrangian formulation to derive continuous-time dynamics that generalize Nesterov's acceleration.
• The dynamics are governed by a second-order ODE derived from the Euler-Lagrange equation of the Bregman Lagrangian.
• A scaling condition $e^{\alpha_t + \beta_t} - \dot{\beta}_t e^{\beta_t} = \sigma(t + b_0)$ is used to tune convergence and regret performance.
• The algorithm is discretized for implementation, with parameters selected to maintain theoretical regret bounds.
• For distributed optimization, a consensus-based update law is applied across agents, with each agent maintaining local estimates of the global solution.
• The method avoids projection and relies on continuous-time dynamics to track time-varying optimal solutions.

Experimental results

Research questions

• RQ1 Can a continuous-time Nesterov accelerated gradient method achieve constant static regret in online convex optimization?
• RQ2 How does the choice of scaling condition affect the regret performance in the proposed continuous-time framework?
• RQ3 Can the proposed method be extended to distributed online optimization with relaxed assumptions?
• RQ4 What is the dynamic regret performance of the continuous-time algorithm under time-varying objectives?
• RQ5 How does the method compare to existing discrete-time online methods in terms of regret and convergence?

Key findings

• The proposed OCT-NAG achieves a constant static regret of $\frac{c}{\sigma}$, which is independent of the time horizon $T$, representing a lower regret than the $O(\log T)$ bound in prior work.
• Under the same assumptions, the dynamic regret is $O(T)$, which is comparable to existing methods and consistent with theoretical expectations.
• In the distributed setting (DOCT-NAG), the algorithm achieves $O(\sqrt{T})$ static regret, matching the performance of state-of-the-art distributed online optimization methods.
• The method does not require gradient boundedness or compact constraint set assumptions, enabling broader applicability to diverse and time-varying optimization problems.
• Simulations confirm that the algorithm outperforms discrete-time counterparts (e.g., Online Gradient Descent, AdaGrad, FTAL) in terms of dynamic regret, with the difference to the optimal value decreasing persistently.
• The distributed version shows high accuracy in tracking the time-varying optimal solution across all agents, with minimal deviation in estimation.