[Paper Review] General Proximal Incremental Aggregated Gradient Algorithms: Better and Novel Results under General Scheme

Tao Sun, Yuejiao Sun+2 more|arXiv (Cornell University)|Oct 11, 2019

Sparse and Compressive Sensing Techniques7 citations

TL;DR

This paper proposes a general proximal incremental aggregated gradient (PIAG) algorithm that extends existing methods by allowing nonconvex objectives, larger stepsizes, and noise resilience. It establishes novel convergence results, including sublinear convergence rates for nonconvex functions and linear convergence with larger stepsizes under general conditions, overcoming limitations of prior work requiring strong convexity and small stepsizes.

ABSTRACT

The incremental aggregated gradient algorithm is popular in network optimization and machine learning research. However, the current convergence results require the objective function to be strongly convex. And the existing convergence rates are also limited to linear convergence. Due to the mathematical techniques, the stepsize in the algorithm is restricted by the strongly convex constant, which may make the stepsize be very small (the strongly convex constant may be small). In this paper, we propose a general proximal incremental aggregated gradient algorithm, which contains various existing algorithms including the basic incremental aggregated gradient method. Better and new convergence results are proved even with the general scheme. The novel results presented in this paper, which have not appeared in previous literature, include: a general scheme, nonconvex analysis, the sublinear convergence rates of the function values, much larger stepsizes that guarantee the convergence, the convergence when noise exists, the line search strategy of the proximal incremental aggregated gradient algorithm and its convergence.

Motivation & Objective

• To address the limitations of existing incremental gradient methods that require strong convexity and small stepsizes.
• To develop a unified framework for proximal incremental aggregated gradient algorithms applicable to general, nonconvex, and noisy settings.
• To establish convergence and convergence rate results under broader conditions than previously possible.
• To introduce a line search strategy that ensures convergence without prior knowledge of Lipschitz constants.

Proposed method

• Proposes a general PIAG algorithm using delayed and noisy gradient approximations via $ v^k = \sum_{i=1}^m \nabla f_i(x^{k-\tau_{i,k}}) + e^k $.
• Introduces a novel Lyapunov function $ F_k(\varepsilon,t) $ to analyze convergence under general convexity and nonconvexity.
• Employs a line search strategy that adaptively selects stepsizes based on descent conditions, avoiding reliance on known Lipschitz constants.
• Uses a general scheme with delays $ \tau_{i,k} $, noise $ e^k $, and proximal updates via $ x^{k+1} = \text{prox}_{\gamma_k g}[x^k - \gamma_k v^k] $.
• Applies technical lemmas to bound the decrease in objective function and establish convergence via recursive inequalities.
• Derives convergence rates by analyzing the decay of the Lyapunov function and bounding the sum of gradient errors over delays.

Experimental results

Research questions

• RQ1 Can convergence be established for the general PIAG algorithm under nonconvex objectives without requiring strong convexity?
• RQ2 What is the largest allowable stepsize that still guarantees convergence in the general PIAG framework?
• RQ3 How do delays and noise affect the convergence behavior of incremental aggregated gradient methods?
• RQ4 Can a line search strategy be designed for PIAG that ensures convergence without prior knowledge of Lipschitz constants?
• RQ5 What convergence rates can be achieved in the general convex and nonconvex cases under the proposed framework?

Key findings

• The paper proves sublinear convergence rates for the objective function value in the general convex case, even without strong convexity.
• Linear convergence is established with larger stepsizes than previously possible, breaking the dependence on the strong convexity constant.
• Convergence is proven under nonconvexity, extending the applicability of PIAG beyond strongly convex problems.
• The algorithm remains convergent in the presence of noise, with convergence rates preserved under bounded noise assumptions.
• A line search strategy is proposed and proven to ensure convergence, enabling adaptive stepsize selection without prior knowledge of Lipschitz constants.
• Novel convergence results are established, including bounds on the decay of gradient errors and the convergence of iterates to a stationary point.