[Paper Review] Perturbed gradient descent with occupation time.

Xin Guo, Jiequn Han+1 more|arXiv (Cornell University)|May 9, 2020

Stochastic Gradient Optimization Techniques41 references2 citations

TL;DR

This paper proposes PGDOT, a perturbed gradient descent algorithm that uses occupation time to adapt perturbations based on historical state visits, ensuring convergence at least as fast as standard PGD while provably avoiding saddle points. The method improves optimization stability in non-convex settings by dynamically adjusting perturbations using cumulative state visitation history.

ABSTRACT

This paper develops further the idea of perturbed gradient descent, by adapting perturbation with the history of state via the notation of occupation time for saddle points. The proposed algorithm PGDOT is shown to converge at least as fast as perturbed gradient descent (PGD) algorithm, and is guaranteed to avoid getting stuck at saddle points. The analysis is corroborated by experimental results.

Motivation & Objective

• To address the challenge of saddle point stagnation in non-convex optimization using adaptive perturbation strategies.
• To improve convergence speed and stability of perturbed gradient descent by incorporating historical state visitation data.
• To develop a provably convergent algorithm that avoids getting trapped at saddle points.
• To formalize the use of occupation time as a mechanism for dynamic perturbation adaptation in optimization.

Proposed method

• The algorithm introduces occupation time, defined as the cumulative time a saddle point has been visited, to guide perturbation magnitude.
• Perturbations are adjusted based on the occupation time of saddle points, increasing when they are frequently visited.
• The method modifies the standard PGD update rule by incorporating a time-dependent perturbation term dependent on occupation time.
• Theoretical analysis shows PGDOT maintains at least the convergence rate of standard PGD under the same assumptions.
• The algorithm is designed to prevent prolonged stays at saddle points by increasing perturbation intensity as occupation time increases.

Experimental results

Research questions

• RQ1 Can occupation time be used to dynamically adapt perturbations in gradient descent to improve saddle point escape?
• RQ2 Does PGDOT maintain or improve convergence speed compared to standard perturbed gradient descent?
• RQ3 Is PGDOT guaranteed to avoid convergence to saddle points under the same conditions as PGD?
• RQ4 How does the use of historical state visitation data affect optimization stability and convergence?

Key findings

• PGDOT converges at least as fast as standard perturbed gradient descent under the same assumptions.
• The algorithm is provably guaranteed to avoid getting stuck at saddle points by adapting perturbations based on occupation time.
• Experimental results corroborate the theoretical analysis, showing improved escape from saddle points.
• The occupation time mechanism effectively increases perturbation strength at frequently visited saddle points, enabling faster escape.