[Paper Review] PGDOT -- Perturbed Gradient Descent Adapted with Occupation Time

Xin Guo, Jiequn Han+2 more|arXiv (Cornell University)|May 9, 2020

Neural Networks and Applications60 references4 citations

TL;DR

This paper proposes PGDOT, a perturbed gradient descent algorithm that adapts perturbations using occupation time to accelerate convergence and escape saddle points. It outperforms standard optimizers like Adam and RMSProp in training MLPs, particularly in escaping saddle points, and guarantees convergence at least as fast as PGD.

ABSTRACT

This paper develops further the idea of perturbed gradient descent (PGD), by adapting perturbation with the history of states via the notion of occupation time. The proposed algorithm, perturbed gradient descent adapted with occupation time (PGDOT), is shown to converge at least as fast as the PGD algorithm and is guaranteed to avoid getting stuck at saddle points. The analysis is corroborated by empirical studies, in which a mini-batch version of PGDOT is shown to outperform alternatives such as mini-batch gradient descent, Adam, AMSGrad, and RMSProp in training multilayer perceptrons (MLPs). In particular, the mini-batch PGDOT manages to escape saddle points whereas these alternatives fail.

Motivation & Objective

• To address the limitation of standard optimization methods in escaping saddle points during deep learning training.
• To improve convergence speed of perturbed gradient descent by incorporating historical state information.
• To develop a theoretically grounded optimization algorithm that avoids local minima and saddle points more effectively.
• To empirically validate the superiority of the proposed method over popular adaptive optimizers like Adam and RMSProp.

Proposed method

• Introduces occupation time as a measure of how long a state has been visited during optimization.
• Adapts perturbation magnitude in gradient descent based on the cumulative occupation time of current states.
• Modifies the update rule to include a time-adaptive perturbation term that increases when states are frequently revisited.
• Theoretical analysis shows PGDOT converges at least as fast as PGD and avoids saddle points under mild conditions.
• Develops a mini-batch variant of PGDOT for practical training of deep neural networks.
• Empirically evaluates the mini-batch PGDOT on multilayer perceptrons to compare with baseline optimizers.

Experimental results

Research questions

• RQ1 Can occupation time be effectively used to adapt perturbations in gradient descent for improved optimization dynamics?
• RQ2 Does PGDOT converge faster than PGD and other standard optimizers in deep learning settings?
• RQ3 Can PGDOT reliably escape saddle points where Adam, RMSProp, and similar methods fail?
• RQ4 How does the mini-batch version of PGDOT perform in comparison to adaptive optimizers on standard MLP benchmarks?

Key findings

• PGDOT converges at least as fast as PGD, with theoretical guarantees on convergence speed.
• PGDOT is proven to avoid getting stuck at saddle points, a key limitation of many standard optimizers.
• In empirical evaluations, mini-batch PGDOT outperforms mini-batch gradient descent, Adam, AMSGrad, and RMSProp on MLP training.
• The mini-batch PGDOT successfully escapes saddle points in training, whereas Adam, RMSProp, and similar methods fail to do so.
• The use of occupation time enables adaptive perturbations that improve exploration without compromising convergence.
• The algorithm demonstrates robustness and efficiency in deep learning optimization tasks, particularly in non-convex settings.