[Paper Review] Maximum Principle Based Algorithms for Deep Learning

Qianxiao Li, Long Chen+2 more|arXiv (Cornell University)|Oct 26, 2017

Model Reduction and Neural Networks82 citations

TL;DR

This paper casts deep learning as a continuous-time optimal control problem and derives Pontryagin’s Maximum Principle (PMP) based training algorithms, notably the Method of Successive Approximations (MSA) and an Extended PMP/MSA with convergence guarantees and favorable early convergence, decoupled layer optimization, and potential robustness to flat landscapes.

ABSTRACT

The continuous dynamical system approach to deep learning is explored in order to devise alternative frameworks for training algorithms. Training is recast as a control problem and this allows us to formulate necessary optimality conditions in continuous time using the Pontryagin's maximum principle (PMP). A modification of the method of successive approximations is then used to solve the PMP, giving rise to an alternative training algorithm for deep learning. This approach has the advantage that rigorous error estimates and convergence results can be established. We also show that it may avoid some pitfalls of gradient-based methods, such as slow convergence on flat landscapes near saddle points. Furthermore, we demonstrate that it obtains favorable initial convergence rate per-iteration, provided Hamiltonian maximization can be efficiently carried out - a step which is still in need of improvement. Overall, the approach opens up new avenues to attack problems associated with deep learning, such as trapping in slow manifolds and inapplicability of gradient-based methods for discrete trainable variables.

Motivation & Objective

• Motivate and formalize deep learning as a continuous-time optimal control problem.
• Derive Pontryagin’s Maximum Principle (PMP) conditions for optimal training.
• Develop numerical schemes (MSA) to solve PMP and provide error/convergence analysis.
• Introduce an extended PMP/MSA to improve convergence and handle feasibility of dynamics.
• Connect the framework to deep residual networks and discuss discretization and mini-batch considerations.

Proposed method

• Define the dynamical system Ẋt = f(t, Xt, θt) with loss Φ(XT) + ∫0T L(θt) dt.
• Introduce the Hamiltonian H(t, x, p, θ) = p·f(t, x, θ) − L(θ) and state PMP conditions (3)-(5).
• Propose Basic MSA: alternately propagate X, solve for P, then update θ by Hamiltonian maximization at each t.
• Modify to Extended PMP with augmented Hamiltonian ṼH to penalize Hamiltonian dynamics feasibility errors; derive Extended MSA (E-MSA) with convergence guarantees.
• Provide discrete-time formulations showing relation to residual networks and backpropagation.
• Discuss mini-batch extensions and practical considerations for Hamiltonian maximization.

Experimental results

Research questions

• RQ1 Can PMP provide a viable, convergent alternative to gradient-based training for deep learning?
• RQ2 Does the Extended PMP/MSA ensure convergence by penalizing Hamiltonian dynamics feasibility errors?
• RQ3 How does PMP-based training compare to SGD/Adam in terms of convergence rate and sensitivity to saddle points?
• RQ4 How can the PMP framework be discretized and related to residual networks and backpropagation?
• RQ5 What are practical considerations for mini-batch training and efficiency of Hamiltonian maximization?

Key findings

• PMP-based training yields forward-backward Hamiltonian dynamics with a layer-wise decoupled Hamiltonian maximization, enabling potential parallelization.
• Basic MSA can diverge; the Extended MSA with an augmented Hamiltonian provides convergence guarantees to the extended PMP for sufficiently large ρ.
• The extended framework yields explicit error control through feasibility terms and descent in the objective J(θ).
• Numerical experiments show favorable initial convergence rate per iteration for E-MSA when Hamiltonian maximization is efficient, and it can mitigate slow convergence on flat landscapes or near saddle points.
• Discrete-time formulations recover traditional residual-network training structures, and softening the maximization step links to gradient-based backpropagation.
• Mini-batch extensions are discussed, with convergence heuristics supported by standard LLN arguments under appropriate conditions.