[Paper Review] A Direct Shooting Method is Equivalent to an Indirect Method

I. Michael Ross|arXiv (Cornell University)|Mar 5, 2020

Spacecraft Dynamics and Control30 references4 citations

TL;DR

This paper establishes a mathematical equivalence between direct shooting methods and indirect methods in optimal control via first-order necessary conditions, showing that direct methods inherently embed indirect elements. The key contribution is the covector mapping principle, which explains why standard nonlinear programming solvers fail on trajectory optimization problems unless Hamiltonian structure is preserved, offering a remedy through Hamiltonian programming for improved convergence and accuracy.

ABSTRACT

We show that a direct shooting method is mathematically equivalent to an indirect method in the sense of certain first-order conditions. Specific mathematical formulas pertaining to the equivalence of a direct shooting method with an indirect method are derived. We also show that a theoretical equivalence does not necessarily translate to practical equivalence if the parameterized optimal control problem is simply patched to a nonlinear programming solver. A mathematical explanation is provided for the successes and failures of such patched nonlinear programming methods. In order to generate the correct solution more consistently, the nonlinear programming solver used in a traditional direct method must be replaced or augmented by a Hamiltonian programming method. The theoretical results derived in this paper further strengthen the connections between computational optimal control, deep learning and automatic differentiation.

Motivation & Objective

• To resolve the long-standing perceived divide between direct and indirect optimal control methods.
• To explain the computational failures of classical direct methods patched to generic nonlinear programming solvers.
• To demonstrate that direct shooting methods mask underlying indirect elements, particularly in first-order optimality conditions.
• To advocate for replacing standard NLP solvers with Hamiltonian programming techniques to achieve reliable convergence and accuracy.
• To strengthen the theoretical link between optimal control, deep learning, and automatic differentiation via the covector mapping principle.

Proposed method

• Derives mathematical equivalence between direct shooting and indirect methods using first-order necessary conditions from Pontryagin's maximum principle.
• Applies forward/backward Euler discretizations to reveal that the gradient of the costate in direct methods corresponds to the adjoint variable in indirect methods.
• Introduces the covector mapping principle as a unifying framework to connect direct and indirect methods through Hamiltonian structure.
• Analyzes the impact of step size h in discretized gradients, showing that omitting h leads to incorrect optimality conditions in standard implementations.
• Reveals that the gradient condition ∂E^N/∂u_k = ∂H/∂u_k (equation 70) is foundational for backpropagation in deep learning and reverse-mode automatic differentiation.
• Proposes replacing generic NLP solvers with Hamiltonian programming methods that preserve the underlying structure of the optimal control problem.

Experimental results

Research questions

• RQ1 Is there a fundamental mathematical equivalence between direct shooting methods and indirect methods in optimal control?
• RQ2 Why do standard direct methods fail when patched to generic nonlinear programming solvers despite theoretical equivalence?
• RQ3 What role does the step size h play in the gradient computation of discretized optimal control problems?
• RQ4 How does the covector mapping principle unify direct and indirect methods under a common theoretical framework?
• RQ5 Can Hamiltonian programming improve convergence and accuracy in trajectory optimization and deep learning?

Key findings

• A direct shooting method is mathematically equivalent to an indirect method up to first-order necessary conditions, as shown through the derivation of the covector mapping principle.
• The failure of classical direct methods when patched to NLP solvers stems from the absence of Hamiltonian structure in the gradient computation, particularly the omission of the step size h in the gradient of the costate.
• Equation (70), ∂E^N/∂u_k = ∂H/∂u_k, is shown to be a special case of the general gradient condition and is foundational for backpropagation in deep learning.
• Theoretical equivalence does not imply computational equivalence unless the underlying Hamiltonian structure is preserved in the numerical implementation.
• Hamiltonian programming methods significantly outperform standard NLP solvers in convergence and accuracy by correctly mapping primal and dual variables through the covector mapping principle.
• The signature optimal control problem in this paper is structurally identical to the loss minimization problem in deep learning, suggesting shared optimization principles across fields.