[Paper Review] Extended Applicability of the Symplectic Pontryagin Method

Mattias Sandberg|ArXiv.org|Jan 29, 2009

Matrix Theory and Algorithms7 references3 citations

TL;DR

This paper extends the Symplectic Pontryagin method for solving optimal control problems by proving existence and convergence under weaker assumptions, eliminating the need for bounded gradient of the dual variable. It achieves an error bound of order $\delta + \Delta t$, allowing independent refinement of regularization and time discretization without error degradation.

ABSTRACT

The Symplectic Pontryagin method was introduced in a previous paper. This work shows that this method is applicable under less restrictive assumptions. Existence of solutions to the Symplectic Pontryagin scheme are shown to exist without the previous assumption on a bounded gradient of the discrete dual variable. The convergence proof uses the representation of solutions to a Hamilton-Jacobi-Bellman equation as the value function of an associated variation problem.

Motivation & Objective

• To establish the existence of solutions to the Symplectic Pontryagin scheme under less restrictive conditions than previous work.
• To remove the prior assumption requiring bounded gradients of the discrete dual variable.
• To improve the error estimate from $\delta + \Delta t + \Delta t^2/\delta$ to $\delta + \Delta t$, enabling independent optimization of $\delta$ and $\Delta t$.
• To provide a convergence proof based on representation of solutions as minimizers of a variation problem and semiconcavity of the value function.

Proposed method

• Use a regularized Hamiltonian $H^\delta$ satisfying $|H^\delta - H| \leq \delta$ to ensure differentiability and enable numerical solution.
• Apply the Symplectic Euler scheme to the regularized Hamiltonian system, with time discretization $\Delta t = T/N$.
• Represent the solution to the Hamilton-Jacobi-Bellman equation as the minimum of a variation problem involving the running cost $L(x,\alpha)$.
• Introduce a modified Hamiltonian $\tilde{H}$ with a quadratic penalty term to ensure compactness of maximizers in the dual variable $\lambda$.
• Leverage local semiconcavity of the value function $u$ to bound the difference between discrete and continuous time derivatives.
• Derive error bounds by comparing the discrete scheme's value with the continuous variation problem, using superdifferential estimates and Lipschitz continuity.

Experimental results

Research questions

• RQ1 Can the Symplectic Pontryagin method be proven to have solutions without assuming bounded gradients of the dual variable?
• RQ2 What is the optimal error bound for the Symplectic Pontryagin method under relaxed regularity assumptions?
• RQ3 Can the regularization parameter $\delta$ and time step $\Delta t$ be adjusted independently without degrading convergence?
• RQ4 How does the representation of the value function as a minimizer of a variation problem support convergence analysis?
• RQ5 What role does semiconcavity of the value function play in bounding the error between discrete and continuous solutions?

Key findings

• Solutions to the Symplectic Pontryagin scheme exist for a broader class of Hamiltonians, without requiring bounded gradients of the dual variable.
• The error bound is established as $\delta + \Delta t$, which is superior to the prior bound of $\delta + \Delta t + \Delta t^2/\delta$.
• The regularization parameter $\delta$ and time step $\Delta t$ can be chosen independently, as the error does not deteriorate with decreasing $\delta$ when $\Delta t$ is fixed.
• The method's convergence is proven via a representation of the value function as the minimum of a variation problem, with the discrete scheme approximating this minimum.
• The use of a penalized Hamiltonian $\tilde{H}$ ensures compactness of maximizers in the dual variable, enabling uniform bounds on $\lambda^*$.
• A lower bound on the difference $\bar{u}(x_0,0) - u(x_0,0)$ is derived as $-\frac{1}{2}C_1C_2(C_3+1)e^{C_2T}T\Delta t$, confirming linear convergence in $\Delta t$.