[Paper Review] Piecewise-Deterministic Optimal Path Planning

Zhengdi Shen, Alexander Vladimirsky|arXiv (Cornell University)|Dec 29, 2015

Risk and Portfolio Optimization36 references3 citations

TL;DR

This paper presents a numerical framework for optimal path planning in piecewise-deterministic systems with random environmental switches, using semi-Lagrangian and Eulerian schemes to solve a weakly-coupled system of Hamilton-Jacobi-Bellman PDEs. The key contribution is demonstrating that asymptotically optimal controls—based on infinite switching rates or zero switching rates—can significantly degrade performance in finite-rate regimes, with the infinite-rate strategy causing collisions in Monte Carlo simulations.

ABSTRACT

We consider piecewise-deterministic optimal control problems in which the environment randomly switches among several deterministic modes, and the goal is to optimize the expected cost up to the termination while taking the likelihood of future mode-switches into account. The dynamic programming approach yields a weakly-coupled system of Hamilton-Jacobi-Bellman PDEs satisfied by the value functions of individual modes. We derive and implement semi-Lagrangian and Eulerian numerical schemes for that system. We further recover simpler, "asymptotically optimal" controls and test their performance in non-asymptotic regimes. Our approach is illustrated on a simple anisotropic path-planning problem: the time-optimal control for a boat affected by randomly switching winds.

Motivation & Objective

• To develop efficient numerical methods for solving optimal control problems in systems with random, instantaneous mode switches.
• To analyze the performance gap between asymptotically optimal controls (infinite or zero switching rates) and the true optimal solution in finite-rate switching environments.
• To quantify the expected performance degradation when using simplified control strategies instead of solving the full weakly-coupled HJB system.
• To demonstrate that optimal trajectories in piecewise-deterministic settings are no longer piecewise straight lines, unlike in deterministic cases.
• To derive a priori bounds on the difference between mode-specific value functions under general switching dynamics.

Proposed method

• Formulates the optimal control problem as a weakly-coupled system of first-order Hamilton-Jacobi-Bellman (HJB) PDEs, where each mode corresponds to a different deterministic dynamics.
• Derives the governing PDE system: $\|\nabla u_i\|s(\mathbf{x}) - \mathbf{w}_i(\mathbf{x})\cdot\nabla u_i = 1 - \sum_{j=1}^n \lambda_{ij}(u_i - u_j)$, encoding expected cost under random switching.
• Implements semi-Lagrangian and Eulerian numerical schemes to solve the coupled HJB system efficiently on a Cartesian grid.
• Proposes two asymptotically optimal control strategies: one assuming instantaneous switching (infinite rate $c \to \infty$), and one assuming no switching (zero rate $c \to 0$).
• Uses Monte Carlo simulations to compare the performance of asymptotic controls against the true optimal solution in a 2D path-planning problem with a single obstacle.
• Derives an upper bound on $\|u_i - u_j\|_\infty$ using hitting time expectations and Lipschitz continuity, showing $|u_i - u_j| = O(c^{-1})$ as $c \to \infty$.

Experimental results

Research questions

• RQ1 How does the performance of asymptotically optimal controls (infinite or zero switching rates) degrade when applied in finite-rate switching environments?
• RQ2 What are the structural differences in optimal trajectories between deterministic and piecewise-deterministic path-planning problems?
• RQ3 Can a priori bounds be derived for the difference between mode-specific value functions in a switching system?
• RQ4 How do numerical schemes for weakly-coupled HJB systems compare in accuracy and efficiency for path-planning under uncertainty?
• RQ5 What is the impact of random environmental switches on the optimality and smoothness of optimal paths?

Key findings

• The asymptotic control strategy based on infinite switching rates ($u^\infty$) results in collisions with obstacles in a significant fraction of Monte Carlo simulations, indicating substantial performance degradation.
• The asymptotic control based on zero switching rates ($\alpha_*^0$) performs better than $u^\infty$, but still incurs noticeable expected time cost increases compared to the true optimal solution.
• Optimal trajectories in the piecewise-deterministic setting are no longer piecewise straight lines, even when the velocity field is constant in each mode, due to the influence of switching dynamics.
• The difference between mode-specific value functions $u_i$ and $u_j$ is bounded by $\bar{C} \cdot \max\{\mathbb{E}[\kappa_{ij}], \mathbb{E}[\kappa_{ji}]\}$, with $\kappa_{ij}$ the hitting time to switch from mode $i$ to $j$.
• As the switching rate $c$ increases, the difference between value functions decays as $O(c^{-1})$, confirming the asymptotic convergence of $u^\infty$ to the true solution.
• The theoretical upper bound on value function differences is derived under mild assumptions: bounded cost, Lipschitz continuity of value functions, and local controllability.