[Paper Review] Motion Planning via Optimal Control for Stochastic Processes

Peyman Mohajerin Esfahani, Debasish Chatterjee+1 more|arXiv (Cornell University)|Nov 6, 2012

Stochastic processes and financial applications21 references3 citations

TL;DR

This paper proposes a weak dynamic programming principle (DPP) for stochastic motion planning involving controlled processes with discontinuous paths, enabling sequential target visits and obstacle avoidance. The key contribution is characterizing the set of initial states that achieve desired maneuvers with pre-specified probability as a level set of discontinuous viscosity solutions to a sequence of PDEs with recursively defined boundary conditions.

ABSTRACT

We study stochastic motion planning problems which involve a controlled process, with possibly discontinuous sample paths, visiting certain subsets of the state-space while avoiding others in a sequential fashion. For this purpose, we first introduce two basic notions of motion planning, and then establish a connection to a class of stochastic optimal control problems concerned with sequential stopping times. A weak dynamic programming principle (DPP) is then proposed, which characterizes the set of initial states that admit a policy enabling the process to execute the desired maneuver with probability no less than some pre-specified value. The proposed DPP comprises auxiliary value functions defined in terms of discontinuous payoff functions. A concrete instance of the use of this novel DPP in the case of diffusion processes is also presented. In this case, we establish that the aforementioned set of initial states can be characterized as the level set of a discontinuous viscosity solution to a sequence of partial differential equations, for which the first one has a known boundary condition, while the boundary conditions of the subsequent ones are determined by the solutions to the preceding steps. Finally, the generality and flexibility of the theoretical results are illustrated on an example involving biological switches.

Motivation & Objective

• To address stochastic motion planning in systems with discontinuous sample paths, where the process must sequentially visit target subsets while avoiding forbidden regions.
• To formalize two fundamental notions of motion planning in a stochastic context, particularly focusing on probabilistic guarantees.
• To establish a connection between motion planning and stochastic optimal control problems involving sequential stopping times.
• To develop a weak DPP that characterizes initial states enabling successful execution of desired maneuvers with probability at least a given threshold.
• To demonstrate the method's generality through an application to biological switch models, illustrating its flexibility in complex systems.

Proposed method

• Introduces a weak dynamic programming principle (DPP) that incorporates auxiliary value functions defined via discontinuous payoff functions.
• Models the motion planning problem as a sequence of stochastic optimal control problems with stopping times, ensuring sequential target visits.
• Applies the DPP to diffusion processes, deriving a system of partial differential equations (PDEs) with recursively defined boundary conditions.
• Characterizes the set of initial states enabling successful maneuvers as a level set of a discontinuous viscosity solution to the PDE system.
• Uses viscosity solution theory to handle the discontinuities in the payoff functions and boundary conditions.
• Employs recursive solution steps: the first PDE has a known boundary condition, and each subsequent PDE’s boundary condition is derived from the solution of the prior step.

Experimental results

Research questions

• RQ1 How can motion planning be formulated for stochastic processes with discontinuous sample paths, ensuring sequential target visits and obstacle avoidance?
• RQ2 What is the role of sequential stopping times in characterizing feasible initial states for desired maneuvers under probabilistic constraints?
• RQ3 How can a weak dynamic programming principle be constructed when payoff functions and boundary conditions are discontinuous?
• RQ4 In the context of diffusion processes, how can the set of initial states enabling a desired maneuver be characterized mathematically?
• RQ5 To what extent can the proposed DPP framework be applied to complex systems such as biological switches?

Key findings

• The set of initial states that admit a policy to execute the desired maneuver with probability at least a pre-specified threshold is characterized as a level set of a discontinuous viscosity solution to a sequence of PDEs.
• The first PDE in the sequence has a known boundary condition, while the boundary conditions for subsequent PDEs are determined recursively by the solutions of earlier equations.
• The method successfully handles discontinuous payoff functions through the use of viscosity solutions, ensuring mathematical rigor despite non-smoothness.
• The framework is general enough to model complex systems, as demonstrated by its application to biological switch dynamics.
• The theoretical results provide a systematic way to compute feasible initial states for motion planning under uncertainty, even when sample paths are discontinuous.
• The recursive structure of the PDE system enables scalable analysis of sequential motion planning tasks in stochastic environments.