[Paper Review] Viscosity Solutions of Stochastic Hamilton-Jacobi-Bellman Equations

Jinniao Qiu|arXiv (Cornell University)|Sep 18, 2017

Stochastic processes and financial applications28 references3 citations

TL;DR

This paper introduces a viscosity solution framework for fully nonlinear stochastic Hamilton-Jacobi-Bellman (HJB) equations arising in optimal stochastic control with random coefficients. It establishes that the value function is the maximal viscosity solution, and proves uniqueness under superparabolic conditions where diffusion coefficients are deterministic functions of time, state, and control.

ABSTRACT

In this paper we study the fully nonlinear stochastic Hamilton-Jacobi-Bellman (HJB) equation for the optimal stochastic control problem of stochastic differential equations with random coefficients. The notion of viscosity solution is introduced, and we prove that the value function of the optimal stochastic control problem is the maximal viscosity solution of the associated stochastic HJB equation. For the superparabolic cases when the diffusion coefficients are deterministic functions of time, states and controls, the uniqueness is addressed as well.

Motivation & Objective

• To address the open problem of existence and uniqueness of solutions for fully nonlinear stochastic HJB equations in the non-Markovian setting with random coefficients.
• To define a rigorous notion of viscosity solution for stochastic HJB equations involving backward stochastic PDEs with random coefficients.
• To establish the value function of the optimal stochastic control problem as the maximal viscosity solution of the associated stochastic HJB equation.
• To prove uniqueness of the viscosity solution under superparabolic conditions where the diffusion coefficient is deterministic.
• To extend the dynamic programming principle to non-Markovian settings via stochastic HJB equations with random coefficients.

Proposed method

• Introduces a viscosity solution concept for fully nonlinear stochastic HJB equations driven by Brownian motion and random coefficients.
• Uses the Doob-Meyer decomposition to define the time and martingale components of the solution, identifying the backward stochastic PDE structure.
• Applies the notion of stochastic differential operators $\mathfrak{d}_t$ and $\mathfrak{d}_\omega$ to characterize the solution in terms of the unknown random fields $u(t,x)$ and $\psi(t,x)$.
• Employs a comparison principle for viscosity solutions to establish that the value function is the maximal solution.
• Applies Itô's formula and a priori estimates to prove regularity and continuity of the value function and the dynamic cost functional.
• Uses a perturbation argument and stability estimates to show continuity in time and space, relying on Lipschitz conditions on coefficients and the Markov property of the state process.

Experimental results

Research questions

• RQ1 Can a viscosity solution framework be developed for fully nonlinear stochastic HJB equations with random coefficients?
• RQ2 Is the value function of the optimal stochastic control problem the maximal viscosity solution of the associated stochastic HJB equation?
• RQ3 Under what conditions is the viscosity solution of the stochastic HJB equation unique?
• RQ4 How does the non-Markovian structure of the coefficients affect the regularity and solvability of the stochastic HJB equation?
• RQ5 Can the dynamic programming principle be rigorously justified in the context of fully nonlinear stochastic HJB equations with random coefficients?

Key findings

• The value function $V(t,x)$ is the maximal viscosity solution of the stochastic HJB equation (1.5) under the given assumptions.
• For superparabolic cases where the diffusion coefficient $\sigma$ is deterministic (i.e., independent of $\omega$), the viscosity solution is unique.
• The value function $V(t,x)$ is continuous in $t$ and Lipschitz continuous in $x$ almost surely, with $|V(t,x) - V(s,y)| \to 0$ as $(s,y) \to (t,x)$.
• The dynamic cost functional $J(t,x;\theta)$ is continuous in $t$ and Lipschitz in $x$, uniformly over controls $\theta$.
• The value function satisfies the stochastic HJB equation in the viscosity sense, with the generator $\mathbb{H}$ defined via essential infimum over controls.
• The solution structure is consistent with the stochastic Feynman-Kac formula and backward stochastic PDE theory, extending classical results to fully nonlinear settings.