[Paper Review] Dynamic Programming Principles for Optimal Stopping with Expectation Constraint

Erhan Bayraktar, Song Yao|arXiv (Cornell University)|Aug 7, 2017

Stochastic processes and financial applications39 references3 citations

TL;DR

This paper establishes a dynamic programming principle (DPP) for optimal stopping problems with an expectation constraint on cumulative costs, proving the value function is continuous and characterizing it as a viscosity solution to a fully nonlinear parabolic Hamilton-Jacobi-Bellman equation. The key innovation is introducing the conditional expected cost as an additional state process, enabling a DPP that resolves an open question in constrained optimal stopping.

ABSTRACT

We analyze an optimal stopping problem with a constraint on the expected cost. When the reward function and cost function are Lipschitz continuous in state variable, we show that the value of such an optimal stopping problem is a continuous function in current state and in budget level. Then we derive a dynamic programming principle (DPP) for the value function in which the conditional expected cost acts as an additional state process. As the optimal stopping problem with expectation constraint can be transformed to a stochastic optimization problem with supermartingale controls, we explore a second DPP of the value function and thus resolve an open question recently raised in [S. Ankirchner, M. Klein, and T. Kruse, A verification theorem for optimal stopping problems with expectation constraints, Appl. Math. Optim., 2017, pp. 1-33]. Based on these two DPPs, we characterize the value function as a viscosity solution to the related fully non-linear parabolic Hamilton-Jacobi-Bellman equation.

Motivation & Objective

• To address the lack of dynamic programming principles in optimal stopping problems with expectation constraints on cumulative costs.
• To establish continuity of the value function in state and budget variables under Lipschitz and non-degeneracy conditions.
• To resolve an open question on DPPs for constrained optimal stopping by introducing the conditional expected cost as an auxiliary state process.
• To characterize the value function as a viscosity solution to a fully nonlinear parabolic HJB equation.
• To connect the constrained optimal stopping problem to stochastic control with supermartingale controls, enabling a second DPP.

Proposed method

• Derives a DPP by treating the conditional expected cost as an additional state process, using shifted stochastic differential equations and flow properties.
• Employs a priori estimates and approximate stopping strategies to prove continuity of the value function in (t, x, y).
• Utilizes regular conditional probability distributions and the Markov property of the state process to establish the DPP's '≤' direction.
• Constructs ε-optimal strategies by pasting local stopping rules, leveraging value function continuity to prove the '≥' direction of the DPP.
• Transforms the constrained optimal stopping problem into an unconstrained stochastic control problem with supermartingale controls starting from budget level y.
• Characterizes the value function as a viscosity solution to a fully nonlinear parabolic HJB equation, including a Monge-Ampère type structure.

Experimental results

Research questions

• RQ1 Can a dynamic programming principle be established for optimal stopping problems with an expectation constraint on the accumulated cost?
• RQ2 Is the value function continuous in the state variable and the budget level under Lipschitz and non-degeneracy conditions?
• RQ3 How can the conditional expected cost be incorporated as a state process to enable a DPP in this constrained setting?
• RQ4 Can the constrained problem be reformulated as a stochastic control problem with supermartingale controls to derive a second DPP?
• RQ5 Is the value function a viscosity solution to a fully nonlinear parabolic Hamilton-Jacobi-Bellman equation?

Key findings

• The value function V(t, x, y) is continuous in (t, x, y) under Lipschitz continuity of f, π, g and non-degeneracy of g.
• A dynamic programming principle is established where the conditional expected cost Y^{t,x,τ}_s acts as an additional state process, yielding a DPP of the form V(t,x,y) = sup_E[1_{τ≤ζ(τ)} R(t,x,τ) + 1_{τ>ζ(τ)} (V(ζ(τ), X^{t,x}_{ζ(τ)}, Y^{t,x,τ}_{ζ(τ)}) + ∫_t^{ζ(τ)} f(r,X^{t,x}_r) dr)].
• The value function is characterized as a viscosity solution to a fully nonlinear parabolic Hamilton-Jacobi-Bellman equation, including a Monge-Ampère type structure.
• A second DPP is derived by transforming the constrained problem into an unconstrained stochastic control problem with supermartingale controls, resolving an open question in [Ankirchner et al., 2017].
• The continuity of V enables the construction of ε-optimal stopping strategies through pasting, which is critical for proving the DPP's reverse inequality.
• The value function is shown to be finite and measurable, with E_t[K_*] < ∞, ensuring the validity of the dynamic programming framework.