[Paper Review] A general necessary and sufficient optimality conditions for singular control problems

Seïd Bahlali|ArXiv.org|Jan 28, 2008

Stochastic processes and financial applications3 citations

TL;DR

This paper establishes a general necessary and sufficient optimality condition for singular stochastic control problems using a novel first-order approach, avoiding second-order expansions. It derives a global stochastic maximum principle for both relaxed-singular and strict-singular controls, with only one adjoint process, under nonconvex control domains and general state dynamics where the control enters both drift and diffusion coefficients.

ABSTRACT

We consider a stochastic control problem where the set of strict (classical) controls is not necessarily convex and the the variable control has two components, the first being absolutely continuous and the second singular. The system is governed by a nonlinear stochastic differential equation, in which the absolutely continuous component of the control enters both the drift and the diffusion coefficients. By introducing a new approach, we establish necessary and sufficient optimality conditions for two models. The first concerns the relaxed-singular controls, who are a pair of processes whose first component is a measure-valued processes. The second is a particular case of the first and relates to strict-singular control problems. These results are given in the form of global stochastic maximum principle by using only the first order expansion and the associated adjoint equation. This improves and generalizes all the previous works on the maximum principle of controlled stochastic differential equations.

Motivation & Objective

• To develop a unified optimality framework for singular stochastic control problems with nonconvex control domains.
• To extend the stochastic maximum principle to relaxed-singular controls by introducing a measure-valued control process.
• To establish necessary and sufficient conditions using only first-order expansions, avoiding second-order variational inequalities.
• To generalize prior results by eliminating the need for convexity assumptions on the control domain or Hamiltonian in the control variable.
• To unify and improve existing maximum principle formulations for strict-singular and relaxed-singular control problems.

Proposed method

• Introduce a new class of relaxed-singular controls by replacing the $U_1$-valued control $v_t$ with a $\mathbb{P}(U_1)$-valued process $q_t$, representing probability measures on the control set.
• Define the Hamiltonian $\mathcal{H}$ using the first-order expansion of the cost functional and the associated adjoint equation.
• Derive necessary optimality conditions via a convex perturbation on the relaxed control, leveraging Ekeland’s variational principle and stability properties.
• Establish the global stochastic maximum principle through the adjoint process $\left(p_t, P_t\right)$ solving a backward SDE with terminal condition tied to the cost function.
• Use the property that the infimum of $\mathcal{H}$ over $\delta(U_1)$ is achieved at the optimal control, ensuring necessary conditions.
• Prove sufficiency under convexity of $g$ and convexity of $x \mapsto H(t,x,q,p,P)$, ensuring the optimal control minimizes the cost functional.

Experimental results

Research questions

• RQ1 Can a necessary and sufficient optimality condition be derived for singular stochastic control problems without relying on second-order expansions?
• RQ2 How can the stochastic maximum principle be extended to relaxed-singular controls with nonconvex control domains?
• RQ3 Is it possible to unify strict-singular and relaxed-singular control problems under a single optimality framework using only first-order analysis?
• RQ4 What are the minimal assumptions on the control domain and Hamiltonian required to ensure optimality conditions hold?
• RQ5 Can the number of adjoint processes be reduced from two to one while preserving the validity of the maximum principle?

Key findings

• The paper establishes a necessary and sufficient optimality condition in the form of a global stochastic maximum principle for relaxed-singular controls using only first-order expansion.
• The optimality condition is expressed as $\mathcal{H}(t,x_t^{(\mu,\xi)},\mu_t,p_t^{(\mu,\xi)},P_t^{(\mu,\xi)}) = \inf_{q_t \in \delta(U_1)} \mathcal{H}(t,x_t^{(\mu,\xi)},q_t,p_t^{(\mu,\xi)},P_t^{(\mu,\xi)})$ a.s., a.e., which ensures the optimal control minimizes the Hamiltonian.
• The condition $\mathbb{P}\left\{\sum_{i=1}^d \mathbf{1}_{\{k_i(t) + G_i^*(t)p_t^{(\mu,\xi)} \geq 0\}} d\xi_t^i = 0\right\} = 1$ ensures the singular control does not act when not beneficial.
• The paper proves that the optimal control $ (u,\xi) $ minimizes the cost functional $ J $ over $ \mathcal{U} $ if the Hamiltonian is minimized over $ \delta(U_1) $ and the adjoint process satisfies the required conditions.
• The result improves upon Bahlali and Mezerdi (2006) by removing the need for second-order expansions and two adjoint processes, reducing complexity.
• The sufficient optimality condition holds without assuming convexity of $ U_1 $ or convexity of $ H $ in $ v $, making it applicable to broader classes of problems.