[Paper Review] Some sensitivity results in stochastic optimal control: A Lagrange multiplier point of view

Julio Backhoff‐Veraguas, F.J.G. Silva|arXiv (Cornell University)|Apr 2, 2014

Stochastic processes and financial applications22 references3 citations

TL;DR

This paper establishes a rigorous connection between Lagrange multipliers and adjoint processes in stochastic optimal control via a weak form of the Pontryagin principle, enabling first-order sensitivity analysis of the value function under infinite-dimensional perturbations of the dynamics, including random coefficients and expectation-type constraints.

ABSTRACT

In this work we provide a first order sensitivity analysis of some parameterized stochastic optimal control problems. The parameters can be given by random processes. The main tool is the one-to-one correspondence between the adjoint states appearing in a weak form of the stochastic Pontryagin principle and the Lagrange multipliers associated to the state equation.

Motivation & Objective

• To establish a functional analytic framework linking adjoint processes in the stochastic Pontryagin principle to Lagrange multipliers in constrained optimization.
• To extend convex duality and Lagrange multiplier techniques to non-convex, non-linear stochastic optimal control problems with random coefficients.
• To enable first-order sensitivity analysis of the value function under infinite-dimensional perturbations of the SDE dynamics, including drift, diffusion, and initial condition parameters.
• To formalize the relationship between weak-Pontryagin multipliers and Lagrange multipliers associated with the state equation constraint.
• To apply the framework to concrete problems such as mean-variance portfolio selection, recovering known explicit sensitivities via the general theory.

Proposed method

• Formulates the stochastic optimal control problem as a constrained optimization problem in a Hilbert space of Itô processes.
• Introduces a weak-Pontryagin multiplier pair (p, q) satisfying first-order optimality conditions instead of full Hamiltonian minimization.
• Defines a Lagrange multiplier process λ(t) = p(0) + ∫₀ᵗ p(s)ds + ∫₀ᵗ q(s)dW(s), showing it is an Itô process and corresponds to the SDE constraint.
• Uses Itô's formula and martingale properties to derive expressions for the derivative of the value function with respect to perturbations.
• Applies the framework to the linear-quadratic (LQ) and mean-variance portfolio selection problems, deriving explicit sensitivity formulas.
• Verifies theoretical results against known explicit solutions in a deterministic, one-dimensional case with r ≡ 0 and bounded μ, σ.

Experimental results

Research questions

• RQ1 How can the adjoint processes in the stochastic Pontryagin principle be interpreted as Lagrange multipliers in a constrained optimization framework?
• RQ2 What is the precise relationship between weak-Pontryagin multipliers and Lagrange multipliers associated with the state equation constraint?
• RQ3 Can first-order sensitivity analysis of the value function be performed under infinite-dimensional perturbations of the SDE coefficients and initial conditions?
• RQ4 How do the derived sensitivity formulas compare with known explicit solutions in the mean-variance portfolio selection problem?
• RQ5 What is the role of the Hilbert space structure on Itô processes in ensuring the Lagrange multiplier is an Itô process?

Key findings

• A one-to-one correspondence is established between weak-Pontryagin multipliers (p, q) and Lagrange multipliers λ(t) = p(0) + ∫₀ᵗ p(s)ds + ∫₀ᵗ q(s)dW(s), which are Itô processes.
• The value function’s derivative with respect to perturbations in initial wealth x, target A, drift μ, and volatility σ is given by Dv(P;ΔP) = E[∫₀ᵀ p(t)π(t)Δμ(t)dt] + E[∫₀ᵀ q(t)π(t)Δσ(t)dt] + p(0)[Δx − ΔA], with explicit expressions derived.
• In the mean-variance portfolio problem with r ≡ 0 and deterministic μ, σ, the sensitivity to initial wealth is Dₓv = 2(x−A)Δx / (e^{∫Σ²ds}−1), matching the explicit derivative of the known value function.
• The sensitivity to the drift parameter μ is Dₘᵤv = −2(A−x)²e^{∫Σ²ds} / (e^{∫Σ²ds}−1)² ∫₀ᵀ (μ(t)Δμ(t))/σ(t)² dt, which is verified to match the analytical derivative.
• The sensitivity to volatility σ is Dσv = 2(A−x)²e^{∫Σ²ds} / (e^{∫Σ²ds}−1)² ∫₀ᵀ (μ(t)²Δσ(t))/σ(t)³ dt, confirming consistency with explicit computation.
• The theoretical framework recovers known explicit sensitivities in the LQ case, validating the method’s accuracy and generality.