[Paper Review] A Maximum Principle for Optimal Control of Stochastic Evolution Equations

Kai Du, Qingxin Meng|arXiv (Cornell University)|Jun 16, 2012

Stochastic processes and financial applications4 citations

TL;DR

This paper establishes a general maximum principle for optimal control of infinite-dimensional stochastic evolution equations with non-convex control sets and control-dependent diffusion, using a novel second-order duality analysis via Lebesgue differentiation and approximation to derive an operator-valued second-order adjoint process. The key contribution is a necessary optimality condition that extends Peng's finite-dimensional maximum principle to the abstract infinite-dimensional setting.

ABSTRACT

A general maximum principle is proved for optimal controls of abstract semilinear stochastic evolution equations. The control variable, as well as linear unbounded operators, acts in both drift and diffusion terms, and the control set need not be convex.

Motivation & Objective

• To close the long-standing gap in deriving a general maximum principle for infinite-dimensional stochastic control systems with non-convex control sets and control-dependent diffusion.
• To extend Peng's finite-dimensional maximum principle to abstract semilinear stochastic evolution equations in Hilbert spaces.
• To develop a new method for second-order duality analysis in the infinite-dimensional setting, overcoming the challenge of solving operator-valued backward stochastic differential equations.
• To establish necessary optimality conditions that apply to a broad class of stochastic partial differential equations with controlled coefficients.
• To provide a theoretical foundation applicable to concrete problems such as controlled SPDEs and stochastic heat equations with non-linear cost functionals.

Proposed method

• Formulates the optimal control problem for a semilinear stochastic evolution equation driven by a Wiener process, with unbounded linear operators and control-dependent drift and diffusion.
• Applies the second-order variation method to derive a variational inequality, focusing on the quadratic term arising from the perturbation analysis.
• Uses the Lebesgue differentiation theorem and an approximation argument to show convergence of the quadratic term to a stochastic bilinear functional.
• Introduces the second-order adjoint process as an operator-valued process that represents the bilinear functional, enabling the derivation of the maximum condition.
• Establishes $L^p$-estimates for the state process and analyzes the regularity and representation of stochastic bilinear functionals.
• Derives the maximum principle by showing that the first-order variation condition leads to a necessary optimality condition involving the adjoint processes and the control.

Experimental results

Research questions

• RQ1 Can a general maximum principle be established for infinite-dimensional stochastic control systems where the control set is non-convex and the diffusion depends on the control?
• RQ2 How can second-order duality analysis be performed in the infinite-dimensional setting when the associated backward stochastic differential equation is operator-valued and potentially unsolvable?
• RQ3 What is the role of the second-order adjoint process in characterizing optimality when the cost functional is non-linear in the state and control?
• RQ4 To what extent can the proposed method be applied to concrete stochastic PDEs with controlled coefficients and non-convex control domains?
• RQ5 Can the method handle cost functionals with quadratic terms in the control that are not covered by existing results in the literature?

Key findings

• The paper proves a general maximum principle for optimal control of abstract semilinear stochastic evolution equations with non-convex control sets and control-dependent diffusion.
• The second-order duality analysis is achieved through an approximation argument and the Lebesgue differentiation theorem, avoiding direct solution of the operator-valued backward stochastic differential equation.
• The quadratic variation term converges to a stochastic bilinear functional represented by an operator-valued second-order adjoint process.
• The necessary optimality condition is derived in the form of a maximum condition involving the first- and second-order adjoint processes and the control perturbation.
• The abstract results are applied to two examples: a controlled super-parabolic SPDE and a stochastic heat equation with non-linear cost, both with non-convex control and quadratic cost terms.
• The method is shown to be applicable to problems not covered by prior works such as [10] and [5], particularly due to the treatment of quadratic cost functionals.