[Paper Review] Classical Adjoints for Ergodic Stochastic Control

Samuel N. Cohen, Victor Fedyashov|arXiv (Cornell University)|Nov 13, 2015

Stochastic processes and financial applications9 references3 citations

TL;DR

This paper establishes a strong duality between the existence of a solution to an infinite-horizon adjoint backward stochastic differential equation (BSDE) and strong dissipativity of the controlled diffusion process in ergodic stochastic control. It proves that strong dissipativity implies irreducibility, the strong Feller property, and exponential ergodicity, thereby justifying the use of adjoint BSDEs for optimality in the strong formulation where drift and volatility are both controlled.

ABSTRACT

In this paper we consider ergodic optimal control of a diffusion process $\{X^u_t\}_{t \geq 0}$, taking values in $\bR^n$, where both drift and volatility are controlled. We establish a novel strong duality between the existence of a unique solution to the infinite horizon adjoint BSDE and strong dissipativity of $X^u$. We then proceed to show that the latter implies irreducibility, the strong Feller property and exponential ergodicity. We conclude by discussing the connection with ergodic BSDEs.

Motivation & Objective

• To address ergodic stochastic control problems where both drift and volatility are controlled, moving beyond weak formulations based on measure changes.
• To establish a rigorous connection between the solvability of an infinite-horizon adjoint BSDE and the ergodic properties of the forward process.
• To prove that strong dissipativity of the drift ensures irreducibility, the strong Feller property, and exponential ergodicity for the controlled diffusion.
• To demonstrate that the adjoint BSDE approach is valid in the strong formulation, even when the control affects both drift and volatility.
• To clarify the limitations of ergodic BSDEs in the strong control setting and show why they cannot capture the full optimality condition when volatility is controlled.

Proposed method

• Proposes a novel infinite-horizon adjoint BSDE for ergodic control, derived from the stochastic maximum principle framework.
• Uses the strong dissipativity condition on the drift: ⟨b(t,x,u)−b(t,y,u),x−y⟩ ≤ −μ‖x−y‖² for μ>0.
• Applies Grönwall’s lemma and stability estimates to establish moment bounds on the forward process under control.
• Links the existence of a solution to the adjoint BSDE to the ergodicity of the forward process via duality.
• Employs the vanishing discount method to construct solutions to the ergodic BSDE and connects them to long-run average costs.
• Uses periodicity assumptions (Assumption 5) to ensure existence of an invariant measure and to identify the ergodic average λᵘ as the long-run cost.

Experimental results

Research questions

• RQ1 Under what conditions does an adjoint BSDE exist for ergodic control when both drift and volatility are controlled?
• RQ2 How is the solvability of the adjoint BSDE related to the ergodic properties of the forward diffusion process?
• RQ3 Can strong dissipativity of the drift imply exponential ergodicity, irreducibility, and the strong Feller property in the general control setting?
• RQ4 Why is the standard ergodic BSDE approach insufficient when volatility is controlled?
• RQ5 What is the connection between the solution of the adjoint BSDE and the long-run average cost in the ergodic control problem?

Key findings

• The existence of a solution to the infinite-horizon adjoint BSDE is equivalent to strong dissipativity of the forward process under all controls.
• Strong dissipativity implies that the controlled diffusion process is irreducible and satisfies the strong Feller property.
• Strong dissipativity leads to exponential ergodicity, meaning the law of the process converges to its invariant measure at an exponential rate.
• The long-run average cost λᵘ, which is the optimal value in the ergodic control problem, equals the solution λᵘ of the ergodic BSDE under periodicity assumptions.
• The adjoint BSDE approach is valid in the strong formulation, even when volatility is controlled, unlike the weak formulation based on measure change.
• The paper shows that ergodic BSDEs alone cannot capture the full optimality condition when both drift and volatility are controlled, due to the lack of a martingale representation under control of volatility.