[Paper Review] Linear Programming Formulations of Singular Stochastic Control Problems: Time-Homogeneous Problems

Thomas G. Kurtz, Richard H. Stockbridge|arXiv (Cornell University)|Jul 28, 2017

Economic theories and models11 references3 citations

TL;DR

This paper establishes the equivalence of time-homogeneous singular stochastic control problems with linear programs over measures representing the expected occupation of state and control processes under both long-term average and discounted criteria. The key contribution is a rigorous characterization of optimal relaxed controls in feedback form, with existence of strict controls proven under closedness and compactness conditions on the control set.

ABSTRACT

Conditions are established under which the optimal control of processes having both absolutely continuous and singular (with respect to time) controls are equivalent to linear programs over a space of measures on the state and control spaces. This paper considers long-term average and discounted criteria and includes budget and resource constraints. The linear programs optimize over measures representing the expected occupation measure of the state and absolutely continuous control processes and a similar expected occupation measure of the state and control when the singular action of the process occurs. The evolution of these processes is characterized through an adjoint equation which the measures must satisfy in relation to the absolutely continuous and singular generators of the process. Existence of optimal relaxed controls of feedback type are established in general while existence of an optimal form of strict control is proven under additional closedness and compactness conditions.

Motivation & Objective

• To establish equivalence between singular stochastic control problems and linear programs over occupation measures.
• To extend linear programming formulations to singular controls under time-homogeneous dynamics.
• To prove existence of optimal strict controls under closedness and compactness conditions on the control set.
• To incorporate budget and resource constraints into the linear programming framework.
• To characterize optimal feedback controls for both long-term average and discounted criteria.

Proposed method

• Formulate the control problem via a singular, controlled martingale problem with generators A (absolutely continuous) and B (singular) acting on test functions in C²_c(ℝ).
• Define two occupation measures: μ₀ for the state process and μ₁ for the singular control actions, both representing expected occupation times.
• Derive an adjoint equation linking the measures μ₀ and μ₁ to the generators A and B, ensuring the martingale condition is satisfied.
• Construct a linear program minimizing cost functions c₀ and c₁ over μ₀ and μ₁, subject to the adjoint equation and constraints on measure support and total mass.
• Use rescaling techniques (as in Remark 3.4) to reformulate the discounted problem with a normalized initial measure ν₀ and discount rate α.
• Apply compactness and closedness conditions to lift relaxed optimal controls to strict feedback controls.

Experimental results

Research questions

• RQ1 Under what conditions is the optimal control of a singular stochastic process equivalent to a linear program over occupation measures?
• RQ2 How can budget and resource constraints be incorporated into the linear programming formulation of singular control problems?
• RQ3 What conditions ensure the existence of an optimal strict control rather than just a relaxed one?
• RQ4 Can the optimal control be represented in feedback form under time-homogeneous dynamics?
• RQ5 How do the long-term average and discounted criteria map to equivalent linear programs with compactly supported measures?

Key findings

• The optimal control problem for time-homogeneous singular diffusions is equivalent to a linear program over measures μ₀ and μ₁ representing expected occupation of the state and singular control actions.
• Existence of an optimal relaxed feedback control is established under general conditions, including the adjoint equation constraint.
• Under closedness and convexity of the control set K(x), an optimal strict control u* exists, as shown in Theorem 4.5.
• For the discounted problem, the linear program is reformulated using a rescaled initial measure ν₀ with total mass 1/α, ensuring finite cost.
• In the inventory control example, the optimal policy is characterized via a linear program with compactly supported μ₁, even when c₁ is not inf-compact.
• The approach generalizes prior results by Kurtz and Stockbridge (1998) and Helmes and Stockbridge (2007) to include singular controls and constraints.