[Paper Review] Instantaneous Control of Brownian Motion with a Positive Lead Time

Zhen Xu, Jiheng Zhang+1 more|arXiv (Cornell University)|Aug 4, 2016

Stochastic processes and financial applications16 references3 citations

TL;DR

This paper introduces a novel framework for instantaneous control of Brownian motion with a positive lead time for upward adjustments, extending $L^{ atural}$-convexity to function spaces to prove that the optimal policy is a state-dependent two-sided reflection that minimally adjusts inventory to keep it within a bounded region, resolving a long-standing challenge in stochastic control with functional state spaces.

ABSTRACT

Consider a storage system where the content is driven by a Brownian motion absent control. At any time, one may increase or decrease the content at a cost proportional to the amount of adjustment. A decrease of the content takes effect immediately, while an increase is realized after a fixed lead time $\lt$. Holding costs are incurred continuously over time and are a convex function of the content. The objective is to find a control policy that minimizes the expected present value of the total costs. Due to the positive lead time for upward adjustments, one needs to keep track of all the outstanding upward adjustments as well as the actual content at time $t$ as there may also be downward adjustments during $[t,t+\lt)$, i.e., the state of the system is a function on $[0,\ell]$. To the best of our knowledge, this is the first paper to study instantaneous control of stochastic systems in such a functional setting. We first extend the concept of $L^ atural$-convexity to function spaces and establish the $L^ atural$-convexity of the optimal cost function. We then derive various properties of the cost function and identify the structure of the optimal policy as a state-dependent two-sided reflection mapping making the minimum amount of adjustment necessary to keep the system states within a certain region.

Motivation & Objective

• To address the challenge of optimal control in a Brownian storage system where upward adjustments have a positive lead time, while downward adjustments are instantaneous.
• To extend the concept of $L^{ atural}$-convexity from finite-dimensional spaces to function spaces, enabling analysis of systems with infinite-dimensional states.
• To characterize the optimal control policy as a state-dependent two-sided reflection that minimally adjusts inventory to prevent entry into costly regions.
• To establish a rigorous mathematical framework for solving stochastic control problems with functional state spaces, overcoming limitations of traditional methods.

Proposed method

• Extend $L^{ atural}$-convexity to function spaces by showing the optimal cost function is the limit of costs from periodic review systems.
• Derive a heuristic Hamilton-Jacobi-Bellman (HJB) equation based on optimality conditions for timing and magnitude of adjustments.
• Use $L^{ atural}$-convexity to identify two distinct regions in the state space where upward or downward adjustments are required.
• Construct a state-dependent two-sided reflection policy that makes the minimal necessary adjustments to keep the system state within a safe region.
• Prove optimality of the policy by establishing key properties: monotonicity, Lipschitz continuity, and complementarity.
• Transform the general case with asymmetric lead times into an equivalent problem with zero downward lead time, reducing complexity.

Experimental results

Research questions

• RQ1 How can $L^{ atural}$-convexity be extended to function spaces to analyze stochastic control problems with infinite-dimensional states?
• RQ2 What is the structure of the optimal control policy when upward adjustments have a positive lead time and downward adjustments are instantaneous?
• RQ3 Can a two-sided reflection policy be proven optimal in a functional state space setting with such lead time asymmetry?
• RQ4 How does the optimal cost function behave under $L^{ atural}$-convexity in the context of Brownian motion with delayed control?
• RQ5 Can the general case with positive lead times for both upward and downward adjustments be reduced to a simpler problem with zero lead time for one direction?

Key findings

• The optimal cost function is $L^{ atural}$-convex in the function space of system states, enabling structural analysis of the optimal policy.
• The optimal control policy is a state-dependent two-sided reflection that makes the minimal necessary adjustments to keep the system state within a bounded region.
• The policy is optimal due to the combination of monotonicity, Lipschitz continuity, and complementarity properties derived from $L^{ atural}$-convexity.
• The problem with asymmetric lead times can be transformed into an equivalent problem with zero downward lead time, preserving optimality up to a constant shift in cost.
• The cost difference between the original and transformed systems is a constant that depends only on the initial state and the downward lead time.
• This is the first paper to solve an instantaneous control problem in a functional state space, providing a new methodology for complex stochastic control systems.