[Paper Review] A probabilistic max-plus numerical method for solving stochastic control problems

Akian, Marianne, Fodjo, Eric|arXiv (Cornell University)|May 9, 2016

Stochastic processes and financial applications1 references3 citations

TL;DR

This paper presents a novel probabilistic max-plus numerical method for solving fully nonlinear Hamilton-Jacobi-Bellman equations arising in stochastic control problems with both finite-set and continuum-valued controls. By combining McEneaney's idempotent max-plus method with Fahim, Touzi, and Warin's probabilistic scheme, the algorithm achieves polynomial-time complexity without pruning, enabling efficient computation of value functions in high-dimensional settings, as validated on a two-dimensional option pricing problem with uncertain volatility.

ABSTRACT

We consider fully nonlinear Hamilton-Jacobi-Bellman equations associated to diffusion control problems involving a finite set-valued (or switching) control and possibly a continuum-valued control. We construct a lower complexity probabilistic numerical algorithm by combining the idempotent expansion properties obtained by McEneaney, Kaise and Han (2011) for solving such problems with a numerical probabilistic method such as the one proposed by Fahim, Touzi and Warin (2011) for solving some fully nonlinear parabolic partial differential equations. Numerical tests on a small example of pricing and hedging an option are presented.

Motivation & Objective

• To develop a low-complexity numerical method for solving fully nonlinear HJB equations in stochastic control with mixed discrete and continuous controls.
• To overcome the limitations of existing probabilistic methods that require strict bounds on diffusion matrices and regularity conditions on the Hamiltonian.
• To integrate idempotent max-plus methods with Monte Carlo-based probabilistic schemes to ensure polynomial-time complexity.
• To enable efficient computation of value functions in high-dimensional problems, such as option pricing with uncertain volatility.
• To validate the method on a benchmark two-dimensional option pricing problem with switching volatility regimes.

Proposed method

• The method combines McEneaney et al.'s idempotent max-plus approach, which represents the value function as a supremum of quadratic forms, with Fahim, Touzi, and Warin's probabilistic scheme based on backward stochastic differential equations.
• It uses Monte Carlo simulation of a small number of uncontrolled stochastic processes to estimate conditional expectations in the backward recursion.
• The algorithm computes the value function at each time step as a supremum over controls, with each term represented as a quadratic form, avoiding exponential growth via probabilistic sampling.
• The method ensures polynomial-time complexity in the number of time steps and sample size, even without pruning, by leveraging the linearity of expectation and the max-plus structure.
• The value function is approximated at time T using a finite supremum of concave quadratic forms, with curvature adjusted to match the payoff function's behavior.
• The scheme is applied to a two-asset option pricing problem with uncertain volatility, where the control switches between two correlation levels.

Experimental results

Research questions

• RQ1 Can a probabilistic numerical method be designed to solve fully nonlinear HJB equations with mixed discrete and continuous controls while maintaining polynomial-time complexity?
• RQ2 Does combining idempotent max-plus methods with probabilistic schemes eliminate the need for pruning and exponential growth in the number of quadratic forms?
• RQ3 Can the method handle cases where the diffusion matrix is not bounded below by a positive definite matrix, thus overcoming limitations of prior probabilistic methods?
• RQ4 How does the method perform on high-dimensional stochastic control problems such as option pricing with uncertain volatility?
• RQ5 What is the impact of sampling parameters on the accuracy and stability of the value function approximation?

Key findings

• The proposed algorithm achieves polynomial-time complexity in the number of time steps and sample size, even without pruning, due to the probabilistic sampling strategy.
• For the two-dimensional option pricing problem with uncertain volatility, the method produced results comparable to those in [5], with the value function closely matching reference solutions.
• When tested with a single control (ρ = -0.8 or 0.8), the method achieved errors of 0.157 (sup-norm) and 0.074 (ℓ¹ norm) at t=0, with ξ₂=50 and ξ₁∈[20,80].
• The method with Nₘ=2 (two control samples) outperformed Nₘ=3 and Nₘ=4, suggesting that higher Nₘ may introduce bias due to maximization over independent random variables.
• The error norms were comparable to those reported in [7] for standard regression-based methods, indicating competitive accuracy despite the non-standard approach.
• The method successfully approximated the non-semiconvex payoff function ψ(ξ) = (ξ₁−ξ₂−K₁)⁺−(ξ₁−ξ₂−K₂)⁺ using quadratic forms with large negative curvature, which previous max-plus methods could not handle.