[Paper Review] FBSDE based Neural Network Algorithms for High-Dimensional Quasilinear Parabolic PDEs

Wenzhong Zhang, Wei Cai|arXiv (Cornell University)|Dec 14, 2020

Advanced Mathematical Modeling in Engineering4 citations

TL;DR

This paper proposes two FBSDE-based deep neural network (DNN) algorithms for solving high-dimensional quasilinear parabolic PDEs, leveraging the pathwise difference between discretized FBSDEs and DNN-approximated solutions to train the network. The method achieves nearly half-order convergence rate comparable to Euler–Maruyama discretization, enabling Richardson extrapolation for enhanced accuracy, and demonstrates improved performance on oscillatory solutions using a multiscale DNN architecture for a 100-dimensional Black–Scholes–Barenblatt equation.

ABSTRACT

In this paper, we propose forward and backward stochastic differential equations (FBSDEs) based deep neural network (DNN) learning algorithms for the solution of high dimensional quasilinear parabolic partial differential equations (PDEs), which are related to the FBSDEs by the Pardoux-Peng theory. The algorithms rely on a learning process by minimizing the pathwise difference between two discrete stochastic processes, defined by the time discretization of the FBSDEs and the DNN representation of the PDE solutions, respectively. The proposed algorithms are shown to generate DNN solutions for a 100-dimensional Black--Scholes--Barenblatt equation, accurate in a finite region in the solution space, and has a convergence rate similar to that of the Euler--Maruyama discretization used for the FBSDEs. As a result, a Richardson extrapolation technique over time discretizations can be used to enhance the accuracy of the DNN solutions. For time oscillatory solutions, a multiscale DNN is shown to improve the performance of the FBSDE DNN for high frequencies.

Motivation & Objective

• To address the curse of dimensionality in solving high-dimensional quasilinear parabolic PDEs using machine learning.
• To develop mathematically consistent DNN training schemes based on forward-backward stochastic differential equations (FBSDEs).
• To improve convergence and accuracy of DNN solutions for high-dimensional PDEs, especially for oscillatory or complex solution structures.
• To enable error reduction through Richardson extrapolation and multiscale DNN architectures for time-oscillatory problems.

Proposed method

• The method formulates the PDE solution as a backward stochastic process linked to a forward SDE via the Pardoux–Peng theory.
• It trains a deep neural network to approximate the solution by minimizing the pathwise difference between the DNN output and the time-discretized FBSDE process.
• The loss function combines the terminal condition of the PDE and the discrepancy between the stochastic processes derived from FBSDE discretization and the DNN output.
• The algorithm uses the Euler–Maruyama scheme for time-discretizing the FBSDEs and trains the DNN via stochastic gradient descent on sampled paths.
• For oscillatory solutions, a multiscale DNN (MscaleDNN) is employed, where inputs are scaled across sub-networks to capture varying temporal frequencies.
• Richardson extrapolation is applied across different time discretization levels to enhance the convergence rate and accuracy of the DNN solution.

Experimental results

Research questions

• RQ1 Can FBSDE-based DNN algorithms achieve convergence rates comparable to the underlying Euler–Maruyama discretization for high-dimensional quasilinear PDEs?
• RQ2 How can the pathwise difference between FBSDE discretization and DNN approximation be effectively minimized to ensure numerical consistency?
• RQ3 Can Richardson extrapolation be successfully applied to DNN solutions of PDEs to improve accuracy, given the observed convergence order?
• RQ4 Does a multiscale DNN architecture significantly improve the approximation of time-oscillatory solutions in high-dimensional PDEs?

Key findings

• The proposed FBSDE-based DNN algorithms achieve a convergence rate nearly matching the half-order strong convergence of the Euler–Maruyama scheme used in FBSDE discretization.
• For the 100-dimensional Black–Scholes–Barenblatt equation, the method produces accurate solutions within a finite region of the solution space.
• Richardson extrapolation reduces the error by approximately half when applied across multiple time discretization levels, confirming the convergence order of the DNN solution.
• The multiscale DNN (MscaleDNN) reduces the overall error by 50% compared to a fully connected DNN for oscillatory time-dependent solutions with parameters α=0.025, β=0.25, γ=32.
• The MscaleDNN enables more accurate prediction of sample paths for oscillatory PDEs, as demonstrated by improved path tracking in numerical experiments.