[Paper Review] Forward-Backward Stochastic Neural Networks: Deep Learning of High-dimensional Partial Differential Equations

Maziar Raissi|arXiv (Cornell University)|Apr 19, 2018

Model Reduction and Neural NetworksPhysics and Astronomy45 citations

TL;DR

The paper proposes Forward-Backward Stochastic Neural Networks (FBSNNs) that approximate the solution to high-dimensional PDEs by representing the solution u(t,x) with a neural network and training via discretized forward-backward SDEs, enabling evaluation of u(t,x) anywhere in space-time.

ABSTRACT

Classical numerical methods for solving partial differential equations suffer from the curse dimensionality mainly due to their reliance on meticulously generated spatio-temporal grids. Inspired by modern deep learning based techniques for solving forward and inverse problems associated with partial differential equations, we circumvent the tyranny of numerical discretization by devising an algorithm that is scalable to high-dimensions. In particular, we approximate the unknown solution by a deep neural network which essentially enables us to benefit from the merits of automatic differentiation. To train the aforementioned neural network we leverage the well-known connection between high-dimensional partial differential equations and forward-backward stochastic differential equations. In fact, independent realizations of a standard Brownian motion will act as training data. We test the effectiveness of our approach for a couple of benchmark problems spanning a number of scientific domains including Black-Scholes-Barenblatt and Hamilton-Jacobi-Bellman equations, both in 100-dimensions.

Motivation & Objective

• Motivate and address the curse of dimensionality in solving high-dimensional PDEs using neural networks.
• Learn the unknown solution u(t,x) as a neural network and obtain its gradients Du(t,x) via automatic differentiation.
• Leverage the connection between forward-backward SDEs and parabolic PDEs to train the network with Euler–Maruyama discretization.
• Achieve a unified framework that provides the full solution surface u(t,x), not just initial-time values.
• Demonstrate applicability to canonical high-dimensional problems such as Black-Scholes-Barenblatt and Hamilton-Jacobi-Bellman equations in 100D.

Proposed method

• Approximate the unknown solution u(t,x) by a deep neural network and compute Du(t,x) via automatic differentiation.
• Relate forward-backward SDEs to parabolic PDEs so that Yt=u(t,Xt) and Zt=Du(t,Xt) hold, enabling a PDE solution via SDE data.
• Discretize the FBSDE system with Euler-Maruyama and define a loss that enforces the dynamics and terminal condition across multiple Brownian realizations.
• Train the network with Adam optimization to minimize the loss, thereby learning the global solution surface u(t,x).
• Use a single neural network for u(t,x) (and thus sharing Du(t,x) parameters) to avoid parameter blow-up across time discretization.

Experimental results

Research questions

• RQ1 Can a neural network representing the PDE solution u(t,x) be trained to accurately approximate high-dimensional parabolic PDEs via forward-backward SDEs?
• RQ2 Does the approach yield the full space-time solution u(t,x) in a single training round, not just the initial value u(0,ξ)?
• RQ3 How does the method perform on 100-dimensional benchmark problems like Black-Scholes-Barenblatt and Hamilton-Jacobi-Bellman equations?
• RQ4 What are the accuracy and computational efficiency characteristics compared with prior methods that only approximate initial-time values?
• RQ5 Can automatic differentiation reliably provide the necessary gradients Du(t,x) for high-dimensional PDEs?

Key findings

• The FBSNN framework can approximate the entire solution function u(t,x) in 100D problems in a single training round.
• The method achieves competitive accuracy and speed relative to prior mesh-based and neural approaches, with explicit results demonstrated on 100D problems.
• Automatic differentiation enables accurate gradient computation Du(t,x) without finite differences, leveraging a shared neural network for u and Du.
• Training uses multiple Brownian motion realizations (e.g., M=100) to drive the loss and enforce the forward-backward dynamics.
• The approach yields observable trajectory estimates Yt=u(t,Xt) across representative high-dimensional paths, not just the initial value.