[Paper Review] Numerically Solving Parametric Families of High-Dimensional Kolmogorov Partial Differential Equations via Deep Learning

Julius Berner, Markus Dablander+1 more|arXiv (Cornell University)|Jan 1, 2020

Stochastic processes and financial applications17 citations

TL;DR

This paper proposes a deep learning method to solve parametric families of high-dimensional Kolmogorov PDEs—such as the heat and Black-Scholes equations—by reformulating the problem as a single statistical learning task using the Feynman-Kac formula. The approach trains a single deep neural network to approximate the solution across all parameters and space-time points, demonstrating robust performance without the curse of dimensionality in both theory and experiments.

ABSTRACT

We present a deep learning algorithm for the numerical solution of parametric families of high-dimensional linear Kolmogorov partial differential equations (PDEs). Our method is based on reformulating the numerical approximation of a whole family of Kolmogorov PDEs as a single statistical learning problem using the Feynman-Kac formula. Successful numerical experiments are presented, which empirically confirm the functionality and efficiency of our proposed algorithm in the case of heat equations and Black-Scholes option pricing models parametrized by affine-linear coefficient functions. We show that a single deep neural network trained on simulated data is capable of learning the solution functions of an entire family of PDEs on a full space-time region. Most notably, our numerical observations and theoretical results also demonstrate that the proposed method does not suffer from the curse of dimensionality, distinguishing it from almost all standard numerical methods for PDEs.

Motivation & Objective

• To address the challenge of numerically solving high-dimensional parametric families of Kolmogorov PDEs, which are intractable with standard methods due to the curse of dimensionality.
• To develop a scalable, data-driven approach that learns entire solution maps across parameter, space, and time domains in a single training procedure.
• To demonstrate that deep learning can avoid the exponential computational cost of classical PDE solvers in high dimensions, both empirically and theoretically.

Proposed method

• Reformulate the solution of a parametric family of Kolmogorov PDEs as a single statistical learning problem using the Feynman-Kac formula, linking PDE solutions to expectations of stochastic processes.
• Train a deep neural network to map parameters, spatial coordinates, and time to the solution value, effectively learning the parametric solution map $\bar{u}(\gamma, x, t) = u_\gamma(x, t)$ across the full domain.
• Use a multilevel architecture with residual connections and batch normalization to improve training stability and generalization performance.
• Employ stochastic gradient descent with adaptive learning rate scheduling and weight decay to optimize the network parameters on simulated data.
• Leverage Monte Carlo approximations for training data generation, ensuring accurate representation of the underlying stochastic processes.
• Validate the method on benchmark problems including the heat equation with paraboloid initial condition and the Black-Scholes model with affine-linear coefficients.

Experimental results

Research questions

• RQ1 Can a single deep neural network learn the solution map of an entire family of high-dimensional Kolmogorov PDEs across all parameters, space, and time?
• RQ2 Does the proposed deep learning method avoid the curse of dimensionality in both approximation and generalization error, as theoretically and empirically demonstrated?
• RQ3 How does the performance of the multilevel architecture with batch normalization compare to standard feed-forward networks in solving parametric PDEs?
• RQ4 To what extent can the method accurately predict not only the solution but also its parametric derivatives, such as Vega in option pricing?
• RQ5 Does the empirical risk minimization procedure used in training scale efficiently with increasing problem dimension?

Key findings

• The proposed deep learning method achieves an $\mathcal{L}^1$-error of approximately $0.0011$ on the Black-Scholes model with paraboloid initial condition, significantly outperforming feed-forward baselines.
• For the heat equation with paraboloid initial condition, the method maintains linear computational cost scaling in the log-log plot, indicating no curse of dimensionality up to dimension $d=15$.
• The multilevel architecture with batch normalization and residual connections reduced the average $\mathcal{L}^1$-error to $0.0012$ on the Black-Scholes model, compared to $0.0526$ for a standard feed-forward network without normalization.
• The method successfully predicts both the solution and its parametric derivative (Vega) with average relative errors below $10^{-2}$, as shown in Figures 6 and 8.
• Empirical results show that the training time per epoch scales sublinearly with dimension, and the model generalizes well across unseen parameter values within the compact set $D$.
• Theoretical analysis confirms that the method does not suffer from the curse of dimensionality in terms of network size and sample size, a key advantage over classical numerical PDE solvers.