[论文解读] Deep neural network approximations for Monte Carlo algorithms

Philipp Grohs, Arnulf Jentzen+1 more|arXiv (Cornell University)|Aug 28, 2019

Model Reduction and Neural Networks参考文献 55被引用 27

一句话总结

本文建立了一般性框架，表明若某函数可通过无维数灾难的蒙特卡洛方案进行近似，则深度神经网络（DNN）同样可无维数灾难地近似该函数。关键结果提供了DNN近似高维PDE解（如科尔莫戈罗夫PDE）所需参数数的显式多项式界，且具有误差控制。

ABSTRACT

Recently, it has been proposed in the literature to employ deep neural networks (DNNs) together with stochastic gradient descent methods to approximate solutions of PDEs. There are also a few results in the literature which prove that DNNs can approximate solutions of certain PDEs without the curse of dimensionality in the sense that the number of real parameters used to describe the DNN grows at most polynomially both in the PDE dimension and the reciprocal of the prescribed approximation accuracy. One key argument in most of these results is, first, to use a Monte Carlo approximation scheme which can approximate the solution of the PDE under consideration at a fixed space-time point without the curse of dimensionality and, thereafter, to prove that DNNs are flexible enough to mimic the behaviour of the used approximation scheme. Having this in mind, one could aim for a general abstract result which shows under suitable assumptions that if a certain function can be approximated by any kind of (Monte Carlo) approximation scheme without the curse of dimensionality, then this function can also be approximated with DNNs without the curse of dimensionality. It is a key contribution of this article to make a first step towards this direction. In particular, the main result of this paper, essentially, shows that if a function can be approximated by means of some suitable discrete approximation scheme without the curse of dimensionality and if there exist DNNs which satisfy certain regularity properties and which approximate this discrete approximation scheme without the curse of dimensionality, then the function itself can also be approximated with DNNs without the curse of dimensionality. As an application of this result we establish that solutions of suitable Kolmogorov PDEs can be approximated with DNNs without the curse of dimensionality.

研究动机与目标

• 建立一个将蒙特卡洛近似方案与DNN近似无维数灾难联系起来的一般理论框架。
• 证明若某函数可被离散方案无维数灾难地近似，则其亦可被DNN无维数灾难地近似。
• 推导出近似科尔莫戈罗夫PDE解所需DNN参数数的显式上界，以实现指定精度。
• 提供DNN复杂度对维数与精度依赖关系的定量估计，确保多项式量级的缩放。

提出的方法

• 作者提出一个一般性近似结果，将离散蒙特卡洛方案与DNN近似在正则性与近似条件下的关系联系起来。
• 他们定义了一类使用修正线性单元（ReLU）的DNN，并分析其实现函数与复合规则。
• 该方法依赖于证明DNN可以以受控误差与参数数量模仿给定蒙特卡洛近似方案的行为。
• 关键技术工具包括高维中随机变量的$L^p$-范数与协方差矩阵的迹范数的界。
• 框架基于对目标函数及其近似方案的光滑性与增长性的假设。
• 通过验证蒙特卡洛欧拉方法满足DNN近似的必要条件，将理论结果应用于科尔莫戈罗夫PDE。

实验结果

研究问题

• RQ1 若同一函数的蒙特卡洛方案可无维数灾难地近似，则DNN是否也能无维数灾难地近似该函数？
• RQ2 为在给定误差容限内近似函数，DNN所需参数数的显式界是什么？
• RQ3 在何种函数及其近似方案条件下，DNN可继承该方案的维数无关收敛性质？
• RQ4 DNN的参数数如何随维数$d$与精度$\varepsilon$变化？
• RQ5 该一般性框架是否可应用于通过多项式参数增长的DNN近似科尔莫戈罗夫PDE解？

主要发现

• 若某函数可被无维数灾难的离散方案近似，且DNN可受控参数数量近似该方案，则该函数本身亦可被DNN无维数灾难地近似。
• DNN的实际参数数在维数$d$与逆精度$\varepsilon^{-1}$上至多以多项式方式增长，且提供了显式的指数。
• 对于科尔莫戈罗夫PDE的解，DNN近似误差可在概率测度下的$L^p$-范数中被控制在$\varepsilon$以内，且参数数满足$\mathcal{P}(\Psi_{d,\varepsilon}) \leq c\,d^c\varepsilon^{-c}$，其中$c>0$为某常数。
• 该框架适用于初始函数与漂移函数多项式增长的PDE，在近似DNN的适当矩与正则性条件下成立。
• 结果对概率测度$\nu_d$的弱假设具有鲁棒性，如矩与迹范数的统一有界性。
• 分析证实DNN可实现高维PDE解的误差控制与多项式复杂度近似，避免了指数级增长。