[论文解读] Variational Bayes Deep Operator Network: A data-driven Bayesian solver for parametric differential equations

Shailesh Garg, Souvik Chakraborty|arXiv (Cornell University)|Jun 12, 2022

Model Reduction and Neural Networks被引用 8

一句话总结

本文提出变分贝叶斯深度算子网络（VB-DeepONet），一种贝叶斯深度算子网络，通过将确定性权重替换为变分后验分布，量化参数微分方程预测中的不确定性。利用变分推断，VB-DeepONet降低了过拟合并提升了泛化能力，在四个基准问题中的三个表现优于确定性DeepONet，同时提供预测置信区间，尽管计算成本略有增加。

ABSTRACT

Neural network based data-driven operator learning schemes have shown tremendous potential in computational mechanics. DeepONet is one such neural network architecture which has gained widespread appreciation owing to its excellent prediction capabilities. Having said that, being set in a deterministic framework exposes DeepONet architecture to the risk of overfitting, poor generalization and in its unaltered form, it is incapable of quantifying the uncertainties associated with its predictions. We propose in this paper, a Variational Bayes DeepONet (VB-DeepONet) for operator learning, which can alleviate these limitations of DeepONet architecture to a great extent and give user additional information regarding the associated uncertainty at the prediction stage. The key idea behind neural networks set in Bayesian framework is that, the weights and bias of the neural network are treated as probability distributions instead of point estimates and, Bayesian inference is used to update their prior distribution. Now, to manage the computational cost associated with approximating the posterior distribution, the proposed VB-DeepONet uses extit{variational inference}. Unlike Markov Chain Monte Carlo schemes, variational inference has the capacity to take into account high dimensional posterior distributions while keeping the associated computational cost low. Different examples covering mechanics problems like diffusion reaction, gravity pendulum, advection diffusion have been shown to illustrate the performance of the proposed VB-DeepONet and comparisons have also been drawn against DeepONet set in deterministic framework.

研究动机与目标

• 为解决确定性DeepONet在求解参数化微分方程时存在的过拟合和缺乏不确定性量化等局限性。
• 开发一种贝叶斯深度算子网络，实现在保持计算效率的同时实现不确定性感知的预测。
• 通过变分推断，将网络权重的点估计替换为概率分布，从而实现鲁棒的泛化和不确定性量化。
• 在涉及常微分方程和偏微分方程的多样化力学问题上评估所提框架。
• 在多个基准问题上对比VB-DeepONet与确定性DeepONet在性能和不确定性校准方面的表现。

提出的方法

• 将神经网络权重和偏置视为概率分布而非点估计，以支持贝叶斯推断。
• 使用变分推断近似真实后验分布，通过最小化变分后验与真实后验之间的KL散度。
• 为网络参数采用标准正态先验和正态变分分布，以支持可扩展的贝叶斯深度学习。
• 结合分支网络（处理输入函数）和主干网络（处理时空坐标），实现函数到函数的映射，与DeepONet一致。
• 使用最小化负变分下界（ELBO）的损失函数进行模型训练，该损失函数同时包含似然项和正则化项。
• 通过从变分后验中进行蒙特卡洛采样，实现对均值输出和预测不确定性的预测。

实验结果

研究问题

• RQ1 DeepONet的贝叶斯形式是否能提升算子学习中参数化微分方程的泛化能力并减少过拟合？
• RQ2 VB-DeepONet的预测不确定性在不同物理系统中与真实值相比如何？
• RQ3 变分推断是否为高维参数空间下贝叶斯DeepONet训练提供了计算上可行的替代MCMC的方法？
• RQ4 在哪些场景下，VB-DeepONet在预测精度和不确定性校准方面优于确定性DeepONet？
• RQ5 VB-DeepONet生成的置信区间在多大程度上可靠，尤其是在与基于MCMC的贝叶斯方法比较时？

主要发现

• 在四个测试案例中的三个，VB-DeepONet的归一化均方误差（NMSE）低于确定性DeepONet：反导数（0.009% vs. 0.016%）、重力摆（0.025% vs. 0.033%）和扩散反应（0.610% vs. 0.458%）。
• 在对流-扩散问题中，VB-DeepONet的NMSE更高（0.355% vs. 0.646%），但仍提供了有意义的不确定性量化。
• 在四个示例中的三个，真实解落在95%预测置信区间内，表明不确定性校准合理。
• 在对流-扩散案例中，真实解偶尔会落在置信区间之外，表明变分推断可能导致过度自信的不确定性估计。
• VB-DeepONet在所有测试案例中成功提供了预测置信区间，为决策者提供了超越点估计的额外洞察。
• 尽管计算成本更高，VB-DeepONet仍表现出更强的鲁棒性和泛化能力，特别是在数据稀缺或噪声较大的场景下，这得益于贝叶斯正则化。