[Paper Review] Approximate Bayesian Neural Operators: Uncertainty Quantification for Parametric PDEs

Emilia Magnani, Nicholas Krämer+3 more|arXiv (Cornell University)|Aug 2, 2022

Model Reduction and Neural Networks4 citations

TL;DR

This paper introduces Approximate Bayesian Neural Operators, a method that provides uncertainty quantification for deep learning models solving parametric partial differential equations (PDEs). By combining Gaussian process theory for the linear case and Laplace approximation for the deep case, the approach enables structured, localized uncertainty estimates that identify regions of high prediction error, even in high-data and low-data regimes.

ABSTRACT

Neural operators are a type of deep architecture that learns to solve (i.e. learns the nonlinear solution operator of) partial differential equations (PDEs). The current state of the art for these models does not provide explicit uncertainty quantification. This is arguably even more of a problem for this kind of tasks than elsewhere in machine learning, because the dynamical systems typically described by PDEs often exhibit subtle, multiscale structure that makes errors hard to spot by humans. In this work, we first provide a mathematically detailed Bayesian formulation of the ''shallow'' (linear) version of neural operators in the formalism of Gaussian processes. We then extend this analytic treatment to general deep neural operators using approximate methods from Bayesian deep learning. We extend previous results on neural operators by providing them with uncertainty quantification. As a result, our approach is able to identify cases, and provide structured uncertainty estimates, where the neural operator fails to predict well.

Motivation & Objective

• Address the lack of uncertainty quantification in neural operators, which is critical for safety-critical and scientific applications of PDE solvers.
• Develop a theoretically grounded Bayesian framework for neural operators to assess predictive confidence and detect failure modes.
• Extend analytic Bayesian treatment from the linear (shallow) case to deep neural operators using approximate inference.
• Enable practical uncertainty estimation with low computational overhead via post-hoc Laplace approximation on pretrained models.
• Demonstrate that uncertainty estimates can detect both global and localized prediction errors, especially in regions with model artifacts or poor generalization.

Proposed method

• Formulate the solution operator of linear parametric PDEs as a Gaussian process (GP) using Green’s functions and kernel methods.
• Use the GP framework to provide exact Bayesian inference for the shallow (single-layer) neural operator case.
• Extend the Bayesian treatment to deep neural operators via Laplace approximation, enabling efficient posterior approximation post-training.
• Apply the Laplace approximation to estimate the posterior covariance of network weights, yielding predictive uncertainty over function outputs.
• Use the predictive variance from the Laplace approximation as a structured uncertainty measure across the input domain.
• Train neural operators on function pairs (f, u) to learn the solution operator G: f ↦ u, then apply Bayesian inference to quantify uncertainty in predictions.

Experimental results

Research questions

• RQ1 Can a Bayesian formulation be rigorously derived for the linear (shallow) version of neural operators using Gaussian processes?
• RQ2 How can uncertainty quantification be effectively extended to deep neural operators where analytic solutions are intractable?
• RQ3 Can approximate Bayesian inference via Laplace approximation detect localized prediction errors in PDE solutions, especially in regions with model artifacts?
• RQ4 Does the proposed uncertainty estimate remain well-calibrated and informative in both low-data and high-data regimes?
• RQ5 Can the uncertainty measure distinguish between regions of high error due to model limitations (e.g., ReLU-based piecewise linear regions) and those due to data sparsity?

Key findings

• The linear case of neural operators admits an exact Bayesian treatment via Gaussian processes, enabling analytic uncertainty quantification.
• The Laplace approximation provides a computationally efficient way to add uncertainty estimates to pretrained deep neural operators with minimal overhead.
• In the low-data regime, the method produces structured uncertainty across the domain, identifying regions where predictions are less reliable.
• In the high-data regime, the uncertainty estimate successfully localizes and delineates a sharp artifact region in the Darcy flow problem where the model fails to generalize well.
• The predictive standard deviation correlates strongly with actual prediction error, demonstrating well-calibrated uncertainty estimates.
• The method enables detection of model-specific failure modes—such as those induced by ReLU activation functions—thereby improving trust in predictions for scientific applications.