[Paper Review] Random Grid Neural Processes for Parametric Partial Differential Equations

Arnaud Vadeboncoeur, Ieva Kazlauskaite+4 more|arXiv (Cornell University)|Jan 26, 2023

Gaussian Processes and Bayesian Inference4 citations

TL;DR

This paper introduces Random Grid Neural Processes, a physics-informed variational inference framework that treats spatial collocation grids as random variables to enable uncertainty-aware, scalable learning for parametric PDEs. By combining spatial statistics, Gaussian processes, and a novel Grid-Invariant Convolutional Network (GICNet), the method improves predictive accuracy and uncertainty quantification in forward and inverse problems for nonlinear Poisson, Burgers, and Navier-Stokes equations, outperforming existing methods in computational efficiency and robustness to noisy, irregularly sampled data.

ABSTRACT

We introduce a new class of spatially stochastic physics and data informed deep latent models for parametric partial differential equations (PDEs) which operate through scalable variational neural processes. We achieve this by assigning probability measures to the spatial domain, which allows us to treat collocation grids probabilistically as random variables to be marginalised out. Adapting this spatial statistics view, we solve forward and inverse problems for parametric PDEs in a way that leads to the construction of Gaussian process models of solution fields. The implementation of these random grids poses a unique set of challenges for inverse physics informed deep learning frameworks and we propose a new architecture called Grid Invariant Convolutional Networks (GICNets) to overcome these challenges. We further show how to incorporate noisy data in a principled manner into our physics informed model to improve predictions for problems where data may be available but whose measurement location does not coincide with any fixed mesh or grid. The proposed method is tested on a nonlinear Poisson problem, Burgers equation, and Navier-Stokes equations, and we provide extensive numerical comparisons. We demonstrate significant computational advantages over current physics informed neural learning methods for parametric PDEs while improving the predictive capabilities and flexibility of these models.

Motivation & Objective

• To develop a probabilistic, scalable framework for solving forward and inverse problems in parametric PDEs that accounts for uncertainty in collocation grids.
• To overcome limitations of fixed-grid deep learning models by treating spatial grids as random variables to be marginalized via variational inference.
• To enable principled incorporation of noisy, irregularly sampled data from arbitrary measurement locations into physics-informed learning.
• To improve computational efficiency and predictive robustness in parametric PDE learning while maintaining strong uncertainty quantification.
• To extend deep learning for PDEs beyond deterministic, grid-locked architectures by integrating spatial statistics and Gaussian process modeling.

Proposed method

• The method assigns probability measures to the spatial domain, treating collocation grids as random variables to be marginalized out during training via variational inference.
• It employs a physics-driven variational inference framework that conditions on random domain partitions to learn joint mappings between physical parameters and solution fields.
• A new class of kernels is introduced to model Gaussian random fields that represent solution fields, enabling uncertainty-aware predictions.
• The Grid-Invariant Convolutional Network (GICNet) architecture is proposed to enable equivariant and robust feature learning across randomly sampled collocation grids.
• Noisy observations from arbitrary, non-fixed grids are incorporated using a probabilistic observation model that aligns with the random grid framework.
• The framework is trained end-to-end using a differentiable loss that combines physics-informed residuals with variational lower bounds on the likelihood of observed data.

Experimental results

Research questions

• RQ1 Can randomizing collocation grids in deep learning for PDEs improve uncertainty quantification and robustness to noisy or irregularly sampled data?
• RQ2 How can a variational inference framework be designed to treat spatial grids as random variables while preserving physical consistency in PDE solutions?
• RQ3 To what extent can a grid-invariant neural architecture (GICNet) improve generalization and computational efficiency in parametric PDE learning compared to standard deep learning models?
• RQ4 How does the proposed method compare in accuracy and uncertainty calibration to existing physics-informed models such as DeepONets, PINNs, and PDDLVM on complex PDEs?
• RQ5 Can the framework effectively handle ill-posed inverse problems by capturing parameter ambiguity through probabilistic modeling?

Key findings

• The proposed method achieves significant computational advantages over existing physics-informed deep learning methods for parametric PDEs, particularly in training and inference speed.
• The model demonstrates improved predictive performance and uncertainty quantification on the nonlinear Poisson problem, Burgers equation, and Navier-Stokes lid-driven cavity flow.
• Incorporating noisy, sparse observations from arbitrary grids leads to more robust predictions than deterministic baselines, especially when measurement locations do not align with fixed meshes.
• The uncertainty estimates reflect model confidence and help practitioners assess prediction reliability, which is critical in black-box deep learning applications.
• The framework successfully handles ill-posed inverse problems by capturing parameter ambiguity through probabilistic modeling, a capability absent in deterministic approaches.
• Empirical results show that the method is competitive with state-of-the-art models such as PDDLVM, modified DeepONets, and Fourier feature networks in accuracy and uncertainty calibration.