[Paper Review] Physics-informed deep generative models

Yibo Yang, Paris Perdikaris|arXiv (Cornell University)|Dec 9, 2018

Model Reduction and Neural Networks22 references59 citations

TL;DR

The paper introduces a physics-informed adversarial variational framework to propagate uncertainty in PDE-governed systems by constraining deep generative models to satisfy underlying physical laws. It demonstrates Burgers equation as a canonical test case.

ABSTRACT

We consider the application of deep generative models in propagating uncertainty through complex physical systems. Specifically, we put forth an implicit variational inference formulation that constrains the generative model output to satisfy given physical laws expressed by partial differential equations. Such physics-informed constraints provide a regularization mechanism for effectively training deep probabilistic models for modeling physical systems in which the cost of data acquisition is high and training data-sets are typically small. This provides a scalable framework for characterizing uncertainty in the outputs of physical systems due to randomness in their inputs or noise in their observations. We demonstrate the effectiveness of our approach through a canonical example in transport dynamics.

Motivation & Objective

• Motivate robust uncertainty quantification for data-limited physical systems.
• Develop a probabilistic, physics-constrained generative model to incorporate PDE laws.
• Provide an adversarial inference framework to train and regularize deep generative models.
• Demonstrate the approach on a Burgers equation transport-dynamics example.

Proposed method

• Formulate a conditional latent variable model p(u|x,t)=∫p(u|x,t,z)p(z|x,t)dz with u_t+N_x u=0 as the physical constraint.
• Use a physics-informed neural network to encode PDE residual r_theta(x,t)=∂t f_theta(x,t)+N_x f_theta(x,t) and minimize data and residual losses.
• Train with a reverse KL objective by relating p_theta(x,t,u) to q(x,t,u) via an adversarial setup using a discriminator T_psi and an encoder q_phi(z|x,t,u).
• Derive a computable entropy lower bound h(p_theta(x,t,u))≥h(p(z))+E[...] [log q_phi(z|x,t,u)], enabling tractable training.
• Optimize a pair of coupled objectives: maximize the discriminator loss L_D(psi) and minimize the generator loss L_G(theta,phi) with PDE residual penalty β L_PDE(theta).
• Demonstrate stability via cycle-consistent latent encoding and entropic regularization controlled by λ.

Experimental results

Research questions

• RQ1 Can a physics-informed probabilistic generative model accurately propagate input noise through nonlinear PDE dynamics?
• RQ2 Does incorporating PDE residuals as a regularizer improve data efficiency and uncertainty quantification in small-data regimes?
• RQ3 How does an adversarial training framework influence posterior sampling and mode coverage in physics-constrained generative models?
• RQ4 What is the impact of PDE constraints on predictive uncertainty concentration in regions with developing shocks (e.g., Burgers equation)?

Key findings

• The proposed p_theta(u|x,t,z) captures uncertainty in spatio-temporal PDE solutions due to input noise propagation.
• Uncertainty concentrates near shock formation in Burgers equation, while providing non-Gaussian predictive statistics.
• Adversarial training with a PDE residual regularizer improves robustness and mitigates mode collapse under limited data.
• The Burgers equation experiment demonstrates effective propagation of initial-condition noise through nonlinear dynamics.
• Systematic studies show robustness to initialization and architecture, and highlight sensitivity to adversarial training hyperparameters.