[Paper Review] Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations

Liu Yang, Dongkun Zhang+1 more|arXiv (Cornell University)|Nov 5, 2018

Model Reduction and Neural Networks32 references59 citations

TL;DR

PI-GANs encode stochastic differential equations into GANs to solve forward, inverse, and mixed SDE problems with limited scattered data, leveraging WGAN-GP for stability. The framework handles high-dimensional stochastic problems without the curse of dimensionality and supports multi-group sensor data.

ABSTRACT

We developed a new class of physics-informed generative adversarial networks (PI-GANs) to solve in a unified manner forward, inverse and mixed stochastic problems based on a limited number of scattered measurements. Unlike standard GANs relying only on data for training, here we encoded into the architecture of GANs the governing physical laws in the form of stochastic differential equations (SDEs) using automatic differentiation. In particular, we applied Wasserstein GANs with gradient penalty (WGAN-GP) for its enhanced stability compared to vanilla GANs. We first tested WGAN-GP in approximating Gaussian processes of different correlation lengths based on data realizations collected from simultaneous reads at sparsely placed sensors. We obtained good approximation of the generated stochastic processes to the target ones even for a mismatch between the input noise dimensionality and the effective dimensionality of the target stochastic processes. We also studied the overfitting issue for both the discriminator and generator, and we found that overfitting occurs also in the generator in addition to the discriminator as previously reported. Subsequently, we considered the solution of elliptic SDEs requiring approximations of three stochastic processes, namely the solution, the forcing, and the diffusion coefficient. We used three generators for the PI-GANs, two of them were feed forward deep neural networks (DNNs) while the other one was the neural network induced by the SDE. Depending on the data, we employed one or multiple feed forward DNNs as the discriminators in PI-GANs. Here, we have demonstrated the accuracy and effectiveness of PI-GANs in solving SDEs for up to 30 dimensions, but in principle, PI-GANs could tackle very high dimensional problems given more sensor data with low-polynomial growth in computational cost.

Motivation & Objective

• Motivate a unified data-driven approach for forward, inverse, and mixed stochastic problems governed by SDEs.
• Integrate known physics into GAN architecture to learn stochastic terms from limited measurements.
• Develop a scalable method that can handle high-dimensional stochastic processes and multiple sensor groups.

Proposed method

• Use two independent feed-forward networks to model stochastic processes like k(x;ω) and u(x;ω).
• Encode the SDE and boundary operators via automatic differentiation to create induced neural networks for f(x;ω) and b(x;ω).
• Apply WGAN-GP with gradient penalty for stable adversarial training and to approximate target stochastic distributions.
• Adopt multiple data groups and discriminators to handle data from different sensor groups and enforce physics across groups.
• Train generators and discriminators iteratively using the Adagrad/Adam optimizer, with a data-driven adversarial loss.
• Extend the framework to forward, inverse, and mixed problems without changing the underlying PI-GAN structure.

Experimental results

Research questions

• RQ1 Can PI-GANs accurately learn stochastic processes and stochastic coefficients from limited, scattered sensor data?
• RQ2 How does encoding SDE physics within GANs affect stability and accuracy compared to data-only GANs?
• RQ3 Can PI-GANs solve forward, inverse, and mixed stochastic problems within a single framework?
• RQ4 What is the impact of multiple sensor groups on learning when data are misaligned or heterogeneous?

Key findings

• WGAN-GP enables stable learning and accurate distribution matching for stochastic processes across varying correlation lengths and sensor counts.
• PI-GANs can approximate stochastic processes up to 30 dimensions with appropriate sensor data, without suffering the curse of dimensionality.
• Overfitting occurs in both the discriminator and generator, including the generator, highlighting the need for careful training and data management.
• The method produces means and standard deviations of the stochastic solution and the diffusion coefficient in good agreement with benchmarks.
• Using multiple data groups with separate discriminators allows leveraging diverse data sources without alignment requirements.
• WGAN-GP outperforms vanilla GANs in cases where the target distribution concentrates on low-dimensional manifolds (e.g., fixed boundary cases).