[Paper Review] Stochastic Physics-Informed Neural Networks (SPINN): A Moment-Matching Framework for Learning Hidden Physics within Stochastic Differential Equations.

Jared O’Leary, Joel A. Paulson+1 more|arXiv (Cornell University)|Sep 3, 2021

Model Reduction and Neural Networks61 references6 citations

TL;DR

SPINN is a novel deep learning framework that learns hidden physics in stochastic differential equations (SDEs) by matching predicted statistical moments—computed via uncertainty propagation—against data-driven estimates. It leverages automatic differentiation and mini-batch optimization to train neural networks that represent constitutive laws, enabling robust discovery of hidden dynamics in multivariate SDEs with multiplicative noise.

ABSTRACT

Stochastic differential equations (SDEs) are used to describe a wide variety of complex stochastic dynamical systems. Learning the hidden physics within SDEs is crucial for unraveling fundamental understanding of the stochastic and nonlinear behavior of these systems. We propose a flexible and scalable framework for training deep neural networks to learn constitutive equations that represent hidden physics within SDEs. The proposed stochastic physics-informed neural network framework (SPINN) relies on uncertainty propagation and moment-matching techniques along with state-of-the-art deep learning strategies. SPINN first propagates stochasticity through the known structure of the SDE (i.e., the known physics) to predict the time evolution of statistical moments of the stochastic states. SPINN learns (deep) neural network representations of the hidden physics by matching the predicted moments to those estimated from data. Recent advances in automatic differentiation and mini-batch gradient descent are leveraged to establish the unknown parameters of the neural networks. We demonstrate SPINN on three benchmark in-silico case studies and analyze the framework's robustness and numerical stability. SPINN provides a promising new direction for systematically unraveling the hidden physics of multivariate stochastic dynamical systems with multiplicative noise.

Motivation & Objective

• To develop a scalable and flexible framework for discovering hidden constitutive laws within stochastic differential equations (SDEs) that govern complex dynamical systems.
• To address the challenge of learning hidden physics in systems with multiplicative noise and nonlinear stochastic dynamics.
• To enable data-driven discovery of underlying physical laws by matching statistical moments predicted by SDEs to empirical data moments.
• To integrate uncertainty propagation and moment-matching techniques with modern deep learning optimization for improved robustness and stability.

Proposed method

• SPINN propagates stochasticity through the known structure of the SDE to analytically or numerically compute the time evolution of statistical moments of the system states.
• It employs deep neural networks to represent unknown constitutive equations that govern the drift and diffusion coefficients in the SDE.
• The framework minimizes the discrepancy between predicted moments (from SDE propagation) and empirically estimated moments from observed data using a loss function based on moment-matching.
• Automatic differentiation is used to backpropagate gradients through the moment computation and neural network parameters for end-to-end training.
• Mini-batch gradient descent is applied to scale the training process and improve convergence efficiency.
• The method is designed to handle multivariate systems and systems with multiplicative noise, ensuring numerical stability and robustness.

Experimental results

Research questions

• RQ1 How can we systematically learn hidden physics in stochastic differential equations when the underlying constitutive laws are unknown but the SDE structure is partially known?
• RQ2 Can moment-matching between predicted and data-estimated statistical moments enable accurate and stable discovery of hidden dynamics in nonlinear, multivariate SDEs?
• RQ3 How does SPINN perform in terms of robustness and numerical stability when applied to SDEs with multiplicative noise and complex nonlinearities?
• RQ4 To what extent can uncertainty propagation through known SDE structures improve the generalization and interpretability of learned neural network models?

Key findings

• SPINN successfully learns hidden physics in three in-silico benchmark case studies, demonstrating accurate recovery of underlying constitutive laws in SDEs with multiplicative noise.
• The framework achieves robust performance across diverse stochastic dynamical systems, maintaining numerical stability even under high noise levels.
• Moment-matching between predicted and data-estimated moments enables effective training of deep neural networks without requiring explicit likelihood estimation.
• Leveraging automatic differentiation and mini-batch optimization allows SPINN to scale efficiently to high-dimensional systems and complex nonlinearities.
• The integration of uncertainty propagation with neural network learning improves the interpretability and physical consistency of the discovered models.
• SPINN provides a systematic and scalable approach for uncovering hidden physics in complex stochastic systems, offering a promising alternative to traditional black-box learning.