[Paper Review] VENI, VINDy, VICI: a generative reduced-order modeling framework with uncertainty quantification

Paolo Conti, Jonas Kneifl+5 more|arXiv (Cornell University)|May 31, 2024

Model Reduction and Neural Networks5 citations

TL;DR

A data-driven, non-intrusive ROM framework combining variational encoding (VENI), variational SINDY (VINDy), and variational inference with certainty intervals (VICI) to reconstruct noise-free high-dimensional solutions and quantify uncertainty. Demonstrated on the Rössler system and high-dimensional PDE benchmarks.

ABSTRACT

The simulation of many complex phenomena in engineering and science requires solving expensive, high-dimensional systems of partial differential equations (PDEs). To circumvent this, reduced-order models (ROMs) have been developed to speed up computations. However, when governing equations are unknown or partially known, typically ROMs lack interpretability and reliability of the predicted solutions. In this work we present a data-driven, non-intrusive framework for building ROMs where the latent variables and dynamics are identified in an interpretable manner and uncertainty is quantified. Starting from a limited amount of high-dimensional, noisy data the proposed framework constructs an efficient ROM by leveraging variational autoencoders for dimensionality reduction along with a newly introduced, variational version of sparse identification of nonlinear dynamics (SINDy), which we refer to as Variational Identification of Nonlinear Dynamics (VINDy). In detail, the method consists of Variational Encoding of Noisy Inputs (VENI) to identify the distribution of reduced coordinates. Simultaneously, we learn the distribution of the coefficients of a pre-determined set of candidate functions by VINDy. Once trained offline, the identified model can be queried for new parameter instances and new initial conditions to compute the corresponding full-time solutions. The probabilistic setup enables uncertainty quantification as the online testing consists of Variational Inference naturally providing Certainty Intervals (VICI). In this work we showcase the effectiveness of the newly proposed VINDy method in identifying interpretable and accurate dynamical system for the Roessler system with different noise intensities and sources. Then the performance of the overall method - named VENI, VINDy, VICI - is tested on PDE benchmarks including structural mechanics and fluid dynamics.

Motivation & Objective

• Address the challenge of building interpretable and reliable ROMs from limited noisy data.
• Integrate dimensionality reduction, system identification, and uncertainty quantification in a unified variational framework.
• Provide offline training to learn latent dynamics and online generation with predictive uncertainty.
• Demonstrate effectiveness on a low-dimensional chaotic system and high-dimensional PDE benchmarks.

Proposed method

• Use VENI to map noisy high-dimensional data to a low-dimensional latent distribution via a variational encoder and Gaussian decoder.
• Apply VINDy to learn a probabilistic, sparse dynamical model of latent coordinates as a linear combination of candidate functions with uncertain coefficients.
• Train offline by jointly optimizing reconstruction, latent dynamics, and coefficient priors within a variational objective.
• Online, use VICI to generate full-field solutions and uncertainty intervals for new parameters/initial conditions.
• Assume Gaussian or Laplacian priors for tractable KL terms and use reparameterization for gradient-based optimization.

Experimental results

Research questions

• RQ1 Can the VENI-VINDy-VICI pipeline recover interpretable latent dynamics from noisy, high-dimensional data?
• RQ2 Does the framework provide accurate noise-robust reconstructions and reliable uncertainty quantification (certainty intervals) for new parameter instances?
• RQ3 How does the method perform on low-dimensional chaotic systems and high-dimensional PDE benchmarks like MEMS beams and unsteady reaction-diffusion problems?
• RQ4 What is the role of offline joint training in stabilizing latent representations and dynamic discovery?

Key findings

• The VINDy component identifies interpretable latent dynamics under varying noise conditions on the Rössler system.
• The integrated VENI–VINDy–VICI framework yields accurate full-field PDE solutions and provides uncertainty via Certainty Intervals during online prediction.
• The method is validated on high-dimensional PDE benchmarks including a MEMS beam resonator and a parametrized reaction-diffusion problem.
• Public source code for VENI, VINDy, VICI is provided in a public repository.
• The offline training jointly optimizes reconstruction, latent dynamics, and coefficient priors to produce a generative ROM with UQ capabilities.