[Paper Review] A fast and accurate physics-informed neural network reduced order model with shallow masked autoencoder

Youngkyu Kim, Youngsoo Choi+2 more|arXiv (Cornell University)|Sep 25, 2020

Model Reduction and Neural Networks85 references42 citations

TL;DR

The paper introduces NM-ROM, a nonlinear-manifold reduced order model using a shallow masked autoencoder to achieve speedups for advection-dominated problems, aided by hyper-reduction and physics-informed training.

ABSTRACT

Traditional linear subspace reduced order models (LS-ROMs) are able to accelerate physical simulations, in which the intrinsic solution space falls into a subspace with a small dimension, i.e., the solution space has a small Kolmogorov n-width. However, for physical phenomena not of this type, e.g., any advection-dominated flow phenomena, such as in traffic flow, atmospheric flows, and air flow over vehicles, a low-dimensional linear subspace poorly approximates the solution. To address cases such as these, we have developed a fast and accurate physics-informed neural network ROM, namely nonlinear manifold ROM (NM-ROM), which can better approximate high-fidelity model solutions with a smaller latent space dimension than the LS-ROMs. Our method takes advantage of the existing numerical methods that are used to solve the corresponding full order models. The efficiency is achieved by developing a hyper-reduction technique in the context of the NM-ROM. Numerical results show that neural networks can learn a more efficient latent space representation on advection-dominated data from 1D and 2D Burgers' equations. A speedup of up to 2.6 for 1D Burgers' and a speedup of 11.7 for 2D Burgers' equations are achieved with an appropriate treatment of the nonlinear terms through a hyper-reduction technique. Finally, a posteriori error bounds for the NM-ROMs are derived that take account of the hyper-reduced operators.

Motivation & Objective

• Address the limitation of linear subspace ROMs for advection-dominated or sharp-gradient problems.
• Develop a nonlinear manifold ROM (NM-ROM) that uses a shallow masked autoencoder for solution representation.
• Leverage hyper-reduction to accelerate NM-ROM computations while maintaining accuracy.
• Derive a posteriori error bounds that account for hyper-reduced operators.
• Demonstrate performance gains on 1D and 2D Burgers’ equations.

Proposed method

• Represent the solution on a nonlinear manifold via a decoder g mapping a low-dimensional latent space to the full space.
• Use a shallow masked autoencoder to learn the nonlinear manifold representation from FOM data.
• Apply NM-Galerkin and NM-LSPG projections to obtain reduced-order evolution equations.
• Incorporate a hyper-reduction technique to efficiently evaluate nonlinear terms during time stepping.
• Train the autoencoder with solution snapshots and normalize data to stabilize learning.
• Derive a posteriori error bounds that incorporate the effects of hyper-reduced operators.

Experimental results

Research questions

• RQ1 Can NM-ROM achieve accurate approximations with a smaller latent dimension than LS-ROMs for advection-dominated problems?
• RQ2 How does hyper-reduction affect the speed and accuracy of NM-ROMs in both 1D and 2D settings?
• RQ3 What are the theoretical error bounds when using hyper-reduced NM-ROMs?
• RQ4 How does a shallow masked autoencoder compare to deeper architectures in the context of NM-ROM efficiency?
• RQ5 What is the role of the nonlinear manifold representation in preserving conservation laws and physical structure?

Key findings

• NM-ROM can learn a more efficient latent space representation on advection-dominated data (1D and 2D Burgers’ equations).
• A speedup of up to 2.6 for 1D Burgers’ and 11.7 for 2D Burgers’ are achieved with appropriate nonlinear-term hyper-reduction.
• A posteriori error bounds for NM-ROMs are derived that take account of hyper-reduced operators.
• The decoder is a shallow masked network, enabling efficient Jacobian computations and integration with hyper-reduction.
• The method leverages existing numerical methods for full-order models to constrain the surrogate and improve fidelity.