[Paper Review] Efficient nonlinear manifold reduced order model

Youngkyu Kim, Youngsoo Choi+2 more|arXiv (Cornell University)|Nov 13, 2020

Model Reduction and Neural Networks55 references32 citations

TL;DR

The paper develops a nonlinear manifold ROM (NM-ROM) using a shallow masked autoencoder and hyper-reduction to accelerate simulations, achieving up to 11.7x speed-up and improved accuracy for advection-dominated 2D Burgers’ equations.

ABSTRACT

Traditional linear subspace reduced order models (LS-ROMs) are able to accelerate physical simulations, in which the intrinsic solution space falls into a subspace with a small dimension, i.e., the solution space has a small Kolmogorov n-width. However, for physical phenomena not of this type, such as advection-dominated flow phenomena, a low-dimensional linear subspace poorly approximates the solution. To address cases such as these, we have developed an efficient nonlinear manifold ROM (NM-ROM), which can better approximate high-fidelity model solutions with a smaller latent space dimension than the LS-ROMs. Our method takes advantage of the existing numerical methods that are used to solve the corresponding full order models (FOMs). The efficiency is achieved by developing a hyper-reduction technique in the context of the NM-ROM. Numerical results show that neural networks can learn a more efficient latent space representation on advection-dominated data from 2D Burgers' equations with a high Reynolds number. A speed-up of up to 11.7 for 2D Burgers' equations is achieved with an appropriate treatment of the nonlinear terms through a hyper-reduction technique.

Motivation & Objective

• Motivate accelerating high-fidelity simulations for design optimization, control, and uncertainty quantification by overcoming LS-ROM limitations.
• Introduce a nonlinear manifold representation to better capture advection-dominated or sharp-gradient solutions.
• Leverage existing full-order solver techniques and develop a hyper-reduction to enable speed-ups.

Proposed method

• Represent the solution on a nonlinear manifold x ≈ x_ref + g(x̂) with g as the decoder of a shallow autoencoder.
• Train the autoencoder on full-order model (FOM) solution data to learn a latent space of dimension ns << Ns.
• Solve the reduced system at each time step via least-squares Petrov–Galerkin (LSPG) projection in the latent space.
• Apply Gauss–Newton to minimize the reduced residual at each time step.
• Introduce hyper-reduction (gappy POD) to approximate the nonlinear residual with far fewer operations, via a residual basis Φr and sampling matrix Z.
• Implement a masked decoder to compute only needed decoder outputs for the selected residual components.

Experimental results

Research questions

• RQ1 Can a nonlinear manifold ROM with a shallow masked autoencoder achieve comparable accuracy to full-order models for advection-dominated PDEs?
• RQ2 Does hyper-reduction enable significant speed-ups in NM-ROM without sacrificing accuracy?
• RQ3 How does NM-ROM compare to traditional LS-ROM and black-box neural networks on 2D Burgers’ equation at high Reynolds numbers?

Key findings

Residual basis n_r	Residual samples n_z	Max. rel. error (%)	Wall-clock time (sec)	Speed-up
55	58	0.93	12.15	11.58
56	59	0.94	12.35	11.39
51	54	0.95	12.09	11.63
53	56	0.97	12.14	11.58
54	57	0.97	12.29	11.44
44	47	0.98	12.01	11.71
59	59	34.38	4.86	26.76
53	58	37.73	5.05	28.02
53	59	37.84	4.86	28.95
53	56	37.95	5.05	27.83
53	55	37.96	4.75	29.61
53	53	37.97	7.18	19.58

• NM-LSPG-HR attains about 11x–12x speed-up over the FOM for 2D Burgers’ with ns = 5 and suitable residual sampling.
• NM-LSPG (nonlinear manifold) yields lower relative errors than LS-LSPG for the same ns, and even exceeds LS ROM projection error in some cases.
• LS-LSPG-HR shows large relative errors (~34–38%), while NM-LSPG-HR achieves around 1% maximum relative error across tested residual bases and samples.
• A well-chosen shallow masked decoder with hyper-reduction can outperform deep learning baselines (BB-NN) in accuracy while providing substantial speed-ups.
• NM-LSPG-HR maintains good agreement with the full-order solution and exhibits a trust region beyond which accuracy degrades gradually.