[Paper Review] POD-DL-ROM: Enhancing deep learning-based reduced order models for nonlinear parametrized PDEs by proper orthogonal decomposition

Stefania Fresca, Andrea Manzoni|arXiv (Cornell University)|Jan 1, 2022

Model Reduction and Neural Networks66 references241 citations

TL;DR

This paper proposes POD-DL-ROM, a hybrid method that combines proper orthogonal decomposition (POD) with deep learning-based reduced order models (DL-ROMs) to drastically accelerate the offline training phase of DL-ROMs for nonlinear parametrized PDEs. By applying randomized POD for dimensionality reduction and multi-fidelity pretraining, the method reduces network complexity and training time while preserving high accuracy and online efficiency, achieving speed-ups of up to 6.59×10⁵ over full-order models.

ABSTRACT

Deep learning-based reduced order models (DL-ROMs) have been recently proposed to overcome common limitations shared by conventional reduced order models (ROMs) – built, e.g., through proper orthogonal decomposition (POD) – when applied to nonlinear time-dependent parametrized partial differential equations (PDEs). These might be related to (i) the need to deal with projections onto high dimensional linear approximating trial manifolds, (ii) expensive hyper-reduction strategies, or (iii) the intrinsic difficulty to handle physical complexity with a linear superimposition of modes. All these aspects are avoided when employing DL-ROMs, which learn in a non-intrusive way both the nonlinear trial manifold and the reduced dynamics, by relying on deep (e.g., feedforward, convolutional, autoencoder) neural networks. Although extremely efficient at testing time, when evaluating the PDE solution for any new testing-parameter instance, DL-ROMs require an expensive training stage, because of the extremely large number of network parameters to be estimated. In this paper we propose a possible way to avoid an expensive training stage of DL-ROMs, by (i) performing a prior dimensionality reduction through POD, and (ii) relying on a multi-fidelity pretraining stage, where different physical models can be efficiently combined. The proposed POD-DL-ROM is tested on several (both scalar and vector, linear and nonlinear) time-dependent parametrized PDEs (such as, e.g., linear advection–diffusion–reaction, nonlinear diffusion–reaction, nonlinear elastodynamics, and Navier–Stokes equations) to show the generality of this approach and its remarkable computational savings.

Motivation & Objective

• Address the high computational cost of training deep learning-based reduced order models (DL-ROMs) for large-scale parametrized PDEs.
• Overcome the curse of dimensionality in DL-ROMs, which scales poorly with full-order model (FOM) dimension Nh.
• Improve training efficiency without sacrificing accuracy or online computational speed.
• Enable application of DL-ROMs to high-fidelity problems (e.g., Navier-Stokes) previously infeasible due to training cost.
• Provide a robust, non-intrusive, and generalizable framework for reduced order modeling of nonlinear time-dependent PDEs.

Proposed method

• Apply randomized POD (rPOD) to FOM snapshots to reduce dimensionality before neural network training, treating rPOD as a first-layer encoder.
• Use a convolutional autoencoder (CAE) to learn a nonlinear trial manifold for the solution space, with the decoder reconstructing the full solution.
• Train a deep feedforward neural network (DFNN) to model the reduced dynamics of generalized coordinates over time and parameters.
• Implement a multi-fidelity pretraining strategy, initializing high-fidelity network weights using parameters from lower-fidelity models (coarser meshes or simplified physics).
• Decouple the offline training phase into dimensionality reduction and pretraining, significantly reducing the number of parameters to be trained from scratch.
• Maintain the non-intrusive nature of DL-ROMs while integrating the mathematical rigor and efficiency of POD.

Experimental results

Research questions

• RQ1 Can POD-DL-ROM reduce the offline training cost of DL-ROMs without increasing network complexity?
• RQ2 How does the combination of rPOD and multi-fidelity pretraining affect the accuracy and convergence of the resulting ROM?
• RQ3 To what extent can POD-DL-ROM handle high-dimensional FOMs (e.g., Nh > 10⁵) while maintaining fast online evaluation?
• RQ4 Can POD-DL-ROM accurately capture complex physical phenomena such as vortex shedding in fluid dynamics?
• RQ5 How does POD-DL-ROM compare in performance to both standard POD-Galerkin ROMs and standard DL-ROMs in terms of accuracy, training time, and online speed-up?

Key findings

• POD-DL-ROM reduced offline training time to 50 minutes on a V100 GPU for a Navier-Stokes problem with Nh = 64,892, compared to significantly longer times for standard DL-ROMs.
• The method achieved a testing time of just 0.1 seconds per parameter instance, resulting in a speed-up of 2.15×10⁵ over the full-order model.
• For a higher-fidelity problem with Nh = 257,528, multi-fidelity pretraining achieved the same relative error (1.4×10⁻²) in only 10 minutes of training, demonstrating scalability.
• The POD-DL-ROM accurately captured vortex shedding at Re = 117 and its absence at Re = 70, with relative errors in the velocity field below 1.5%.
• Flow rate predictions over the outlet boundary matched the FOM with a maximum relative error of 1.65%, confirming fidelity in output quantities of interest.
• The method achieved a speed-up of 6.59×10⁵ at testing time for the high-fidelity case, enabling real-time or faster-than-real-time simulation.