[Paper Review] Long-time prediction of nonlinear parametrized dynamical systems by deep learning-based reduced order models

Federico Fatone, Stefania Fresca+1 more|arXiv (Cornell University)|Jan 25, 2022

Model Reduction and Neural Networks4 citations

TL;DR

This paper proposes the μt-POD-LSTM-ROM, a deep learning-based reduced order model that combines proper orthogonal decomposition (POD) with long short-term memory (LSTM) networks to enable long-term prediction of nonlinear parametrized dynamical systems. By decoupling parameter and time prediction into two separate LSTM architectures, the framework achieves accurate time extrapolation up to 15 times beyond the training window, with a mean relative error of 9.029×10⁻⁵ and 52% faster inference than prior POD-DL-ROMs.

ABSTRACT

Deep learning-based reduced order models (DL-ROMs) have been recently proposed to overcome common limitations shared by conventional ROMs - built, e.g., exclusively through proper orthogonal decomposition (POD) - when applied to nonlinear time-dependent parametrized PDEs. In particular, POD-DL-ROMs can achieve extreme efficiency in the training stage and faster than real-time performances at testing, thanks to a prior dimensionality reduction through POD and a DL-based prediction framework. Nonetheless, they share with conventional ROMs poor performances regarding time extrapolation tasks. This work aims at taking a further step towards the use of DL algorithms for the efficient numerical approximation of parametrized PDEs by introducing the $μt$-POD-LSTM-ROM framework. This novel technique extends the POD-DL-ROM framework by adding a two-fold architecture taking advantage of long short-term memory (LSTM) cells, ultimately allowing long-term prediction of complex systems' evolution, with respect to the training window, for unseen input parameter values. Numerical results show that this recurrent architecture enables the extrapolation for time windows up to 15 times larger than the training time domain, and achieves better testing time performances with respect to the already lightning-fast POD-DL-ROMs.

Motivation & Objective

• Address the critical limitation of conventional reduced order models (ROMs) in extrapolating solutions beyond the training time domain.
• Overcome the poor time extrapolation performance of POD-based ROMs while maintaining high efficiency for online simulations.
• Enable fast, accurate, and robust prediction of complex nonlinear parametrized dynamical systems, including fluid dynamics problems, with unseen parameter values and extended time horizons.
• Develop a non-intrusive framework compatible with black-box high-fidelity solvers, reducing offline computational costs by enabling shorter FOM snapshot windows.
• Achieve faster-than-real-time performance for time-critical applications such as control, optimization, and uncertainty quantification.

Proposed method

• Decompose the solution approximation into two stages: (1) prediction of the solution for a new parameter instance using a parameter-aware LSTM, and (2) time evolution prediction using a time-aware LSTM.
• Use proper orthogonal decomposition (POD) to extract a low-dimensional reduced manifold from high-fidelity FOM snapshots.
• Train a two-branch LSTM architecture: one for parameter-dependent solution mapping and another for temporal dynamics, both leveraging long short-term memory cells to capture long-range dependencies.
• Employ a 3-layer feedforward regressor to predict future generalized coordinates from LSTM-encoded latent states, enabling end-to-end time extrapolation.
• Integrate the decoder of a convolutional autoencoder (CAE) to reconstruct full-order solutions from reduced coordinates, ensuring spatial fidelity.
• Train the model in a non-intrusive manner using only FOM snapshots, allowing compatibility with black-box solvers and minimizing implementation complexity.

Experimental results

Research questions

• RQ1 Can a deep learning-based ROM framework achieve reliable long-term time extrapolation for nonlinear parametrized PDEs beyond the training time window?
• RQ2 How does the proposed μt-POD-LSTM-ROM compare in accuracy and speed to existing POD-DL-ROMs, particularly in terms of online inference time and generalization to unseen parameters?
• RQ3 To what extent can the framework reduce offline computational costs by enabling shorter FOM snapshot windows without sacrificing prediction accuracy?
• RQ4 Can the model maintain high accuracy in complex, aperiodic, or vortex-dominated flows such as unsteady Navier-Stokes equations with laminar flow?
• RQ5 Does the decoupled architecture (parameter and time prediction) improve extrapolation performance compared to joint modeling approaches in nonlinear dynamical systems?

Key findings

• The μt-POD-LSTM-ROM achieved time extrapolation up to 15 times longer than the training time domain, demonstrating robustness on complex fluid dynamics problems such as unsteady Navier-Stokes equations.
• The mean relative error for the test case with Re=117 (vortex shedding) was 5.806×10⁻², with a 95% bootstrap confidence interval of [9.029×10⁻⁵, 1.053×10⁻⁴], indicating high accuracy despite limited training data.
• The framework reduced online prediction time by 52% compared to the baseline POD-DL-ROM, achieving faster-than-real-time simulation performance for problems with time scales in the range of tenths of a second.
• The model maintained high accuracy even when trained only on velocity fields, without access to pressure data, highlighting its robustness and generalization capability.
• The training of the μt-POD-LSTM-ROM required only 2415 epochs (5692s), and the inference was accelerated via a compact architecture with 1.6 million parameters, enabling local deployment without cloud GPU clusters.
• The framework demonstrated successful extrapolation on both periodic and aperiodic systems, including a 3-species Lotka-Volterra model and a linear advection-diffusion-reaction equation, confirming broad applicability.