[Paper Review] Model order reduction with neural networks: Application to laminar and turbulent flows

Kai Fukami, Kazuto Hasegawa+3 more|arXiv (Cornell University)|Nov 20, 2020

Model Reduction and Neural Networks120 references71 citations

TL;DR

This study evaluates a hybrid autoencoder (AE) combining convolutional neural networks (CNNs) and multi-layer perceptrons (MLPs) for model order reduction in fluid dynamics. It demonstrates that AE performance is highly sensitive to hyperparameters like latent dimension, activation functions, and model size, with optimal configurations varying significantly across laminar and turbulent flows—highlighting the need for flow-specific tuning and offering insights for robust, data-driven reduced-order modeling in complex fluid systems.

ABSTRACT

We investigate the capability of neural network-based model order reduction, i.e., autoencoder (AE), for fluid flows. As an example model, an AE which comprises of a convolutional neural network and multi-layer perceptrons is considered in this study. The AE model is assessed with four canonical fluid flows, namely: (1) two-dimensional cylinder wake, (2) its transient process, (3) NOAA sea surface temperature, and (4) $y-z$ sectional field of turbulent channel flow, in terms of a number of latent modes, a choice of nonlinear activation functions, and a number of weights contained in the AE model. We find that the AE models are sensitive against the choice of the aforementioned parameters depending on the target flows. Finally, we foresee the extensional applications and perspectives of machine learning based order reduction for numerical and experimental studies in fluid dynamics community.

Motivation & Objective

• To systematically assess the impact of key hyperparameters—latent dimension, activation functions, and number of weights—on autoencoder-based model order reduction in fluid dynamics.
• To investigate the sensitivity of autoencoder performance to flow characteristics, including laminar vs. turbulent flows and transient dynamics.
• To evaluate the effectiveness of pruning as a method for reducing model complexity while preserving reconstruction accuracy.
• To provide practical guidelines for deploying neural network-based reduced-order models in numerical and experimental fluid dynamics studies.
• To address open questions regarding the limitations and generalization potential of autoencoders in complex, high-dimensional fluid systems.

Proposed method

• A hybrid autoencoder architecture is employed, combining convolutional neural networks (CNNs) for spatial feature extraction and multi-layer perceptrons (MLPs) for latent space compression and reconstruction.
• The CNN encodes high-dimensional flow fields into a lower-dimensional latent space using learnable filters and pooling operations, followed by fully connected layers for dimensionality reduction.
• The decoder reconstructs the original flow field from the latent representation using transposed convolutions and up-sampling layers.
• The model is trained end-to-end using mean squared error (MSE) loss between input and reconstructed flow fields.
• Hyperparameter sensitivity is evaluated across four canonical fluid flow datasets: 2D cylinder wake, transient flow, NOAA SST, and turbulent channel flow.
• Model pruning is applied by setting a sparsity threshold to remove less significant weights, followed by re-evaluation of reconstruction error.

Experimental results

Research questions

• RQ1 How does the number of latent modes affect reconstruction accuracy and generalization in autoencoder-based reduced-order models for different flow types?
• RQ2 How do different nonlinear activation functions (e.g., ReLU, ELU, SELU) influence the performance and stability of the autoencoder across laminar and turbulent flows?
• RQ3 What is the impact of model size (number of weights) on training dynamics, reconstruction error, and generalization capability?
• RQ4 Can model pruning effectively reduce parameter count without degrading reconstruction quality, and how does this vary across flow regimes?
• RQ5 What are the inherent limitations of autoencoders in representing complex turbulent flow dynamics compared to laminar or transient flows?

Key findings

• The number of latent modes required for accurate reconstruction varies significantly with flow complexity: laminar flows (e.g., cylinder wake) require fewer modes than turbulent or transient flows.
• The choice of activation function has a strong, flow-dependent impact on reconstruction accuracy, with ReLU and ELU showing superior performance in most cases, while SELU led to instability in turbulent flow reconstruction.
• AEs trained on turbulent flows exhibit more uniformly distributed weight magnitudes across the network compared to laminar flows, where a few large weights dominate, suggesting different learning dynamics.
• Pruning up to 10–20% of the least significant weights maintains low reconstruction error (L2 norm), indicating that model compression is feasible without significant accuracy loss.
• The reconstruction error for the NOAA sea surface temperature dataset showed intermediate behavior between laminar and turbulent cases, reflecting its intermediate complexity.
• The study confirms that no single AE configuration is universally optimal—hyperparameter selection must be tailored to the specific flow characteristics, especially for high-dimensional turbulent systems.