[Paper Review] On the use of recurrent neural networks for predictions of turbulent flows

Luca Guastoni, Prem A. Srinivasan+3 more|arXiv (Cornell University)|Feb 4, 2020

Model Reduction and Neural Networks13 references12 citations

TL;DR

This paper evaluates recurrent neural networks, particularly Long Short-Term Memory (LSTM) networks, for predicting the temporal dynamics of a low-order model of near-wall turbulence. Using a simplified nine-mode system at Re=400, LSTMs achieve sub-1% relative errors in mean and fluctuating velocity statistics, outperforming MLPs and GRUs, especially when loss functions incorporate statistical metrics for improved training and generalization.

ABSTRACT

In this paper, the prediction capabilities of recurrent neural networks are assessed in the low-order model of near-wall turbulence by Moehlis {\it et al.} (New J. Phys. {\bf 6}, 56, 2004). Our results show that it is possible to obtain excellent predictions of the turbulence statistics and the dynamic behavior of the flow with properly trained long short-term memory (LSTM) networks, leading to relative errors in the mean and the fluctuations below $1\%$. We also observe that using a loss function based only on the instantaneous predictions of the flow may not lead to the best predictions in terms of turbulence statistics, and it is necessary to define a stopping criterion based on the computed statistics. Furthermore, more sophisticated loss functions, including not only the instantaneous predictions but also the averaged behavior of the flow, may lead to much faster neural network training.

Motivation & Objective

• To assess the feasibility of using recurrent neural networks (RNNs) for predicting the temporal dynamics of turbulent shear flows.
• To evaluate whether LSTM networks can accurately reproduce turbulence statistics and dynamical features such as Poincaré maps and Lyapunov exponents in a low-order model.
• To investigate the impact of different loss functions—particularly those incorporating statistical averages—on training efficiency and prediction accuracy.
• To compare the performance of LSTMs and gated recurrent units (GRUs) in capturing turbulent flow statistics with limited training data.
• To determine optimal hyperparameters, including time step size and network architecture, for robust and accurate predictions.

Proposed method

• A low-order model of near-wall turbulence with nine spatial modes is used, derived from Galerkin projection of the Navier-Stokes equations, yielding a system of nine coupled ODEs.
• Over 10,000 time series of 4,000 time units each are generated for training and validation, ensuring statistical convergence of turbulence statistics.
• Long Short-Term Memory (LSTM) networks are trained to predict the time evolution of the nine mode amplitudes using sequences of past amplitudes as input.
• Loss functions are designed to include both instantaneous prediction errors and statistical metrics (mean and variance of velocity fields), with training stopped based on statistical error thresholds.
• Alternative architectures, including multilayer perceptrons (MLPs) and gated recurrent units (GRUs), are evaluated for comparison.
• TensorFlow is used for model implementation, with training conducted on CPU and hyperparameters tuned for optimal performance.

Experimental results

Research questions

• RQ1 Can LSTM networks accurately predict the temporal dynamics and turbulence statistics of a low-order model of near-wall turbulence?
• RQ2 Does using a loss function that includes statistical averages of the flow (e.g., mean and variance) lead to faster convergence and better generalization than a standard instantaneous prediction loss?
• RQ3 How do LSTMs compare to GRUs and MLPs in predicting turbulence statistics and dynamical features such as Poincaré maps and Lyapunov exponents?
• RQ4 What is the optimal time step size and network architecture (e.g., number of layers, units, sequence length) for minimizing prediction errors in turbulence statistics?
• RQ5 Can small training datasets (e.g., 100 time series) yield reliable predictions when using advanced loss functions and architectural choices?

Key findings

• LSTM networks trained with a loss function incorporating both instantaneous predictions and statistical averages of the flow achieve relative errors of 0.26% for the mean velocity and 0.59% for the velocity fluctuations.
• The best-performing LSTM configuration (LSTM1–1–100) achieves sub-1% errors in both mean and fluctuating velocity statistics, significantly outperforming MLPs (3.21% and 18.61% error) and GRUs.
• Training with a loss function based solely on instantaneous predictions leads to poor statistical accuracy; thus, a stopping criterion based on statistical error metrics is essential for optimal performance.
• More sophisticated loss functions that include statistical behavior accelerate training and improve generalization, even with limited data.
• GRUs, despite having fewer parameters and similar computational cost, produce significantly higher statistical errors and exhibit non-physical behavior with small datasets, making them less suitable than LSTMs for this task.
• The optimal time step size for network performance is Δt = 1, with larger steps degrading prediction accuracy.