[Paper Review] Comparing recurrent and convolutional neural networks for predicting wave propagation

Stathi Fotiadis, Eduardo Pignatelli+4 more|arXiv (Cornell University)|Feb 20, 2020

Meteorological Phenomena and Simulations17 references29 citations

TL;DR

This paper compares recurrent (LSTM-based) and convolutional (U-Net, PredRNN++) neural networks for predicting surface wave propagation governed by the Saint-Venant equations. It demonstrates that convolutional networks achieve performance on par with recurrent models in long-term prediction while offering significantly faster inference—up to 241× faster than numerical solvers—making them highly efficient alternatives for spatiotemporal PDE prediction in engineering and environmental modeling.

ABSTRACT

Dynamical systems can be modelled by partial differential equations and numerical computations are used everywhere in science and engineering. In this work, we investigate the performance of recurrent and convolutional deep neural network architectures to predict the surface waves. The system is governed by the Saint-Venant equations. We improve on the long-term prediction over previous methods while keeping the inference time at a fraction of numerical simulations. We also show that convolutional networks perform at least as well as recurrent networks in this task. Finally, we assess the generalisation capability of each network by extrapolating in longer time-frames and in different physical settings.

Motivation & Objective

• To evaluate and compare the performance of recurrent and convolutional neural networks in predicting long-term surface wave propagation governed by the Saint-Venant equations.
• To assess the generalization capability of these models beyond the training distribution, including extrapolation to longer times and different physical settings.
• To determine whether convolutional architectures can match or exceed the performance of recurrent models in spatiotemporal sequence prediction tasks involving PDEs.
• To investigate whether learned representations in convolutional networks encode physically meaningful information, such as tank size or wave speed.
• To provide a fast, accurate deep learning alternative to traditional numerical solvers, reducing inference time by orders of magnitude.

Proposed method

• Trained four architectures: LSTM (baseline), ConvLSTM, PredRNN++, and U-Net on sequences of water surface height maps from simulated wave propagation.
• Used a physical simulation of the Saint-Venant equations as the ground truth data generator, with varying tank sizes and initial conditions.
• Applied standard deep learning training protocols with early stopping, learning rate scheduling, and weight initialization, using mean squared error loss.
• Employed a U-Net encoder head to extract latent representations and fine-tuned it to predict tank size from 5 consecutive frames.
• Compared inference speed and prediction accuracy across models using RMSE at 20- and 80-time-step horizons.
• Conducted ablation studies on generalization by testing models on unseen tank sizes and wave dynamics not present in training.

Experimental results

Research questions

• RQ1 Can convolutional neural networks achieve comparable long-term prediction accuracy to recurrent networks for wave propagation governed by the Saint-Venant equations?
• RQ2 How do different deep learning architectures (LSTM, ConvLSTM, PredRNN++, U-Net) compare in terms of inference speed and prediction accuracy?
• RQ3 To what extent can trained models generalize to longer prediction horizons and different physical settings such as altered tank sizes?
• RQ4 Do the learned representations in convolutional networks encode physically relevant information, such as wave speed or system size?
• RQ5 What is the trade-off between model complexity, inference speed, and accuracy in deep learning-based PDE solvers for wave dynamics?

Key findings

• The U-Net and PredRNN++ models achieved the lowest RMSE at 80 time-steps ahead (0.07 and 0.09, respectively), significantly outperforming the baseline LSTM (0.19).
• The U-Net model achieved a 241× speed-up over the numerical simulator, computing one frame in 2.6ms compared to 630.7ms for the solver.
• Convolutional networks (U-Net, PredRNN++) matched or exceeded the performance of recurrent models (LSTM, ConvLSTM) in long-term prediction accuracy.
• The pre-trained U-Net encoder could predict tank size with a mean error of 0.14, indicating it learned physically relevant features, while a random encoder performed poorly (error 2.27).
• Both U-Net and PredRNN++ showed limited generalization to smaller and larger tanks, with errors exceeding the baseline dummy regressor, indicating failure to extrapolate physical scale.
• Feature map visualization revealed that U-Net uses spatially localized features (peaks, troughs, boundaries), while PredRNN++ builds predictions progressively through temporal memory.