[Paper Review] Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting

Xingjian Shi, Zhourong Chen+4 more|arXiv (Cornell University)|Jun 13, 2015

Meteorological Phenomena and Simulations27 references6,619 citations

TL;DR

The paper introduces ConvLSTM, a convolutional extension of LSTM for spatiotemporal precipitation nowcasting, and shows it outperforms FC-LSTM and ROVER on radar echo data.

ABSTRACT

The goal of precipitation nowcasting is to predict the future rainfall intensity in a local region over a relatively short period of time. Very few previous studies have examined this crucial and challenging weather forecasting problem from the machine learning perspective. In this paper, we formulate precipitation nowcasting as a spatiotemporal sequence forecasting problem in which both the input and the prediction target are spatiotemporal sequences. By extending the fully connected LSTM (FC-LSTM) to have convolutional structures in both the input-to-state and state-to-state transitions, we propose the convolutional LSTM (ConvLSTM) and use it to build an end-to-end trainable model for the precipitation nowcasting problem. Experiments show that our ConvLSTM network captures spatiotemporal correlations better and consistently outperforms FC-LSTM and the state-of-the-art operational ROVER algorithm for precipitation nowcasting.

Motivation & Objective

• Formulate precipitation nowcasting as a spatiotemporal sequence forecasting problem.
• Develop ConvLSTM to preserve spatial structure in input, state, and output transitions.
• Create an end-to-end trainable encoding-forecasting network for multi-step nowcasting.
• Evaluate on real radar echo data and synthetic Moving-MNIST to assess spatiotemporal modeling advantages.

Proposed method

• Extend FC-LSTM to ConvLSTM with convolutional input-to-state and state-to-state transitions.
• Represent inputs, cell outputs, hidden states, and gates as 3D tensors with spatial dimensions.
• Stack ConvLSTM layers in an encoding-forecasting architecture to produce multi-step predictions.
• Use a 1x1 convolution to generate final multi-frame predictions from concatenated forecasting states.
• Train end-to-end with backpropagation through time using cross-entropy loss on sequences.

Experimental results

Research questions

• RQ1 Can ConvLSTM better capture spatiotemporal correlations in radar echo data compared to FC-LSTM?
• RQ2 Does increasing the state-to-state convolutional kernel improve modeling of spatiotemporal motion patterns?
• RQ3 How does ConvLSTM perform relative to the ROVER optical-flow based method for precipitation nowcasting?
• RQ4 Do deeper ConvLSTM networks yield better predictions with manageable parameter counts?
• RQ5 Is ConvLSTM robust in out-of-domain and varied-motion scenarios?

Key findings

• ConvLSTM outperforms FC-LSTM in capturing spatiotemporal correlations on synthetic and radar data.
• Larger state-to-state convolutional kernels (e.g., 5x5, 9x9) improve motion pattern modeling over 1x1.
• Deeper ConvLSTM models can achieve better results with fewer parameters.
• ConvLSTM outperforms the ROVER optical-flow based method on precipitation nowcasting metrics.
• On the radar dataset, ConvLSTM achieved higher CSI and lower false alarms than ROVER, and better rainfall-mse and correlation.