[Paper Review] Deep Learning for Precipitation Nowcasting: A Benchmark and A New Model

Xingjian Shi, Zhihan Gao+5 more|arXiv (Cornell University)|Jun 12, 2017

Flood Risk Assessment and Management20 references616 citations

TL;DR

The paper introduces Trajectory GRU (TrajGRU), a learnable location-variant recurrent structure for precipitation nowcasting, and provides a large-scale HKO-7 benchmark with online/offline evaluation and balanced loss functions.

ABSTRACT

With the goal of making high-resolution forecasts of regional rainfall, precipitation nowcasting has become an important and fundamental technology underlying various public services ranging from rainstorm warnings to flight safety. Recently, the Convolutional LSTM (ConvLSTM) model has been shown to outperform traditional optical flow based methods for precipitation nowcasting, suggesting that deep learning models have a huge potential for solving the problem. However, the convolutional recurrence structure in ConvLSTM-based models is location-invariant while natural motion and transformation (e.g., rotation) are location-variant in general. Furthermore, since deep-learning-based precipitation nowcasting is a newly emerging area, clear evaluation protocols have not yet been established. To address these problems, we propose both a new model and a benchmark for precipitation nowcasting. Specifically, we go beyond ConvLSTM and propose the Trajectory GRU (TrajGRU) model that can actively learn the location-variant structure for recurrent connections. Besides, we provide a benchmark that includes a real-world large-scale dataset from the Hong Kong Observatory, a new training loss, and a comprehensive evaluation protocol to facilitate future research and gauge the state of the art.

Motivation & Objective

• Motivate accurate, high-resolution nowcasting of regional rainfall for real-world public services.
• Address limitations of ConvLSTM by learning location-variant recurrent structures.
• Provide a large-scale benchmark (HKO-7) with online and offline evaluation and balanced loss functions.
• Encourage standardized evaluation protocols and online learning applicability to video prediction in meteorology.

Proposed method

• Propose Trajectory GRU (TrajGRU) where a subnetwork outputs a learned set of local, potentially non-grid recurrent connections parameterized by learned optical flows.
• Replace fixed state-to-state convolution with a learned, location-variant connectivity pattern via warp-based sampling of previous states.
• Introduce an encoding-forecasting network structure with downsampling/upsampling to separately capture global and local spatiotemporal information.
• Develop the HKO-7 dataset from Hong Kong radar CAPPI images (2009–2015) with noise filtering and rain-rate based preprocessing.
• Define Balanced MSE (B-MSE) and Balanced MAE (B-MAE) losses to address rain-rate imbalances.
• Provide an evaluation protocol including offline and online settings and multiple rain-rate thresholds.

Experimental results

Research questions

• RQ1 Can a learnable, location-variant recurrent structure improve precipitation nowcasting over ConvGRU and other baselines?
• RQ2 Does online fine-tuning (online learning) improve nowcasting performance compared to offline training?
• RQ3 Do balanced loss functions (B-MSE, B-MAE) better reflect performance at heavier rainfall thresholds?
• RQ4 How does TrajGRU perform on a large, real-world dataset (HKO-7) across multiple rain-rate thresholds?

Key findings

• TrajGRU outperforms ConvGRU, 2D CNN, 3D CNN, and DFN on MovingMNIST++ with fewer parameters than some baselines.
• On the MovingMNIST++ synthetic dataset, TrajGRU with 13 links achieves better test MSE than ConvGRU with larger fixed kernels and fewer parameters.
• In the HKO-7 precipitation nowcasting benchmark, deep learning models with balanced losses surpass optical-flow baselines, with TrajGRU achieving the best overall performance.
• Online fine-tuning consistently improves nowcasting scores across models compared to offline training.
• Balanced loss functions (B-MSE, B-MAE) show stronger correlation with practical skill scores (CSI, HSS) than vanilla MSE/MAE.
• TrajGRU demonstrates that learning trajectory-based recurrent connections is beneficial for rotation and other location-variant motion patterns in rainfall fields.