[Paper Review] Super-resolution Probabilistic Rain Prediction from Satellite Data Using 3D U-Nets and EarthFormers

Yang Li, Haiyu Dong+3 more|arXiv (Cornell University)|Dec 6, 2022

Precipitation Measurement and Analysis4 citations

TL;DR

This paper presents a super-resolution probabilistic rain prediction model using 3D U-Nets and EarthFormers for 8-hour forecasts from multi-band satellite data, achieving strong performance in the Weather4cast 2022 NeurIPS competition. It leverages optimal spatial context, class-imbalanced loss functions, model ensembling, and threshold tuning, with 8.7 μm (cloudtop phase) and 10.8/13.4 μm (cloud-top height) bands identified as most predictive for rainfall.

ABSTRACT

Accurate and timely rain prediction is crucial for decision making and is also a challenging task. This paper presents a solution which won the 2 nd prize in the Weather4cast 2022 NeurIPS competition using 3D U-Nets and EarthFormers for 8-hour probabilistic rain prediction based on multi-band satellite images. The spatial context effect of the input satellite image has been deeply explored and optimal context range has been found. Based on the imbalanced rain distribution, we trained multiple models with different loss functions. To further improve the model performance, multi-model ensemble and threshold optimization were used to produce the final probabilistic rain prediction. Experiment results and leaderboard scores demonstrate that optimal spatial context, combined loss function, multi-model ensemble, and threshold optimization all provide modest model gain. A permutation test was used to analyze the effect of each satellite band on rain prediction, and results show that satellite bands signifying cloudtop phase (8.7 um) and cloud-top height (10.8 and 13.4 um) are the best predictors for rain prediction. The source code is available at https://github.com/bugsuse/weather4cast-2022-stage2.

Motivation & Objective

• To improve short-term probabilistic rain prediction using high-resolution satellite data for better decision-making in weather forecasting.
• To address the challenge of imbalanced rainfall distribution in satellite-based rain prediction models.
• To identify the most informative satellite bands for predicting precipitation through systematic analysis.
• To enhance model performance through multi-model ensemble strategies and threshold optimization.
• To achieve state-of-the-art results in the Weather4cast 2022 NeurIPS competition using deep learning architectures on 3D spatiotemporal data.

Proposed method

• Employed 3D U-Nets to model spatiotemporal dynamics in multi-band satellite images, capturing long-range spatial and temporal dependencies.
• Integrated EarthFormers to enhance modeling of long-range spatial context in the input satellite data, improving feature representation.
• Trained multiple models with different loss functions tailored to the imbalanced distribution of rainfall events to improve detection of rare precipitation events.
• Applied multi-model ensemble learning to combine predictions from diverse models, reducing variance and improving robustness.
• Optimized inference thresholds using validation data to maximize probabilistic forecast performance metrics such as Brier score and AUC.
• Conducted permutation tests to evaluate the contribution of individual satellite bands to rain prediction performance.

Experimental results

Research questions

• RQ1 Which spatial context range in satellite imagery yields optimal performance for 8-hour probabilistic rain prediction?
• RQ2 How do different loss functions impact model performance under class-imbalanced rainfall data?
• RQ3 Which satellite bands—particularly in the infrared and water vapor spectrum—contribute most significantly to accurate rain prediction?
• RQ4 To what extent does model ensembling and threshold optimization improve probabilistic forecast reliability and accuracy?
• RQ5 How do 3D U-Nets and EarthFormers jointly enhance spatiotemporal representation learning for precipitation forecasting?

Key findings

• Optimal spatial context range was identified through ablation, significantly improving model performance on the Weather4cast leaderboard.
• The use of a combined loss function addressing class imbalance led to measurable gains in detecting light and moderate rainfall events.
• The 8.7 μm band, indicating cloudtop phase, and the 10.8 μm and 13.4 μm bands, indicating cloud-top height, were found to be the most predictive for rainfall occurrence.
• Multi-model ensemble and threshold optimization provided modest but consistent improvements in probabilistic forecast scores, including Brier score and AUC.
• Permutation testing confirmed that infrared and water vapor bands are more informative than visible bands for precipitation prediction.
• The model achieved a competitive leaderboard score in the Weather4cast 2022 NeurIPS competition, securing 2nd place with the proposed architecture and training strategy.