[Paper Review] Precipitation nowcasting with generative diffusion models

Andrea Asperti, Fabio Merizzi+4 more|arXiv (Cornell University)|Aug 13, 2023

Climate variability and models13 citations

TL;DR

This paper introduces Generative Diffusion Ensemble (GDE) for precipitation nowcasting using DDIM diffusion with a post-processing U-net, achieving improved performance over WF-UNet on ERA-5 data. It trains on 2016–2020 data and evaluates on 2021 data with 15 diffusion steps.

ABSTRACT

In recent years traditional numerical methods for accurate weather prediction have been increasingly challenged by deep learning methods. Numerous historical datasets used for short and medium-range weather forecasts are typically organized into a regular spatial grid structure. This arrangement closely resembles images: each weather variable can be visualized as a map or, when considering the temporal axis, as a video. Several classes of generative models, comprising Generative Adversarial Networks, Variational Autoencoders, or the recent Denoising Diffusion Models have largely proved their applicability to the next-frame prediction problem, and is thus natural to test their performance on the weather prediction benchmarks. Diffusion models are particularly appealing in this context, due to the intrinsically probabilistic nature of weather forecasting: what we are really interested to model is the probability distribution of weather indicators, whose expected value is the most likely prediction. In our study, we focus on a specific subset of the ERA-5 dataset, which includes hourly data pertaining to Central Europe from the years 2016 to 2021. Within this context, we examine the efficacy of diffusion models in handling the task of precipitation nowcasting. Our work is conducted in comparison to the performance of well-established U-Net models, as documented in the existing literature. Our proposed approach of Generative Ensemble Diffusion (GED) utilizes a diffusion model to generate a set of possible weather scenarios which are then amalgamated into a probable prediction via the use of a post-processing network. This approach, in comparison to recent deep learning models, substantially outperformed them in terms of overall performance.

Motivation & Objective

• Motivate the use of diffusion-based generative models for probabilistic precipitation nowcasting on regular gridded weather data.
• Develop a Generative Diffusion Ensemble (GDE) that produces multiple plausible future rainfall fields and aggregates them into accurate forecasts.
• Incorporate additional meteorological features and conditioning to guide the diffusion process for short-term forecasts.
• Compare diffusion-based forecasts to established models such as WF-UNet and demonstrate improved performance on ERA-5 data.

Proposed method

• Utilize Denoising Diffusion Implicit Models (DDIM) with a denoising U-net as the core denoiser.
• Train with conditioning in a classifier-free manner by concatenating past weather frames and auxiliary fields to the noisy input.
• Define conditioning information as past precipitation, wind components, land-sea mask, and geopotential, integrated along the channel dimension.
• Adopt Generative Diffusion Ensemble (GED) by generating multiple samples and either averaging them or feeding them into a post-processing U-net to produce the final three-hour forecasts.
• Implement a post-processing U-net that takes fifteen diffusion outputs (three-hour forecasts per output) and outputs a refined three-hour precipitation forecast.
• Train with MAE on noise for the diffusion model and MSE for the post-processing step; evaluate using MSE after denormalization.
• Experiment with a U-net size [64, 128, 256, 384] for the denoiser and sinusoidal embeddings of the noise variance.

Experimental results

Research questions

• RQ1 Can a generative diffusion model capture the stochastic nature of precipitation nowcasting better than deterministic models?
• RQ2 Does averaging multiple diffusion samples or using a post-processing network improve nowcasting accuracy?
• RQ3 How does conditioning on past weather frames and ancillary fields affect diffusion-based nowcasting performance?
• RQ4 How does the proposed GED approach compare to WF-UNet on ERA-5 based precipitation nowcasting?
• RQ5 What is the effect of diffusion step count and training settings on performance and stability?

Key findings

• The Generative Diffusion Ensemble (GED) approach outperforms WF-UNet on precipitation nowcasting in the ERA-5 dataset.
• Mean of multiple diffusion generations yields better forecasts than using a single diffusion sample.
• A post-processing U-net trained on diffusion outputs further improves prediction accuracy over simple averaging.
• The study reports a 15-step DDIM diffusion process with 3-hour ahead forecasts evaluated on 2021 data.
• Training configurations include batch size 2 over 40 epochs, AdamW optimization, and MAE loss for diffusion noise prediction, with a subsequent 10-epoch fine-tuning.