[Paper Review] Deep Learning for Day Forecasts from Sparse Observations

Marcin Andrychowicz, Lasse Espeholt+6 more|arXiv (Cornell University)|Jun 6, 2023

Meteorological Phenomena and Simulations40 citations

TL;DR

MetNet-3 is an observation-based neural weather model that predicts precipitation, wind, temperature, and dew point up to 24 hours ahead at high spatial (1 km) and temporal (2 min) resolution by learning from both dense and sparse observations and densifying outputs.

ABSTRACT

Deep neural networks offer an alternative paradigm for modeling weather conditions. The ability of neural models to make a prediction in less than a second once the data is available and to do so with very high temporal and spatial resolution, and the ability to learn directly from atmospheric observations, are just some of these models' unique advantages. Neural models trained using atmospheric observations, the highest fidelity and lowest latency data, have to date achieved good performance only up to twelve hours of lead time when compared with state-of-the-art probabilistic Numerical Weather Prediction models and only for the sole variable of precipitation. In this paper, we present MetNet-3 that extends significantly both the lead time range and the variables that an observation based neural model can predict well. MetNet-3 learns from both dense and sparse data sensors and makes predictions up to 24 hours ahead for precipitation, wind, temperature and dew point. MetNet-3 introduces a key densification technique that implicitly captures data assimilation and produces spatially dense forecasts in spite of the network training on extremely sparse targets. MetNet-3 has a high temporal and spatial resolution of, respectively, up to 2 minutes and 1 km as well as a low operational latency. We find that MetNet-3 is able to outperform the best single- and multi-member NWPs such as HRRR and ENS over the CONUS region for up to 24 hours ahead setting a new performance milestone for observation based neural models. MetNet-3 is operational and its forecasts are served in Google Search in conjunction with other models.

Motivation & Objective

• Demonstrate that an observation-based neural network can forecast multiple weather variables up to 24 hours ahead.
• Develop a densification method to convert sparse observations into dense forecasts without relying on dense ground truth across the domain.
• Evaluate MetNet-3 against state-of-the-art numerical weather prediction baselines (ENS, HREF, HRRR, HRES) across CONUS.
• Show that the model operates with low latency and scales to high spatial and temporal resolutions while leveraging sparse data sources.

Proposed method

• Use a three-part MetNet-3 architecture with topographical embeddings, a U-Net backbone, and a MaxViT transformer to capture local and long-range interactions.
• Introduce densification that masks input stations during training and uses spatial parameter sharing to output dense forecasts.
• Predict probabilistic distributions via softmax outputs for precipitation and surface variables, enabling CRPS and CSI evaluation, while using MSE for assimilated state channels.
• Encode lead time as a one-hot embedding and apply it additively and multiplicatively to inputs and hidden states to condition forecasts on lead time.
• Train with cross-entropy losses for precipitation and surface variables, plus MSE for assimilated state, and employ gradient rescaling to balance multi-output losses.
• Employ model parallelism across 16 TPU cores to handle large spatiotemporal inputs.

Figure 1 : Abstract depiction of densification aspects. (a) During training a fraction of the weather stations are masked out from the input, while kept in the target. (b) To evaluate generalization to untrained locations, a set of weather stations represented by squares is never trained on and only

Experimental results

Research questions

• RQ1 Can an observation-based neural network forecast multiple atmospheric variables up to 24 hours ahead at high resolution using sparse observations?
• RQ2 Does a learned densification process enable dense field forecasts from sparsely observed ground truth locations?
• RQ3 How does MetNet-3 compare to ensemble and deterministic NWP baselines across lead times and variables?
• RQ4 What are the impacts of lead-time conditioning and training strategies on long-range forecast accuracy?],
• RQ5 key_findings=[

Key findings

• MetNet-3 outperforms multi-member baselines ENS and HREF on CRPS up to about 19 hours for instantaneous precipitation and up to 18 hours for moderate precipitation when using CSI-based thresholds.
• MetNet-3 extends the lead-time advantage over prior observation-based models (MetNet-2) from 12 hours to 24 hours for multiple variables.
• MetNet-3 provides probabilistic predictions (full distributions) for precipitation and surface variables, with better CRPS than ENS across lead times.
• Densification allows high-resolution forecasts at 1 km for precipitation and 4 km for surface variables, using sparse observations from 942 weather stations spread over CONUS.
• The model achieves high temporal resolution (2 minutes) and operational latency near 1 second per lead time, serving forecasts in Google Search.
• Training employs a randomized masking strategy of ground stations and a specific lead-time sampling scheme to improve long-lead performance.

Figure 2 : Case study for Sat Apr 23 2022 12:00 UTC featuring the Rocky Mountains of Colorado showing the mean of the ENS and MetNet-3 6 hour wind speed forecasts (top, left and center) along with the OMO stations ground truth (top, right) and the error of ENS and MetNet-3 on the individual weather