[Paper Review] Data Assimilation with Machine Learning Surrogate Models: A Case Study with FourCastNet

Melissa Adrian, Daniel Sanz-Alonso+1 more|arXiv (Cornell University)|May 21, 2024

Big Data Technologies and ApplicationsDecision Sciences3 citations

TL;DR

This paper proposes integrating FourCastNet, a deep learning weather surrogate model, into a 3DVar variational data assimilation framework to enable accurate, stable long-term weather state estimation and forecasting from sparse, noisy observations. Despite FourCastNet's inherent long-term instability, the method achieves stable reconstruction error and high-fidelity forecasts over a full year, demonstrating viability for operational use in extreme event prediction.

ABSTRACT

Modern data-driven surrogate models for weather forecasting provide accurate short-term predictions but inaccurate and nonphysical long-term forecasts. This paper investigates online weather prediction using machine learning surrogates supplemented with partial and noisy observations. We empirically demonstrate and theoretically justify that, despite the long-time instability of the surrogates and the sparsity of the observations, filtering estimates can remain accurate in the long-time horizon. As a case study, we integrate FourCastNet, a weather surrogate model, within a variational data assimilation framework using partial, noisy ERA5 data. Our results show that filtering estimates remain accurate over a year-long assimilation window and provide effective initial conditions for forecasting tasks, including extreme event prediction.

Motivation & Objective

• To evaluate the utility of machine learning weather surrogate models in operational data assimilation tasks under realistic conditions of sparse, noisy observations.
• To investigate whether unstable surrogate models like FourCastNet can still produce stable and accurate filtering estimates over long time horizons when combined with variational data assimilation.
• To assess the effectiveness of 3DVar-initialized forecasts using FourCastNet for short-term and extreme event prediction, particularly for typhoons.
• To provide theoretical justification for the observed stability of filtering estimates despite model instability and observational sparsity.
• To explore the potential of using coarse NWP forecasts as inputs to data assimilation with surrogates to reduce computational cost while maintaining accuracy.

Proposed method

• Utilizes FourCastNet as a fast, differentiable weather surrogate model to generate short-term forecasts in a global, 0.25° resolution atmospheric state space.
• Applies a 3DVar variational data assimilation framework to combine FourCastNet forecasts with low-resolution, noisy ERA5 reanalysis data as observational proxies.
• Employs a 3DVar cost function that balances forecast model dynamics and observational constraints, minimizing the misfit between observations and model states.
• Uses a fixed background error covariance matrix $ C $, with potential for future improvement via physics-informed tuning.
• Performs long-term assimilation over a 12-month window to evaluate stability and accuracy of filtering estimates.
• Conducts ensemble forecasting from assimilated states to assess uncertainty quantification and extreme event prediction performance.

(a) a

Experimental results

Research questions

• RQ1 Can a machine learning weather surrogate model with known long-term instability still produce stable and accurate filtering estimates when combined with variational data assimilation?
• RQ2 How does 3DVar initialization using noisy, coarse observations compare to direct interpolation in terms of forecast accuracy for extreme weather events?
• RQ3 Can data assimilation with FourCastNet produce physically realistic and stable reconstructions over long time horizons despite observational sparsity and noise?
• RQ4 What is the impact of using coarse NWP forecasts as background fields in the assimilation process for surrogate-based systems?
• RQ5 How effective are 3DVar-initialized forecasts from surrogate models in predicting the intensity and track of extreme events like typhoons?

Key findings

• The 3DVar filtering process with FourCastNet maintains stable reconstruction error over a full 12-month assimilation window, despite the surrogate model's inherent long-term instability.
• 3DVar analyses initialized from noisy, interpolated $4.5^\circ$ observations produce more accurate and physically realistic forecasts than direct interpolation of observations, particularly in capturing the eye and intensity of Typhoon Mawar.
• The 3DVar-initialized forecasts of Typhoon Mawar's 10m wind speed and mean sea level pressure closely match the ground truth ERA5 data, outperforming forecasts initialized from raw interpolated observations.
• Visualizations show that 3DVar analyses produce realistic weather patterns and maintain fidelity in both wind speed and pressure fields over multiple forecast steps.
• The method enables effective uncertainty quantification through ensemble forecasting, supporting real-time decision-making for extreme weather events.
• Theoretical justification is provided for the stability of filtering estimates, even when the underlying surrogate model is unstable, due to the regularizing effect of the variational framework.

(b) b