[Paper Review] Fixing the Double Penalty in Data-Driven Weather Forecasting Through a Modified Spherical Harmonic Loss Function

Christopher Subich, Syed Zahid Husain+2 more|ArXiv.org|Jan 31, 2025

Meteorological Phenomena and Simulations3 citations

TL;DR

The paper introduces a parameter-free loss function AMSE that separates spectral amplitude errors from decorrelation in data-driven weather forecasts, and shows how fine-tuning GraphCast with AMSE yields sharper forecasts, higher effective resolution, improved ensemble spread, and better tropical cyclone predictions.

ABSTRACT

Recent advancements in data-driven weather forecasting models have delivered deterministic models that outperform the leading operational forecast systems based on traditional, physics-based models. However, these data-driven models are typically trained with a mean squared error loss function, which causes smoothing of fine scales through a "double penalty" effect. We develop a simple, parameter-free modification to this loss function that avoids this problem by separating the loss attributable to decorrelation from the loss attributable to spectral amplitude errors. Fine-tuning the GraphCast model with this new loss function results in sharp deterministic weather forecasts, an increase of the model's effective resolution from 1,250km to 160km, improvements to ensemble spread, and improvements to predictions of tropical cyclone strength and surface wind extremes.

Motivation & Objective

• Address the smoothing (double penalty) problem in deterministic data-driven weather forecasts trained with MSE.
• Propose a parameter-free loss modification that decouples spectral amplitude errors from decorrelation.
• Demonstrate that fine-tuning GraphCast with the modified loss yields sharper forecasts and higher effective resolution.
• Assess improvements in ensemble spread and tropical cyclone/intensity predictions.
• Discuss limitations, extensions, and applicability to other grids and fields.

Proposed method

• Derive a spectrally decomposed MSE using spherical harmonics to separate amplitude and coherence terms.
• Define AMSE as an adjusted MSE: AMSE(x,y)= sum_k (sqrt(PSD_k(x)) - sqrt(PSD_k(y)))^2 + 2 max(PSD_k(x), PSD_k(y)) (1 - Coh_k(x,y)).
• Fine-tune an operational GraphCast checkpoint (¼°, 13-level) starting from an existing baseline, using the AMSE loss in a staged curriculum (single-step to multi-step forecasts).
• Maintain per-variable, per-level weighting and normalization to handle heterogeneous units and scales.
• Evaluate on 2022 data with HRES as ground truth, using lagged-ensemble verification and CRPS/eRMSE/spread metrics.
• Provide discussion on potential extensions to other grids and to ensemble/ML forecasting.

Experimental results

Research questions

• RQ1 Can a spectrally aware, parameter-free loss function improve sharpness and effective resolution in deterministic data-driven weather forecasts?
• RQ2 Does AMSE retain or improve predictive skill while increasing spectral variance at fine scales compared to MSE?
• RQ3 What are the impacts of AMSE fine-tuning on ensemble spread and tropical cyclone intensity forecasts?
• RQ4 Is the approach transferable to other grid types or turbulence-like applications beyond global atmospheric forecasts?
• RQ5 How does autoregressive multi-step training interact with the AMSE objective in terms of maintaining variability?

Key findings

• AMSE fine-tuning yields sharper forecasts and an effective resolution improvement from about 1250 km to 160 km (at longer leads).
• Lagged ensemble verification shows consistent CRPS improvements with AMSE, indicating better-calibrated dispersion without sacrificing predictive skill (eRMSE largely unchanged).
• AMSE improves tropical cyclone intensity predictions and wind extremes relative to the control model, aligning more closely with ground-truth analyses.
• The AMSE AR12 model maintains storm location accuracy comparable to the baseline while restoring storm intensity realism and reducing weak-intensity bias.
• Autoregressive multi-step training with AMSE introduces some smoothing at moderate scales but preserves fine-scale variability better than the unmodified MSE approach.
• The method is presented as parameter-free (no scale cutoffs) and potentially applicable to other spectral-decomposable fields or turbulence-like problems.