[论文解读] Challenges of learning multi-scale dynamics with AI weather models: Implications for stability and one solution

Ashesh Chattopadhyay, Sun, Y. Qiang+1 more|arXiv (Cornell University)|Apr 14, 2023

Meteorological Phenomena and Simulations被引用 14

一句话总结

本文将谱偏差识别为湍流数据驱动数字孪生长期不稳定性的普遍原因，并引入 FouRKS 以在气候模拟中实现长期、物理一致的稳定性。它在 QG 和 ERA5 数据上展示了从数百到数万天的稳定预测。

ABSTRACT

Long-term stability and physical consistency are critical properties for AI-based weather models if they are going to be used for subseasonal-to-seasonal forecasts or beyond, e.g., climate change projection. However, current AI-based weather models can only provide short-term forecasts accurately since they become unstable or physically inconsistent when time-integrated beyond a few weeks or a few months. Either they exhibit numerical blow-up or hallucinate unrealistic dynamics of the atmospheric variables, akin to the current class of autoregressive large language models. The cause of the instabilities is unknown, and the methods that are used to improve their stability horizons are ad-hoc and lack rigorous theory. In this paper, we reveal that the universal causal mechanism for these instabilities in any turbulent flow is due to extit{spectral bias} wherein, extit{any} deep learning architecture is biased to learn only the large-scale dynamics and ignores the small scales completely. We further elucidate how turbulence physics and the absence of convergence in deep learning-based time-integrators amplify this bias, leading to unstable error propagation. Finally, using the quasi-geostrophic flow and European Center for Medium-Range Weather Forecasting (ECMWF) Reanalysis data as test cases, we bridge the gap between deep learning theory and numerical analysis to propose one mitigative solution to such unphysical behavior. We develop long-term physically-consistent data-driven models for the climate system and demonstrate accurate short-term forecasts, and hundreds of years of time-integration with accurate mean and variability.

研究动机与目标

• 识别湍流数据驱动数字孪生中不稳定性的普遍原因。
• 提出一个受物理启发、与架构无关的长期稳定性缓解框架（FouRKS）。
• 在 QG 和 ERA5 数据集上证明长期稳定且物理一致的仿真。
• 评估 FouRKS 在长期预报中如何保持均值、概率密度函数（PDFs）和变率。

提出的方法

• 将谱偏差作为基于深度学习的数字孪生不稳定性的来源进行分析。
• 开发基于傅里叶的谱正则化，在训练过程中惩罚高波数误差。
• 将收敛的四阶 Runge-Kutta 时间积分器嵌入模型内部作为一个可微分层。
• 在自回归预测过程中实现自监督谱纠正策略。
• 使 FouRKS 对架构无关，以与任何预测偏微分方程残差的神经动力学仿真器协同工作。

Figure 1: Long-term instabilities in a FourCastNet pathak2022fourcastnet and U-NET-based digital twin (section 4.3 ) trained on both $0.25^{\circ}$ and $2^{\circ}$ ERA5 data (section 4.1 ) and U-NET on QG simulations (section 4.2 ). (a) Z500 values from ERA5 on day 1 from year $2018$ . (b) correspon

实验结果

研究问题

• RQ1 湍流数据驱动数字孪生长期不稳定性的根本原因是什么？
• RQ2 一个有原则的方法框架能否缓解谱偏差并产生收敛的长期预测？
• RQ3 FouRKS 在 QG 和 ERA5 数据中的物理意义长时统计（均值、PDF、变率）表现如何？
• RQ4 FouRKS 能在多长时间尺度内提供数百到数千天的稳定预测？

主要发现

• 谱偏差被确认为湍流数据驱动数字孪生不稳定性的普遍原因。
• 基于 Fourier 的谱正则化、RK4 积分器以及自监督谱纠正的协同作用实现长期稳定性。
• FouRKS 结合 U-NET 的两层 QG 系统实现了 20,000 天的稳定自回归仿真。
• 结合 ERA5 数据的 FouRKS 可实现长达 5,200 天的稳定且物理一致的自回归预测。
• 该框架产生的预测在各尺度上的傅里叶谱与真实谱相匹配，并保持关键变量的均值和 PDFs。
• 与基线模型相比，FouRKS 在长期稳定性和预测的物理真实感方面显著提升。

Figure 2: Schematics for each component of the FouRKS framework and the baseline U-NET. More details about each of the components can be found in section 4.3 and section 4.4 .