[论文解读] Can we integrate spatial verification methods into neural-network loss functions for atmospheric science?

Ryan Lagerquist, Imme Ebert‐Uphoff|arXiv (Cornell University)|Mar 21, 2022

Meteorological Phenomena and Simulations被引用 6

一句话总结

本文提出空间增强损失函数（SELFs），将空间验证方法整合至大气科学的神经网络训练中，通过邻域或光谱滤波器（傅里叶/小波）增强常见的评分指标（如Brier评分）。结果表明，使用光谱SELFs训练的模型在概率预报校准方面显著优于像素级损失，尤其在从孤立单体对流到热带气旋的多样化对流尺度上表现更优。

ABSTRACT

In the last decade, much work in atmospheric science has focused on spatial verification (SV) methods for gridded prediction, which overcome serious disadvantages of pixelwise verification. However, neural networks (NN) in atmospheric science are almost always trained to optimize pixelwise loss functions, even when ultimately assessed with SV methods. This establishes a disconnect between model verification during vs. after training. To address this issue, we develop spatially enhanced loss functions (SELF) and demonstrate their use for a real-world problem: predicting the occurrence of thunderstorms (henceforth, "convection") with NNs. In each SELF we use either a neighbourhood filter, which highlights convection at scales larger than a threshold, or a spectral filter (employing Fourier or wavelet decomposition), which is more flexible and highlights convection at scales between two thresholds. We use these filters to spatially enhance common verification scores, such as the Brier score. We train each NN with a different SELF and compare their performance at many scales of convection, from discrete storm cells to tropical cyclones. Among our many findings are that (a) for a low (high) risk threshold, the ideal SELF focuses on small (large) scales; (b) models trained with a pixelwise loss function perform surprisingly well; (c) however, models trained with a spectral filter produce much better-calibrated probabilities than a pixelwise model. We provide a general guide to using SELFs, including technical challenges and the final Python code, as well as demonstrating their use for the convection problem. To our knowledge this is the most in-depth guide to SELFs in the geosciences.

研究动机与目标

• 弥合大气神经网络训练（像素级损失）与评估（空间验证）之间的差距。
• 开发空间增强损失函数（SELFs），将空间验证原理直接整合至训练过程中。
• 在多种空间尺度下，对真实世界对流（雷暴）预测任务评估SELFs的性能。
• 为在地球科学应用中实现SELFs提供实用且开源的指南。
• 证明SELFs相较于标准像素级训练，可生成更校准的概率预报。

提出的方法

• 通过在空间验证评分（如Brier评分）上应用邻域或光谱滤波器（傅里叶/小波），设计SELFs以实现空间增强。
• 利用空间滤波器突出特定尺度的对流——通过邻域滤波强调较小尺度，通过光谱分解强调中到大尺度。
• 通过将标准损失函数中的像素级比较替换为经空间滤波、具备尺度感知能力的验证评分，对损失函数进行修改。
• 使用不同的SELFs训练神经网络，并在多种对流强度阈值和空间尺度下比较其性能。
• 实现并发布SELFs的完整Python代码，包含数值稳定性与计算效率的技术考量。
• 基于观测数据和再分析数据，在真实世界对流预测任务中验证该方法的有效性。

实验结果

研究问题

• RQ1 空间验证方法能否在大气科学的神经网络损失函数中有效集成？
• RQ2 不同空间滤波器（邻域 vs. 光谱）在不同对流尺度下如何影响模型性能？
• RQ3 与标准像素级损失相比，使用SELFs训练是否能带来更校准的概率预报？
• RQ4 在对流预测中，针对不同风险阈值，SELFs的最佳尺度聚焦为何？
• RQ5 SELFs能否在地球科学应用中实现实际部署并具备良好的泛化能力？

主要发现

• 对于低风险阈值，理想的SELF聚焦于小尺度对流；而对于高风险阈值，则更偏好大尺度特征。
• 尽管与空间验证存在脱节，使用像素级损失函数训练的模型在许多指标上表现仍出人意料地好。
• 基于光谱滤波的SELFs在中到大尺度上显著提升了概率预报的校准性，优于像素级训练。
• 使用光谱滤波器（傅里叶或小波）相比固定邻域滤波器，能实现更灵活且具备尺度自适应性的空间增强。
• 所提出的SELF框架具备可扩展性，并为地球科学应用提供了实用且开源的实现方案。
• SELFs在多个空间尺度上提升了预报的可靠性，从孤立雷暴单体到热带气旋，均通过空间验证评分得到验证。