[Paper Review] Achieving Conservation of Energy in Neural Network Emulators for Climate Modeling

Tom Beucler, Stephan Rasp+2 more|arXiv (Cornell University)|Jun 15, 2019

Meteorological Phenomena and Simulations9 references103 citations

TL;DR

The paper introduces two methods to enforce linear conservation laws in neural network emulators for climate models: constrain the loss function and constrain the network architecture, showing architecture constraints yield precise conservation and better generalization to climate change scenarios.

ABSTRACT

Artificial neural-networks have the potential to emulate cloud processes with higher accuracy than the semi-empirical emulators currently used in climate models. However, neural-network models do not intrinsically conserve energy and mass, which is an obstacle to using them for long-term climate predictions. Here, we propose two methods to enforce linear conservation laws in neural-network emulators of physical models: Constraining (1) the loss function or (2) the architecture of the network itself. Applied to the emulation of explicitly-resolved cloud processes in a prototype multi-scale climate model, we show that architecture constraints can enforce conservation laws to satisfactory numerical precision, while all constraints help the neural-network better generalize to conditions outside of its training set, such as global warming.

Motivation & Objective

• Motivate the use of data-driven neural emulators for cloud processes in climate models while addressing energy/mass conservation issues.
• Propose two methods to enforce physical constraints in neural networks: loss-function constraints and architecture-based conservation layers.
• Demonstrate the methods on convective parametrization within a prototype multi-scale climate model.
• Evaluate how constraints affect numerical conservation and generalization to warming scenarios.

Proposed method

• Formalize physical constraints as a linear system C[x; y] = 0 relating inputs and outputs.
• Train standard neural networks (NNU) with MSE as the objective.
• Introduce constrained-loss (NNL) by adding a penalty P = ||C[x; y_NN]|| to the loss with weight alpha.
• Introduce architecture-constrained networks (NNA) with n conservation layers that enforce C[x; y_NN] = 0 by solving for constrained output components during forward pass.
• Apply to emulation of cloud processes in a climate model, using a 5-layer, 512-node network with leaky ReLU activations, trained on 3 months of 30-minute climate data.
• Evaluate against two validation datasets: (+0K) and (+4K) ocean-world warming scenarios, focusing on conservation penalties and MSE.

Experimental results

Research questions

• RQ1 Can linear conservation laws be enforced in neural network emulators of physical climate processes?
• RQ2 How do loss-function penalties versus architectural constraint layers compare in preserving energy/mass and improving generalization under warming?
• RQ3 Does constraining architecture yield numerically precise conservation and better radiative-forcing predictions under climate perturbations?

Key findings

• Architecture-constrained networks (NNA) enforce conservation laws to numerical precision, yielding very small conservation penalties across validation sets.
• Constrained loss with balanced or small penalty weights can also improve generalization, sometimes outperforming unconstrained models on warming scenarios.
• All constrained approaches outperform a baseline multiple-linear regression model and reduce energy/mass/radiation conservation violations compared to the unconstrained network.
• Networks incorporating physical constraints generalize better to unseen warming conditions (+4K) than the unconstrained model, as evidenced by higher radiative-forecast fidelity (R^2 for outgoing longwave radiation).
• Using a very small penalty in the loss function (alpha ≈ 0.01) achieved best performance on the reference dataset while still preserving favorable generalization properties.