[Paper Review] Prediction of Extreme Events in Multiscale Simulations of Geophysical Turbulence using Reinforcement Learning

Yifei Guan, Lucas Amoudruz+5 more|arXiv (Cornell University)|Feb 26, 2026

Meteorological Phenomena and Simulations0 citations

TL;DR

The paper develops a SMARL-based framework to learn subgrid-scale closures for geophysical turbulence using enstrophy spectrum as reward, enabling stable LES that reproduce extreme-event statistics with orders-of-magnitude fewer training samples.

ABSTRACT

Accurate subgrid-scale closures are essential for weather/climate models, where predicting extreme events is critical. Traditional closures have structural errors, e.g., producing excessive diffusion that dampens extremes. Artificial intelligence has gained attention for closure modeling, but the prediction of extreme events remains challenging. Supervised offline learning needs abundant high-fidelity training data and can lead to instabilities. Online learning algorithms are emerging as an alternative, but reliance on differentiable numerical solvers or scalable optimizers hinders broad use. Here, we introduce SMARL to develop closures for canonical prototypes of atmospheric/oceanic turbulence, using only the enstrophy spectrum, estimated from a few high-fidelity samples, as reward. This reward ensures that the model captures the cascades of scales in these simulations. These online-learned closures enable stable simulations, with up to five orders of magnitude fewer degrees of freedom, that reproduce high-fidelity simulation statistics and capture in particular extremes. We interpret the closures by analyzing the SMARL policy and demonstrate generalization to other flows. The results highlight SMARL as a potent tool for developing closures capable of capturing extremes in atmospheric/oceanic flows, opening new capabilities for effective climate modeling.

Motivation & Objective

• Motivate accurate prediction of extreme events in weather/climate models at affordable computational cost.
• Develop online, data-efficient closures for LES that reduce reliance on high-fidelity data.
• Propose a SMARL framework to learn Leith-type closures for geophysical turbulence.
• Demonstrate stability and accuracy of LES with learned closures across multiple cases and Reynolds numbers.

Proposed method

• Use SMARL to learn a dynamic Leith closure coefficient c_l as a function of the enstrophy spectrum up to the LES cutoff, c_l = f_DNN(hatZ(k,t)).
• Deploy n_Ax × n_Ay agents distributed on the LES grid that output c_l via a shared policy.
• Define the state as the enstrophy spectrum hatZ_LES up to k_c and the action as the local c_l, interpolated to the grid.
• Train with online interactions between agents and a low-resolution LES solver, without requiring a differentiable solver.
• Optimize a reward r(t) proportional to 1 / ||log(hatZ DNS) - log(hatZ LES)||, guiding spectrum matching.
• Evaluate against DNS and traditional SGS models by comparing kinetic energy spectra and vorticity PDFs.

Experimental results

Research questions

• RQ1 Can SMARL learn SGS closures for geophysical turbulence that reproduce DNS statistics using very few high-fidelity samples?
• RQ2 Do SMARL closures capture both diffusion and backscatter to accurately represent interscale transfers?
• RQ3 Is the learned closure robust and generalizable to higher Reynolds numbers and different flow regimes?
• RQ4 How does the SMARL-based closure compare to dynamic Leith and Smagorinsky closures in extreme-event statistics?

Key findings

• RL-Leith closures outperform DSmag and DLeith in matching DNS vorticity PDFs, including tails that represent extreme events.
• RL-Leith better reproduces interscale enstrophy transfer, indicating improved backscattering and reduced excessive diffusion.
• The learned c_l distribution spans a wider range and includes negative values, enabling both diffusion and backscatter modeling.
• Sobol analysis shows the enstrophy at low wavenumbers and near the cutoff k_c most influence the closure, aligning with physical transfer regions.
• A SMARL closure trained at lower Re generalizes to a case with 15× higher Re, retaining improved spectra and PDF tails compared to baselines.