[Paper Review] Adaptive learning of effective dynamics: Adaptive real-time, online modeling for complex systems

Ivica Kičić, Pantelis R. Vlachas+4 more|arXiv (Cornell University)|Apr 4, 2023

Model Reduction and Neural Networks4 citations

TL;DR

AdaLED is a novel adaptive, online learning framework that accelerates complex multiscale simulations by dynamically switching between a high-fidelity computational solver and a learned surrogate model. It uses an autoencoder for dimensionality reduction and an ensemble of probabilistic RNNs as a latent time-stepper, continuously adapting to new dynamics through online training while maintaining accuracy and achieving up to 2.9x net speed-up on Navier-Stokes flows across varying Reynolds numbers.

ABSTRACT

Predictive simulations are essential for applications ranging from weather forecasting to material design. The veracity of these simulations hinges on their capacity to capture the effective system dynamics. Massively parallel simulations predict the systems dynamics by resolving all spatiotemporal scales, often at a cost that prevents experimentation. On the other hand, reduced order models are fast but often limited by the linearization of the system dynamics and the adopted heuristic closures. We propose a novel systematic framework that bridges large scale simulations and reduced order models to extract and forecast adaptively the effective dynamics (AdaLED) of multiscale systems. AdaLED employs an autoencoder to identify reduced-order representations of the system dynamics and an ensemble of probabilistic recurrent neural networks (RNNs) as the latent time-stepper. The framework alternates between the computational solver and the surrogate, accelerating learned dynamics while leaving yet-to-be-learned dynamics regimes to the original solver. AdaLED continuously adapts the surrogate to the new dynamics through online training. The transitions between the surrogate and the computational solver are determined by monitoring the prediction accuracy and uncertainty of the surrogate. The effectiveness of AdaLED is demonstrated on three different systems - a Van der Pol oscillator, a 2D reaction-diffusion equation, and a 2D Navier-Stokes flow past a cylinder for varying Reynolds numbers (400 up to 1200), showcasing its ability to learn effective dynamics online, detect unseen dynamics regimes, and provide net speed-ups. To the best of our knowledge, AdaLED is the first framework that couples a surrogate model with a computational solver to achieve online adaptive learning of effective dynamics. It constitutes a potent tool for applications requiring many expensive simulations.

Motivation & Objective

• Address the high computational cost of large-scale simulations for complex systems such as fluid flows and material dynamics.
• Overcome limitations of traditional reduced-order models that rely on linearization or heuristic closures.
• Develop a framework that enables continuous, online adaptation of surrogate models to unseen dynamic regimes during simulation.
• Achieve significant computational speed-up while maintaining predictive accuracy in multiscale systems.
• Enable real-time, adaptive coupling between high-fidelity solvers and data-driven surrogates for dynamic system modeling.

Proposed method

• Employ a convolutional autoencoder to learn a low-dimensional, effective representation of the system's state from high-dimensional data.
• Use an ensemble of probabilistic recurrent neural networks (RNNs) as a latent time-stepper to propagate the system dynamics in the reduced space.
• Dynamically switch between the high-fidelity computational solver and the surrogate model based on real-time prediction uncertainty and accuracy monitoring.
• Continuously retrain the surrogate model online using newly generated data from the computational solver to adapt to new dynamic regimes.
• Define transition criteria between solver and surrogate based on prediction error and uncertainty estimates to balance speed and accuracy.
• Integrate the framework within a hybrid simulation loop that alternates between micro-scale (solver) and macro-scale (surrogate) propagation.

Experimental results

Research questions

• RQ1 Can a surrogate model be trained and adapted online during simulation to capture evolving dynamics in complex multiscale systems?
• RQ2 How can the trade-off between computational speed-up and predictive accuracy be effectively managed in real time?
• RQ3 Can a single adaptive surrogate model generalize across vastly different dynamical regimes without catastrophic forgetting?
• RQ4 What criteria enable reliable, dynamic switching between high-fidelity solvers and learned surrogates during simulation?
• RQ5 To what extent can online learning improve the robustness and efficiency of reduced-order modeling in fluid dynamics and other complex systems?

Key findings

• AdaLED achieved a net speed-up of 2.9x in a 3-day simulation of 2D Navier-Stokes flow past a cylinder across Reynolds numbers from 400 to 1200.
• The model maintained a mean square error of only ~1% on the velocity field and a root mean square error of ~5% on the cylinder force, with a cross-correlation of 0.99.
• The framework successfully learned and adapted to previously unseen dynamic regimes without requiring retraining or reinitialization.
• The surrogate model demonstrated robustness to distributional shifts and avoided catastrophic forgetting during online adaptation.
• The online training mechanism enabled continuous improvement of the surrogate as new data from the solver were integrated.
• The framework provides a tunable trade-off between speed-up and accuracy by adjusting thresholds for prediction uncertainty and error.