[Paper Review] Machine learning prediction of critical transition and system collapse

Ling-Wei Kong, Huawei Fan+2 more|arXiv (Cornell University)|Dec 2, 2020

Neural Networks and Reservoir Computing1 references4 citations

TL;DR

This paper proposes a model-free, machine learning framework using parameter-aware reservoir computing to predict critical transitions and system collapse in nonlinear dynamical systems. By incorporating the bifurcation parameter as an input channel, the method accurately predicts the critical transition point and average transient lifetime before collapse, even when trained only on data from the stable, chaotic regime.

ABSTRACT

To predict a critical transition due to parameter drift without relying on model is an outstanding problem in nonlinear dynamics and applied fields. A closely related problem is to predict whether the system is already in or if the system will be in a transient state preceding its collapse. We develop a model free, machine learning based solution to both problems by exploiting reservoir computing to incorporate a parameter input channel. We demonstrate that, when the machine is trained in the normal functioning regime with a chaotic attractor (i.e., before the critical transition), the transition point can be predicted accurately. Remarkably, for a parameter drift through the critical point, the machine with the input parameter channel is able to predict not only that the system will be in a transient state, but also the average transient time before the final collapse.

Motivation & Objective

• To develop a model-free, data-driven method for predicting critical transitions and system collapse in nonlinear dynamical systems without relying on known system equations.
• To address the challenge of detecting when a system has already entered a transient chaotic state preceding inevitable collapse, especially when traditional methods fail to distinguish it from sustained chaos.
• To enable early prediction of the critical parameter value at which a system transitions into transient chaos, allowing for timely intervention before collapse.
• To predict the average lifetime of transient chaos after the critical point, providing a quantitative estimate of system survival time.
• To demonstrate the method's effectiveness on real-world systems such as electrical power grids and ecological food chains where catastrophic collapse is preceded by transient chaos.

Proposed method

• Adapt reservoir computing by adding a dedicated input channel for the bifurcation parameter, enabling the network to learn the dependence of system dynamics on parameter variations.
• Train the reservoir machine exclusively on time series data collected from the normal, chaotic regime (before the critical transition), using multiple distinct parameter values (e.g., K = 0.97, 0.98, 0.99).
• Use the trained reservoir to predict the critical transition point by identifying the parameter value at which the system's behavior shifts from sustained chaos to transient chaos.
• Estimate the average transient lifetime by analyzing the predicted dynamics after the critical point, using ensemble averaging over multiple reservoir realizations.
• Leverage the reservoir's ability to generalize beyond the training parameter range, enabling prediction of critical transitions and transient lifetimes even for parameter values not used during training.
• Validate predictions using return maps and transient lifetime distributions, comparing predicted outcomes with true values from numerical simulations.

Experimental results

Research questions

• RQ1 Can a machine learning model trained only on data from the stable, chaotic regime predict the critical parameter value at which a system undergoes a catastrophic transition to transient chaos?
• RQ2 Can the model accurately predict the average duration of transient chaos before system collapse, even without access to post-transition data?
• RQ3 How well can the reservoir machine distinguish between sustained chaos and transient chaos when both exhibit similar observable behaviors?
• RQ4 To what extent can the model generalize to parameter values beyond the training range, and how does prediction accuracy depend on training proximity to the critical point?
• RQ5 Can this framework be applied to real-world systems with complex, non-sparse dynamics, such as power systems and ecological food chains, where system equations are unknown?

Key findings

• The reservoir machine trained at K = 0.97, 0.98, and 0.99 predicted the critical transition point in the three-species food chain model as K*c = 0.9997 ± 4×10⁻⁴, closely matching the true value of Kc = 0.99976.
• The predicted average transient lifetime at K = Kc + 2×10⁻⁴ was 1.35×10³, in excellent agreement with the true value of 1.33×10³.
• The method successfully predicted the critical transition and transient lifetime in an electrical power system susceptible to voltage collapse via transient chaos.
• The framework demonstrated robustness in predicting system behavior beyond the training parameter range, with prediction accuracy improving when training points were closer to the critical point.
• Ensemble averaging over 500 reservoir realizations significantly reduced prediction error, showing that collective predictions are more accurate than individual ones.
• The method provides a reliable early warning system for systems already in transient chaos, enabling detection of inevitable collapse even when no post-critical data are available.