[Paper Review] Forecasting Sequential Data using Consistent Koopman Autoencoders

Omri Azencot, N. Benjamin Erichson+2 more|arXiv (Cornell University)|Mar 4, 2020

Model Reduction and Neural Networks44 references31 citations

TL;DR

This paper proposes a Consistent Koopman Autoencoder (PCL) that learns forward and backward Koopman dynamics in a latent space to forecast high-dimensional time series with improved accuracy and stability, even under noise.

ABSTRACT

Recurrent neural networks are widely used on time series data, yet such models often ignore the underlying physical structures in such sequences. A new class of physics-based methods related to Koopman theory has been introduced, offering an alternative for processing nonlinear dynamical systems. In this work, we propose a novel Consistent Koopman Autoencoder model which, unlike the majority of existing work, leverages the forward and backward dynamics. Key to our approach is a new analysis which explores the interplay between consistent dynamics and their associated Koopman operators. Our network is directly related to the derived analysis, and its computational requirements are comparable to other baselines. We evaluate our method on a wide range of high-dimensional and short-term dependent problems, and it achieves accurate estimates for significant prediction horizons, while also being robust to noise.

Motivation & Objective

• Motivate time-series forecasting with physics-informed structure rather than purely data-driven RNNs.
• Introduce a Consistent Koopman Autoencoder (PCL) that enforces forward/backward dynamics in latent space.
• Provide a theoretical and empirical framework linking consistency of latent dynamics to Koopman operators.
• Demonstrate efficiency and robustness on high-dimensional, noisy, and short-term dependent datasets.

Proposed method

• Encode observations into a low-dimensional latent space with an autoencoder (encoder chi_e and decoder chi_d).
• Model forward and backward dynamics in latent space via linear Koopman operators C and D without biases.
• Train with a loss that combines reconstruction, forward prediction, backward prediction, and forward-backward consistency penalties.
• Enforce consistency through a Koopman-based penalty E_con that couples C and D to approximate inverse relations.
• Allow multi-step forecasting by propagating powers of C and D and using Chi to map back to observation space.
• Provide theoretical conditions for continuous and discrete consistency of forward/backward dynamics and relate them to C, D (Eqs. 12, 15 and Prop. 1–2).

Experimental results

Research questions

• RQ1 How can forward and backward Koopman dynamics be learned in a single architecture for time-series forecasting?
• RQ2 What regularization/consistency constraints ensure stable and invertible-like behavior between forward and backward maps?
• RQ3 Can a physics-constrained learning (PCL) framework improve forecasting accuracy and robustness on high-dimensional datasets compared to existing Koopman-based nets?
• RQ4 How do continuous vs discrete consistency conditions translate into practical training penalties?
• RQ5 What is the impact of including backward dynamics on prediction quality and stability under noise?

Key findings

• The Consistent Koopman Autoencoder (PCL) achieves accurate long-horizon forecasts on high-dimensional and noisy time-series.
• Joint Forward (C) and Backward (D) Koopman operators with a consistency penalty improve stability and predictions compared to forward-only baselines.
• Including backward dynamics and consistency constraints yields more robust forecasts under noise and across different dynamical regimes.
• Stability is improved via eigenvalue regularization, yielding more unit-circle-like spectra for the learned operators.
• Empirical results on pendulum, cylinder flow, curved-domain vortex flow, and climate data demonstrate strong performance relative to Dynamic AE (DAE) baselines and RNNs/FFNs.