[Paper Review] Koopa: Learning Non-stationary Time Series Dynamics with Koopman Predictors

Yong Liu, Chenyu Li+2 more|arXiv (Cornell University)|May 30, 2023

Model Reduction and Neural Networks37 citations

TL;DR

Koopa introduces modular Koopman Predictors with Fourier Filter to disentangle time-invariant and time-variant dynamics for non-stationary time series forecasting, achieving competitive accuracy with substantial efficiency gains.

ABSTRACT

Real-world time series are characterized by intrinsic non-stationarity that poses a principal challenge for deep forecasting models. While previous models suffer from complicated series variations induced by changing temporal distribution, we tackle non-stationary time series with modern Koopman theory that fundamentally considers the underlying time-variant dynamics. Inspired by Koopman theory of portraying complex dynamical systems, we disentangle time-variant and time-invariant components from intricate non-stationary series by Fourier Filter and design Koopman Predictor to advance respective dynamics forward. Technically, we propose Koopa as a novel Koopman forecaster composed of stackable blocks that learn hierarchical dynamics. Koopa seeks measurement functions for Koopman embedding and utilizes Koopman operators as linear portraits of implicit transition. To cope with time-variant dynamics that exhibits strong locality, Koopa calculates context-aware operators in the temporal neighborhood and is able to utilize incoming ground truth to scale up forecast horizon. Besides, by integrating Koopman Predictors into deep residual structure, we ravel out the binding reconstruction loss in previous Koopman forecasters and achieve end-to-end forecasting objective optimization. Compared with the state-of-the-art model, Koopa achieves competitive performance while saving 77.3% training time and 76.0% memory.

Motivation & Objective

• Disentangle time-variant and time-invariant dynamics in non-stationary time series using Koopman theory.
• Develop a modular, hierarchical Koopman-based forecaster (Koopa) with Fourier Filter to separate dynamics.
• Enable end-to-end forecasting by integrating Koopman Predictors into a deep residual architecture.
• Improve forecast horizon scalability by leveraging globally shared and locally adapted operators.

Proposed method

• Use Fourier Filter to split input into time-invariant (X_inv) and time-variant (X_var) components.
• Implement Time-invariant KP to learn a global Koopman embedding and linear operator (K_inv).
• Implement Time-variant KP to compute localized operators (K_var) within lookback windows using segment-wise embeddings.
• Stack Koopa Blocks to learn hierarchical dynamics by feeding residuals to subsequent blocks and summing outputs.
• Train with an end-to-end MSE objective on Y and ground truth to avoid reconstruction losses common in Koopman Autoencoders.
• Enable scaling up forecast horizon by freezing learned components and adapting K_var with incoming ground-truth snapshots.

Experimental results

Research questions

• RQ1 Can Koopman-based predictors effectively separate and model time-invariant and time-variant dynamics in non-stationary time series?
• RQ2 Do modular, hierarchical Koopman Predictors improve forecasting performance and efficiency compared to state-of-the-art methods?
• RQ3 Can the model scale to longer forecast horizons by leveraging ground-truth-adaptive operator updates?
• RQ4 What is the impact of Fourier-based disentanglement on capturing varying temporal dependencies?
• RQ5 How does Koopa compare to existing Koopman-based forecasters and deep learning baselines on real-world datasets?

Key findings

• Koopa achieves competitive forecasting performance across six real-world multivariate benchmarks and shows strong univariate results on M4.
• Koopa delivers significant efficiency gains, saving substantial training time (average 22.7% across datasets) and memory (average 24.0%) versus PatchTST.
• Time-variant and time-invariant KP components are complementary; removing either degrades performance.
• Fourier Filter-based disentanglement reveals higher variation in time-variant components, supporting the design choice.
• A ground-truth-driven operator adaptation mechanism improves horizon scaling, especially on non-stationary data.