[Paper Review] Learning Koopman Invariant Subspaces for Dynamic Mode Decomposition

Naoya Takeishi, Yoshinobu Kawahara+1 more|arXiv (Cornell University)|Oct 12, 2017

Model Reduction and Neural Networks14 references78 citations

TL;DR

A fully data-driven method to learn Koopman invariant subspaces for dynamic mode decomposition using RSS loss and neural networks, enabling Koopman spectral analysis without predefined observables.

ABSTRACT

Spectral decomposition of the Koopman operator is attracting attention as a tool for the analysis of nonlinear dynamical systems. Dynamic mode decomposition is a popular numerical algorithm for Koopman spectral analysis; however, we often need to prepare nonlinear observables manually according to the underlying dynamics, which is not always possible since we may not have any a priori knowledge about them. In this paper, we propose a fully data-driven method for Koopman spectral analysis based on the principle of learning Koopman invariant subspaces from observed data. To this end, we propose minimization of the residual sum of squares of linear least-squares regression to estimate a set of functions that transforms data into a form in which the linear regression fits well. We introduce an implementation with neural networks and evaluate performance empirically using nonlinear dynamical systems and applications.

Motivation & Objective

• Motivate analysis of nonlinear dynamics via Koopman operator and DMD.
• Develop a fully data-driven approach to learn Koopman invariant subspaces from observed data.
• Avoid manual selection of observables by optimizing a regression-based loss.
• Incorporate state reconstruction and measurement back-projection to ensure physical relevance.
• Demonstrate empirical performance on nonlinear systems and applications.

Proposed method

• Define observables g that span a Koopman invariant subspace and minimize the residual sum of squares of a linear regression between Y1 and Y0.
• Use an empirical risk perspective with delay embedding to handle partial observations via a learned linear delay embedder ϕ.
• Introduce a reconstruction term to ensure y can be approximated from g, via a reconstructor h.
• Implement the framework with neural networks (MLPs) to model g and h and optimize via gradient descent.
• Offer a procedure to perform standard DMD on the learned observables by computing A = Y1 Y0† and extracting eigenpairs.
• Provide guidance on training with non-decomposable RSS loss and batch normalization for stability.

Experimental results

Research questions

• RQ1 Can we learn a set of observables that span a Koopman invariant subspace directly from data without hand-crafted features?
• RQ2 Does minimizing the RSS loss yield a finite-dimensional Koopman operator that accurately captures nonlinear dynamics?
• RQ3 Can delay embedding be learned jointly to reconstruct latent states from partial measurements?
• RQ4 Does incorporating a measurement reconstruction loss prevent trivial or degenerate solutions for g?
• RQ5 How does LKIS-DMD compare with existing DMD variants on nonlinear systems and practical applications?

Key findings

• LKIS-DMD identifies Koopman eigenvalues corresponding to the target invariant subspace and remains robust to observation noise.
• LKIS-DMD produces eigenvalues that align with theoretical expectations for fixed-point, limit-cycle, and multi-basin systems.
• The learned eigenfunctions can reveal basins of attraction and unstable phenomena via near-zero or small-magnitude eigenvalues.
• The method provides competitive or superior prediction accuracy for chaotic time-series compared with LSTM and Hankel DMD baselines.
• A reconstruction term enables physically meaningful back-projection from learned observables to original measurements.