[Paper Review] Sharp Spectral Rates for Koopman Operator Learning

Vladimir R. Kostic, Karim Lounici+2 more|arXiv (Cornell University)|Feb 3, 2023

Model Reduction and Neural Networks4 citations

TL;DR

This paper presents the first non-asymptotic learning bounds for Koopman operator eigenvalues and eigenfunctions using Extended Dynamic Mode Decomposition (EDMD) and Reduced Rank Regression (RRR). It introduces metric distortion as a critical factor alongside operator norm error, revealing that EDMD incurs higher bias than RRR, which explains its slower learning rates and susceptibility to spurious eigenvalues in practice.

ABSTRACT

Nonlinear dynamical systems can be handily described by the associated Koopman operator, whose action evolves every observable of the system forward in time. Learning the Koopman operator and its spectral decomposition from data is enabled by a number of algorithms. In this work we present for the first time non-asymptotic learning bounds for the Koopman eigenvalues and eigenfunctions. We focus on time-reversal-invariant stochastic dynamical systems, including the important example of Langevin dynamics. We analyze two popular estimators: Extended Dynamic Mode Decomposition (EDMD) and Reduced Rank Regression (RRR). Our results critically hinge on novel {minimax} estimation bounds for the operator norm error, that may be of independent interest. Our spectral learning bounds are driven by the simultaneous control of the operator norm error and a novel metric distortion functional of the estimated eigenfunctions. The bounds indicates that both EDMD and RRR have similar variance, but EDMD suffers from a larger bias which might be detrimental to its learning rate. Our results shed new light on the emergence of spurious eigenvalues, an issue which is well known empirically. Numerical experiments illustrate the implications of the bounds in practice.

Motivation & Objective

• To establish non-asymptotic learning bounds for Koopman eigenvalues and eigenfunctions in time-reversal-invariant stochastic dynamical systems.
• To analyze the statistical performance of EDMD and RRR estimators in terms of spectral estimation error.
• To identify the causes of spurious eigenvalues in Koopman operator learning, particularly in relation to bias and metric distortion.
• To provide a theoretical framework for detecting spurious eigenvalues from data using novel error bounds.
• To derive minimax optimal operator norm error bounds for finite-rank Koopman operators.

Proposed method

• Introduces a novel metric distortion functional to quantify norm changes of eigenfunctions between the RKHS and the ambient $L^2_\pi$ space.
• Derives sharp non-asymptotic bounds for the operator norm error of Koopman operator estimators, achieving minimax optimality for finite-rank operators.
• Uses perturbation theory and spectral decomposition to bound eigenvalue and eigenvector estimation errors in terms of operator norm error and metric distortion.
• Applies the bounds to both EDMD (via PCR) and RRR, showing that RRR has lower bias and better learning rates.
• Proposes a data-driven detection method for spurious eigenvalues based on the spectral bound in Theorem 4.
• Employs concentration inequalities and random matrix theory to control the error in high-dimensional, finite-sample settings.

Experimental results

Research questions

• RQ1 How do the spectral estimation errors of EDMD and RRR compare in terms of bias and variance?
• RQ2 What role does metric distortion play in the accuracy of Koopman eigenfunction estimation?
• RQ3 Why do spurious eigenvalues emerge in Koopman operator learning despite small operator norm error?
• RQ4 Can the proposed bounds be used to detect spurious eigenvalues from finite data?
• RQ5 What are the sharp, non-asymptotic learning rates for Koopman eigenvalues and eigenfunctions under general regularity conditions?

Key findings

• EDMD exhibits larger bias than RRR, leading to slower spectral learning rates despite similar variance.
• The operator norm error is minimized in a minimax-optimal way, with bounds scaling as $O(n^{-\frac{\alpha}{2(\alpha+\beta)}})$ under regularity conditions.
• Metric distortion is essential for spectral error control; ignoring it leads to misleadingly small operator norm errors even with spurious eigenvalues.
• Spurious eigenvalues arise when the estimated eigenvalues are not aligned with true Koopman eigenvalues, even when operator norm error is small—this is explained by high metric distortion and bias.
• Numerical experiments on Langevin dynamics confirm that RRR achieves lower eigenvalue and eigenfunction errors than EDMD, with decay rates of approximately $n^{-0.35}$ for eigenvalues and $n^{-0.45}$ for eigenfunctions.
• The proposed detection method based on Theorem 4 successfully identifies spurious eigenvalues in the Alanine dipeptide dataset using kernel selection and error thresholding.