[Paper Review] Deep reconstruction of strange attractors from time series

William Gilpin|arXiv (Cornell University)|Feb 14, 2020

Neural dynamics and brain function77 references35 citations

TL;DR

This paper presents a neural autoencoder-based method with a false-nearest-neighbor latent-space regularizer to reconstruct high-dimensional attractors from low-dimensional time series, enabling consistent dynamics and forecasting across known and unknown systems.

ABSTRACT

Experimental measurements of physical systems often have a limited number of independent channels, causing essential dynamical variables to remain unobserved. However, many popular methods for unsupervised inference of latent dynamics from experimental data implicitly assume that the measurements have higher intrinsic dimensionality than the underlying system---making coordinate identification a dimensionality reduction problem. Here, we study the opposite limit, in which hidden governing coordinates must be inferred from only a low-dimensional time series of measurements. Inspired by classical analysis techniques for partial observations of chaotic attractors, we introduce a general embedding technique for univariate and multivariate time series, consisting of an autoencoder trained with a novel latent-space loss function. We show that our technique reconstructs the strange attractors of synthetic and real-world systems better than existing techniques, and that it creates consistent, predictive representations of even stochastic systems. We conclude by using our technique to discover dynamical attractors in diverse systems such as patient electrocardiograms, household electricity usage, neural spiking, and eruptions of the Old Faithful geyser---demonstrating diverse applications of our technique for exploratory data analysis.

Motivation & Objective

• Address partial observations by inferring hidden governing coordinates from low-dimensional time series.
• Develop a general embedding technique for univariate and multivariate series that recovers the attractor's structure.
• Introduce a sparsity-promoting latent-space regularizer to estimate embedding dimensionality close to the true system dimensionality.
• Demonstrate robustness to noise and applicability to diverse real-world datasets.
• Provide metrics to compare reconstructed attractors with original dynamics and show forecasting capability.

Proposed method

• Train a stacked autoencoder on a Hankel matrix derived from time series to learn latent coordinates of the attractor.
• Use a novel latent-space loss, the false-nearest-neighbor loss, to promote embeddings with dimensionality close to the true attractor dimension (d_E ≈ d).
• Represent the loss as a batch-averaged false-nearest-neighbors fraction across latent units, weighting latent activity to discourage unnecessary dimensions.
• Align reconstructions with the original attractor via Procrustes analysis and evaluate using multiple similarity metrics (Euclidean distance, DTW, forecasting via cross-mapping, neighbor accuracy, dimensionality similarity, topological and fractal measures).
• Experiment with LSTM and MLP encoders on various chaotic and real-world datasets, comparing to baselines like ETD, tICA, and unregularized autoencoders.

Experimental results

Research questions

• RQ1 Can a latent-space regularized autoencoder reconstruct unseen higher-dimensional dynamical coordinates from a univariate or low-dimensional time series?
• RQ2 Does the false-nearest-neighbor regularizer yield embeddings with appropriate intrinsic dimensionality and better attractor similarity than baseline methods?
• RQ3 How robust is the method to noise and non-stationarity, and can it forecast long-horizon dynamics?
• RQ4 Do embeddings reveal consistent attractors in diverse real-world datasets (ECG, geyser eruptions, electricity usage, neuronal spiking)?

Key findings

• The false-nearest-neighbor regularizer consistently improves attractor similarity across datasets compared to baselines.
• Embedding dimensionality accuracy (S_dim) improves with regularization and remains robust across regimes, with d_E ≈ d.
• Regularized models show slower decay of forecast accuracy under noise and longer horizons than non-regularized ones.
• Embeddings of unknown datasets reveal interpretable attractor structures, such as nested loops in ECG and quasi-periodic torus in geyser data.
• The approach remains effective for both low- and high-dimensional chaotic systems and enables exploratory analysis of real-world time series.