[Paper Review] Reservoir computing approaches for representation and classification of multivariate time series

Filippo Maria Bianchi, Simone Scardapane+2 more|arXiv (Cornell University)|Mar 21, 2018

Neural Networks and Reservoir Computing4 citations

TL;DR

This paper proposes a novel reservoir model space representation for multivariate time series (MTS) classification using reservoir computing (RC), where MTS are encoded via parameters of a model predicting future reservoir states rather than future inputs. The method achieves state-of-the-art accuracy with drastically faster training than deep learning models, enabled by a low-dimensional embedding of reservoir dynamics through a tensor-PCA variant for efficient, regularized representation learning.

ABSTRACT

Classification of multivariate time series (MTS) has been tackled with a large variety of methodologies and applied to a wide range of scenarios. Reservoir Computing (RC) provides efficient tools to generate a vectorial, fixed-size representation of the MTS that can be further processed by standard classifiers. Despite their unrivaled training speed, MTS classifiers based on a standard RC architecture fail to achieve the same accuracy of fully trainable neural networks. In this paper we introduce the reservoir model space, an unsupervised approach based on RC to learn vectorial representations of MTS. Each MTS is encoded within the parameters of a linear model trained to predict a low-dimensional embedding of the reservoir dynamics. Compared to other RC methods, our model space yields better representations and attains comparable computational performance, thanks to an intermediate dimensionality reduction procedure. As a second contribution we propose a modular RC framework for MTS classification, with an associated open-source Python library. The framework provides different modules to seamlessly implement advanced RC architectures. The architectures are compared to other MTS classifiers, including deep learning models and time series kernels. Results obtained on benchmark and real-world MTS datasets show that RC classifiers are dramatically faster and, when implemented using our proposed representation, also achieve superior classification accuracy.

Motivation & Objective

• Address the gap in accuracy of standard RC-based MTS classifiers despite their fast training.
• Overcome the limitation of existing model-space approaches that focus on predicting future inputs, which may miss key dynamical information.
• Develop a representation that captures the full dynamical system underlying MTS by modeling future reservoir states instead.
• Design a unified, modular RC framework for MTS classification with open-source Python support.
• Demonstrate that the proposed representation enables RC classifiers to match or exceed deep learning models in accuracy while maintaining training speed advantages.

Proposed method

• Introduce the reservoir model space: a representation derived from the parameters of a one-step-ahead predictor that forecasts the next reservoir state, not the next input.
• Apply a modified tensor-based PCA to reduce the dimensionality of reservoir state sequences, separating variation across time steps and data samples.
• Use the low-dimensional embedding of reservoir dynamics as input to a linear classifier, enabling efficient and regularized representation learning.
• Design a modular RC framework with four independent components: reservoir architecture, dimensionality reduction, representation method, and readout classifier.
• Implement and evaluate multiple RC architectures using the proposed framework, including bidirectional reservoirs and deep readouts.
• Train the predictor model in an unsupervised manner on the embedded reservoir dynamics to generate fixed-size, informative representations.

Experimental results

Research questions

• RQ1 Can modeling future reservoir states instead of future inputs yield more informative representations for MTS classification?
• RQ2 Does the proposed reservoir model space representation improve classification accuracy compared to standard RC and model-space baselines?
• RQ3 To what extent does dimensionality reduction of reservoir dynamics enhance computational efficiency and generalization in RC-based MTS classifiers?
• RQ4 How do advanced RC architectures (e.g., bidirectional reservoirs, deep readouts) interact with the proposed representation in terms of accuracy and speed?
• RQ5 Can the proposed method achieve competitive performance on univariate time series classification, despite being designed for multivariate data?

Key findings

• The reservoir model space representation, based on predicting future reservoir states, yields significantly more informative features than standard model-space methods that predict future inputs.
• On benchmark MTS datasets, the proposed method achieves state-of-the-art classification accuracy, with mean F1-scores exceeding 95% on several datasets including UCR-UEA and ItalyPower.
• The method is dramatically faster than deep learning baselines, with training times in seconds—e.g., rmESN model trained in under 10 seconds on most datasets.
• The use of tensor-PCA for dimensionality reduction reduces memory usage and prevents overfitting, especially when combined with complex discriminative classifiers.
• Even on univariate time series, the proposed RC classifier (rmESN with model space) achieves competitive accuracy, outperforming or matching complex deep learning models like LSTM-FCN on average.
• Adding bidirectional reservoirs or deep readouts provides only marginal accuracy gains when combined with the reservoir model space, indicating the representation is already highly informative.