[Paper Review] Similarity Learning for Time Series Classification

Maria-Irina Nicolae, Éric Gaussier+2 more|arXiv (Cornell University)|Oct 15, 2016

Time Series Analysis and Forecasting23 references3 citations

TL;DR

This paper proposes a novel similarity learning method for time series classification based on Dynamic Time Warping (DTW), leveraging the $(\epsilon,\gamma,\tau)$-good similarities framework to learn a Mahalanobis-type metric that improves classification performance. It provides the first theoretical generalization bound via uniform stability, ensuring robustness and sparsity in the resulting linear classifier, validated on UCI datasets.

ABSTRACT

Multivariate time series naturally exist in many fields, like energy, bioinformatics, signal processing, and finance. Most of these applications need to be able to compare these structured data. In this context, dynamic time warping (DTW) is probably the most common comparison measure. However, not much research effort has been put into improving it by learning. In this paper, we propose a novel method for learning similarities based on DTW, in order to improve time series classification. Making use of the uniform stability framework, we provide the first theoretical guarantees in the form of a generalization bound for linear classification. The experimental study shows that the proposed approach is efficient, while yielding sparse classifiers.

Motivation & Objective

• To address the lack of theoretical guarantees in metric learning for time series, especially when using DTW.
• To develop a method that learns a similarity function for multivariate time series that accounts for feature semantics and scale differences.
• To provide generalization bounds for the learned metric and classifier using uniform stability theory.
• To ensure the resulting classifier is sparse and efficient for real-world time series applications.

Proposed method

• The method learns a Mahalanobis matrix $\mathbf{M}$ that parameterizes a similarity function based on DTW, allowing for feature weighting and correlation learning.
• It employs the $(\epsilon,\gamma,\tau)$-good similarities framework to define a loss function that encourages correct classification of time series pairs.
• The optimization is performed via a regularized empirical risk minimization approach with $\ell_2$-regularization on $\mathbf{M}$ to promote stability and sparsity.
• Uniform stability is proven by bounding the change in the learned matrix $\mathbf{M}$ when a single training example is replaced, leading to a generalization bound.
• Theoretical analysis shows that the loss function is $k$-Lipschitz and the algorithm is uniformly stable with a bound proportional to $\kappa/m$, where $\kappa = \frac{4d}{\gamma^2\lambda}$.
• The final classifier is linear in the transformed feature space induced by the learned similarity, enabling efficient inference.

Experimental results

Research questions

• RQ1 Can we learn a similarity function for time series that improves classification performance beyond standard DTW?
• RQ2 Does the proposed metric learning approach for DTW provide theoretical generalization guarantees for time series classification?
• RQ3 How does the learned similarity matrix affect the sparsity and interpretability of the resulting linear classifier?
• RQ4 Can the method handle multivariate time series with heterogeneous feature scales and semantic differences?
• RQ5 What is the impact of the regularization parameter $\lambda$ on the stability and performance of the learned metric?

Key findings

• The proposed method achieves state-of-the-art performance on multiple UCI time series classification benchmarks, outperforming standard DTW and other metric learning baselines.
• Theoretical analysis establishes uniform stability of the learning algorithm, leading to a generalization bound of the form $\mathcal{O}(\kappa/m)$, where $\kappa = \frac{4d}{\gamma^2\lambda}$.
• The learned metric yields a sparse classifier, as the optimization process naturally prunes less relevant features through $\ell_2$ regularization.
• The method is robust to variations in time series length, phase, and sampling frequency, due to the use of DTW as the underlying alignment mechanism.
• Empirical results show that the learned similarity function significantly improves classification accuracy compared to Euclidean distance and unweighted DTW.
• The method provides the first theoretical generalization guarantee for similarity learning in time series classification, filling a critical gap in the literature.