[Paper Review] Clustering Noisy Signals with Structured Sparsity Using Time-Frequency Representation

Tom Hope, Avishai Wagner+1 more|arXiv (Cornell University)|Oct 18, 2015

Time Series Analysis and Forecasting19 references3 citations

TL;DR

This paper introduces SPARCWave, a novel sparse clustering framework that leverages structured sparsity and time-frequency representations—specifically the scattering transform and wavelet-based multi-scale analysis—to improve clustering of low-SNR, noisy time-series signals. By jointly performing cross-signal smoothing and dimensionality reduction while preserving clustering-informative features, SPARCWave achieves higher accuracy than existing methods on real-world datasets including growth curves, phonemes, and wheat spectra.

ABSTRACT

We propose a simple and efficient time-series clustering framework particularly suited for low Signal-to-Noise Ratio (SNR), by simultaneous smoothing and dimensionality reduction aimed at preserving clustering information. We extend the sparse K-means algorithm by incorporating structured sparsity, and use it to exploit the multi-scale property of wavelets and group structure in multivariate signals. Finally, we extract features invariant to translation and scaling with the scattering transform, which corresponds to a convolutional network with filters given by a wavelet operator, and use the network's structure in sparse clustering. By promoting sparsity, this transform can yield a low-dimensional representation of signals that gives improved clustering results on several real datasets.

Motivation & Objective

• Address the challenge of clustering noisy, low-SNR time-series signals where traditional distance-based methods fail due to unreliable pairwise distances.
• Overcome limitations of single-signal denoising, which may discard clustering-relevant features by ignoring global cross-signal information.
• Develop a method that preserves clustering structure by integrating structured sparsity with time-frequency representations such as wavelets and scattering transforms.
• Achieve invariance to common signal transformations like translation, scaling, and small deformations, which degrade standard clustering performance.
• Improve clustering accuracy on real-world datasets by combining sparse K-means with multi-scale, group-sparse feature representations derived from the scattering transform.

Proposed method

• Apply the scattering transform to time-series signals, producing a multi-scale, time-frequency representation that is invariant to translation, dilation, and small deformations.
• Use the natural group structure of scattering coefficients (organized by scale and time) to enforce group sparsity in the sparse K-means framework.
• Integrate wavelet-based multi-resolution analysis to capture features across different scales, emphasizing coarse-scale coefficients that carry clustering-relevant information.
• Formulate a sparse clustering optimization problem that selects only the most informative wavelet and scattering coefficients, reducing dimensionality while preserving clustering structure.
• Leverage cross-signal information during coefficient selection to improve smoothing and feature extraction beyond individual signal denoising.
• Implement the method using sparse K-means with structured sparsity constraints, where non-zero weights are learned to emphasize features that best separate clusters.

Experimental results

Research questions

• RQ1 Can structured sparsity in time-frequency representations improve clustering performance for low-SNR time-series signals compared to standard sparse clustering?
• RQ2 To what extent does the scattering transform’s invariance to translation and deformation enhance clustering accuracy in noisy, real-world datasets?
• RQ3 Does incorporating global cross-signal information during coefficient selection outperform individual signal denoising in preserving clustering structure?
• RQ4 How does group sparsity over wavelet and scattering coefficients affect clustering accuracy across different signal types and noise levels?
• RQ5 Can the proposed framework achieve superior clustering results on diverse real datasets—such as growth curves, phoneme log-periodograms, and wheat spectra—compared to baseline methods?

Key findings

• SPARCWave achieved an Adjusted Rand Index (ARI) of 0.91 on the growth curve dataset, significantly outperforming standard K-means (ARI = 0.87) and other baselines.
• On the Phoneme dataset, SPARCWave achieved an ARI of 0.73, surpassing the next best method (Giacofci [14]) with an ARI of 0.69.
• For the Wheat spectra dataset, SPARCWave achieved an ARI of 0.46, outperforming all other methods, including Antoniadis [2] (ARI = 0.35) and Giacofci [14] (ARI = 0.30).
• The use of non-zero scattering features as a feature selection step improved clustering accuracy from 0.87 to 0.91 on the growth curve dataset, demonstrating the value of SPARCWave as a feature selection tool.
• Group-sparse clustering did not improve performance on the Phoneme and Wheat datasets, likely due to large sample sizes or limited informative features, but still maintained competitive results.
• The framework’s success is attributed to its ability to jointly smooth signals and reduce dimensionality using cross-signal information, preserving clustering structure even at low SNR.