[Paper Review] Classification of Time-Series Images Using Deep Convolutional Neural Networks

Nima Hatami, Yann Gavet+1 more|arXiv (Cornell University)|Oct 2, 2017

Time Series Analysis and Forecasting21 references89 citations

TL;DR

This paper encodes time-series as recurrence-plot images and uses a two-stage CNN to jointly learn features and a classifier for time-series classification, reporting competitive results on the UCR archive.

ABSTRACT

Convolutional Neural Networks (CNN) has achieved a great success in image recognition task by automatically learning a hierarchical feature representation from raw data. While the majority of Time-Series Classification (TSC) literature is focused on 1D signals, this paper uses Recurrence Plots (RP) to transform time-series into 2D texture images and then take advantage of the deep CNN classifier. Image representation of time-series introduces different feature types that are not available for 1D signals, and therefore TSC can be treated as texture image recognition task. CNN model also allows learning different levels of representations together with a classifier, jointly and automatically. Therefore, using RP and CNN in a unified framework is expected to boost the recognition rate of TSC. Experimental results on the UCR time-series classification archive demonstrate competitive accuracy of the proposed approach, compared not only to the existing deep architectures, but also to the state-of-the art TSC algorithms.

Motivation & Objective

• Motivate time-series classification (TSC) and review traditional TSC approaches.
• Propose converting time-series to recurrence plot images to leverage CNNs.
• Develop a two-stage CNN architecture to learn features and perform classification end-to-end.
• Evaluate the approach on the UCR time-series dataset archive and compare with state-of-the-art methods.

Proposed method

• Transform 1D time-series into 2D recurrence texture images without binarizing (RP images).
• Use a two-stage CNN with 32(5)-2-32(5)-2-128-c architecture and ReLU activations.
• Train with stochastic gradient descent using categorical-crossentropy loss and Adam optimizer.
• Input image sizes are 28x28, 56x56 or 64x64 with 2 convolutional layers, 2 pooling layers, and a 128-neuron hidden layer.
• Compare performance against 1-NN DTW, Shapelet, BoP, SAX-VSM, TFRP, MCNN, and GAF-MTF baselines.
• Code is available and experiments use the UCR archive with 20 selected datasets.

Experimental results

Research questions

• RQ1 Can recurrence-plot based image representations enable CNNs to perform competitive TSC?
• RQ2 How does the RP+CNN pipeline compare to existing TSC methods and other CNN-based approaches on standard benchmarks?
• RQ3 What is the impact of RP-based encoding and CNN architecture choices on classification accuracy?

Key findings

• The RP+CNN approach achieves 10 wins out of 20 on the 20 selected UCR datasets.
• Average rank of the proposed method is 2.15, competitive with MCNN (2.36) and SAX-VSM (3.0).
• RP images with CNN outperform traditional RP-based SVM/textured descriptors (TFRP) and GAF-MTF CNN approaches.
• The study shows RP encoding yields more discriminative features for TSC than some alternative image-based encodings.
• The proposed pipeline demonstrates the importance of the encoding choice for time-series images in CNN-based TSC.
• CNN learns features and classifier jointly, improving performance over hand-crafted or separately trained features.