[Paper Review] Data augmentation using synthetic data for time series classification with deep residual networks

Hassan Ismail Fawaz, Germain Forestier+3 more|arXiv (Cornell University)|Aug 7, 2018

Time Series Analysis and Forecasting15 references73 citations

TL;DR

The paper applies a DTW-based DBA-weighted data augmentation to augment time series for a ResNet in time series classification, showing large gains on some small datasets and improved robustness when combined in an ensemble.

ABSTRACT

Data augmentation in deep neural networks is the process of generating artificial data in order to reduce the variance of the classifier with the goal to reduce the number of errors. This idea has been shown to improve deep neural network's generalization capabilities in many computer vision tasks such as image recognition and object localization. Apart from these applications, deep Convolutional Neural Networks (CNNs) have also recently gained popularity in the Time Series Classification (TSC) community. However, unlike in image recognition problems, data augmentation techniques have not yet been investigated thoroughly for the TSC task. This is surprising as the accuracy of deep learning models for TSC could potentially be improved, especially for small datasets that exhibit overfitting, when a data augmentation method is adopted. In this paper, we fill this gap by investigating the application of a recently proposed data augmentation technique based on the Dynamic Time Warping distance, for a deep learning model for TSC. To evaluate the potential of augmenting the training set, we performed extensive experiments using the UCR TSC benchmark. Our preliminary experiments reveal that data augmentation can drastically increase deep CNN's accuracy on some datasets and significantly improve the deep model's accuracy when the method is used in an ensemble approach.

Motivation & Objective

• Address overfitting and limited training data in time series classification (TSC) with deep networks.
• Evaluate a Dynamic Time Warping (DTW)–based synthetic data augmentation method for TSC.
• Assess the impact of augmentation on a deep ResNet architecture using the UCR TSC benchmark.
• Explore ensemble strategies to mitigate potential negatives of data augmentation.

Proposed method

• Use a deep Residual Network (ResNet) architecture tailored for univariate time series with three residual blocks.
• Apply a DTW-based weighted averaging (DBA) augmentation to generate synthetic time series from the training set, selecting samples via an Average Selected scheme.
• Generate synthetic data equal to twice the size of the most represented class.
• Train models with and without augmentation under consistent initializations and optimization settings.
• Combine predictions from augmented and non-augmented ResNets via ensemble averaging of posterior probabilities.

Experimental results

Research questions

• RQ1 Does DTW-based synthetic data augmentation improve ResNet performance on time series classification tasks in the UCR archive?
• RQ2 On which datasets does augmentation help or hurt performance, and how significant are these effects?
• RQ3 Can an ensemble of augmented and non-augmented models provide more robust improvements across datasets?
• RQ4 What is the impact of augmentation on small, hard datasets like DiatomSizeReduction and Wine?

Key findings

• Data augmentation can drastically improve deep model accuracy on some datasets (e.g., DiatomSizeReduction: 30% to 96%).
• Augmentation can have small negative effects on some datasets, but overall does not significantly reduce accuracy.
• An ensemble of augmented and non-augmented ResNets reduces the number of datasets with degraded performance and preserves gains on others.
• On the DiatomSizeReduction dataset, which has only 16 training instances, augmentation yields a large gain, while the 1-NN with DTW achieves 97% accuracy indicating dataset easiness for simpler methods.
• Wilcoxon signed-rank test shows a significant difference (p-value < 0.0005) for the ensemble improvement over single models.
• The Wine dataset also shows important improvement with augmentation, indicating dataset-dependent benefits.