[Paper Review] Data Augmentation for Time-Series Classification: An Extensive Empirical Study and Comprehensive Survey

Zijun Gao, Liu, Haibao+1 more|arXiv (Cornell University)|Oct 16, 2023

Time Series Analysis and Forecasting4 citations

TL;DR

This paper presents a comprehensive survey and empirical evaluation of data augmentation (DA) techniques for time-series classification (TSC), introducing a novel five-category taxonomy: Transformation-Based, Pattern-Based, Generative, Decomposition-Based, and Automated DA. It evaluates 15 DA methods across 8 UCR datasets using ResNet, achieving a benchmark accuracy of 88.94 ± 11.83%, with Permutation emerging as the most effective strategy (89.20 ± 11.95% accuracy), while Rotation degraded performance.

ABSTRACT

Data Augmentation (DA) has become a critical approach in Time Series Classification (TSC), primarily for its capacity to expand training datasets, enhance model robustness, introduce diversity, and reduce overfitting. However, the current landscape of DA in TSC is plagued with fragmented literature reviews, nebulous methodological taxonomies, inadequate evaluative measures, and a dearth of accessible and user-oriented tools. This study addresses these challenges through a comprehensive examination of DA methodologies within the TSC domain.Our research began with an extensive literature review spanning a decade, revealing significant gaps in existing surveys and necessitating a detailed analysis of over 100 scholarly articles to identify more than 60 distinct DA techniques. This rigorous review led to the development of a novel taxonomy tailored to the specific needs of DA in TSC, categorizing techniques into five primary categories: Transformation-Based, Pattern-Based, Generative, Decomposition-Based, and Automated Data Augmentation. This taxonomy is intended to guide researchers in selecting appropriate methods with greater clarity. In response to the lack of comprehensive evaluations of foundational DA techniques, we conducted a thorough empirical study, testing nearly 20 DA strategies across 15 diverse datasets representing all types within the UCR time-series repository. Using ResNet and LSTM architectures, we employed a multifaceted evaluation approach, including metrics such as Accuracy, Method Ranking, and Residual Analysis, resulting in a benchmark accuracy of 84.98 +- 16.41% in ResNet and 82.41 +- 18.71% in LSTM. Our investigation underscored the inconsistent efficacies of DA techniques, for instance, methods like RGWs and Random Permutation significantly improved model performance, whereas others, like EMD, were less effective.

Motivation & Objective

• Address the fragmented and inconsistent literature on data augmentation (DA) in time-series classification (TSC), which lacks unified taxonomies and holistic evaluations.
• Identify and categorize over 60 unique DA techniques from a decade of research to establish a structured, purpose-built taxonomy for TSC.
• Conduct a rigorous empirical evaluation of 15 leading DA methods across 8 UCR time-series datasets using ResNet to establish a performance benchmark and reveal method-specific efficacy.
• Provide data-driven, dataset-specific recommendations for selecting optimal DA strategies based on intrinsic dataset characteristics such as length, class distribution, and temporal structure.
• Develop and release a user-friendly Python library integrating all evaluated DA techniques to accelerate research and application in TSC.

Proposed method

• Conducted an extensive literature review of over 100 scholarly articles from the past decade to identify and analyze more than 60 distinct data augmentation techniques for TSC.
• Proposed a novel five-tier taxonomy for DA in TSC: Transformation-Based, Pattern-Based, Generative, Decomposition-Based, and Automated Data Augmentation, designed to reflect methodological and functional distinctions.
• Evaluated 15 representative DA techniques using a standardized experimental setup on 8 UCR time-series benchmark datasets, employing ResNet as the base model for consistent performance comparison.
• Applied a multi-faceted evaluation protocol combining Accuracy, Method Ranking (via Friedman test), and Residual Analysis to assess model performance and generalization across diverse datasets.
• Analyzed the impact of dataset-specific attributes—such as sequence length, number of classes, and temporal correlation—on the effectiveness of each DA method to inform context-aware selection.
• Implemented and open-sourced a comprehensive Python library integrating all evaluated DA techniques, enabling easy application and reproducibility for future research.

Experimental results

Research questions

• RQ1 Which data augmentation techniques are most effective for time-series classification, and how do their performances vary across different datasets?
• RQ2 How do the intrinsic characteristics of time-series datasets—such as length, class imbalance, and temporal structure—influence the success of specific data augmentation strategies?
• RQ3 What is the relative performance of different DA categories (e.g., Transformation-Based vs. Generative) when evaluated under a standardized benchmark with a deep learning model?
• RQ4 Can a unified, comprehensive taxonomy be developed to systematically classify and guide the selection of DA methods in TSC, improving methodological clarity and research coherence?
• RQ5 To what extent do combined or automated DA strategies outperform individual techniques, and how can their design be optimized for efficiency and effectiveness?

Key findings

• The Permutation method achieved the highest average rank (4.125) and the best performance (89.20 ± 11.95% accuracy), outperforming other methods in 2 out of 8 UCR datasets.
• The Rotation method significantly degraded model performance, achieving only 84.75 ± 12.92% accuracy, indicating that not all augmentation techniques are universally beneficial.
• Transformation-Based methods consistently demonstrated strong and stable performance across diverse datasets, particularly excelling in preserving data integrity and temporal structure.
• Pattern-Based methods were most effective for datasets with high feature richness and complex class boundaries, such as ECG5000, where class-specific augmentation improved generalization.
• Generative methods like GANs showed negligible performance gains (close to baseline), suggesting they require careful hyperparameter tuning and alignment with original data distribution to be effective.
• The benchmark model accuracy across all experiments was 88.94 ± 11.83%, serving as a reliable reference point for future comparisons in TSC with data augmentation.