[Paper Review] Hyperspectral Data Augmentation

Jakub Nalepa, Michał Myller+1 more|arXiv (Cornell University)|Mar 13, 2019

Remote-Sensing Image Classification13 references4 citations

TL;DR

This paper proposes a novel online hyperspectral data augmentation technique that generates synthetic samples during inference rather than during training, improving deep neural network generalization without increasing training time. It introduces a PCA-based augmentation method that preserves data distribution and inter-class relationships, achieving state-of-the-art classification accuracy when combined with offline techniques, with real-time inference performance.

ABSTRACT

Data augmentation is a popular technique which helps improve generalization capabilities of deep neural networks. It plays a pivotal role in remote-sensing scenarios in which the amount of high-quality ground truth data is limited, and acquiring new examples is costly or impossible. This is a common problem in hyperspectral imaging, where manual annotation of image data is difficult, expensive, and prone to human bias. In this letter, we propose online data augmentation of hyperspectral data which is executed during the inference rather than before the training of deep networks. This is in contrast to all other state-of-the-art hyperspectral augmentation algorithms which increase the size (and representativeness) of training sets. Additionally, we introduce a new principal component analysis based augmentation. The experiments revealed that our data augmentation algorithms improve generalization of deep networks, work in real-time, and the online approach can be effectively combined with offline techniques to enhance the classification accuracy.

Motivation & Objective

• To address the challenge of limited, costly-to-acquire, and imbalanced hyperspectral image (HSI) training data in remote sensing.
• To improve deep neural network generalization in HSI classification and segmentation without increasing training time.
• To propose a novel online data augmentation method that generates synthetic samples at inference time, rather than pre-training.
• To develop a PCA-based augmentation technique that preserves data distribution and inter-class relationships.
• To evaluate the effectiveness of combining online and offline augmentation for enhanced classification accuracy.

Proposed method

• Proposes online data augmentation by generating synthetic HSI samples during inference using noise injection and PCA-based transformation.
• Applies a PCA-based augmentation method that projects data to a lower-dimensional space, adds perturbations, and projects back to the original space using all principal components.
• Uses a voting scheme over original and augmented samples to determine final predictions, improving robustness.
• Trains PCA on the original training set and applies it both offline (to augment training data) and online (to generate samples at test time).
• Employs a lightweight 1D spectral CNN for efficient inference, with batch normalization and fully connected layers for classification.
• Combines online augmentation (PCA-ON, Noise-ON) with offline techniques (PCA, GAN, Noise) to enhance model performance.

Experimental results

Research questions

• RQ1 Can online data augmentation during inference improve the generalization of deep neural networks in hyperspectral image classification?
• RQ2 How effective is PCA-based data augmentation in preserving the original data distribution and inter-class relationships in HSI data?
• RQ3 What is the impact of combining online and offline data augmentation on classification accuracy in HSI benchmarks?
• RQ4 Can online augmentation be performed in real-time without degrading inference speed?
• RQ5 How does online augmentation compare to traditional offline augmentation in terms of performance and training efficiency?

Key findings

• Online data augmentation using PCA-ON and noise injection significantly improves classification accuracy across all three HSI benchmarks: Salinas Valley, Indian Pines, and Pavia University.
• The combination of offline PCA and online PCA-ON (PCA/PCA-ON) achieves the highest overall accuracy, with improvements of up to 2.5% over baseline models.
• The proposed online method maintains real-time inference speed, making it suitable for deployment in resource-constrained environments such as satellite platforms.
• PCA-based augmentation preserves the original data distribution and inter-class relationships, as confirmed by visualization and quantitative analysis.
• The integration of GAN-based offline augmentation with online PCA-ON further enhances performance, demonstrating the complementary nature of both approaches.
• The method outperforms existing state-of-the-art offline augmentation techniques, particularly in handling imbalanced and limited training sets.