[Paper Review] HSI-CNN: A Novel Convolution Neural Network for Hyperspectral Image

Yanan Luo, Jie Zou+3 more|arXiv (Cornell University)|Feb 28, 2018

Remote-Sensing Image ClassificationEngineering26 references16 citations

TL;DR

This paper proposes HSI-CNN, a novel 3D convolutional neural network framework that extracts spectral-spatial features from hyperspectral images by first generating 1D feature maps from a target pixel and its neighbors, then stacking them into a 2D matrix for standard CNN processing. The method achieves state-of-the-art classification accuracy of 99.28% on KSC, 99.09% on IP, 99.42% on PU, and 98.95% on SA datasets, demonstrating superior performance under limited training samples.

ABSTRACT

With the development of deep learning, the performance of hyperspectral image (HSI) classification has been greatly improved in recent years. The shortage of training samples has become a bottleneck for further improvement of performance. In this paper, we propose a novel convolutional neural network framework for the characteristics of hyperspectral image data, called HSI-CNN. Firstly, the spectral-spatial feature is extracted from a target pixel and its neighbors. Then, a number of one-dimensional feature maps, obtained by convolution operation on spectral-spatial features, are stacked into a two-dimensional matrix. Finally, the two-dimensional matrix considered as an image is fed into standard CNN. This is why we call it HSI-CNN. In addition, we also implements two depth network classification models, called HSI-CNN+XGBoost and HSI-CapsNet, in order to compare the performance of our framework. Experiments show that the performance of hyperspectral image classification is improved efficiently with HSI-CNN framework. We evaluate the model's performance using four popular HSI datasets, which are the Kennedy Space Center (KSC), Indian Pines (IP), Pavia University scene (PU) and Salinas scene (SA). As far as we concerned, HSI-CNN has got the state-of-art accuracy among all methods we have known on these datasets of 99.28%, 99.09%, 99.42%, 98.95% separately.

Motivation & Objective

• To address the challenge of limited training samples in hyperspectral image (HSI) classification.
• To develop a deep learning framework tailored to the unique spectral-spatial characteristics of HSI data.
• To improve classification accuracy by effectively leveraging both spectral and spatial information in HSI.
• To evaluate the proposed HSI-CNN framework against existing models on standard HSI benchmarks.
• To explore extensions using XGBoost and CapsNet for comparative performance analysis.

Proposed method

• Extract spectral-spatial features from a central pixel and its surrounding neighborhood using local spatial and spectral patterns.
• Apply 1D convolution operations on the extracted spectral-spatial features to generate multiple 1D feature maps.
• Stack the 1D feature maps into a 2D matrix, treating it as a pseudo-image input for standard 2D CNNs.
• Utilize standard convolutional neural network layers to learn hierarchical spatial and spectral representations from the 2D matrix.
• Implement two variant models—HSI-CNN+XGBoost and HSI-CapsNet—for comparative evaluation.
• Train the models end-to-end using standard backpropagation with cross-entropy loss on four benchmark HSI datasets.

Experimental results

Research questions

• RQ1 Can a hybrid spectral-spatial feature extraction strategy improve classification accuracy in hyperspectral images with limited training samples?
• RQ2 How does the 2D matrix transformation of 1D spectral-spatial features enhance CNN learning compared to direct 3D or 2D processing?
• RQ3 What is the performance gain of HSI-CNN over existing state-of-the-art methods on standard HSI benchmarks?
• RQ4 How do the additional models HSI-CNN+XGBoost and HSI-CapsNet compare in terms of accuracy and robustness?
• RQ5 Does the proposed framework maintain high performance across diverse HSI datasets with varying spatial and spectral characteristics?

Key findings

• HSI-CNN achieves a classification accuracy of 99.28% on the Kennedy Space Center (KSC) dataset, outperforming existing methods.
• On the Indian Pines (IP) dataset, HSI-CNN attains 99.09% accuracy, demonstrating strong generalization under limited samples.
• The model reaches 99.42% accuracy on the Pavia University (PU) scene, indicating high effectiveness in urban environments.
• For the Salinas (SA) scene, HSI-CNN achieves 98.95% accuracy, confirming robustness across diverse land cover types.
• The framework consistently outperforms baseline models and established state-of-the-art methods across all four benchmark datasets.
• The ablation study confirms that the 2D matrix transformation of 1D spectral-spatial features significantly enhances feature representation and classification performance.