[Paper Review] Successive Embedding and Classification Loss for Aerial Image Classification

Jiayun Wang, Patrick Virtue+1 more|arXiv (Cornell University)|Dec 5, 2017

Advanced Neural Network Applications36 references7 citations

TL;DR

This paper proposes a dual-loss training strategy for aerial image classification that combines classification loss with an embedding loss applied in the classifier space—just before softmax—significantly improving generalization. By optimizing both semantic classification and inter-class separation in the classifier space, the method reduces overfitting and achieves state-of-the-art performance on MSTAR (98.2% test accuracy) and AID (97.0% with VGG16) benchmarks.

ABSTRACT

Deep neural networks can be effective means to automatically classify aerial images but is easy to overfit to the training data. It is critical for trained neural networks to be robust to variations that exist between training and test environments. To address the overfitting problem in aerial image classification, we consider the neural network as successive transformations of an input image into embedded feature representations and ultimately into a semantic class label, and train neural networks to optimize image representations in the embedded space in addition to optimizing the final classification score. We demonstrate that networks trained with this dual embedding and classification loss outperform networks with classification loss only. %We also study placing the embedding loss on different network layers. We also find that moving the embedding loss from commonly-used feature space to the classifier space, which is the space just before softmax nonlinearization, leads to the best classification performance for aerial images. Visualizations of the network's embedded representations reveal that the embedding loss encourages greater separation between target class clusters for both training and testing partitions of two aerial image classification benchmark datasets, MSTAR and AID. Our code is publicly available on GitHub.

Motivation & Objective

• Address overfitting in aerial image classification due to limited training data and domain shift between training and test environments.
• Improve generalization of deep neural networks by enhancing discriminative feature representations beyond standard classification loss.
• Investigate the optimal location for applying embedding loss—feature space vs. classifier space—for improved robustness and accuracy.
• Evaluate the effectiveness of different embedding loss functions (contrastive loss, center loss) in the context of SAR and aerial scene classification.
• Demonstrate that moving the embedding loss to the classifier space leads to better performance than applying it in earlier feature spaces.

Proposed method

• Train a deep neural network with a dual objective: standard cross-entropy classification loss and an additional contrastive or center loss.
• Apply the embedding loss specifically in the classifier space—the linear projection layer just before the softmax nonlinearity—rather than in the earlier feature space.
• Use contrastive loss to maximize inter-class margin and minimize intra-class variance in the classifier space.
• Use center loss to pull features of the same class toward their class centers in the classifier space.
• Optimize the network end-to-end using stochastic gradient descent with both classification and embedding losses.
• Visualize embedded representations in the classifier space to analyze cluster separation and intra-class compactness.

Experimental results

Research questions

• RQ1 Does applying an embedding loss in the classifier space improve generalization in aerial image classification compared to applying it in the feature space?
• RQ2 How does the choice of embedding loss function (contrastive vs. center loss) affect classification accuracy and outlier rejection in SAR and aerial scene recognition?
• RQ3 Can the dual-loss strategy reduce the gap between training and test accuracy, especially under domain shift or data augmentation?
• RQ4 Does moving the embedding loss to the classifier space lead to better cluster separation and reduced intra-class variance in the final representation?
• RQ5 To what extent does the dual-loss approach improve performance on benchmark datasets like MSTAR and AID?

Key findings

• Applying the embedding loss in the classifier space yields 98.2% test accuracy on the MSTAR SAR dataset, significantly outperforming the baseline with only classification loss (91.4%).
• On the AID aerial scene classification dataset, the dual-loss method with center loss in the classifier space achieves 97.0% accuracy using VGG16, a 2% improvement over the baseline.
• Contrastive loss applied in the classifier space reduces false alarm probability to 20.5% at 90% detection rate, outperforming center loss and feature-space embedding losses.
• Visualizations show that the classifier-space embedding loss increases inter-class separation and reduces intra-class variance, especially for test samples.
• The method reduces overfitting, as evidenced by a smaller gap between training and test accuracy, and improves robustness to data augmentation and domain shift.
• Center loss applied in the feature space provides no performance gain, while the same loss in the classifier space yields consistent improvements across both datasets.