[Paper Review] Unsupervised Deep Features for Remote Sensing Image Matching via Discriminator Network

Mohbat Tharani, Numan Khurshid+1 more|arXiv (Cornell University)|Oct 15, 2018

Advanced Image and Video Retrieval Techniques21 references4 citations

TL;DR

This paper proposes an unsupervised deep feature learning framework for remote sensing image matching (RSIM) using a residual encoder-decoder network to extract compact, discriminative features without labeled data. It replaces traditional distance metrics with a deep discriminator network that learns a non-linear decision boundary to distinguish similar from dissimilar image pairs, achieving a state-of-the-art mAP of 81.2% on benchmark datasets and a 12% improvement in content-based remote sensing image retrieval (CBRSIR).

ABSTRACT

The advent of deep perceptual networks brought about a paradigm shift in machine vision and image perception. Image apprehension lately carried out by hand-crafted features in the latent space have been replaced by deep features acquired from supervised networks for improved understanding. However, such deep networks require strict supervision with a substantial amount of the labeled data for authentic training process. These methods perform poorly in domains lacking labeled data especially in case of remote sensing image retrieval. Resolving this, we propose an unsupervised encoder-decoder feature for remote sensing image matching (RSIM). Moreover, we replace the conventional distance metrics with a deep discriminator network to identify the similarity of the image pairs. To the best of our knowledge, discriminator network has never been used before for solving RSIM problem. Results have been validated with two publicly available benchmark remote sensing image datasets. The technique has also been investigated for content-based remote sensing image retrieval (CBRSIR); one of the widely used applications of RSIM. Results demonstrate that our technique supersedes the state-of-the-art methods used for unsupervised image matching with mean average precision (mAP) of 81%, and image retrieval with an overall improvement in mAP score of about 12%.

Motivation & Objective

• Address the scarcity of labeled remote sensing data in deep learning applications.
• Overcome limitations of supervised CNNs that require large labeled datasets for training.
• Replace conventional distance metrics (e.g., Euclidean, Cosine) with a learned discriminative mechanism for improved image matching.
• Develop a robust, unsupervised feature extraction method suitable for diverse remote sensing image domains.
• Enhance content-based remote sensing image retrieval (CBRSIR) performance without requiring iterative relevance feedback.

Proposed method

• Utilize a residual encoder-decoder architecture to learn unsupervised deep features from remote sensing images.
• Extract features from the bottleneck layer of the autoencoder as compact visual descriptors.
• Train a deep residual discriminator network to classify image feature pairs as similar or dissimilar.
• Use the discriminator’s output as a learned similarity score instead of traditional metrics.
• Train the discriminator end-to-end with contrastive loss to learn a non-linear decision boundary between positive and negative pairs.
• Apply the framework to both RSIM and CBRSIR tasks using two public remote sensing datasets.

Experimental results

Research questions

• RQ1 Can unsupervised deep features extracted via an autoencoder architecture effectively represent remote sensing images for matching tasks?
• RQ2 Does a deep discriminator network outperform traditional distance metrics (e.g., Euclidean, Cosine) in distinguishing similar from dissimilar image pairs in an unsupervised setting?
• RQ3 Can the proposed framework achieve state-of-the-art performance in content-based remote sensing image retrieval without requiring labeled data or relevance feedback?
• RQ4 How robust is the method to geometric variations such as rotation and translation in remote sensing imagery?
• RQ5 To what extent does the discriminator network generalize across diverse remote sensing image classes?

Key findings

• The proposed method achieves a mean average precision (mAP) of 81.2% on the LandUse dataset, surpassing state-of-the-art unsupervised methods.
• The framework improves mAP by approximately 12% compared to existing approaches in content-based remote sensing image retrieval.
• Top-10 retrieval precision reaches 99.2% on the LandUse dataset and 99.8% on the SatScene dataset, indicating high relevance of retrieved images.
• The model maintains strong performance even on challenging classes like rotated or translated objects (e.g., intersections, storage tanks), demonstrating robustness.
• The confusion matrix shows that aeroplane and storage tank classes are most frequently confused, and tennis courts with freeways, due to visual similarities.
• The method eliminates the need for iterative relevance feedback, significantly reducing annotation burden in remote sensing applications.