[논문 리뷰] In-domain representation learning for remote sensing

Maxim Neumann, André Susano Pinto외 2명|arXiv (Cornell University)|2019. 11. 15.

Remote-Sensing Image Classification참고 문헌 31인용 수 34

한 줄 요약

The paperTrainin g uses five remote sensing datasets to study in-domain representation learning, showing that fine-tuning in-domain representations yields state-of-the-art results across tasks, especially with limited labeled data.

ABSTRACT

Given the importance of remote sensing, surprisingly little attention has been paid to it by the representation learning community. To address it and to establish baselines and a common evaluation protocol in this domain, we provide simplified access to 5 diverse remote sensing datasets in a standardized form. Specifically, we investigate in-domain representation learning to develop generic remote sensing representations and explore which characteristics are important for a dataset to be a good source for remote sensing representation learning. The established baselines achieve state-of-the-art performance on these datasets.

연구 동기 및 목표

• Develop general remote sensing representations via in-domain supervised fine-tuning.
• Provide standardized access to five diverse remote sensing datasets.
• Establish a common evaluation protocol and strong baselines for benchmarking.

제안 방법

• Standardize five remote sensing datasets in TensorFlow Datasets with fixed train/val/test splits.
• Use ResNet-50 v2 as the backbone across experiments for fair comparison.
• Compare transfer performance of in-domain representations against ImageNet-pretrained and scratch-trained models across varying downstream data sizes.
• Evaluate using top-1 accuracy for multi-class and mean average precision for multi-label tasks, with logit-transformed accuracy for aggregation.
• Analyze which dataset characteristics (e.g., diversity, label quality) influence representation learning effectiveness.

실험 결과

연구 질문

• RQ1 Can in-domain supervised fine-tuning produce more transferable remote sensing representations than ImageNet pre-training or from-scratch training?
• RQ2 How do different remote sensing datasets serve as sources for learning general representations, considering data size, diversity, and label quality?
• RQ3 What downstream performance gains arise when transferring in-domain representations to unseen remote sensing tasks with limited labeled data?

주요 결과

• In-domain representations generally outperform ImageNet baselines when transferring to unseen remote sensing tasks, especially with small downstream data budgets.
• RESISC-45 emerged as a particularly effective source for general representations across aerial and satellite data, while BigEarthNet and So2Sat sometimes lag due to weak labeling.
• Fine-tuning in-domain representations consistently yielded the best downstream performance in 4 of 5 datasets evaluated, with one exception (BigEarthNet at full data).
• Larger weakly labeled datasets did not always outperform smaller, diverse, human-curated datasets for representation learning.
• A multi-resolution dataset and label quality/diversity factors significantly influence the quality of learned representations.