[Paper Review] xView: Objects in Context in Overhead Imagery

Darius Lam, Richard S. Kuzma+6 more|arXiv (Cornell University)|Feb 22, 2018

Advanced Neural Network ApplicationsComputer Science17 references86 citations

TL;DR

xView introduces a large-scale overhead imagery object-detection dataset with 60 classes and ~1 million labeled objects across 1,400+ km2, plus a multi-stage annotation and quality-control pipeline and baseline SSD experiments.

ABSTRACT

We introduce a new large-scale dataset for the advancement of object detection techniques and overhead object detection research. This satellite imagery dataset enables research progress pertaining to four key computer vision frontiers. We utilize a novel process for geospatial category detection and bounding box annotation with three stages of quality control. Our data is collected from WorldView-3 satellites at 0.3m ground sample distance, providing higher resolution imagery than most public satellite imagery datasets. We compare xView to other object detection datasets in both natural and overhead imagery domains and then provide a baseline analysis using the Single Shot MultiBox Detector. xView is one of the largest and most diverse publicly available object-detection datasets to date, with over 1 million objects across 60 classes in over 1,400 km^2 of imagery.

Motivation & Objective

• Develop a large, diverse, publicly available overhead imagery object-detection dataset to advance research across multiple CV frontiers.
• Provide a rigorous annotation and quality-control workflow to ensure high-quality bounding boxes and labels.
• Enable multi-scale detection and fine-grained classification reflective of real-world overhead imagery.
• Facilitate comparisons with natural-image datasets and prior overhead datasets, and encourage downstream remote-sensing applications.

Proposed method

• Assemble imagery from WorldView-3 at 0.3m GSD to maximize resolution and diversity across 60 object classes.
• Organize classes hierarchically into seven parent categories with multiple fine-grained child classes to capture context and variability.
• Employ a three-stage quality-control annotation pipeline (worker, supervisory, expert) and use an in-house QGIS-based tool for axis-aligned bounding boxes.
• Create 1 km2 chips via a UTM grid across diverse AOIs, ensuring uniform scene-type distribution and geographic variety.
• Conduct a rigorous data-ground-truth process including gold-standard benchmarking against expert labels (precision 0.75, recall 0.95 at IoU 0.5).
• Benchmark baseline detection using the Single Shot Multibox Detector (SSD) with multi-scale features, evaluating vanilla, multi-resolution, and augmented datasets.

Experimental results

Research questions

• RQ1 What is the scale, diversity, and quality required for a robust overhead-imagery object-detection dataset?
• RQ2 How do multi-scale representations and data augmentations impact detection performance across a large, varied overhead dataset?
• RQ3 How does an overhead-detection dataset with fine-grained classes compare to natural-image benchmarks in terms of learning challenges and evaluation?
• RQ4 Can the dataset support future research directions such as few-shot learning and domain adaptation in geospatial contexts?

Key findings

• xView comprises over 1 million labeled objects, 60 classes, and covers more than 1,400 km2 of imagery.
• Three-stage quality control (worker, supervisory, expert) and gold-standard evaluation ensure labeling precision (0.75) and recall (0.95) at IoU 0.5.
• SSD baselines show that multi-resolution training improves mean AP (mAP) over vanilla and augmented datasets, with multi-resolution achieving the best overall performance.
• Detection performance is higher for larger, contextually easier classes and degrades for small or highly variable classes, underscoring scale and background-context challenges.
• The dataset’s geographic and contextual diversity motivates the need for domain adaptation and few-shot learning approaches in overhead imagery.