[Paper Review] EAGLE: Large-scale Vehicle Detection Dataset inReal-World Scenarios using Aerial Imagery

Seyed Majid Azimi, Reza Bahmanyar+2 more|arXiv (Cornell University)|Jul 12, 2020

Advanced Neural Network Applications42 references4 citations

TL;DR

EAGLE is a large-scale, real-world aerial imagery dataset with 215,986 oriented vehicle instances annotated using four-point bounding boxes and orientation labels, enabling multi-class vehicle detection under diverse conditions such as varying weather, lighting, occlusion, and camera angles. It establishes a new benchmark for aerial object detection by supporting three tasks—horizontal, rotated, and oriented bounding box detection—and provides state-of-the-art baselines through extensive evaluation of SOTA models.

ABSTRACT

Multi-class vehicle detection from airborne imagery with orientation estimation is an important task in the near and remote vision domains with applications in traffic monitoring and disaster management. In the last decade, we have witnessed significant progress in object detection in ground imagery, but it is still in its infancy in airborne imagery, mostly due to the scarcity of diverse and large-scale datasets. Despite being a useful tool for different applications, current airborne datasets only partially reflect the challenges of real-world scenarios. To address this issue, we introduce EAGLE (oriEnted vehicle detection using Aerial imaGery in real-worLd scEnarios), a large-scale dataset for multi-class vehicle detection with object orientation information in aerial imagery. It features high-resolution aerial images composed of different real-world situations with a wide variety of camera sensor, resolution, flight altitude, weather, illumination, haze, shadow, time, city, country, occlusion, and camera angle. The annotation was done by airborne imagery experts with small- and large-vehicle classes. EAGLE contains 215,986 instances annotated with oriented bounding boxes defined by four points and orientation, making it by far the largest dataset to date in this task. It also supports researches on the haze and shadow removal as well as super-resolution and in-painting applications. We define three tasks: detection by (1) horizontal bounding boxes, (2) rotated bounding boxes, and (3) oriented bounding boxes. We carried out several experiments to evaluate several state-of-the-art methods in object detection on our dataset to form a baseline. Experiments show that the EAGLE dataset accurately reflects real-world situations and correspondingly challenging applications.

Motivation & Objective

• Address the lack of large-scale, diverse, and realistic aerial imagery datasets for multi-class vehicle detection with orientation estimation.
• Overcome limitations of existing datasets that fail to represent real-world challenges such as haze, shadows, varying flight altitudes, and sensor types.
• Provide a comprehensive benchmark for evaluating state-of-the-art object detection models under realistic conditions using high-resolution aerial images.
• Enable research in related tasks such as haze/shadow removal, super-resolution, and image in-painting through the dataset's diverse and rich annotations.
• Establish standardized evaluation protocols for three detection tasks: horizontal, rotated, and oriented bounding box detection.

Proposed method

• Collect high-resolution aerial images from diverse real-world scenarios, including multiple cities, countries, flight altitudes, weather conditions, and illumination levels.
• Employ airborne imagery experts to annotate 215,986 vehicle instances using oriented bounding boxes defined by four corner points and orientation angles.
• Design three distinct detection tasks: (1) horizontal bounding boxes, (2) rotated bounding boxes, and (3) oriented bounding boxes to evaluate model robustness.
• Ensure dataset diversity by including variations in camera sensor types, resolution, time of day, and levels of occlusion, haze, and shadows.
• Support downstream applications by including data suitable for haze and shadow removal, super-resolution, and image in-painting tasks.
• Train and evaluate multiple state-of-the-art object detection models on the EAGLE dataset to establish performance baselines.

Experimental results

Research questions

• RQ1 To what extent does the EAGLE dataset reflect real-world aerial detection challenges such as varying weather, lighting, and occlusion?
• RQ2 How do state-of-the-art object detection models perform on oriented vehicle detection in aerial imagery under diverse real-world conditions?
• RQ3 What performance gains can be achieved by using oriented bounding boxes compared to horizontal or rotated bounding boxes in aerial vehicle detection?
• RQ4 Can the EAGLE dataset serve as a reliable benchmark for advancing research in aerial image understanding, including haze removal and super-resolution?
• RQ5 How do variations in flight altitude, sensor type, and environmental conditions affect detection accuracy in aerial imagery?

Key findings

• The EAGLE dataset contains 215,986 vehicle instances annotated with oriented bounding boxes and orientation, making it the largest dataset of its kind for aerial vehicle detection.
• Experiments on EAGLE demonstrate that models trained on this dataset achieve improved robustness across diverse real-world conditions, including haze, shadows, and occlusion.
• The use of oriented bounding boxes leads to significant performance improvements over horizontal and rotated bounding box baselines, particularly in scenarios with long, narrow vehicles.
• State-of-the-art object detection models show measurable performance degradation on real-world variations in the dataset, confirming its challenge and realism.
• The dataset supports advanced research beyond detection, including haze and shadow removal, super-resolution, and image in-painting, due to its high-resolution and diverse annotations.
• Baseline results on the three defined tasks (horizontal, rotated, oriented) provide a strong reference point for future method development in aerial object detection.