[Paper Review] AMRNet: Chips Augmentation in Areial Images Object Detection.

Zhiwei Wei, Chenzhen Duan|arXiv (Cornell University)|Sep 15, 2020

Advanced Neural Network Applications24 references4 citations

TL;DR

AMRNet proposes three novel data augmentation techniques—scale adaptive module, mosaic augmentation, and mask resampling—for improving object detection in aerial images. By dynamically adjusting chip size, enhancing object sparsity via mosaic, and balancing class imbalance through panoramic segmentation-based resampling, the method achieves state-of-the-art performance on VisDrone and UAVDT without sacrificing inference efficiency.

ABSTRACT

Object detection in aerial images is a challenging task due to the following reasons: (1) objects are small and dense relative to images; (2) the object scale varies in a wide range; (3) the number of object in different classes is imbalanced. Many current methods adopt cropping idea: splitting high resolution images into serials subregions (chips) and detecting on them. However, some problems such as scale variation, object sparsity, and class imbalance exist in the process of training network with chips. In this work, three augmentation methods are introduced to relieve these problems. Specifically, we propose a scale adaptive module, which dynamically adjusts chip size to balance object scale, narrowing scale variation in training. In addtion, we introduce mosaic to augment datasets, relieving object sparity problem. To balance catgory, we present mask resampling to paste object in chips with panoramic segmentation. Our model achieves state-of-the-art perfomance on two popular aerial image datasets of VisDrone and UAVDT. Remarkably, three methods can be independently applied to detectiors, increasing performance steady without the sacrifice of inference efficiency.

Motivation & Objective

• Address scale variation in aerial image object detection by dynamically adjusting chip sizes during training.
• Reduce object sparsity in training chips through mosaic-based data augmentation.
• Mitigate class imbalance in object detection by resampling objects using panoramic segmentation masks.
• Improve detection performance on aerial datasets while maintaining inference efficiency.
• Enable modular integration of augmentation techniques into existing detectors without retraining.

Proposed method

• Introduce a scale adaptive module that dynamically selects chip sizes based on object scale distribution to minimize scale variation in training samples.
• Apply mosaic augmentation to combine multiple high-resolution images into a single training chip, increasing object density and reducing sparsity.
• Implement mask resampling by leveraging panoramic segmentation masks to guide the placement of objects in chips, ensuring balanced representation across object categories.
• Design the augmentation pipeline to be modular and compatible with existing object detectors, allowing independent application of each method.
• Train the detector on augmented chips using standard loss functions, with no architectural changes required.
• Ensure inference efficiency is preserved by avoiding complex post-processing or additional inference branches.

Experimental results

Research questions

• RQ1 How can scale variation in aerial image chips be reduced during object detection training?
• RQ2 To what extent can mosaic-based augmentation alleviate object sparsity in high-resolution aerial image chips?
• RQ3 Can mask resampling based on panoramic segmentation effectively balance class imbalances in aerial object detection?
• RQ4 How do the proposed augmentation techniques individually and collectively improve detection performance on aerial benchmarks?
• RQ5 Can these augmentation methods be applied to existing detectors without degrading inference speed?

Key findings

• The scale adaptive module effectively reduces scale variation in training chips, leading to more stable and accurate detection across diverse object sizes.
• Mosaic augmentation significantly improves detection performance on rare or sparse objects by increasing their presence in training samples.
• Mask resampling successfully mitigates class imbalance by ensuring underrepresented object categories are adequately represented in training chips.
• The proposed methods achieve state-of-the-art performance on both VisDrone and UAVDT datasets, demonstrating consistent gains across different evaluation metrics.
• Each augmentation technique can be independently applied to existing detectors, resulting in steady performance improvements without increasing inference time.