[Paper Review] Multi-adversarial Faster-RCNN for Unrestricted Object Detection

Zhenwei He, Lei Zhang|arXiv (Cornell University)|Jul 24, 2019

Domain Adaptation and Few-Shot Learning42 references42 citations

TL;DR

The paper introduces MAF, a Faster-RCNN based detector that uses hierarchical multi-adversarial domain feature alignment and aggregated proposal features with a weighted gradient reversal layer to enable unrestricted object detection across labeled source and unlabeled target domains.

ABSTRACT

Conventional object detection methods essentially suppose that the training and testing data are collected from a restricted target domain with expensive labeling cost. For alleviating the problem of domain dependency and cumbersome labeling, this paper proposes to detect objects in an unrestricted environment by leveraging domain knowledge trained from an auxiliary source domain with sufficient labels. Specifically, we propose a multi-adversarial Faster-RCNN (MAF) framework for unrestricted object detection, which inherently addresses domain disparity minimization for domain adaptation in feature representation. The paper merits are in three-fold: 1) With the idea that object detectors often becomes domain incompatible when image distribution resulted domain disparity appears, we propose a hierarchical domain feature alignment module, in which multiple adversarial domain classifier submodules for layer-wise domain feature confusion are designed; 2) An information invariant scale reduction module (SRM) for hierarchical feature map resizing is proposed for promoting the training efficiency of adversarial domain adaptation; 3) In order to improve the domain adaptability, the aggregated proposal features with detection results are feed into a proposed weighted gradient reversal layer (WGRL) for characterizing hard confused domain samples. We evaluate our MAF on unrestricted tasks, including Cityscapes, KITTI, Sim10k, etc. and the experiments show the state-of-the-art performance over the existing detectors.

Motivation & Objective

• Address domain shift in object detection by enabling training on a labeled source domain and testing on an unlabeled target domain.
• Learn domain-invariant feature representations through multi-adversarial alignment at multiple convolutional blocks.
• Improve training efficiency with a scale reduction module (SRM) for hierarchical feature maps.
• Enhance domain confusion for proposals by aggregating proposal features with detection outputs and applying a weighted gradient reversal layer.
• Demonstrate state-of-the-art performance on unrestricted object detection across Cityscapes, Foggy Cityscapes, KITTI, and SIM10K.

Proposed method

• Build on Faster-RCNN with a VGG16 backbone.
• Introduce hierarchical domain feature alignment using multiple adversarial domain classifier submodules placed in convolution blocks 3, 4, and 5.
• Apply a scale reduction module to down-sample feature maps without losing information, improving training efficiency.
• Add an aggregated proposal feature alignment module that concatenates proposal features with classification scores and bounding box coordinates before domain discrimination.
• Use a weighted gradient reversal layer to emphasize hard-to-confuse samples during adversarial training, guiding domain adaptation.
• Train end-to-end with a combined loss including detection loss on the source domain and adversarial domain alignment loss across hierarchical and proposal features.

Experimental results

Research questions

• RQ1 How can domain disparity between source and target domains be minimized in object detection models?
• RQ2 Does hierarchical domain feature alignment across multiple CNN blocks improve cross-domain detection more than single-block alignment?
• RQ3 Can aggregating proposal-level features with detection outputs and applying a weighted GRL enhance domain confusion for object detectors?
• RQ4 What is the impact of SRM on training efficiency and performance in adversarial domain adaptation for detection?
• RQ5 How does MAF perform on unrestricted object detection acrossCityscapes, Foggy Cityscapes, KITTI, and SIM10K compared to prior methods?

Key findings

• MAF achieves state-of-the-art results on unrestricted object detection among Cityscapes, Foggy Cityscapes, KITTI, and SIM10K datasets.
• Using multi-adversarial domain classifiers across multiple CNN blocks improves domain-invariant feature learning beyond single-block alignment.
• SRM reduces feature map scale to accelerate training without sacrificing information, enhancing adversarial learning efficiency.
• Aggregated proposal features with detection outputs plus WGRL further boosts domain confusion and mAP for target domains.
• MAF outperforms the prior DAF method across several cross-domain settings, including synthetic-to-real and cross-scene scenarios.