[Paper Review] Domain Adaptation for Object Detection via Style Consistency

Adrian Rodriguez, Krystian Mikolajczyk|arXiv (Cornell University)|Nov 22, 2019

Domain Adaptation and Few-Shot Learning45 references60 citations

TL;DR

The paper proposes a two-step domain adaptation method for object detection: (1) pixel-level style transfer to bridge low-level style gaps with a feature-consistency constraint, and (2) robust pseudo-labeling to adapt high-level features, evaluated on SSD300 across VOC and style-variant target domains.

ABSTRACT

We propose a domain adaptation approach for object detection. We introduce a two-step method: the first step makes the detector robust to low-level differences and the second step adapts the classifiers to changes in the high-level features. For the first step, we use a style transfer method for pixel-adaptation of source images to the target domain. We find that enforcing low distance in the high-level features of the object detector between the style transferred images and the source images improves the performance in the target domain. For the second step, we propose a robust pseudo labelling approach to reduce the noise in both positive and negative sampling. Experimental evaluation is performed using the detector SSD300 on PASCAL VOC extended with the dataset proposed in arxiv:1803.11365 where the target domain images are of different styles. Our approach significantly improves the state-of-the-art performance in this benchmark.

Motivation & Objective

• Motivate domain adaptation for object detection when source and target domains differ in low- and high-level features.
• Develop a pixel-level style transfer approach to reduce low-level domain gaps while enforcing high-level feature consistency.
• Introduce a robust pseudo-labeling strategy to adapt detectors to high-level differences in the target domain.
• Integrate style transfer, feature consistency, and robust pseudo-labeling into a unified training objective.
• Empirically validate the approach on SSD300 with PASCAL VOC and style-diverse target domains (Clipart, Watercolor, Comic).

Proposed method

• Use Adaptive Instance Normalization style transfer to generate multiple style-transferred variants x_S→T from each source image x_S.
• Enforce feature consistency by minimizing distance between high-level SSD feature maps f_k(x_S) and f_k(x_S→T) across K last feature maps.
• Train SSD with standard loss on both x_S and x_S→T to promote low-level robustness.
• Apply robust pseudo-labeling to target domain images by selecting high-confidence detections as positives using a threshold on h(b) and sampling negatives with a separate threshold to reduce noise.
• Generate pseudo-labels for x_T and x_T→T and retrain using both source and target style-augmented data.
• Combine losses into a single objective L = λ1 L_ST + λ2 L_Cons + λ3 L_S + λ4 L_RPL, with two-step training (initial stage using source and style-transferred data, second stage including pseudo-labeled target data).

Experimental results

Research questions

• RQ1 Can style-based pixel adaptation alone close the domain gap for object detection across visually different styles?
• RQ2 Does enforcing high-level feature consistency between original and style-transferred images improve detector robustness to low-level changes?
• RQ3 Does incorporating robust pseudo-labeling of target-domain images further adapt classifiers to high-level domain shifts?
• RQ4 What is the effect of combining style transfer, feature consistency, and robust pseudo-labeling on cross-domain detection performance?

Key findings

• Style transfer with feature consistency improves detection performance across style-diverse domains versus using domain transferred images alone.
• Enforcing high-level feature consistency yields additional gains across Clipart1k, Watercolor2k, and Comic2k datasets. (Quantitative specifics are in the paper’s results.)
• Using robust pseudo-labeling with target-domain and style-transferred target-domain images further improves performance over using only style transfer or pseudo-labeling alone.
• The combined approach achieves state-of-the-art improvements on the evaluated benchmarks compared to several prior domain-adaptation methods.