[Paper Review] Gated Domain-Invariant Feature Disentanglement for Domain Generalizable Object Detection

Haozhuo Zhang, Huimin Yu+2 more|arXiv (Cornell University)|Mar 22, 2022

Domain Adaptation and Few-Shot Learning4 citations

TL;DR

This paper proposes Gated Domain-Invariant Feature Disentanglement (GDIFD), a novel method for domain generalizable object detection that disentangles domain-invariant representations (DIR) from domain-specific representations (DSR) by learning channel-wise gate signals via a Channel Gate Module (CGM). The CGM produces binary-like gate signals (near 0 or 1) to mask out DSR-exclusive channels, enabling cleaner, channel-wise disentanglement and achieving state-of-the-art performance on standard benchmarks.

ABSTRACT

For Domain Generalizable Object Detection (DGOD), Disentangled Representation Learning (DRL) helps a lot by explicitly disentangling Domain-Invariant Representations (DIR) from Domain-Specific Representations (DSR). Considering the domain category is an attribute of input data, it should be feasible for networks to fit a specific mapping which projects DSR into feature channels exclusive to domain-specific information, and thus much cleaner disentanglement of DIR from DSR can be achieved simply on channel dimension. Inspired by this idea, we propose a novel DRL method for DGOD, which is termed Gated Domain-Invariant Feature Disentanglement (GDIFD). In GDIFD, a Channel Gate Module (CGM) learns to output channel gate signals close to either 0 or 1, which can mask out the channels exclusive to domain-specific information helpful for domain recognition. With the proposed GDIFD, the backbone in our framework can fit the desired mapping easily, which enables the channel-wise disentanglement. In experiments, we demonstrate that our approach is highly effective and achieves state-of-the-art DGOD performance.

Motivation & Objective

• To improve domain generalizable object detection (DGOD) by enabling cleaner disentanglement of domain-invariant representations (DIR) from domain-specific representations (DSR).
• To address the limitation of existing DRL methods that rely on stacked convolutions without guaranteeing exclusive channel assignment for DSR.
• To leverage the domain category as an attribute to explicitly assign specific channels to DSR, enabling more effective disentanglement along the channel dimension.
• To achieve superior DGOD performance through a lightweight, trainable gate mechanism that enforces binary-like channel selection.

Proposed method

• A Channel Gate Module (CGM) is introduced to output channel-wise gate signals $ S_{di} $ that are constrained to be close to 0 or 1 via a novel gate loss.
• The gate signals are applied via element-wise multiplication with backbone features $ F_b $, masking out channels dedicated to domain-specific information.
• The CGM is initialized with values near 1 to encourage minimal use of channels for DSR, promoting efficient DIR learning.
• A gate loss is proposed to enforce binary-like behavior in gate signals, ensuring that channels are exclusively assigned to either DIR or DSR.
• Adversarial alignment is used to further promote independence between DIR and DSR.
• The framework uses a shared backbone and only a single shallow encoder for DIR, enabling efficient and scalable disentanglement.

Experimental results

Research questions

• RQ1 Can channel-wise disentanglement of domain-invariant and domain-specific features be effectively achieved through learnable binary-like gate signals?
• RQ2 Does enforcing gate signals to be near 0 or 1 significantly improve the quality of feature disentanglement compared to conventional attention mechanisms?
• RQ3 Can a lightweight, channel-gating mechanism outperform complex multi-backbone or reconstruction-based DRL methods in DGOD?
• RQ4 How does the combination of special initialization and gate loss affect the performance and disentanglement quality of the CGM?
• RQ5 Can the proposed GDIFD method generalize to unsupervised domain adaptation (UDAOD) settings with comparable or better performance?

Key findings

• The proposed CGM with both special initialization and gate loss achieves the best performance, with a 38.3% mAP on the B&C to F benchmark, outperforming all ablation variants.
• Without the gate loss, gate signals remain gray (intermediate values), indicating entanglement between DIR and DSR, which leads to performance degradation.
• The model learns to assign only one channel to domain classification in the B&C to F setting when both special initialization and gate loss are applied, minimizing DSR interference.
• The ablation study confirms that both the gate loss and special initialization are essential for achieving clean, binary-like channel gating and optimal disentanglement.
• In the UDAOD setting, GDIFD improves mAP by 1.2% over CFA (41.4% vs. 40.2%) using the same FCOS baseline, demonstrating strong transferability.
• The method achieves state-of-the-art performance on multiple DGOD benchmarks, validating its effectiveness and generalization capability.