[Paper Review] Feature Stylization and Domain-aware Contrastive Learning for Domain Generalization

Seogkyu Jeon, Kibeom Hong+3 more|arXiv (Cornell University)|Aug 19, 2021

Domain Adaptation and Few-Shot Learning60 references4 citations

TL;DR

This paper proposes a domain generalization framework that stylizes deep features using learned domain-specific statistics to generate novel domains while preserving class semantics. By decomposing features into high- and low-frequency components, stylizing only the low-frequency components, and applying a domain-aware contrastive loss, the method achieves state-of-the-art performance on PACS and Office-Home benchmarks, outperforming prior methods in robustness and generalization.

ABSTRACT

Domain generalization aims to enhance the model robustness against domain shift without accessing the target domain. Since the available source domains for training are limited, recent approaches focus on generating samples of novel domains. Nevertheless, they either struggle with the optimization problem when synthesizing abundant domains or cause the distortion of class semantics. To these ends, we propose a novel domain generalization framework where feature statistics are utilized for stylizing original features to ones with novel domain properties. To preserve class information during stylization, we first decompose features into high and low frequency components. Afterward, we stylize the low frequency components with the novel domain styles sampled from the manipulated statistics, while preserving the shape cues in high frequency ones. As the final step, we re-merge both components to synthesize novel domain features. To enhance domain robustness, we utilize the stylized features to maintain the model consistency in terms of features as well as outputs. We achieve the feature consistency with the proposed domain-aware supervised contrastive loss, which ensures domain invariance while increasing class discriminability. Experimental results demonstrate the effectiveness of the proposed feature stylization and the domain-aware contrastive loss. Through quantitative comparisons, we verify the lead of our method upon existing state-of-the-art methods on two benchmarks, PACS and Office-Home.

Motivation & Objective

• To address domain shift in deep learning without access to target domain data during training.
• To overcome limitations of existing domain generalization methods that struggle with optimization or semantic distortion during domain synthesis.
• To preserve class discriminative information while generating diverse, novel domain styles during training.
• To enhance model robustness through feature and output consistency using a novel contrastive learning objective.

Proposed method

• The method samples novel domain styles from manipulated batch-wise feature statistics, emphasizing outlying style vectors through re-scaling of standard deviation.
• Features are decomposed into high-frequency (shape cues) and low-frequency (texture/stylistic) components using a frequency decomposition technique.
• Only the low-frequency components are stylized using affine transformation with sampled domain styles, preserving high-frequency structural information.
• The stylized low-frequency and original high-frequency components are recombined to form novel domain features.
• A domain-aware supervised contrastive loss is introduced to enforce feature and output consistency across domains while enhancing class discriminability.
• The feature stylization block is strategically placed after the second residual block in a ResNet architecture to balance low-level and high-level feature representation.

Experimental results

Research questions

• RQ1 Can feature stylization based on statistical domain styles improve generalization to unseen domains without access to target data?
• RQ2 How can class semantics be preserved during feature stylization when synthesizing novel domains?
• RQ3 What is the optimal location and component for applying feature stylization within a deep neural network?
• RQ4 Does a domain-aware contrastive loss improve model robustness and generalization compared to standard contrastive learning?
• RQ5 How does the scale of feature re-scaling affect the trade-off between shape and texture information in stylized features?

Key findings

• The proposed method achieves an average accuracy of 85.86% on the Office-Home benchmark, surpassing the baseline by 2.32 percentage points.
• On the PACS dataset, the method attains an average accuracy of 83.95%, outperforming the baseline by 0.41 percentage points.
• The ablation study confirms that stylizing only the low-frequency components yields the best performance, with a 0.39% improvement over using high-frequency components.
• The optimal scale parameter for feature re-scaling is found to be 10, which balances shape and texture cue utilization.
• Placing the feature stylization block after the second residual block yields the highest performance, indicating this layer captures an optimal balance of structural and semantic information.
• The domain-aware contrastive loss significantly improves generalization, with a 2.26% average accuracy gain over standard contrastive learning when combined with stylization.