[Paper Review] Feature Alignment and Restoration for Domain Generalization and Adaptation

Xin Jin, Cuiling Lan+2 more|arXiv (Cornell University)|Jun 22, 2020

Domain Adaptation and Few-Shot Learning61 references35 citations

TL;DR

The paper introduces FAR, a unified framework that performs feature alignment across domains and restores task-relevant information from residuals to improve both domain generalization (DG) and unsupervised domain adaptation (UDA).

ABSTRACT

For domain generalization (DG) and unsupervised domain adaptation (UDA), cross domain feature alignment has been widely explored to pull the feature distributions of different domains in order to learn domain-invariant representations. However, the feature alignment is in general task-ignorant and could result in degradation of the discrimination power of the feature representation and thus hinders the high performance. In this paper, we propose a unified framework termed Feature Alignment and Restoration (FAR) to simultaneously ensure high generalization and discrimination power of the networks for effective DG and UDA. Specifically, we perform feature alignment (FA) across domains by aligning the moments of the distributions of attentively selected features to reduce their discrepancy. To ensure high discrimination, we propose a Feature Restoration (FR) operation to distill task-relevant features from the residual information and use them to compensate for the aligned features. For better disentanglement, we enforce a dual ranking entropy loss constraint in the FR step to encourage the separation of task-relevant and task-irrelevant features. Extensive experiments on multiple classification benchmarks demonstrate the high performance and strong generalization of our FAR framework for both domain generalization and unsupervised domain adaptation.

Motivation & Objective

• Motivate and address the loss of discriminative power during cross-domain feature alignment.
• Propose a unified framework that combines feature alignment with a restoration step to preserve task-relevant information.
• Disentangle task-relevant and task-irrelevant residual features via a dual ranking entropy loss.
• Demonstrate state-of-the-art performance on DG and UDA benchmarks and analyze design choices.

Proposed method

• Apply adaptive feature alignment (FA) on attentively selected features to reduce cross-domain distribution discrepancies.
• Introduce a feature restoration (FR) step that distills task-relevant information from the residual between original and aligned features.
• Disentangle residuals into task-relevant R+ and task-irrelevant R− using a gating mechanism based on spatial and channel attention.
• Enforce a dual ranking entropy loss to promote discrimination after restoration and suppression of interference.
• Train with a shared domain classifier guided by domain-specific expert Teachers and use consistency constraints across classifiers.
• Optionally use per-domain teachers during training with an L1-consistency term; inference uses a single shared classifier.

Experimental results

Research questions

• RQ1 Can feature alignment improve generalization across multiple domains without sacrificing discriminative power?
• RQ2 Does restoring task-relevant information from aligned features improve DG and UDA performance?
• RQ3 Does disentangling residual information with a dual ranking entropy loss enhance feature separation between classes?
• RQ4 How do attention-guided feature selection and teacher-guided training affect cross-domain classification?

Key findings

• FAR outperforms state-of-the-art methods on DG and UDA benchmarks across multiple datasets.
• Feature alignment alone can degrade discrimination, but restoration recovers discriminative power.
• The dual ranking entropy loss improves disentanglement of task-relevant and task-irrelevant information.
• Attention-guided adaptive feature selection for FA yields better alignment than simple conv-based mappings.
• Teacher-guided training with a shared classifier provides performance gains and reduces inference complexity.
• Ablation studies confirm the importance of the FR module and the DRE loss for overall performance.