[Paper Review] Return of Frustratingly Easy Domain Adaptation

Baochen Sun, Jiashi Feng+1 more|arXiv (Cornell University)|Nov 17, 2015

Domain Adaptation and Few-Shot LearningComputer Science35 references108 citations

TL;DR

This paper proposes CORAL, a simple yet highly effective unsupervised domain adaptation method that aligns the second-order statistics of source and target domains by re-coloring whitened source features using the target domain's covariance matrix. Despite its minimal implementation (four lines of code), CORAL achieves state-of-the-art performance on image recognition and sentiment analysis benchmarks, especially with deep features, outperforming complex methods that require retraining or hyperparameter tuning.

ABSTRACT

Unlike human learning, machine learning often fails to handle changes between training (source) and test (target) input distributions. Such domain shifts, common in practical scenarios, severely damage the performance of conventional machine learning methods. Supervised domain adaptation methods have been proposed for the case when the target data have labels, including some that perform very well despite being "frustratingly easy" to implement. However, in practice, the target domain is often unlabeled, requiring unsupervised adaptation. We propose a simple, effective, and efficient method for unsupervised domain adaptation called CORrelation ALignment (CORAL). CORAL minimizes domain shift by aligning the second-order statistics of source and target distributions, without requiring any target labels. Even though it is extraordinarily simple--it can be implemented in four lines of Matlab code--CORAL performs remarkably well in extensive evaluations on standard benchmark datasets.

Motivation & Objective

• To address domain shift in machine learning where training (source) and test (target) data distributions differ, a common issue that degrades model performance.
• To develop an unsupervised domain adaptation method that does not require labeled target data, which is often unavailable in real-world applications.
• To create a method that is both simple to implement and highly effective, especially when applied to deep neural network features.
• To demonstrate that aligning second-order statistics (covariance) between domains can yield superior performance compared to more complex adaptation techniques.

Proposed method

• CORAL minimizes domain shift by aligning the covariance matrices of source and target feature distributions.
• It first whitens the source features to remove internal correlations, then re-colors them using the target domain's covariance matrix.
• The transformation is a linear operation defined as X_s' = W_T^{-1/2} X_s W_T^{1/2}, where W_T is the target covariance matrix.
• The method requires only two steps: computing the covariance matrices of source and target features, and applying the whitening and re-coloring transformation.
• After transformation, standard supervised learning is applied on the adapted source features, with no need for target labels.
• The approach is computationally efficient and can be implemented in just four lines of MATLAB code.

Experimental results

Research questions

• RQ1 Can a simple, non-iterative method based on second-order statistics outperform complex, deep learning-based domain adaptation methods?
• RQ2 How effective is covariance alignment in reducing domain shift when no target labels are available?
• RQ3 Does the performance gain of CORAL scale with feature complexity, particularly for deep features?
• RQ4 How does CORAL compare to state-of-the-art methods like TCA, GFK, and SA in both vision and NLP tasks?

Key findings

• On the Office-Caltech10 dataset with SURF features, CORAL achieved 64.0% accuracy, significantly outperforming the next best method (SA at 49.1%) and the no-adaptation baseline.
• With deep CNN features (DECAF-fc7), CORAL achieved 64.0% accuracy on Office-Caltech10, surpassing all prior methods, including complex adaptation networks.
• In sentiment analysis on Amazon reviews, CORAL achieved the highest average accuracy across all domain shifts, outperforming even state-of-the-art methods like TCA and GFK.
• The performance gap between CORAL and other methods was substantially larger on deep features than on shallow features, indicating greater benefit in deep learning settings.
• CORAL’s improvement was most pronounced on image data, where features are denser and more correlated, compared to sparse bag-of-words text features.
• Even with full training data, CORAL maintained strong performance, suggesting robustness to dataset size and generalization capability.