[Paper Review] Domain Adaptive Neural Networks for Object Recognition

Muhammad Ghifary, W. Bastiaan Kleijn+1 more|arXiv (Cornell University)|Sep 21, 2014

Domain Adaptation and Few-Shot LearningComputer Science17 references96 citations

TL;DR

This paper proposes Domain Adaptive Neural Networks (DaNN) that use Maximum Mean Discrepancy (MMD) as a regularization loss to reduce distribution mismatch between source and target domains in the latent space of deep neural networks. The method achieves state-of-the-art performance on the Office dataset using raw image pixels, outperforming SVM-based baselines and recent models like GFK and TSC, especially when combined with denoising auto-encoder pretraining.

ABSTRACT

We propose a simple neural network model to deal with the domain adaptation problem in object recognition. Our model incorporates the Maximum Mean Discrepancy (MMD) measure as a regularization in the supervised learning to reduce the distribution mismatch between the source and target domains in the latent space. From experiments, we demonstrate that the MMD regularization is an effective tool to provide good domain adaptation models on both SURF features and raw image pixels of a particular image data set. We also show that our proposed model, preceded by the denoising auto-encoder pretraining, achieves better performance than recent benchmark models on the same data sets. This work represents the first study of MMD measure in the context of neural networks.

Motivation & Objective

• To address domain shift in object recognition where training and test data come from different distributions.
• To reduce distribution mismatch between source and target domains without relying on handcrafted features like SURF.
• To investigate the effectiveness of Maximum Mean Discrepancy (MMD) as a regularization in deep neural networks for domain adaptation.
• To evaluate whether denoising auto-encoder (DAE) pretraining enhances domain adaptation performance.
• To establish a simple yet effective neural network framework for domain adaptation on raw image pixels.

Proposed method

• Introduces a neural network model with MMD regularization to align hidden layer representations of source and target domains.
• Uses MMD as a non-parametric measure of distribution discrepancy between two domains in the latent space.
• Employs a kernelized MMD estimator based on the unit ball in a universal Reproducing Kernel Hilbert Space (RKHS).
• Applies MMD regularization during backpropagation to minimize the distribution gap between source and target features.
• Pre-trains the network using denoising auto-encoders (DAE) before end-to-end fine-tuning with MMD regularization.
• Uses raw image pixels as input, avoiding reliance on handcrafted features like SURF.

Experimental results

Research questions

• RQ1 Can MMD regularization effectively reduce domain shift in deep neural networks for object recognition?
• RQ2 Does the proposed DaNN model outperform existing domain adaptation methods on the Office dataset when using raw image pixels?
• RQ3 How does DAE pretraining affect the performance of domain adaptive neural networks?
• RQ4 Is MMD regularization more effective in unsupervised or semi-supervised domain adaptation settings?
• RQ5 Can a deep neural network with MMD regularization generalize well without handcrafted feature extractors?

Key findings

• The DaNN model with MMD regularization achieves state-of-the-art performance on the Office dataset, outperforming SVM-based baselines and models like GFK and TSC across all domain adaptation pairs.
• When using raw image pixels as input, the DaNN with DAE pretraining achieves a mean accuracy of 77.9% on the dslr domain, surpassing the best in-domain performance of 74.2% on the same domain.
• The MMD regularization significantly improves performance in unsupervised settings, with the largest performance gain observed in the W→D (webcam to dslr) pair.
• In semi-supervised settings, the performance gap between DaNN and standard NN decreases, indicating that labeled target data reduces the relative benefit of MMD regularization.
• The DAE pretraining consistently improves performance across all domain pairs, especially when combined with MMD regularization.
• The model demonstrates that MMD regularization is effective even without handcrafted features, achieving strong results directly on raw pixels.