[Paper Review] Scalable domain adaptation of convolutional neural networks

Adrian Popescu, Etienne Gadeski+1 more|arXiv (Cornell University)|Dec 7, 2015

Advanced Image and Video Retrieval Techniques18 references3 citations

TL;DR

This paper proposes a scalable, fully automated method for domain adaptation of convolutional neural networks (CNNs) using noisy web-collected images from Flickr, without manual labeling. By applying weakly-supervised reranking and training a CNN from scratch on these images, the approach achieves state-of-the-art retrieval performance on tourism-focused datasets like Oxford5k and INRIA Holidays, outperforming both generic ImageNet models and prior CNN methods requiring manual supervision.

ABSTRACT

Convolutional neural networks (CNNs) tend to become a standard approach to solve a wide array of computer vision problems. Besides important theoretical and practical advances in their design, their success is built on the existence of manually labeled visual resources, such as ImageNet. The creation of such datasets is cumbersome and here we focus on alternatives to manual labeling. We hypothesize that new resources are of uttermost importance in domains which are not or weakly covered by ImageNet, such as tourism photographs. We first collect noisy Flickr images for tourist points of interest and apply automatic or weakly-supervised reranking techniques to reduce noise. Then, we learn domain adapted models with a standard CNN architecture and compare them to a generic model obtained from ImageNet. Experimental validation is conducted with publicly available datasets, including Oxford5k, INRIA Holidays and Div150Cred. Results show that low-cost domain adaptation improves results compared to the use of generic models but also compared to strong non-CNN baselines such as triangulation embedding.

Motivation & Objective

• To address the lack of domain-specific visual resources in domains like tourism, where ImageNet coverage is limited.
• To develop a scalable, low-cost alternative to manual dataset creation for training domain-specific CNNs.
• To investigate whether CNNs can be effectively trained on noisy, web-sourced images without manual annotation.
• To compare the performance of automatically adapted CNNs against generic models and strong non-CNN baselines in retrieval tasks.
• To evaluate the impact of data quality, diversity, and feature engineering on domain-adapted CNN performance.

Proposed method

• Automatically collect large-scale, noisy Flickr images for points of interest (POIs) using keyword-based queries.
• Apply weakly-supervised reranking techniques to reduce noise and improve image quality in the training set.
• Train a standard CNN architecture from scratch on the reranked, domain-specific image collections.
• Use PCA-based dimensionality reduction to compress CNN features for improved scalability without significant accuracy loss.
• Apply spatial search and data augmentation to enhance feature robustness and retrieval performance.
• Implement query expansion using top-ranked results to refine retrieval queries and improve mAP.

Experimental results

Research questions

• RQ1 Can CNNs be effectively trained on large-scale, noisy web images without manual labeling, particularly in low-resource domains like tourism?
• RQ2 How does the performance of a domain-adapted CNN compare to generic ImageNet-pretrained models and non-CNN baselines in image retrieval?
• RQ3 To what extent does data diversity compensate for noise in the training set when fine-tuning CNNs?
• RQ4 What is the impact of feature compression and tuning techniques (e.g., PCA, spatial search) on retrieval performance and scalability?
• RQ5 Does weakly-supervised reranking improve model performance compared to raw noisy collections?

Key findings

• The proposed method achieves a mAP of 69.2% on Oxford5k and 78.5% on INRIA Holidays using PCA-compressed features (256 dimensions), outperforming prior CNN-based methods.
• Even with noisy data, training a CNN from scratch on reranked Flickr images yields better results than relying on generic ImageNet models.
• The best performance is achieved with 256-dimensional PCA compression, showing that feature compression improves scalability without sacrificing accuracy.
• Feature tuning via spatial search and data augmentation improves mAP by over 10% on Oxford5k and Holidays, confirming the effectiveness of preprocessing.
• Query expansion improves mAP to 81.5% on Oxford5k, but shows no benefit on Holidays and Div150Cred, indicating domain-dependent effectiveness.
• The results suggest that data diversity can compensate for noise, implying that manual annotation may not be essential for effective CNN training in new domains.