[Paper Review] A Baseline for Visual Instance Retrieval with Deep Convolutional Networks

Ali Sharif Razavian, Josephine Sullivan+2 more|arXiv (Cornell University)|Dec 20, 2014

Advanced Image and Video Retrieval TechniquesComputer Science24 references92 citations

TL;DR

This paper introduces a simple, non-fine-tuned ConvNet-based image representation pipeline that outperforms state-of-the-art methods on six standard visual instance retrieval benchmarks. It achieves strong performance with a compact 32-byte representation, demonstrating that pre-trained ConvNets alone can serve as a powerful baseline without domain-specific adaptation.

ABSTRACT

This paper presents a simple pipeline for visual instance retrieval exploiting image representations based on convolutional networks (ConvNets), and demonstrates that ConvNet image representations outperform other state-of-the-art image representations on six standard image retrieval datasets for the first time. Unlike existing design choices, our image representation does not require fine-tuning or learning with data similar to the test set. Furthermore, we consider the challenge Can you construct a tiny image representation with memory requirements less than or equal to 32 bytes that can successfully perform retrieval? We report the promising performance of our tiny ConvNet based representation.

Motivation & Objective

• To establish a strong, simple baseline for visual instance retrieval using deep convolutional networks.
• To evaluate whether pre-trained ConvNet features can outperform existing state-of-the-art image representations without fine-tuning.
• To investigate the feasibility of constructing a highly compact image representation with memory ≤32 bytes for effective retrieval.
• To demonstrate that powerful retrieval performance is achievable without adapting models to test-set data or using complex architectures.

Proposed method

• The method uses pre-trained convolutional neural network (ConvNet) features extracted from the last fully connected layer of a deep network.
• Features are L2-normalized to improve retrieval performance and reduce sensitivity to scale variations.
• The pipeline avoids fine-tuning and does not require training data similar to the test set.
• A compact representation is constructed by quantizing or compressing the feature vector to fit within 32 bytes.
• Image retrieval is performed via cosine similarity between query and gallery features.
• The approach is evaluated on six standard benchmark datasets for visual instance retrieval.

Experimental results

Research questions

• RQ1 Can a pre-trained ConvNet-based representation outperform existing state-of-the-art image representations in visual instance retrieval?
• RQ2 Does a non-fine-tuned ConvNet representation achieve strong performance without adaptation to the test data distribution?
• RQ3 Is it possible to construct a 32-byte image representation that still enables effective retrieval?
• RQ4 How does the proposed method compare to existing methods in terms of accuracy and memory efficiency?

Key findings

• The proposed ConvNet-based representation outperforms all prior state-of-the-art methods on six standard visual instance retrieval benchmarks.
• The method achieves strong performance without any fine-tuning or adaptation to the test set, demonstrating generalization capability.
• A compact image representation with memory requirements ≤32 bytes is successfully constructed and shown to be effective for retrieval.
• The results establish a new baseline for visual instance retrieval using simple, pre-trained deep features.