[Paper Review] Deep Supervised Discrete Hashing

Qi Li, Zhenan Sun+2 more|arXiv (Cornell University)|May 31, 2017

Advanced Image and Video Retrieval Techniques25 references163 citations

TL;DR

A deep hashing method that directly learns binary codes in a single stream by leveraging both pairwise semantic labels and classification information, optimizing via alternating minimization to preserve similarity while being ideal for classification.

ABSTRACT

With the rapid growth of image and video data on the web, hashing has been extensively studied for image or video search in recent years. Benefit from recent advances in deep learning, deep hashing methods have achieved promising results for image retrieval. However, there are some limitations of previous deep hashing methods (e.g., the semantic information is not fully exploited). In this paper, we develop a deep supervised discrete hashing algorithm based on the assumption that the learned binary codes should be ideal for classification. Both the pairwise label information and the classification information are used to learn the hash codes within one stream framework. We constrain the outputs of the last layer to be binary codes directly, which is rarely investigated in deep hashing algorithm. Because of the discrete nature of hash codes, an alternating minimization method is used to optimize the objective function. Experimental results have shown that our method outperforms current state-of-the-art methods on benchmark datasets.

Motivation & Objective

• Motivate learning binary hash codes directly within a deep framework using semantic pairwise and label information.
• Develop a one-stream CNN-based hashing model that integrates classification to improve code quality.
• Maintain the discrete nature of hash codes during optimization to reduce quantization error.
• Demonstrate state-of-the-art retrieval performance on CIFAR-10 and NUS-WIDE datasets.

Proposed method

• Use CNN-F as the base architecture to learn the hash function and representations.
• Formulate a loss combining pairwise likelihood of similarity with a classifier-based term linking codes to labels.
• Constrain the last layer outputs to be binary codes and introduce an auxiliary variable to enable discrete optimization.
• Optimize via alternating minimization with updates to the last-layer outputs, the classifier weights, and the binary codes through discrete cyclic coordinate descent.
• Incorporate a joint objective that enforces both similarity preservation and classification suitability of the codes.

Experimental results

Research questions

• RQ1 Can binary hash codes learned in a single-stream CNN framework simultaneously preserve semantic similarity and be optimal for classification?
• RQ2 Does enforcing discreteness of hash codes during training improve retrieval performance compared to continuous relaxations?
• RQ3 How does a unified, one-stream approach compare to two-stream methods in leveraging label information for hashing?
• RQ4 What is the impact of jointly learning representations, hashes, and a linear classifier on standard image retrieval benchmarks?

Key findings

• The proposed method consistently outperforms state-of-the-art deep hashing methods on CIFAR-10 and NUS-WIDE across multiple bit-lengths.
• Using both pairwise similarity and classification information within one stream yields substantial improvements over A/B variants and two-stream approaches.
• Directly constraining the last layer to binary codes with discrete optimization reduces quantization error and improves retrieval accuracy.
• The method achieves higher MAP scores under various experimental settings, indicating robustness to different training/test splits.