[Paper Review] Unsupervised Semantic Deep Hashing

Sheng Jin|arXiv (Cornell University)|Mar 19, 2018

Advanced Image and Video Retrieval Techniques2 references3 citations

TL;DR

This paper proposes Unsupervised Semantic Deep Hashing (USDH), an end-to-end unsupervised deep hashing method that leverages semantic information from VGG-19 features to learn binary hashing codes without labels. By enforcing four loss components—semantic, quantization, entropy maximization, and rotation invariance—USDH achieves state-of-the-art performance on image retrieval and fine-grained classification, outperforming prior unsupervised methods by up to 10.7% in accuracy.

ABSTRACT

In recent years, deep hashing methods have been proved to be efficient since it employs convolutional neural network to learn features and hashing codes simultaneously. However, these methods are mostly supervised. In real-world application, it is a time-consuming and overloaded task for annotating a large number of images. In this paper, we propose a novel unsupervised deep hashing method for large-scale image retrieval. Our method, namely unsupervised semantic deep hashing ( extbf{USDH}), uses semantic information preserved in the CNN feature layer to guide the training of network. We enforce four criteria on hashing codes learning based on VGG-19 model: 1) preserving relevant information of feature space in hashing space; 2) minimizing quantization loss between binary-like codes and hashing codes; 3) improving the usage of each bit in hashing codes by using maximum information entropy, and 4) invariant to image rotation. Extensive experiments on CIFAR-10, NUSWIDE have demonstrated that extbf{USDH} outperforms several state-of-the-art unsupervised hashing methods for image retrieval. We also conduct experiments on Oxford 17 datasets for fine-grained classification to verify its efficiency for other computer vision tasks.

Motivation & Objective

• Address the high cost of manual annotation in supervised deep hashing by developing an unsupervised alternative.
• Preserve semantic similarity from deep features in the hashing code space without relying on class labels.
• Improve hashing code quality by enforcing quantization loss, information entropy maximization, and rotation invariance.
• Demonstrate effectiveness beyond image retrieval, including in fine-grained classification tasks.

Proposed method

• Utilizes the VGG-19 network's mid-level features as semantic descriptors to guide hashing code learning.
• Introduces a semantic loss that encourages hashing codes to preserve similarity relationships computed from feature descriptors.
• Applies a quantization loss to minimize the difference between real-valued codes and their binary approximations.
• Maximizes information entropy of each bit in the hashing code to ensure balanced and effective bit usage.
• Enforces rotation invariance by minimizing the distance between hashing codes of original and rotated images.
• Trains the network end-to-end using a composite loss function combining all four components.

Experimental results

Research questions

• RQ1 Can semantic information from deep features be effectively leveraged in an unsupervised setting to improve hashing code quality?
• RQ2 How does enforcing rotation invariance impact the performance of unsupervised deep hashing?
• RQ3 To what extent does combining semantic, quantization, entropy, and rotation losses improve retrieval accuracy compared to prior unsupervised methods?
• RQ4 Can unsupervised hashing codes learned without labels be effective for fine-grained classification tasks?

Key findings

• On CIFAR-10, USDH achieves a mean average precision (MAP) of 65.87% with 64-bit codes, outperforming DeepBit by 27.57 percentage points.
• On NUSWIDE, USDH achieves a MAP of 64.07% for 64-bit codes, significantly exceeding other unsupervised baselines.
• The semantic loss alone improves performance by 7.62% over DeepBit, and adding quantization loss further boosts accuracy by 4.07%.
• USDH achieves 81.3% accuracy on the Oxford-17 fine-grained flower classification dataset, outperforming DeepBit by 6.2% and SIFT-Internal by 10.7%.
• The method is robust to hyperparameter settings, with MAP varying by less than 0.2% across different values of the semantic loss parameter ρ.
• USDH maintains fast inference speed (0.07 seconds per sample), comparable to DeepBit and significantly faster than traditional hand-crafted descriptors.