[Paper Review] Learning Hash Codes via Hamming Distance Targets

Martin Loncaric, Bowei Liu+1 more|arXiv (Cornell University)|Oct 1, 2018

Advanced Image and Video Retrieval Techniques16 references4 citations

TL;DR

This paper proposes a novel loss function, Hamming Distance Target (HDT), that improves learning of binary hash codes by modeling the probability of two inputs falling within a target Hamming distance using log-likelihood loss. The method uses efficient mini-batch pair sampling and multi-indexing for fast retrieval, achieving state-of-the-art performance with 84% MAP on ImageNet and 78.1% recall at 12,709 distance comparisons on SIFT-1M, outperforming prior methods by up to 8x in query efficiency.

ABSTRACT

We present a powerful new loss function and training scheme for learning binary hash codes with any differentiable model and similarity function. Our loss function improves over prior methods by using log likelihood loss on top of an accurate approximation for the probability that two inputs fall within a Hamming distance target. Our novel training scheme obtains a good estimate of the true gradient by better sampling inputs and evaluating loss terms between all pairs of inputs in each minibatch. To fully leverage the resulting hashes, we use multi-indexing. We demonstrate that these techniques provide large improvements to a similarity search tasks. We report the best results to date on competitive information retrieval tasks for ImageNet and SIFT 1M, improving MAP from 73% to 84% and reducing query cost by a factor of 2-8, respectively.

Motivation & Objective

• To address the poor performance of data-to-results hashing methods in similarity search by improving the training signal for binary hash code learning.
• To develop a differentiable, general-purpose loss function that models the probability of two inputs being within a target Hamming distance.
• To reduce query cost in approximate nearest neighbor search while maintaining or improving recall through better hash code learning and multi-indexing.
• To demonstrate state-of-the-art performance on competitive benchmarks like ImageNet and SIFT-1M using end-to-end learned hash codes.

Proposed method

• The HDT loss function uses log-likelihood of the probability that two inputs fall within a Hamming distance target, approximated via a sigmoid-based model of the pre-binarized output distribution.
• The training scheme improves gradient estimation by sampling all pairs of inputs within each minibatch and computing loss terms between them.
• The method is compatible with any differentiable model and similarity function, enabling end-to-end training of neural networks for binary hashing.
• Multi-indexing is used to efficiently retrieve results within the Hamming distance target, enabling fast and scalable similarity search.
• The model uses a temperature-scheduled sigmoid to encourage outputs to converge to ±1, improving binarization stability.
• Hyperparameters such as the Hamming distance target (r), loss ratio for false positives (λ), and hash bit length (n) are tuned to optimize recall and query cost.

Experimental results

Research questions

• RQ1 Can a differentiable loss function that models the probability of Hamming distance proximity improve binary hash code learning?
• RQ2 Does improved gradient estimation through full pair-wise loss computation in mini-batches lead to better generalization in hashing?
• RQ3 Can HDT achieve state-of-the-art performance on standard benchmarks like ImageNet and SIFT-1M while reducing query cost?
• RQ4 Why does HDT perform better on shorter hash lengths, defying the expectation that longer hashes should always outperform shorter ones?
• RQ5 How does the statistical modeling of Hamming distance probability compare to standard loss functions like cross-entropy or L2 quantization loss?

Key findings

• HDT achieved 83.8% MAP on ImageNet at 16 bits, a 10.5% absolute improvement over the next best method, with 84% MAP reported as the best result to date.
• On SIFT-1M, HDT-E achieved 78.1% recall at 12,709 distance comparisons, outperforming product quantization (PQ) which required 101,158 comparisons for 74.4% recall.
• HDT reduced query cost by a factor of 2–8 on SIFT-1M and improved MAP by 11 percentage points on ImageNet compared to prior methods.
• The method achieved higher recall than PQ even in the high-recall regime, demonstrating superior efficiency and effectiveness.
• HDT showed strong performance at low bit lengths (e.g., 16 bits), suggesting high representational efficiency and potential for overfitting at longer lengths.
• The use of multi-indexing enabled fast retrieval within the Hamming distance target, making HDT practical for large-scale applications.