[Paper Review] Sparse similarity-preserving hashing

Jonathan Masci, Alex Bronstein+3 more|arXiv (Cornell University)|Dec 19, 2013

Advanced Image and Video Retrieval Techniques28 references17 citations

TL;DR

This paper proposes SparseHash, a novel similarity-preserving hashing method that enforces sparsity in binary hash codes to achieve higher recall at comparable precision to state-of-the-art dense hashing methods. By using an ISTA-inspired feed-forward neural network with $oldsymbol{ ext{ℓ}}_1$ regularization, the method enables efficient Hamming distance computation while maintaining low false positive and false negative rates, outperforming dense hashing in both visual and multi-modal retrieval tasks.

ABSTRACT

In recent years, a lot of attention has been devoted to efficient nearest neighbor search by means of similarity-preserving hashing. One of the plights of existing hashing techniques is the intrinsic trade-off between performance and computational complexity: while longer hash codes allow for lower false positive rates, it is very difficult to increase the embedding dimensionality without incurring in very high false negatives rates or prohibiting computational costs. In this paper, we propose a way to overcome this limitation by enforcing the hash codes to be sparse. Sparse high-dimensional codes enjoy from the low false positive rates typical of long hashes, while keeping the false negative rates similar to those of a shorter dense hashing scheme with equal number of degrees of freedom. We use a tailored feed-forward neural network for the hashing function. Extensive experimental evaluation involving visual and multi-modal data shows the benefits of the proposed method.

Motivation & Objective

• Address the trade-off between performance and computational complexity in similarity-preserving hashing by introducing sparsity into hash codes.
• Overcome the limitations of dense hashing, where long codes increase false negatives and computational cost, while short codes reduce precision.
• Develop a neural network-based framework that learns sparse, discriminative hash codes through $oldsymbol{ ext{ℓ}}_1$-regularized optimization.
• Extend the method to multi-modal and cross-modal retrieval tasks, enabling joint learning across image and text data.
• Demonstrate that sparse codes with the same number of degrees of freedom as dense codes achieve significantly better recall without sacrificing precision.

Proposed method

• Formulate the hashing problem as an $oldsymbol{ ext{ℓ}}_1$-regularized minimization of the aggregate false positive and false negative rates to enforce sparsity in the hash codes.
• Design a feed-forward neural network architecture inspired by iterative shrinkage thresholding algorithms (ISTA), emulating the structure of sparse coding solvers.
• Train the network using stochastic gradient descent, enabling scalability to large-scale datasets.
• Use a differentiable, differentiable activation function to allow end-to-end backpropagation through the sparse coding layers.
• Integrate the network into a similarity-preserving hashing framework where the output codes preserve pairwise similarities from the input data.
• Extend the method to multi-modal learning by jointly embedding images and text into a shared sparse binary space using a unified loss function.

Experimental results

Research questions

• RQ1 Can enforcing sparsity in binary hash codes improve recall while maintaining low false positive rates compared to dense hashing?
• RQ2 Does a neural network architecture inspired by ISTA algorithms effectively learn discriminative sparse codes for similarity-preserving hashing?
• RQ3 Can sparse hashing achieve comparable or better precision-recall performance than state-of-the-art dense hashing methods with the same number of degrees of freedom?
• RQ4 How does the proposed method perform in cross-modal retrieval tasks involving images and text?
• RQ5 Can the sparsity of the hash codes lead to lower computational cost in Hamming distance computation, especially for non-zero radius nearest neighbor search?

Key findings

• SparseHash achieves 84.24% mean average precision (mAP) on the CIFAR-10 dataset with 256-bit codes, matching or exceeding state-of-the-art methods like DH and SSH while maintaining low false positive and false negative rates.
• On the NUS-WIDE multi-modal dataset, MM-SparseHash achieves 61.52% mAP for image-to-tag retrieval and 59.52% for tag-to-image retrieval, outperforming CM-SSH and AGH in cross-modal recall.
• With only 64-bit codes, SparseHash achieves 74.17% mAP on CIFAR-10, significantly outperforming dense methods like DH and SSH at the same code length.
• The method reduces false positive and false negative rates by up to 50% compared to dense hashing with equivalent degrees of freedom, particularly evident in 64-bit codes.
• The proposed network architecture enables efficient partial collision detection in Hamming balls, reducing computational cost compared to dense counterparts.
• Visual inspection of retrieval results shows that MM-SparseHash retrieves semantically relevant tags even when they were not in the original image tags, indicating robust semantic generalization.