[Paper Review] Locality-Sensitive Hashing with Margin Based Feature Selection

Makiko Konoshima, Yui Noma|arXiv (Cornell University)|Sep 26, 2012

Advanced Image and Video Retrieval Techniques10 references3 citations

TL;DR

This paper proposes S-LSH, a locality-sensitive hashing method that uses margin-based feature selection to learn hyperplanes from high-dimensional data, improving search accuracy in scenarios with many labels and sparse label distributions. By selecting optimal bits from a larger set of candidate hyperplanes, S-LSH achieves superior performance over LSH, MLH, and SH, especially on fingerprint, speech, and image data with high label diversity and low label cardinality.

ABSTRACT

We propose a learning method with feature selection for Locality-Sensitive Hashing. Locality-Sensitive Hashing converts feature vectors into bit arrays. These bit arrays can be used to perform similarity searches and personal authentication. The proposed method uses bit arrays longer than those used in the end for similarity and other searches and by learning selects the bits that will be used. We demonstrated this method can effectively perform optimization for cases such as fingerprint images with a large number of labels and extremely few data that share the same labels, as well as verifying that it is also effective for natural images, handwritten digits, and speech features.

Motivation & Objective

• To address the challenge of low search accuracy in locality-sensitive hashing when dealing with high-dimensional data that have many labels and sparse label distributions.
• To overcome limitations of existing learning-based LSH methods like MLH, which require differentiable loss functions and suffer from slow convergence and poor scalability with large bit counts.
• To develop a feature selection mechanism that avoids exhaustive optimization over large solution spaces while maintaining high search accuracy.
• To demonstrate effectiveness across diverse data types, including fingerprint images, speech features, handwritten digits, and natural images.
• To show that margin-based bit selection from a larger pool of hyperplanes leads to better generalization and robustness in 1:N biometric identification tasks.

Proposed method

• The method generates a large set of candidate hyperplanes (denoted as $ ilde{B} $) using random projections, creating bit arrays of length $ ilde{B} $ from input feature vectors.
• From these $ ilde{B} $ bits, the method performs feature selection to identify a smaller subset of $ B $ bits (e.g., $ B = 1024 $) that maximizes margin-based separation between same-label and different-label data pairs.
• The selection process uses a margin-based criterion that prioritizes bits where the Hamming distance between same-label pairs is minimized and between different-label pairs is maximized.
• The final bit array is formed by selecting only the $ B $ most discriminative bits from the $ ilde{B} $-bit representation, improving Hamming distance correlation with true similarity.
• The method does not assume differentiability of the objective function, enabling efficient optimization even for large bit counts.
• The learning process is linear in the number of training samples and $ ilde{B} $, making it scalable compared to optimization-based methods like MLH.

Experimental results

Research questions

• RQ1 Can a margin-based feature selection strategy improve the accuracy of locality-sensitive hashing in high-dimensional data with many labels and low label cardinality?
• RQ2 How does the proposed S-LSH method compare to conventional LSH and learning-based methods like MLH and SH in terms of error rate and scalability?
• RQ3 Does selecting from a larger pool of candidate hyperplanes (i.e., $ ilde{B} > B $) lead to better generalization than directly optimizing $ B $ hyperplanes?
• RQ4 In what types of data—such as fingerprint images, speech, handwritten digits, or natural images—does the proposed method show the most significant improvement over baseline methods?
• RQ5 How does the method perform when label distribution is highly imbalanced, with many labels and only a few samples per label?

Key findings

• S-LSH achieved the lowest error rate across all tested datasets (LabelMe, MNIST, speech, fingerprint) at 1,024 bits, outperforming LSH, SH, and MLH.
• For fingerprint data, S-LSH reduced the error rate below that of LSH and MLH as the number of bits increased, with the largest improvement observed at higher bit counts.
• On the MNIST dataset, MLH slightly outperformed S-LSH, but S-LSH was the top performer on all other datasets, including those with high label counts and low label cardinalities.
• The processing time for S-LSH was significantly higher than LSH (e.g., 9,131.7s for fingerprint data) but remained feasible and scalable, with linear dependence on training data size and $ ilde{B} $.
• S-LSH demonstrated robustness in high-label, low-cardinality scenarios—such as fingerprint images with ~1,300 labels and only 7 samples per label—where conventional methods like MLH failed to improve performance.
• The method’s performance ranking (Table 2) confirmed S-LSH as the top method for LabelMe (rank 1), MNIST (rank 2), speech (rank 1), and fingerprint (rank 1), indicating broad applicability.