[Paper Review] Generative Semantic Hashing Enhanced via Boltzmann Machines

Lin Zheng, Qinliang Su+2 more|arXiv (Cornell University)|Jun 16, 2020

Advanced Image and Video Retrieval Techniques28 references4 citations

TL;DR

This paper proposes a novel generative semantic hashing model that introduces correlations among hash code bits using a Boltzmann machine as the variational posterior, overcoming the independence assumption in prior methods. By reparameterizing the Boltzmann distribution and deriving an asymptotically exact ELBO lower bound, the model enables efficient end-to-end training and achieves state-of-the-art retrieval performance across multiple benchmarks without degradation as code length increases.

ABSTRACT

Generative semantic hashing is a promising technique for large-scale information retrieval thanks to its fast retrieval speed and small memory footprint. For the tractability of training, existing generative-hashing methods mostly assume a factorized form for the posterior distribution, enforcing independence among the bits of hash codes. From the perspectives of both model representation and code space size, independence is always not the best assumption. In this paper, to introduce correlations among the bits of hash codes, we propose to employ the distribution of Boltzmann machine as the variational posterior. To address the intractability issue of training, we first develop an approximate method to reparameterize the distribution of a Boltzmann machine by augmenting it as a hierarchical concatenation of a Gaussian-like distribution and a Bernoulli distribution. Based on that, an asymptotically-exact lower bound is further derived for the evidence lower bound (ELBO). With these novel techniques, the entire model can be optimized efficiently. Extensive experimental results demonstrate that by effectively modeling correlations among different bits within a hash code, our model can achieve significant performance gains.

Motivation & Objective

• To address the limitation of independent bit assumptions in existing generative semantic hashing models, which restrict representational capacity and code space efficiency.
• To enable effective modeling of complex correlations among hash code bits to improve semantic representation quality.
• To develop a tractable training method for Boltzmann machine-based variational posteriors, which are otherwise intractable due to normalization term issues.
• To achieve end-to-end, label-free training of semantic hashing models while maintaining computational efficiency.

Proposed method

• Proposes using the Boltzmann machine distribution as a non-factorized variational posterior to model dependencies among binary hash code bits.
• Reformulates the Boltzmann distribution as a hierarchical concatenation of a Gaussian-like and a Bernoulli distribution to enable reparameterization for gradient-based optimization.
• Derives an asymptotically exact lower bound for the evidence lower bound (ELBO) to handle the intractable normalization constant in Boltzmann machines.
• Employs a low-rank perturbation of the covariance matrix in the Gaussian component to balance model complexity and computational cost.
• Uses a mixture of k components to approximate the posterior, with the tightness of the bound increasing with k.
• Enables end-to-end training via backpropagation using the reparameterized gradients and the derived ELBO bound.

Experimental results

Research questions

• RQ1 Can modeling correlations among hash code bits improve retrieval performance in generative semantic hashing?
• RQ2 How can the intractable normalization term in Boltzmann machine posteriors be addressed for efficient training?
• RQ3 Does introducing structured dependencies between bits prevent performance degradation as code length increases?
• RQ4 Can a non-factorized posterior based on Boltzmann machines be reparameterized effectively for gradient-based optimization?
• RQ5 How do the hyperparameters v (low-rank dimension) and k (mixture components) affect model performance and trade-offs?

Key findings

• The proposed model achieves the highest retrieval precision among all compared models on all three public datasets, with a 0.8465 precision on the Reuters dataset using 64-bit codes.
• Performance improves monotonically with increasing rank parameter v, with the largest gain observed between v=0 and v=1, indicating strong benefit from even low-order correlations.
• The model maintains or improves performance as code length increases, contrary to the typical degradation seen in independent-bit models.
• Using more mixture components (k) leads to tighter ELBO bounds and better retrieval precision, with k=10 achieving the best performance.
• The model with v=10 and k=10 achieves a 0.8465 precision on Reuters, significantly outperforming NASH (0.7812 at v=0) and showing minimal improvement beyond v=10.
• Visualization on 20Newsgroups shows that the proposed model generates the most semantically separable hash codes, with distinct clusters per category.