[Paper Review] Transfer Adversarial Hashing for Hamming Space Retrieval

Zhangjie Cao, Mingsheng Long+2 more|arXiv (Cornell University)|Dec 13, 2017

Advanced Image and Video Retrieval Techniques4 citations

TL;DR

This paper proposes Transfer Adversarial Hashing (TAH), a deep learning framework that enables efficient Hamming space retrieval across domain-shifted data by combining a pairwise t-distribution cross-entropy loss to concentrate similar hash codes within Hamming radius 2 and an adversarial network to align source and target domain distributions. TAH achieves state-of-the-art performance on NUS-WIDE and VisDA2017 benchmarks, significantly improving retrieval accuracy under transfer settings where source and target domains differ.

ABSTRACT

Hashing is widely applied to large-scale image retrieval due to the storage and retrieval efficiency. Existing work on deep hashing assumes that the database in the target domain is identically distributed with the training set in the source domain. This paper relaxes this assumption to a transfer retrieval setting, which allows the database and the training set to come from different but relevant domains. However, the transfer retrieval setting will introduce two technical difficulties: first, the hash model trained on the source domain cannot work well on the target domain due to the large distribution gap; second, the domain gap makes it difficult to concentrate the database points to be within a small Hamming ball. As a consequence, transfer retrieval performance within Hamming Radius 2 degrades significantly in existing hashing methods. This paper presents Transfer Adversarial Hashing (TAH), a new hybrid deep architecture that incorporates a pairwise $t$-distribution cross-entropy loss to learn concentrated hash codes and an adversarial network to align the data distributions between the source and target domains. TAH can generate compact transfer hash codes for efficient image retrieval on both source and target domains. Comprehensive experiments validate that TAH yields state of the art Hamming space retrieval performance on standard datasets.

Motivation & Objective

• To address the challenge of large distribution gaps between source and target domains in deep hashing for image retrieval.
• To improve Hamming space retrieval performance within radius 2 when training and database data come from different but related domains.
• To develop a unified deep architecture that jointly learns similarity relationships and aligns domain distributions for transferable hash codes.
• To overcome the degradation in retrieval quality caused by sparse distribution of similar points in target domain Hamming balls.

Proposed method

• Introduces a pairwise t-distribution cross-entropy loss to encourage similar data points to be concentrated within a Hamming radius 2 ball in the hash space.
• Employs an adversarial network to align the feature distributions of source and target domains, reducing domain shift.
• Designs an end-to-end deep neural network that jointly optimizes for both similarity preservation and domain alignment.
• Utilizes unsupervised target domain data to enhance domain adaptation without requiring labeled data in the target domain.
• Leverages the hybrid architecture to generate compact, transferable hash codes that maintain semantic similarity across domains.
• Applies adversarial training to minimize the distribution gap between source and target domains in the latent space.

Experimental results

Research questions

• RQ1 Can a deep hashing model trained on a source domain generalize effectively to a target domain with a different data distribution?
• RQ2 How can hash codes be optimized to maintain high retrieval accuracy within Hamming radius 2 under domain shift?
• RQ3 To what extent does a t-distribution-based similarity loss improve concentration of similar points in Hamming space compared to standard cross-entropy?
• RQ4 Does adversarial domain alignment outperform distribution matching methods like MMD in transfer hashing?
• RQ5 How do the components of TAH—similarity learning and domain alignment—contribute independently to retrieval performance?

Key findings

• TAH achieves state-of-the-art mean average precision (MAP) on both NUS-WIDE and VisDA2017 datasets under transfer retrieval settings.
• On the VisDA2017 dataset, TAH achieves a MAP of 0.784 at 48 bits for the real→synthetic task, outperforming the next best method by 0.067.
• The variant TAH-t, which replaces the t-distribution loss with standard cross-entropy, shows a 0.031 average MAP drop, confirming the superiority of the t-distribution loss.
• The variant TAH-A, which omits adversarial learning, suffers a 0.078 average MAP drop on the synthetic→real task, proving the importance of domain alignment.
• Precision remains stable while recall increases across retrieval thresholds, indicating that TAH’s t-distribution loss effectively concentrates relevant points within Hamming radius 2.
• Visualizations show that TAH retrieves more relevant and user-desired images than DHN and HashNet, especially in cross-domain settings.