[Paper Review] Probability Weighted Compact Feature for Domain Adaptive Retrieval

Fuxiang Huang, Lei Zhang+2 more|arXiv (Cornell University)|Mar 6, 2020

Domain Adaptation and Few-Shot Learning33 references4 citations

TL;DR

This paper proposes Probability Weighted Compact Feature Learning (PWCF), a novel method for cross-domain image retrieval that leverages Bayesian perspective-derived losses and a Histogram Feature of Neighbors (HFON) to model inter-domain correlations and mitigate distribution discrepancies. By integrating BP-induced focal-triplet, quantization, and classification losses with a shared manifold structure, PWCF achieves state-of-the-art performance in both cross-domain and single-domain retrieval, outperforming existing methods on benchmark datasets with compact binary codes.

ABSTRACT

Domain adaptive image retrieval includes single-domain retrieval and cross-domain retrieval. Most of the existing image retrieval methods only focus on single-domain retrieval, which assumes that the distributions of retrieval databases and queries are similar. However, in practical application, the discrepancies between retrieval databases often taken in ideal illumination/pose/background/camera conditions and queries usually obtained in uncontrolled conditions are very large. In this paper, considering the practical application, we focus on challenging cross-domain retrieval. To address the problem, we propose an effective method named Probability Weighted Compact Feature Learning (PWCF), which provides inter-domain correlation guidance to promote cross-domain retrieval accuracy and learns a series of compact binary codes to improve the retrieval speed. First, we derive our loss function through the Maximum A Posteriori Estimation (MAP): Bayesian Perspective (BP) induced focal-triplet loss, BP induced quantization loss and BP induced classification loss. Second, we propose a common manifold structure between domains to explore the potential correlation across domains. Considering the original feature representation is biased due to the inter-domain discrepancy, the manifold structure is difficult to be constructed. Therefore, we propose a new feature named Histogram Feature of Neighbors (HFON) from the sample statistics perspective. Extensive experiments on various benchmark databases validate that our method outperforms many state-of-the-art image retrieval methods for domain adaptive image retrieval. The source code is available at https://github.com/fuxianghuang1/PWCF

Motivation & Objective

• Address the performance drop in image retrieval when queries and databases come from different domains due to distribution discrepancies.
• Improve retrieval accuracy in cross-domain scenarios where traditional methods assume similar data distributions.
• Develop a compact binary feature representation to accelerate retrieval speed while preserving cross-domain semantic similarity.
• Mitigate the impact of domain shift by modeling inter-domain sample correlations through a shared manifold structure.
• Propose a robust similarity metric, Histogram Feature of Neighbors (HFON), to better capture cross-domain neighborhood relationships.

Proposed method

• Derive three Bayesian Perspective (BP)-induced loss functions: focal-triplet loss, quantization loss, and classification loss to optimize binary feature learning.
• Construct a common manifold structure across domains to preserve meaningful nearest neighbor relationships in Hamming space.
• Introduce Histogram Feature of Neighbors (HFON) as a domain-robust feature representation to reduce bias from inter-domain distribution shifts.
• Use Maximum A Posteriori (MAP) estimation to derive the loss functions, enhancing feature discrimination and reducing quantization error.
• Employ a variable alternating optimization strategy to jointly train the model components and ensure convergence.
• Integrate all components into a unified framework that jointly optimizes feature learning, quantization, and cross-domain correlation modeling.

Experimental results

Research questions

• RQ1 Can Bayesian-optimized loss functions improve cross-domain image retrieval performance compared to standard triplet or classification losses?
• RQ2 How effective is the proposed Histogram Feature of Neighbors (HFON) in modeling similarity between cross-domain samples under distribution shifts?
• RQ3 To what extent does a shared manifold structure enhance inter-domain correlation learning in binary feature space?
• RQ4 Does the integration of multiple BP-induced losses lead to better compact binary code representation than individual components?
• RQ5 How does PWCF perform across diverse datasets and code lengths in both cross-domain and single-domain retrieval settings?

Key findings

• PWCF achieves a mean average precision (mAP) of 73.89% on the Office-Home dataset with 128-bit codes, significantly outperforming state-of-the-art methods.
• On the MNIST&USPS dataset, PWCF reaches 71.64% mAP at 64 bits for cross-domain retrieval, demonstrating strong generalization.
• Ablation studies confirm that removing any component—especially the BP-induced focal-triplet loss or HFON—leads to a significant drop in performance.
• The proposed method converges quickly, with objective function curves showing stable convergence within a few iterations on multiple datasets.
• PWCF maintains high performance across various code lengths, showing robustness and scalability in practical deployment.
• The inclusion of the manifold loss and HFON consistently improves retrieval accuracy, proving their effectiveness in modeling cross-domain relationships.