[Paper Review] Automatic Feature Weight Determination using Indexing and Pseudo-Relevance Feedback for Multi-feature Content-Based Image Retrieval

Asheet Kumar, Shivam Choudhary+3 more|arXiv (Cornell University)|Dec 10, 2018

Image Retrieval and Classification Techniques4 citations

TL;DR

This paper proposes a novel multi-feature content-based image retrieval (CBIR) framework that automatically determines optimal feature weights using indexing and pseudo-relevance feedback. By combining color, texture, and shape descriptors (CDH, LBP, CLD, EOH) with multiclass SVM-based indexing and a relevance ratio/mean difference technique, the method reduces search space by 81.25% and achieves superior retrieval accuracy, with the combined feature approach yielding the highest AUC (0.951) on Dataset D3 using Canberra distance.

ABSTRACT

Content-based image retrieval (CBIR) is one of the most active research areas in multimedia information retrieval. Given a query image, the task is to search relevant images in a repository. Low level features like color, texture, and shape feature vectors of an image are always considered to be an important attribute in CBIR system. Thus the performance of the CBIR system can be enhanced by combining these feature vectors. In this paper, we propose a novel CBIR framework by applying to index using multiclass SVM and finding the appropriate weights of the individual features automatically using the relevance ratio and mean difference. We have taken four feature descriptors to represent color, texture and shape features. During retrieval, feature vectors of query image are combined, weighted and compared with feature vectors of images in the database to rank order the results. Experiments were performed on four benchmark datasets and performance is compared with existing techniques to validate the superiority of our proposed framework.

Motivation & Objective

• Address the challenge of optimal feature weight assignment in multi-feature CBIR without requiring labeled training data.
• Reduce computational cost in large-scale image retrieval by leveraging image indexing to limit search space.
• Improve retrieval accuracy by automatically identifying the most relevant feature descriptors for each query image.
• Develop a framework that combines multiple low-level features (color, texture, shape) with dynamic weighting to enhance retrieval performance.
• Validate the proposed method on multiple benchmark datasets against state-of-the-art techniques.

Proposed method

• Employ multiclass Support Vector Machine (SVM) for image indexing to predict top 3 relevant image categories, reducing the search space by 81.25%.
• Use four feature descriptors: Color Difference Histogram (CDH), Local Binary Pattern (LBP), Co-occurrence Local Descriptor (CLD), and Edge Orientation Histogram (EOH) for color, texture, and shape representation.
• Apply relevance ratio and mean difference techniques to automatically compute optimal weights for individual features based on their performance in precision-recall curves.
• Combine weighted feature vectors using distance metrics (Canberra, chi-square, Euclidean) to rank images in the database.
• Utilize pseudo-relevance feedback by selecting top-ranked images as pseudo-relevant to refine feature weight assignment iteratively.
• Normalize initial weights using area under the precision-recall curve (AUC) for each feature descriptor to ensure balanced initialization.

Experimental results

Research questions

• RQ1 How can feature weights be automatically determined in a multi-feature CBIR system without requiring labeled training data?
• RQ2 To what extent does image indexing using multiclass SVM reduce the search space in large-scale CBIR systems?
• RQ3 Which combination of feature descriptors (CDH, LBP, CLD, EOH) yields the highest retrieval accuracy across diverse datasets?
• RQ4 How does the proposed automatic weight assignment method compare to equal-weighting or fixed-weight strategies in terms of precision and recall?
• RQ5 Can the relevance ratio and mean difference technique effectively identify the most discriminative feature descriptor for a given query image?

Key findings

• The proposed method reduced the search space by 81.25% through multiclass SVM-based image indexing, significantly improving retrieval efficiency.
• The combined feature descriptor with automatically assigned weights achieved the highest area under the precision-recall curve (AUC = 0.951) on Dataset D3 using Canberra distance.
• Color Difference Histogram (CDH) alone achieved the highest individual AUC (0.97) on Dataset D1, but the combined approach outperformed all individual features across all datasets.
• The relevance ratio-based weight assignment technique consistently improved retrieval accuracy compared to equal-weighting or fixed-weight strategies, with the best performance observed using Canberra distance.
• The method successfully identified the most relevant feature descriptor for each query image, with CLD receiving the highest weights on Dataset D2 and D3, while CDH dominated on D1.
• On average, the combined feature approach improved AUC by 8.5% compared to the best individual feature across all datasets and distance metrics.