[Paper Review] A novel model for query expansion using pseudo-relevant web knowledge

Hiteshwar Kumar Azad, Akshay Deepak|arXiv (Cornell University)|Aug 27, 2019

Information Retrieval and Search Behavior53 references4 citations

TL;DR

This paper proposes a novel Web Knowledge-based Query Expansion (WKQE) model that enhances information retrieval by leveraging pseudo-relevant web content from Google, Bing, and DuckDuckGo to generate expanded query terms. Using a three-tiered weighting strategy—tf-itf, kNN-based cosine similarity, and correlation scoring—the model improves retrieval effectiveness, achieving a 25.89% gain in MAP and 30.83% in GMAP over unexpanded queries on the FIRE dataset.

ABSTRACT

In the field of information retrieval, query expansion (QE) has long been used as a technique to deal with the fundamental issue of word mismatch between a user's query and the target information. In the context of the relationship between the query and expanded terms, existing weighting techniques often fail to appropriately capture the term-term relationship and term to the whole query relationship, resulting in low retrieval effectiveness. Our proposed QE approach addresses this by proposing three weighting models based on (1) tf-itf, (2) k-nearest neighbor (kNN) based cosine similarity, and (3) correlation score. Further, to extract the initial set of expanded terms, we use pseudo-relevant web knowledge consisting of the top N web pages returned by the three popular search engines namely, Google, Bing, and DuckDuckGo, in response to the original query. Among the three weighting models, tf-itf scores each of the individual terms obtained from the web content, kNN-based cosine similarity scores the expansion terms to obtain the term-term relationship, and correlation score weighs the selected expansion terms with respect to the whole query. The proposed model, called web knowledge based query expansion (WKQE), achieves an improvement of 25.89% on the MAP score and 30.83% on the GMAP score over the unexpanded queries on the FIRE dataset. A comparative analysis of the WKQE techniques with other related approaches clearly shows significant improvement in the retrieval performance. We have also analyzed the effect of varying the number of pseudo-relevant documents and expansion terms on the retrieval effectiveness of the proposed model.

Motivation & Objective

• To address the vocabulary mismatch problem in information retrieval caused by short, ambiguous user queries.
• To improve query expansion by capturing both term-term relationships and term-to-query relevance more effectively than existing weighting models.
• To explore the effectiveness of combining pseudo-relevant results from multiple search engines (Google, Bing, DuckDuckGo) as a source for expansion terms.
• To evaluate the impact of varying numbers of pseudo-relevant documents and expansion terms on retrieval performance.
• To develop and validate a hybrid weighting framework that integrates tf-itf, kNN-based similarity, and correlation scoring for term selection.

Proposed method

• Collects the top N web results from three major search engines (Google, Bing, DuckDuckGo) for each user query to form a pseudo-relevant document pool.
• Extracts candidate expansion terms from the aggregated content of these top-ranked web pages.
• Applies a tf-itf weighting scheme to score individual terms based on their frequency and inverse term frequency in the web content.
• Uses k-nearest neighbor (kNN) based cosine similarity to identify semantically related expansion terms and model term-term relationships.
• Computes a correlation score to evaluate how well each expansion term aligns with the entire original query.
• Combines the three weighting scores to rank and select the final set of expansion terms for query reformulation.

Experimental results

Research questions

• RQ1 How does combining web content from multiple search engines improve query expansion effectiveness compared to single-source approaches?
• RQ2 To what extent do the three proposed weighting models—tf-itf, kNN-based cosine similarity, and correlation score—improve term relevance and retrieval performance?
• RQ3 How does the number of pseudo-relevant documents and expansion terms affect the overall retrieval effectiveness of the WKQE model?
• RQ4 How does the WKQE model compare to baseline and state-of-the-art query expansion techniques on standard evaluation metrics?
• RQ5 Can the proposed three-tiered weighting strategy better capture both term-term and term-to-query relationships than existing methods?

Key findings

• The WKQE model achieves a 25.89% improvement in Mean Average Precision (MAP) and a 30.83% improvement in GMAP over unexpanded queries on the FIRE dataset.
• The kNN-based cosine similarity and correlation scoring models significantly outperform individual term frequency methods in capturing semantic relationships.
• The combination of results from three search engines (Google, Bing, DuckDuckGo) yields more diverse and relevant expansion terms than using a single engine.
• The model's performance is sensitive to the number of expansion terms, with optimal results observed at 15–20 terms, beyond which performance plateaus or declines.
• The proposed GBDQE variant, which integrates all three weighting models, consistently outperforms other configurations and baseline methods across all evaluation metrics.
• Case studies show that the model generates semantically coherent and contextually relevant expansion terms, such as adding 'nasa', 'mars', and 'space' for a query about life in space.