[Paper Review] Improving Query Expansion Using WordNet

Dipasree Pal, Mandar Mitra+1 more|arXiv (Cornell University)|Sep 19, 2013

Topic Modeling30 references4 citations

TL;DR

This paper proposes a novel query expansion method that leverages WordNet to enhance term usefulness assessment by combining semantic similarity from WordNet definitions with term distribution and statistical association in pseudo-relevant documents. The method significantly outperforms existing WordNet-based and standard QE techniques like KLD and RM3 across multiple TREC collections, demonstrating consistent improvements in mean average precision (MAP).

ABSTRACT

This study proposes a new way of using WordNet for Query Expansion (QE). We choose candidate expansion terms, as usual, from a set of pseudo relevant documents; however, the usefulness of these terms is measured based on their definitions provided in a hand-crafted lexical resource like WordNet. Experiments with a number of standard TREC collections show that this method outperforms existing WordNet based methods. It also compares favorably with established QE methods such as KLD and RM3. Leveraging earlier work in which a combination of QE methods was found to outperform each individual method (as well as other well-known QE methods), we next propose a combination-based QE method that takes into account three different aspects of a candidate expansion term's usefulness: (i) its distribution in the pseudo relevant documents and in the target corpus, (ii) its statistical association with query terms, and (iii) its semantic relation with the query, as determined by the overlap between the WordNet definitions of the term and query terms. This combination of diverse sources of information appears to work well on a number of test collections, viz., TREC123, TREC5, TREC678, TREC robust new and TREC910 collections, and yields significant improvements over competing methods on most of these collections.

Motivation & Objective

• To address the limited effectiveness of WordNet in query expansion by integrating it more meaningfully with retrieval statistics.
• To overcome the shortcomings of prior WordNet-based methods that rely solely on lexical similarity or co-occurrence without considering term rarity or document context.
• To develop a robust combination method that integrates multiple sources of evidence—distribution, association, and semantics—for more reliable candidate expansion term selection.
• To validate that combining diverse signal types (distributional, statistical, semantic) yields better performance than individual approaches.

Proposed method

• Candidate expansion terms are extracted from top-ranked pseudo-relevant documents retrieved using the original query.
• Term usefulness is assessed via three distinct features: (1) distribution in pseudo-relevant vs. full corpus, (2) statistical association with query terms, and (3) semantic similarity via WordNet definition overlap.
• A weighted combination method (KLWNET) integrates these three features using a fusion formula (Equation 9), where each component contributes based on its reliability.
• Semantic similarity is computed by measuring overlap between WordNet definitions of the candidate term and query terms, using a lexical overlap metric.
• The method preserves original query term weights while intelligently adjusting expansion term weights based on their combined signal strength.
• The final expanded query is formed using a weighted sum of original and expanded terms, with weights derived from the fusion of all three components.

Experimental results

Research questions

• RQ1 Can WordNet definitions be effectively used to improve the selection of candidate expansion terms beyond simple synonymy?
• RQ2 Does combining distributional, statistical, and semantic signals lead to better query expansion performance than using any single signal alone?
• RQ3 How does the proposed method compare to established QE techniques like KLD and RM3 in terms of MAP across diverse TREC collections?
• RQ4 Can a hybrid approach that fuses multiple signal types outperform individual methods, especially in cases where one method fails?

Key findings

• The proposed KLWNET method achieves significantly higher mean average precision (MAP) than MII_mp, KLD, and RM3 on TREC7 and TREC8 collections, with improvements confirmed to be statistically significant.
• On the TREC678 collection, KLWNET outperforms both P-WNET and KLDLCA on 40 queries, and achieves intermediate performance on the remaining 110, demonstrating robustness across query types.
• For query 316 (polygamy, polyandry, polygyny), KLWNET maintains higher original term weight (1.00) compared to KLDLCA (1.00), while assigning a lower weight to the less relevant expansion term 'children' (0.48 vs. 0.69), leading to better performance.
• For query 361 (clothing sweatshops), KLWNET successfully includes the useful term 'shop' with intermediate weight, avoiding the overemphasis seen in P-WNET and underuse in KLDLCA, resulting in balanced performance.
• The combination method KLWNET consistently outperforms the baseline (original query) across all tested TREC collections (TREC123, TREC5, TREC678, TREC robust new, TREC910), with significant MAP improvements.
• The method shows that integrating semantic information from WordNet definitions with distributional and statistical features leads to a more reliable and effective query expansion strategy than any single-source approach.