[Paper Review] Fast Passage Re-ranking with Contextualized Exact Term Matching and Efficient Passage Expansion

Shengyao Zhuang, Guido Zuccon|arXiv (Cornell University)|Aug 19, 2021

Topic Modeling45 references48 citations

TL;DR

TILDEv2 replaces TILDE with contextualized exact term matching and passage expansion to achieve state-of-the-art CPU-only passage re-ranking, reducing index size by up to 99% and maintaining latency under 100 ms while improving effectiveness over TILDE.

ABSTRACT

BERT-based information retrieval models are expensive, in both time (query latency) and computational resources (energy, hardware cost), making many of these models impractical especially under resource constraints. The reliance on a query encoder that only performs tokenization and on the pre-processing of passage representations at indexing, has allowed the recently proposed TILDE method to overcome the high query latency issue typical of BERT-based models. This however is at the expense of a lower effectiveness compared to other BERT-based re-rankers and dense retrievers. In addition, the original TILDE method is characterised by indexes with a very high memory footprint, as it expands each passage into the size of the BERT vocabulary. In this paper, we propose TILDEv2, a new model that stems from the original TILDE but that addresses its limitations. TILDEv2 relies on contextualized exact term matching with expanded passages. This requires to only store in the index the score of tokens that appear in the expanded passages (rather than all the vocabulary), thus producing indexes that are 99% smaller than those of TILDE. This matching mechanism also improves ranking effectiveness by 24%, without adding to the query latency. This makes TILDEv2 the state-of-the-art passage re-ranking method for CPU-only environments, capable of maintaining query latency below 100ms on commodity hardware.

Motivation & Objective

• Address the high query latency of BERT-based re-rankers by enabling CPU-friendly second-stage ranking.
• Reduce index memory footprint compared to TILDE without sacrificing effectiveness.
• Introduce contextualized exact term matching to replace query-likelihood matching.
• Propose a fast passage expansion method to mitigate vocabulary mismatch.
• Demonstrate state-of-the-art performance on MS MARCO and DL2019/2020 datasets under CPU constraints.

Proposed method

• Tokenizer-based query encoder that encodes queries into sparse, query-length feature vectors using the BERT tokenizer (no model inference at query time).
• Contextualized exact term matching where passage tokens are assigned scalar weights via a BERT-based projection, enabling exact term matching with the passage’s tokens.
• Use of Noise-contrastive Estimation (NCE) loss for training with negative samples (S(q,p+), S(q,p−)).
• Passage expansion at indexing time to mitigate vocabulary mismatch by appending semantically related tokens derived from a TILDE-based expansion (replacing docT5query).
• Expansion uses the original TILDE model to generate token likelihoods, selecting top-m tokens not in the passage or stopword list for expansion (Algorithm 1).
• Index stored as a lightweight structure containing only tokens present in passages with their max contextualized term weights (drastically reducing index size).

Experimental results

Research questions

• RQ1 RQ1: Is contextualized exact term matching more effective and efficient than the original TILDE’s query-likelihood matching?
• RQ2 RQ2: How does TILDEv2 compare to baselines (BM25, docT5query, DeepImpact, uniCOIL, RepBERT, ANCE, EPIC, BERT-based re-rankers) in effectiveness and latency?
• RQ3 RQ3: What is the effectiveness-efficiency trade-off of TILDEv2 relative to a strong BERT re-ranker under varying cut-offs?
• RQ4 RQ4: How effective and efficient is the proposed passage expansion based on TILDE compared with docT5query?

Key findings

• Contextualized exact term matching in TILDEv2 yields higher effectiveness than TILDE’s query-likelihood matching, with up to 24% improvement on MS MARCO when re-ranking BM25 (and 20% when re-ranking docT5query).
• TILDEv2 maintains CPU-friendly latency (<100 ms) and adds only a few milliseconds to BM25 or docT5query pipelines, while achieving competitive effectiveness.
• TILDEv2 significantly reduces index size (up to 99% smaller than TILDE) by storing only passage tokens with max contextualized weights, instead of the full vocabulary.
• Passage expansion using the original TILDE (instead of docT5query) enables faster expansion (7.3 hours for MS MARCO with docT5query vs a fraction of that with TILDE-based expansion) and incurs less than 1% effectiveness loss.
• On MS MARCO and DL2019/DL2020, TILDEv2 is on par with or better than baselines in effectiveness while offering substantially lower latency, especially on CPU (≤80 ms).
• A three-stage pipeline (BM25 → TILDEv2 → BERT-large re-ranker) can achieve similar or better effectiveness with much lower latency than using BERT-large alone on top passages.