[Paper Review] Selective Term Proximity Scoring Via BP-ANN

Yang Ju, Jiancong Tong+5 more|arXiv (Cornell University)|Jun 23, 2016

Data Management and Algorithms10 references3 citations

TL;DR

This paper proposes a selective term proximity (TP) scoring model using a backpropagation artificial neural network (BP-ANN) to predict which queries will benefit from TP-based ranking. By learning query-specific features, the model selectively applies TP scoring only when effective, improving ranking quality while reducing computational overhead—achieving better MAP and throughput than always-on TP scoring.

ABSTRACT

When two terms occur together in a document, the probability of a close relationship between them and the document itself is greater if they are in nearby positions. However, ranking functions including term proximity (TP) require larger indexes than traditional document-level indexing, which slows down query processing. Previous studies also show that this technique is not effective for all types of queries. Here we propose a document ranking model which decides for which queries it would be beneficial to use a proximity-based ranking, based on a collection of features of the query. We use a machine learning approach in determining whether utilizing TP will be beneficial. Experiments show that the proposed model returns improved rankings while also reducing the overhead incurred as a result of using TP statistics.

Motivation & Objective

• To address the inefficiency and inconsistent effectiveness of term proximity (TP) scoring across all queries in information retrieval.
• To reduce computational overhead from always-on TP scoring by intelligently selecting only beneficial queries.
• To improve ranking quality by applying TP scoring only when it enhances relevance, based on query-specific features.
• To balance retrieval effectiveness and efficiency through machine learning-driven query selection.

Proposed method

• The model uses a BP-ANN to classify queries based on a set of 13 query features, predicting whether TP scoring will improve ranking.
• Features include query length, term frequency, inverse document frequency, and proximity-related statistics such as average distance between query terms.
• The BP-ANN is trained to predict a binary label: 1 if TP improves ranking (beneficial), 0 otherwise.
• The model is integrated into ranking pipelines, enabling selective application of TP scoring only for predicted beneficial queries.
• The approach avoids full TP computation for non-beneficial queries, reducing index size and query processing time.
• The model is evaluated using standard IR metrics (MAP, NDCG, precision) and throughput on real test collections.

Experimental results

Research questions

• RQ1 Which queries truly benefit from term proximity (TP) scoring in document ranking?
• RQ2 Can a machine learning model accurately predict whether TP scoring will improve ranking effectiveness for a given query?
• RQ3 Does selective TP application reduce computational overhead while maintaining or improving ranking quality?
• RQ4 How does the performance of the selective model compare to always-on TP and no-TP baselines across different query types and lengths?

Key findings

• The selective TP model (_tpS) achieved higher MAP and mean NDCG than always-on TP (_tpAll), indicating better ranking quality.
• For the MRF ranking formula, _tpS achieved a MAP of 0.4924, outperforming _tpAll (0.4883) and approaching the oracle performance (0.5362).
• The selective model improved throughput by 15%–25% compared to _tpAll, with _tpS processing 477.82 queries per second (Q/s) vs. 334.56 Q/s for _tpAll under EXP.
• In k=1 precision, _tpS outperformed _tpAll across all query lengths, showing stronger effectiveness for exact-match queries.
• The model reduced the number of queries using TP from 143 to 80 (EXP) and 69 (MRF), indicating effective filtering of non-beneficial queries.
• The BP-ANN model achieved a 98.57 Q/s throughput for the BM25TP model with selective scoring, significantly outperforming the 352.71 Q/s of _tpAll.