[Paper Review] Interpreting search result rankings through intent modeling

Jaspreet Singh, Avishek Anand|arXiv (Cornell University)|Sep 13, 2018

Explainable Artificial Intelligence (XAI)26 references4 citations

TL;DR

This paper proposes a post-hoc, model-agnostic framework to interpret complex neural ranking models by using query expansion and a simple retrieval model to approximate the black-box ranker's intent. It demonstrates that the expanded query terms serve as interpretable explanations for ranking decisions, enabling detection of model biases and pairwise document ranking differences with high local fidelity and improved global accuracy through strategic sampling of preference pairs.

ABSTRACT

Given the recent interest in arguably accurate yet non-interpretable neural models, even with textual features, for document ranking we try to answer questions relating to how to interpret rankings. In this paper we take first steps towards a framework for the interpretability of retrieval models with the aim of answering 3 main questions "What is the intent of the query according to the ranker?", "Why is a document ranked higher than another for the query?" and "Why is a document relevant to the query?" Our framework is predicated on the assumption that text based retrieval model behavior can be estimated using query expansions in conjunction with a simpler retrieval model irrespective of the underlying ranker. We conducted experiments with the Clueweb test collection. We show how our approach performs for both simpler models with a closed form notation (which allows us to measure the accuracy of the interpretation) and neural ranking models. Our results indicate that we can indeed interpret more complex models with reasonable accuracy under certain simplifying assumptions. In a case study we also show our framework can be employed to interpret the results of the DRMM neural retrieval model in various scenarios.

Motivation & Objective

• To address the lack of interpretability in complex neural ranking models used in information retrieval.
• To develop a post-hoc, model-agnostic method that explains why a document is ranked higher than another or deemed relevant to a query.
• To estimate the underlying query intent as perceived by a black-box ranker using query expansion and a simpler retrieval model.
• To detect biases—especially temporal and model-specific—by analyzing the terms in the interpreted intent.
• To enable developers and users to understand, debug, and improve ranking models through interpretable, human-readable explanations.

Proposed method

• Use query expansion to generate a set of intent terms that reflect the black-box ranker’s perceived query intent.
• Train a simple, interpretable retrieval model (e.g., BM25 or TF-IDF) on the expanded query to approximate the original ranker’s output.
• Formulate the interpretation task as a preference learning problem using pairwise document comparisons from the original ranking.
• Optimize the expanded query by minimizing the ranking disagreement between the black-box model and the simple model using preference pairs.
• Apply document perturbation techniques to reduce false positive terms in the explanation, especially in weakly supervised settings.
• Visualize explanations using term importance scores (e.g., via term frequency and document length) to show why one document ranks higher than another.

Experimental results

Research questions

• RQ1 What is the actual query intent as perceived by a black-box ranking model?
• RQ2 Why is document A ranked above document B for a given query?
• RQ3 Why is a specific document considered relevant to the query according to the model’s internal logic?
• RQ4 How can we accurately approximate the behavior of complex neural rankers using interpretable, query-expanded explanations?
• RQ5 Can the framework detect temporal and model-specific biases in ranking decisions?

Key findings

• The framework achieves high local fidelity in interpreting rankings by focusing on preference pairs from the top-k results, though this sacrifices some global accuracy.
• Sampling additional preference pairs from lower-ranked documents significantly improves global fidelity, even at the cost of reduced local precision.
• The sampling strategy of top-k + rank random consistently outperforms purely random or rank-biased sampling, indicating a balance between local and global fidelity is optimal.
• Document perturbation is essential in weakly agnostic settings to reduce false positive terms in the explanation, improving reliability.
• Case studies on DRMM and DESM reveal clear temporal biases: DRMM favors 2004-era news events (e.g., Afghan war, Gore’s 2000 election), while DESM reflects more generic semantic concepts (e.g., flag, nurse).
• Visual explanations based on term importance scores effectively illustrate pairwise ranking differences, such as why a document with a rare but relevant term (e.g., 'intestine') ranks higher than a semantically similar but less specific one.