[Paper Review] Explainable Product Search with a Dynamic Relation Embedding Model

Qingyao Ai, Yongfeng Zhang+2 more|arXiv (Cornell University)|Sep 16, 2019

Advanced Graph Neural Networks75 references4 citations

TL;DR

This paper proposes the Dynamic Relation Embedding Model (DREM), a novel approach for explainable product search that constructs session-specific knowledge graphs from multi-relational product data and uses dynamic relation embeddings to model user-item interactions. DREM improves retrieval performance and generates human-interpretable explanations through logic inference and soft matching on the knowledge graph, significantly outperforming state-of-the-art baselines.

ABSTRACT

Product search is one of the most popular methods for customers to discover products online. Most existing studies on product search focus on developing effective retrieval models that rank items by their likelihood to be purchased. They, however, ignore the problem that there is a gap between how systems and customers perceive the relevance of items. Without explanations, users may not understand why product search engines retrieve certain items for them, which consequentially leads to imperfect user experience and suboptimal system performance in practice. In this work, we tackle this problem by constructing explainable retrieval models for product search. Specifically, we propose to model the "search and purchase" behavior as a dynamic relation between users and items, and create a dynamic knowledge graph based on both the multi-relational product data and the context of the search session. Ranking is conducted based on the relationship between users and items in the latent space, and explanations are generated with logic inferences and entity soft matching on the knowledge graph. Empirical experiments show that our model, which we refer to as the Dynamic Relation Embedding Model (DREM), significantly outperforms the state-of-the-art baselines and has the ability to produce reasonable explanations for search results.

Motivation & Objective

• To address the gap between system-perceived and user-perceived relevance in product search by introducing explainable retrieval models.
• To model user-item relationships as dynamic, session-specific relations rather than static knowledge graph links.
• To generate human-readable, logically grounded explanations for search results using structured inference on multi-relational product data.
• To jointly optimize retrieval effectiveness and explanation quality in e-commerce search systems.
• To explore the impact of explanation quality on user trust and transaction behavior in product search.

Proposed method

• Construct a dynamic knowledge graph that evolves per search session using multi-relational product data (e.g., categories, brands, co-purchase, view-through relationships).
• Learn dynamic relation embeddings for users and items in a shared latent space, capturing session-specific interactions.
• Use a Soft Matching Algorithm to identify relevant explanation paths in the knowledge graph based on query and item features.
• Apply logic inference to derive explanations by tracing paths from query to retrieved items, ensuring semantic coherence.
• Score explanation paths using heuristic functions (Equations 13 and 14) to rank and select the most relevant explanations.
• Integrate retrieval and explanation generation into a unified end-to-end framework trained via joint optimization.

Experimental results

Research questions

• RQ1 Can a dynamic knowledge graph representation improve retrieval performance in product search compared to static knowledge graphs or baseline models?
• RQ2 How effective are the explanations generated by the model in terms of relevance and persuasiveness to human users?
• RQ3 To what extent does the quality of explanations correlate with retrieval performance or user transaction likelihood?
• RQ4 Which types of relationships (e.g., also-bought, also-viewed) contribute most meaningfully to explanation quality in different product domains?
• RQ5 Can joint optimization of retrieval and explanation generation lead to better overall system effectiveness?

Key findings

• DREM significantly outperforms state-of-the-art product retrieval baselines in terms of standard retrieval metrics such as NDCG and MAP.
• Explanations generated by DREM are highly effective: over 84% of useful explanations (Q3 ≥ 1) provide relevant item or query information (Q1+ or Q2+), indicating strong informational relevance.
• Even when explanations are judged unhelpful (Q3 = 0), less than 16% of them fail to provide basic relevance to the query or item, showing robustness in explanation quality.
• Annotators found that relationships like 'also-bought' and 'also-viewed' were less useful in niche domains (e.g., Kindle Store, CDs & Vinyl), where familiarity with the product category affects explanation utility.
• There is no strong correlation between retrieval performance and explanation usefulness, suggesting that explanation quality is a distinct and separable dimension of system effectiveness.
• Annotators reported that explanations were particularly ineffective when the top-ranked item was irrelevant, highlighting the need for alignment between item relevance and explanation quality.