[Paper Review] Towards Personalized and Semantic Retrieval: An End-to-End Solution for E-commerce Search via Embedding Learning

Han Zhang, Songlin Wang+6 more|arXiv (Cornell University)|Jun 3, 2020

Recommender Systems and Techniques30 references4 citations

TL;DR

This paper proposes DPSR, a deep learning-based end-to-end system for e-commerce search that enhances both semantic and personalized retrieval through learned embeddings. By employing a two-tower neural network with multi-head query encoding, attention-based loss, and user-specific features, DPSR improves retrieval relevance—especially for long-tail queries—achieving a 1.29% increase in conversion rate and 10.03% improvement on long-tail queries in online A/B tests, with low latency suitable for production deployment at scale.

ABSTRACT

Nowadays e-commerce search has become an integral part of many people's shopping routines. Two critical challenges stay in today's e-commerce search: how to retrieve items that are semantically relevant but not exact matching to query terms, and how to retrieve items that are more personalized to different users for the same search query. In this paper, we present a novel approach called DPSR, which stands for Deep Personalized and Semantic Retrieval, to tackle this problem. Explicitly, we share our design decisions on how to architect a retrieval system so as to serve industry-scale traffic efficiently and how to train a model so as to learn query and item semantics accurately. Based on offline evaluations and online A/B test with live traffics, we show that DPSR model outperforms existing models, and DPSR system can retrieve more personalized and semantically relevant items to significantly improve users' search experience by +1.29% conversion rate, especially for long tail queries by +10.03%. As a result, our DPSR system has been successfully deployed into JD.com's search production since 2019.

Motivation & Objective

• To address the limitations of traditional inverted index-based candidate retrieval in e-commerce search, which fails to capture semantic similarity and personalization.
• To design a scalable, production-ready deep learning system that supports real-time, low-latency inference at industrial scale.
• To improve retrieval performance for long-tail queries, which are often underserved by exact-term matching systems.
• To integrate user-specific features (e.g., gender, purchase power) into the retrieval process without relying on hand-crafted rules.
• To validate the effectiveness of the proposed model through offline evaluation and online A/B testing with live traffic.

Proposed method

• Proposes a two-tower neural network architecture where queries and items are embedded into a shared vector space for semantic matching.
• Employs a multi-head query tower to capture diverse aspects of user intent, enhancing representation learning.
• Introduces an attention-based loss function to prioritize hard negative samples during training, improving model generalization.
• Uses a customized negative sampling strategy to balance positive and negative examples, increasing training stability and effectiveness.
• Implements an efficient training algorithm tailored for large-scale embedding learning, leveraging distributed data and model parallelism.
• Incorporates human-annotated supervision data to refine model predictions and align with human judgment of relevance.

Experimental results

Research questions

• RQ1 Can a deep learning-based retrieval system outperform traditional inverted index methods in capturing semantic relevance for e-commerce search?
• RQ2 How can user-specific personalization be effectively integrated into the retrieval pipeline without relying on rule-based heuristics?
• RQ3 To what extent does the proposed model improve retrieval performance on long-tail queries, which are challenging for keyword-based systems?
• RQ4 What is the latency and throughput performance of the proposed system under real-world production workloads?
• RQ5 Can the model achieve consistent improvements across key business metrics such as conversion rate and gross merchandise value?

Key findings

• DPSR achieved a +1.29% improvement in user conversion rate (UCVR) in online A/B testing on 10% of live traffic, demonstrating strong business impact.
• The system improved gross merchandise value (GMV) by +2.19% and reduced query rewrite rate (QRR) by 4.29%, indicating higher user satisfaction.
• On long-tail queries, DPSR delivered a +10.03% improvement in conversion rate, highlighting its effectiveness in handling ambiguous or rare search terms.
• Offline human evaluation showed a 6% reduction in 'bad' relevance cases, with a shift from 17.86% to 13.70% bad results on 500 long-tail queries.
• The system achieved sub-10ms retrieval latency on CPU and sub-1ms search latency on GPU, with 14x higher QPS on GPU, proving real-time feasibility.
• The 2-head query tower and personalized variant (1-head-p13n) both outperformed the vanilla 1-head model, confirming the value of architectural enhancements.