[Paper Review] Optimizing Gross Merchandise Volume via DNN-MAB Dynamic Ranking Paradigm

Yan Yan, Wentao Guo+3 more|arXiv (Cornell University)|Aug 14, 2017

Advanced Bandit Algorithms Research21 references17 citations

TL;DR

This paper proposes DNN-MAB, a dynamic ranking framework that combines a pairwise deep neural network (DNN) pre-ranker with a revised Thompson sampling multi-armed bandit (MAB) post-ranker to optimize gross merchandise volume (GMV). By incorporating real-time user feedback—such as clicks and conversions—the system dynamically adjusts rankings to improve user intent alignment, significantly boosting GMV in production deployment at JD.com.

ABSTRACT

With the transition from people's traditional `brick-and-mortar' shopping to online mobile shopping patterns in web 2.0 $\mathit{era}$, the recommender system plays a critical role in E-Commerce and E-Retails. This is especially true when designing this system for more than $\mathbf{236~million}$ daily active users. Ranking strategy, the key module of the recommender system, needs to be precise, accurate, and responsive for estimating customers' intents. We propose a dynamic ranking paradigm, named as DNN-MAB, that is composed of a pairwise deep neural network (DNN) $\mathit{pre}$-ranker connecting a revised multi-armed bandit (MAB) dynamic $\mathit{post}$-ranker. By taking into account of explicit and implicit user feedbacks such as impressions, clicks, conversions, etc. DNN-MAB is able to adjust DNN $\mathit{pre}$-ranking scores to assist customers locating items they are interested in most so that they can converge quickly and frequently. To the best of our knowledge, frameworks like DNN-MAB have not been discussed in the previous literature to either E-Commerce or machine learning audiences. In practice, DNN-MAB has been deployed to production and it easily outperforms against other state-of-the-art models by significantly lifting the gross merchandise volume (GMV) which is the objective metrics at JD.

Motivation & Objective

• To address the limitations of static ranking in E-commerce recommender systems, where user intent and feedback are not dynamically incorporated.
• To improve conversion rates and gross merchandise volume (GMV) by enabling real-time adaptation of item rankings based on user behavior.
• To design a scalable, production-ready dynamic ranking framework that balances exploration and exploitation in real-time recommendation scenarios.
• To overcome the shortcomings of traditional learning-to-rank and MAB models by integrating deep learning with contextual bandits in a two-stage ranking pipeline.
• To validate the framework’s effectiveness in a large-scale E-commerce setting with over 236 million active users, focusing on real-world performance metrics like GMV and DCG.

Proposed method

• The framework uses a pairwise DNN as a pre-ranker to generate initial item relevance scores based on user and item features.
• A revised Thompson sampling algorithm acts as a post-ranker that dynamically adjusts the pre-ranked scores using real-time feedback (impressions, clicks, conversions).
• The MAB component employs a novel initialization strategy to accelerate convergence and reduce early regret in production deployment.
• The system processes user interactions in real time, enabling online learning and continuous adaptation of rankings to current user intent.
• The two-stage architecture separates static relevance modeling (DNN) from dynamic feedback adaptation (MAB), improving scalability and responsiveness.
• The framework is deployed in production at JD.com, where it continuously learns from user interactions and updates rankings in real time.

Experimental results

Research questions

• RQ1 How can a deep learning-based pre-ranker be effectively combined with a contextual bandit post-ranker to improve dynamic ranking in E-commerce?
• RQ2 What impact does real-time feedback incorporation (e.g., clicks, conversions) have on GMV and ranking effectiveness compared to static models?
• RQ3 How does a revised Thompson sampling algorithm with custom initialization improve convergence speed and performance in a production-scale recommender system?
• RQ4 Can a hybrid DNN-MAB framework outperform state-of-the-art models in terms of GMV and DCG while maintaining scalability?
• RQ5 What are the trade-offs between optimizing for GMV and other KPIs (e.g., order count), and how can multi-objective optimization be addressed in dynamic ranking?

Key findings

• The DNN-MAB framework significantly outperformed baseline models in production, leading to a measurable and substantial increase in gross merchandise volume (GMV), the primary business metric at JD.
• The system achieved improved performance on DCG (Discounted Cumulative Gain), indicating better ranking quality and user satisfaction.
• The revised Thompson sampling algorithm with custom initialization enabled faster convergence and reduced early regret, enhancing responsiveness to user feedback.
• The framework demonstrated robustness in real-world deployment, handling over 236 million active users and billions of daily interactions.
• Despite improvements in GMV and DCG, the model showed temporary negative impacts on order count in some periods, indicating a need for multi-KPI optimization.
• The integration of DNN pre-ranking with MAB post-ranking proved effective and scalable, with no significant scalability issues reported in the production environment.