[Paper Review] Large-scale Open Dataset, Pipeline, and Benchmark for Bandit Algorithms

Yuta Saito, Shunsuke Aihara+2 more|arXiv (Cornell University)|Aug 17, 2020

Advanced Bandit Algorithms Research21 references4 citations

TL;DR

This paper introduces the Open Bandit Dataset and Open Bandit Pipeline, a large-scale, real-world dataset and open-source toolkit derived from ZOZOTOWN's e-commerce platform, designed to enable reproducible, large-scale evaluation of bandit algorithms and off-policy evaluation (OPE) estimators. The dataset includes ground-truth performance of multiple bandit policies, allowing fair benchmarking of OPE methods and revealing key challenges for future research.

ABSTRACT

We build and publicize the Open Bandit Dataset to facilitate scalable and reproducible research on bandit algorithms. It is especially suitable for off-policy evaluation (OPE), which attempts to estimate the performance of hypothetical policies using data generated by a different policy.We construct the dataset based on experiments and implementations on a large-scale fashion e-commerce platform, ZOZOTOWN. The data contain the ground-truth about the performance of several bandit policies and enable fair comparisons of different OPE estimators. We also build a Python package called the Open Bandit Pipeline to streamline implementations of bandit algorithms and OPE estimators. Our open data and pipeline will allow researchers and practitioners to easily evaluate and compare their bandit algorithms and OPE estimators with others in a large, real-world setting. Using our data and pipeline, we provide extensive benchmark experiments of existing OPE estimators. Our experiments open up essential challenges and new avenues for future OPE research. Our pipeline and example data are available at this https URL You can follow the updates of the whole project at this https URL

Motivation & Objective

• To address the lack of large-scale, real-world datasets for evaluating bandit algorithms and off-policy evaluation (OPE) methods.
• To enable fair and reproducible benchmarking of OPE estimators using real-world data with known ground-truth policy performances.
• To reduce barriers for researchers and practitioners by providing a standardized, open-source pipeline for implementing and testing bandit algorithms and OPE estimators.
• To uncover practical challenges and limitations in existing OPE methods through extensive benchmarking on real-world data.

Proposed method

• The dataset is constructed from real experiments and production logs on ZOZOTOWN, a large-scale fashion e-commerce platform, capturing user interactions and policy decisions.
• The dataset includes logged interactions with multiple bandit policies, along with ground-truth performance metrics for each policy, enabling accurate OPE evaluation.
• A Python package called the Open Bandit Pipeline is developed to standardize the implementation of bandit algorithms and OPE estimators, ensuring reproducibility and ease of use.
• The pipeline supports data loading, policy evaluation, and OPE estimator benchmarking, streamlining end-to-end experimentation.
• The dataset and pipeline are publicly released to promote transparency, reproducibility, and community-driven research in bandit learning.

Experimental results

Research questions

• RQ1 How do existing OPE estimators perform in a large-scale, real-world e-commerce setting with known ground-truth policy performance?
• RQ2 What are the key failure modes and biases of OPE estimators when applied to real-world data with complex user behavior?
• RQ3 How does the performance of OPE estimators vary across different types of bandit policies and data distributions in real-world settings?
• RQ4 What practical challenges emerge when applying OPE to real-world data, and how can future methods be improved to address them?

Key findings

• The Open Bandit Dataset reveals significant discrepancies between estimated and true policy performance for several OPE estimators, highlighting their limitations in real-world settings.
• Some widely used OPE estimators exhibit high variance and bias when applied to real-world data, even with large sample sizes.
• The benchmarking results demonstrate that no single OPE estimator consistently outperforms others across all scenarios, indicating the need for context-aware estimator selection.
• The dataset and pipeline expose practical challenges such as non-uniform behavior policies and complex user feedback, which are often underrepresented in synthetic benchmarks.