[Paper Review] RPC: A Large-Scale Retail Product Checkout Dataset

Xiu-Shen Wei, Quan Cui+3 more|arXiv (Cornell University)|Jan 22, 2019

Advanced Neural Network Applications30 references111 citations

TL;DR

This paper introduces the Retail Product Checkout (RPC) dataset, the largest benchmark for automatic checkout (ACO) in terms of SKUs and images, and benchmarks cross-domain detection baselines using exemplar and checkout images.

ABSTRACT

Over recent years, emerging interest has occurred in integrating computer vision technology into the retail industry. Automatic checkout (ACO) is one of the critical problems in this area which aims to automatically generate the shopping list from the images of the products to purchase. The main challenge of this problem comes from the large scale and the fine-grained nature of the product categories as well as the difficulty for collecting training images that reflect the realistic checkout scenarios due to continuous update of the products. Despite its significant practical and research value, this problem is not extensively studied in the computer vision community, largely due to the lack of a high-quality dataset. To fill this gap, in this work we propose a new dataset to facilitate relevant research. Our dataset enjoys the following characteristics: (1) It is by far the largest dataset in terms of both product image quantity and product categories. (2) It includes single-product images taken in a controlled environment and multi-product images taken by the checkout system. (3) It provides different levels of annotations for the check-out images. Comparing with the existing datasets, ours is closer to the realistic setting and can derive a variety of research problems. Besides the dataset, we also benchmark the performance on this dataset with various approaches. The dataset and related resources can be found at \url{https://rpc-dataset.github.io/}.

Motivation & Objective

• Motivate automatic checkout research by addressing large-scale, fine-grained, and domain-shift challenges in real-world retail scenarios.
• Introduce RPC, a dataset with 200 SKUs, 53,739 exemplar images, and 30,000 checkout images, across three clutter levels.
• Provide hierarchical meta-categories and weak-to-strong annotations to enable diverse learning settings.
• Benchmark baseline methods to establish a feasibility baseline and identify room for improvement.

Proposed method

• Define the ACO task and data requirements: single-product exemplars for training and checkout images for evaluation.
• Construct RPC with two image types (exemplar and checkout) and three clutter levels (easy, medium, hard) to reflect realistic conditions.
• Provide weak to strong annotations on checkout images (shopping list, point-level, bounding boxes) to support weakly-supervised learning.
• Implement four cross-domain detection baselines (Single, Syn, Render, Syn+Render) using exemplar-trained detectors and data augmentation with synthesis and Cycle-GAN-based domain translation.
• Evaluate detectors with custom ACO metrics (cAcc, ACD, mCCD, mCIoU) plus standard detection metrics (mAP50, mmAP).
• Analyze the impact of synthesis and rendering, showing substantial gains from domain translation and mixed synthetic/real data.

Experimental results

Research questions

• RQ1 Can a large-scale, multi-category dataset with realistic checkout clutter support effective automatic checkout research?
• RQ2 How do domain gaps between exemplar single-product images and checkout scenes affect detector performance, and can synthesis and domain translation close this gap?
• RQ3 What annotations and supervision levels are useful for advancing ACO tasks (weak to strong supervision)?
• RQ4 How do different training data strategies (single, synthesized, rendered, combined) perform across easy, medium, and hard checkout clutter levels?
• RQ5 What are the practical failure modes of ACO detectors, and which approaches best mitigate them?

Key findings

• RPC comprises 200 SKUs, 53,739 exemplar images, and 30,000 checkout images, enabling large-scale evaluation.
• Directly training on exemplar images yields near-zero cAcc, especially in easy clutter; synthetic data improves cAcc substantially.
• Rendering (domain translation) dramatically improves performance, increasing easy-mode cAcc from 0.02% (Single) to 63.19% (Render) and further to 73.17% (Syn+Render).
• Combining synthesized and rendered data yields the best results across clutter levels, with cAcc of 73.17% (Easy), 54.69% (Medium), and 42.48% (Hard) for Syn+Render in table results; mmAP reaches up to 73.83% Averaged.
• Standard detection metrics (mAP50, mmAP) show strong gains with Render and Syn+Render, e.g., Averaged mmAP 73.83% with Syn+Render.
• The study confirms substantial room for improvement, highlighting practical challenges such as missed detections, dense placement, fine-grained distinctions, and false positives.