[Paper Review] One Million Scenes for Autonomous Driving: ONCE Dataset

Jiageng Mao, Minzhe Niu+11 more|arXiv (Cornell University)|Jun 21, 2021

Advanced Neural Network ApplicationsComputer Science56 references133 citations

TL;DR

The paper introduces the ONCE dataset with 1 million LiDAR scenes and 7 million images for 3D object detection, plus a benchmark evaluating self-/semi-/unsupervised methods on 3D detection using ONCE. It also analyzes data quality, diversity, and domain adaptation potential compared to existing datasets.

ABSTRACT

Current perception models in autonomous driving have become notorious for greatly relying on a mass of annotated data to cover unseen cases and address the long-tail problem. On the other hand, learning from unlabeled large-scale collected data and incrementally self-training powerful recognition models have received increasing attention and may become the solutions of next-generation industry-level powerful and robust perception models in autonomous driving. However, the research community generally suffered from data inadequacy of those essential real-world scene data, which hampers the future exploration of fully/semi/self-supervised methods for 3D perception. In this paper, we introduce the ONCE (One millioN sCenEs) dataset for 3D object detection in the autonomous driving scenario. The ONCE dataset consists of 1 million LiDAR scenes and 7 million corresponding camera images. The data is selected from 144 driving hours, which is 20x longer than the largest 3D autonomous driving dataset available (e.g. nuScenes and Waymo), and it is collected across a range of different areas, periods and weather conditions. To facilitate future research on exploiting unlabeled data for 3D detection, we additionally provide a benchmark in which we reproduce and evaluate a variety of self-supervised and semi-supervised methods on the ONCE dataset. We conduct extensive analyses on those methods and provide valuable observations on their performance related to the scale of used data. Data, code, and more information are available at https://once-for-auto-driving.github.io/index.html.

Motivation & Objective

• Address data inadequacy for autonomous driving by providing a large-scale, diverse 3D scene dataset.
• Enable exploration of unlabeled data through a self-/semi-/unsupervised learning benchmark for 3D detection.
• Promote cross-domain studies and analysis of data quality, diversity, and generalization in 3D perception.

Proposed method

• Collect and downsample LiDAR and camera data to create 1M 3D scenes and 7M images across 144 driving hours.
• Annotate 16k scenes with 3D boxes for 5 categories and project to 2D boxes for images.
• Provide weather, time, and area labels for all scenes and split into train/val/test with a large unlabeled pool.
• Benchmark 3D detectors (single- and multi-modality) under a unified setting on ONCE.
• Reproduce and evaluate self-supervised, semi-supervised, and unsupervised domain adaptation methods on 3D detection.
• Analyze data quality and diversity via pretraining effects and distributional comparisons.

Experimental results

Research questions

• RQ1 How does pretraining on ONCE influence downstream 3D detection performance compared to nuScenes and Waymo?
• RQ2 What is the impact of using unlabeled data via self-/semi-supervised methods on 3D object detection on ONCE?
• RQ3 How do different self-supervised and semi-supervised strategies perform in 3D detection under varying data scale?
• RQ4 Can unsupervised domain adaptation improve cross-dataset 3D detection involving ONCE?
• RQ5 What is the role of data diversity (weather, time, area) in detection performance on autonomous driving scenes?

Key findings

• ONCE yields superior pretraining benefits; models pretrained on ONCE show higher 3D mAP on KITTI when fine-tuned, compared to nuScenes/Waymo pretraining.
• Self-/semi-supervised methods improve 3D detection when unlabeled ONCE data is used, with performance gains increasing with more unlabeled data.
• Clustering-based self-supervised methods (SwAV, DeepCluster) generally outperform contrastive methods (BYOL, PointContrast) on ONCE in large-scale settings.
• Semi-supervised methods (Mean Teacher, SESS, 3DIoUMatch) achieve notable gains, with Mean Teacher reaching up to 59.99% mAP on large unlabeled data.
• Unsupervised domain adaptation from/to ONCE demonstrates meaningful improvements over Source Only baselines, but gaps remain to Oracle performance.