[Paper Review] Revisiting Domain-Adaptive 3D Object Detection by Reliable, Diverse and Class-balanced Pseudo-Labeling

Zhuoxiao Chen, Yadan Luo+3 more|arXiv (Cornell University)|Jul 16, 2023

Domain Adaptation and Few-Shot Learning4 citations

TL;DR

This paper proposes ReDB, a novel framework for domain-adaptive 3D object detection that generates reliable, diverse, and class-balanced pseudo-labels to address performance drops in multi-class settings. By introducing cross-domain examination (CDE) for label reliability, overlapped boxes counting (OBC) for geometric diversity, and class-balanced self-training, ReDB improves mAP by 23.15% on the nuScenes→KITTI benchmark over prior SOTA methods.

ABSTRACT

Unsupervised domain adaptation (DA) with the aid of pseudo labeling techniques has emerged as a crucial approach for domain-adaptive 3D object detection. While effective, existing DA methods suffer from a substantial drop in performance when applied to a multi-class training setting, due to the co-existence of low-quality pseudo labels and class imbalance issues. In this paper, we address this challenge by proposing a novel ReDB framework tailored for learning to detect all classes at once. Our approach produces Reliable, Diverse, and class-Balanced pseudo 3D boxes to iteratively guide the self-training on a distributionally different target domain. To alleviate disruptions caused by the environmental discrepancy (e.g., beam numbers), the proposed cross-domain examination (CDE) assesses the correctness of pseudo labels by copy-pasting target instances into a source environment and measuring the prediction consistency. To reduce computational overhead and mitigate the object shift (e.g., scales and point densities), we design an overlapped boxes counting (OBC) metric that allows to uniformly downsample pseudo-labeled objects across different geometric characteristics. To confront the issue of inter-class imbalance, we progressively augment the target point clouds with a class-balanced set of pseudo-labeled target instances and source objects, which boosts recognition accuracies on both frequently appearing and rare classes. Experimental results on three benchmark datasets using both voxel-based (i.e., SECOND) and point-based 3D detectors (i.e., PointRCNN) demonstrate that our proposed ReDB approach outperforms existing 3D domain adaptation methods by a large margin, improving 23.15% mAP on the nuScenes $ ightarrow$ KITTI task. The code is available at https://github.com/zhuoxiao-chen/ReDB-DA-3Ddet.

Motivation & Objective

• Address the significant performance drop in multi-class 3D domain adaptation due to low-quality pseudo-labels and class imbalance.
• Mitigate the impact of environmental and object shifts in cross-domain 3D detection.
• Develop a self-training framework that enhances detection robustness across diverse geometric and distributional shifts.
• Ensure reliable pseudo-labeling by validating predictions across source and target domains.
• Achieve balanced recognition of both frequent and rare classes through progressive, class-balanced data augmentation.

Proposed method

• Introduce Cross-Domain Examination (CDE), which copies pseudo-labeled target objects into the source domain and measures prediction consistency via IoU to filter unreliable labels.
• Design the Overlapped Boxes Counting (OBC) metric to quantify geometric diversity and enable uniform downsampling of pseudo-labeled objects across scales, densities, and occlusion levels.
• Implement a class-balanced self-training strategy that progressively augments target point clouds with a balanced mix of pseudo-labeled instances and source objects to improve rare-class detection.
• Apply CDE with a threshold δ_cde to discard pseudo-labels with low consistency between source and target domain predictions.
• Use OBC to downsample pseudo-labels such that objects with high diversity in geometric features (e.g., size, distance, occlusion) are preserved.
• Integrate the ReDB framework with both voxel-based (e.g., SECOND) and point-based (e.g., PointRCNN) detectors, demonstrating model-agnostic performance gains.

Experimental results

Research questions

• RQ1 How can pseudo-label reliability be improved in the presence of environmental and object shifts during 3D domain adaptation?
• RQ2 What metrics can effectively capture geometric diversity in pseudo-labeled 3D objects to prevent model collapse on similar object types?
• RQ3 Can class-balanced self-training significantly improve detection performance on rare classes in multi-class 3D detection under domain shift?
• RQ4 To what extent does the proposed method generalize across different 3D detector architectures (voxel-based vs. point-based)?
• RQ5 How sensitive is the performance of ReDB to hyperparameters such as the CDE IoU threshold and OBC downsampling rate?

Key findings

• ReDB achieves a 23.15% improvement in mAP on the nuScenes→KITTI benchmark compared to the prior SOTA method ST3D.
• The method improves mAP by 23.32% and 24.11% on PointRCNN at moderate and hard detection thresholds, respectively, surpassing ST3D++ and specialized single-class methods.
• Ablation studies confirm that removing any component of ReDB—CDE, OBC, or class-balanced training—results in a clear drop in mAP, validating each module’s contribution.
• The CDE mechanism reduces error accumulation by filtering out 30–40% of low-consistency pseudo-labels, significantly improving label quality.
• The OBC-based downsampling preserves diverse geometric features, with high-OBC objects (e.g., large, distant, or occluded) being preferentially retained.
• The method is robust to hyperparameter variations, with only 2.69% fluctuation in mAP when δ_cde varies between 0.3 and 0.6, and acceptable performance drop even at high downsampling rates (d=10).