[Paper Review] Labels Are Not Perfect: Improving Probabilistic Object Detection via Label Uncertainty

Di Feng, Lars Rosenbaum+2 more|arXiv (Cornell University)|Aug 10, 2020

Advanced Neural Network Applications20 references4 citations

TL;DR

This paper proposes leveraging label uncertainty—estimated from inherent variability in ground truth bounding box annotations—to improve probabilistic LiDAR-based object detection. By incorporating this uncertainty into training, the method enhances detection accuracy, achieving up to 3.6% higher Average Precision on the KITTI dataset compared to baselines and heuristic-based approaches.

ABSTRACT

Reliable uncertainty estimation is crucial for robust object detection in autonomous driving. However, previous works on probabilistic object detection either learn predictive probability for bounding box regression in an un-supervised manner, or use simple heuristics to do uncertainty regularization. This leads to unstable training or suboptimal detection performance. In this work, we leverage our previously proposed method for estimating uncertainty inherent in ground truth bounding box parameters (which we call label uncertainty) to improve the detection accuracy of a probabilistic LiDAR-based object detector. Experimental results on the KITTI dataset show that our method surpasses both the baseline model and the models based on simple heuristics by up to 3.6% in terms of Average Precision.

Motivation & Objective

• To address the instability and suboptimal performance in probabilistic object detection caused by imperfect ground truth labels.
• To improve uncertainty estimation in object detection by modeling the inherent uncertainty in ground truth bounding box annotations.
• To enhance detection accuracy of a LiDAR-based detector through uncertainty-aware training.
• To move beyond heuristic uncertainty regularization and unsupervised uncertainty learning in probabilistic detection.

Proposed method

• The method estimates label uncertainty from the variability in ground truth bounding box annotations, treating it as a source of reliable uncertainty prior.
• It integrates this label uncertainty into the training objective of a probabilistic object detector, using it as a regularizer during optimization.
• The approach uses a differentiable loss function that accounts for both prediction uncertainty and label uncertainty in bounding box regression.
• The model is trained end-to-end on the KITTI dataset using LiDAR point clouds and annotated bounding boxes.
• Label uncertainty is computed from multiple annotations of the same object, capturing natural variability in human-annotated bounding boxes.
• The method improves detection performance by encouraging the model to learn more robust uncertainty estimates aligned with real-world annotation variability.

Experimental results

Research questions

• RQ1 Can modeling label uncertainty improve the reliability and accuracy of probabilistic object detection in autonomous driving?
• RQ2 How does incorporating ground-truth label uncertainty compare to heuristic or unsupervised uncertainty regularization in detection performance?
• RQ3 To what extent does label uncertainty reduce training instability in probabilistic detectors?
• RQ4 Can label uncertainty serve as a meaningful prior for improving uncertainty estimation in LiDAR-based object detection?

Key findings

• The proposed method improves Average Precision by up to 3.6% over the baseline model on the KITTI dataset.
• The model outperforms approaches using simple heuristics for uncertainty regularization, demonstrating superior detection accuracy.
• Incorporating label uncertainty leads to more stable training compared to unsupervised uncertainty learning methods.
• The method achieves better uncertainty calibration by aligning model predictions with the natural variability in human-annotated bounding boxes.
• The improvement is consistent across multiple detection metrics, indicating robust performance gains.
• The results validate that label uncertainty is a valuable signal for improving probabilistic object detection in real-world scenarios.