[Paper Review] 3D IoU-Net: IoU Guided 3D Object Detector for Point Clouds

Jiale Li, Shujie Luo+5 more|arXiv (Cornell University)|Apr 10, 2020

Advanced Neural Network Applications41 references31 citations

TL;DR

Introduces 3D IoU-Net, a two-stage 3D detector that learns IoU-aware features for improved proposal confidence using Attentive Corner Aggregation and Corner Geometry Encoding, plus an IoU alignment at inference.

ABSTRACT

Most existing point cloud based 3D object detectors focus on the tasks of classification and box regression. However, another bottleneck in this area is achieving an accurate detection confidence for the Non-Maximum Suppression (NMS) post-processing. In this paper, we add a 3D IoU prediction branch to the regular classification and regression branches. The predicted IoU is used as the detection confidence for NMS. In order to obtain a more accurate IoU prediction, we propose a 3D IoU-Net with IoU sensitive feature learning and an IoU alignment operation. To obtain a perspective-invariant prediction head, we propose an Attentive Corner Aggregation (ACA) module by aggregating a local point cloud feature from each perspective of eight corners and adaptively weighting the contribution of each perspective with different attentions. We propose a Corner Geometry Encoding (CGE) module for geometry information embedding. To the best of our knowledge, this is the first time geometric embedding information has been introduced in proposal feature learning. These two feature parts are then adaptively fused by a multi-layer perceptron (MLP) network as our IoU sensitive feature. The IoU alignment operation is introduced to resolve the mismatching between the bounding box regression head and IoU prediction, thereby further enhancing the accuracy of IoU prediction. The experimental results on the KITTI car detection benchmark show that 3D IoU-Net with IoU perception achieves state-of-the-art performance.

Motivation & Objective

• Motivate the need for accurate IoU-based confidence for 3D bounding boxes in LiDAR point clouds.
• Develop an IoU-centric detection framework that surpasses classification-based confidence.
• Introduce modules to learn IoU sensitive features from partial visibility due to perspective.
• Propose an IoU alignment procedure to resolve training-inference mismatches without extra training.

Proposed method

• Two-stage 3D detection with a PointNet++ backbone for proposal generation.
• Attentive Corner Aggregation (ACA) aggregates corner-perspective features with perspective-channel attention for IoU-sensitive representation.
• Corner Geometry Encoding (CGE) injects absolute corner geometry into IoU prediction.
• IoU alignment: re-evaluates IoU on refined boxes via a second inference for aligned IoU scores.
• Three prediction heads share IoU-sensitive features: classification, bbox regression, and IoU perception.
• Auxiliary corner loss and IoU loss guide IoU predictor training.

Experimental results

Research questions

• RQ1 Can IoU-based detection confidence improve 3D NMS and overall AP on KITTI over traditional classification confidence?
• RQ2 Do ACA and CGE provide perspective-invariant IoU features that improve IoU prediction?
• RQ3 Does incorporating bounding box geometry via CGE improve IoU perception?
• RQ4 Does IoU alignment at inference mitigate IoU assignment mismatches without extra training?

Key findings

• 3D IoU-Net achieves state-of-the-art 3D detection performance on KITTI Car benchmark among LiDAR-based methods.
• IoU-aware predictions with ACA and CGE yield higher 3D AP than baselines relying on classification confidence.
• IoU alignment at inference further improves IoU prediction accuracy without additional training.
• On KITTI test set, 3D IoU-Net attains 87.96% Easy, 79.03% Moderate, 72.78% Hard 3D AP and 94.76% Easy, 88.38% Moderate, 81.93% Hard BEV AP.
• Ablations show significant mAP gains from aligning IoU predictions and from perspective-aware ACA features.