[Paper Review] IPOD: Intensive Point-based Object Detector for Point Cloud

Zetong Yang, Yanan Sun+3 more|arXiv (Cornell University)|Dec 13, 2018

Remote Sensing and LiDAR ApplicationsEnvironmental Science33 references132 citations

TL;DR

IPOD seeds object proposals from every point in raw point clouds, uses a point-based backbone to extract proposal features with context, and predicts 3D bounding boxes end-to-end, achieving state-of-the-art results on KITTI especially in hard cases.

ABSTRACT

We present a novel 3D object detection framework, named IPOD, based on raw point cloud. It seeds object proposal for each point, which is the basic element. This paradigm provides us with high recall and high fidelity of information, leading to a suitable way to process point cloud data. We design an end-to-end trainable architecture, where features of all points within a proposal are extracted from the backbone network and achieve a proposal feature for final bounding inference. These features with both context information and precise point cloud coordinates yield improved performance. We conduct experiments on KITTI dataset, evaluating our performance in terms of 3D object detection, Bird's Eye View (BEV) detection and 2D object detection. Our method accomplishes new state-of-the-art , showing great advantage on the hard set.

Motivation & Objective

• Motivate 3D object detection directly on raw point clouds without voxelization or projection.
• Develop a per-point proposal generation strategy to maintain localization fidelity and high recall.
• Design an end-to-end architecture that extracts proposal features with both context and precise point coordinates.
• Address redundancy and ambiguity in point-based proposals through novel labeling and alignment schemes.
• Demonstrate state-of-the-art performance on KITTI for Car, Pedestrian, and Cyclist tasks, especially under occlusion and clutter.

Proposed method

• Seed object proposals at the center of each point with multi-scale, multi-angle shifts.
• Use a subsampling network to filter background points and maintain high recall (96.0% on KITTI).
• Employ a PointNet++ backbone to extract per-point features from the raw point cloud.
• Generate proposal features by combining high-level context features with canonized point coordinates and T-Net centered residuals.
• Predict class, size ratio, center residuals, and orientation per proposal with a multi-task loss (L_cls, L_loc, L_ang, L_cor, L_corner).
• Align proposals and assign positive/negative labels using PointsIoU, not plain box IoU, to better reflect point-level overlap.

Experimental results

Research questions

• RQ1 Can a per-point proposal generation paradigm on raw point clouds achieve higher recall and better 3D detection, BEV, and 2D metrics without voxelization or projection?
• RQ2 Does incorporating context features and canonized point coordinates within proposal representations improve localization and classification?
• RQ3 How does PointsIoU-based labeling affect training stability and final detection performance compared to traditional IoU-based labeling?
• RQ4 What is the impact of subsampling, proposal feature design, and backbone choice on KITTI Car, Pedestrian, and Cyclist detection performance?

Key findings

• Achieves state-of-the-art results on KITTI, with notable gains on the hard subset for 2D, BEV, and 3D AP compared to prior methods.
• Outperforms F-PointNet and multi-view methods, especially for pedestrians and crowded scenes.
• Demonstrates high recall (96.0%) without projection-based preprocessing.
• Ablations show that PointsIoU labeling and the combination of high-level context features with canonized coordinates significantly improve AP (Table 3 and Table 5).
• Beats VoxelNet and AVOD baselines in 3D and BEV detections on KITTI val set (Car, Pedestrian, Cyclist) across Easy/Moderate/Hard levels.