[Paper Review] 3DSSD: Point-based 3D Single Stage Object Detector

Zetong Yang, Yanan Sun+2 more|arXiv (Cornell University)|Feb 24, 2020

Advanced Neural Network Applications43 references78 citations

TL;DR

Introduces a lightweight, point-based 3D single-stage detector that removes upsampling and refinement stages, using a fusion sampling strategy and an anchor-free head to achieve fast, accurate 3D detection on KITTI and nuScenes.

ABSTRACT

Currently, there have been many kinds of voxel-based 3D single stage detectors, while point-based single stage methods are still underexplored. In this paper, we first present a lightweight and effective point-based 3D single stage object detector, named 3DSSD, achieving a good balance between accuracy and efficiency. In this paradigm, all upsampling layers and refinement stage, which are indispensable in all existing point-based methods, are abandoned to reduce the large computation cost. We novelly propose a fusion sampling strategy in downsampling process to make detection on less representative points feasible. A delicate box prediction network including a candidate generation layer, an anchor-free regression head with a 3D center-ness assignment strategy is designed to meet with our demand of accuracy and speed. Our paradigm is an elegant single stage anchor-free framework, showing great superiority to other existing methods. We evaluate 3DSSD on widely used KITTI dataset and more challenging nuScenes dataset. Our method outperforms all state-of-the-art voxel-based single stage methods by a large margin, and has comparable performance to two stage point-based methods as well, with inference speed more than 25 FPS, 2x faster than former state-of-the-art point-based methods.

Motivation & Objective

• Motivate efficient, accurate 3D object detection directly on raw point clouds without voxelization or two-stage refinement.
• Develop a lightweight point-based single-stage framework that eliminates expensive FP layers and refinement modules.
• Propose fusion sampling to preserve interior points during downsampling and enable robust detection.
• Design a candidate generation layer and an anchor-free regression head with 3D center-ness to balance accuracy and speed.
• Demonstrate state-of-the-art or competitive performance on KITTI and nuScenes with high inference speed.

Proposed method

• Backbone with multiple set abstraction layers using Fusion Sampling (FS) to retain both positive interior and representative negative points.
• Feature-FPS (F-FPS) combines spatial distance and feature distance to select points, mitigating loss of foreground points during downsampling.
• Candidate Generation (CG) layer shifts F-FPS points to generate candidate centers and gathers surrounding points for feature extraction.
• Anchor-free regression head predicts 3D box offsets, size, and orientation for each candidate point in a single stage.
• 3D center-ness assignment scores candidates by proximity to instance centers using a 3D geometric center-ness formula to guide classification.
• Loss combines classification, regression (distance, size, angle, corners) and shifting supervision for CG.

Experimental results

Research questions

• RQ1 Can a fully point-based 3D detector achieve competitive accuracy without FP layers or a refinement module?
• RQ2 Does a fusion sampling strategy improve foreground point retention and overall detection performance on challenging datasets?
• RQ3 Is an anchor-free, center-ness-guided head sufficient for accurate 3D bounding box regression in a single-stage framework?

Key findings

• Outperforms state-of-the-art voxel-based single-stage detectors on KITTI and competes with two-stage point-based methods while running at high speed (38 ms per scene on KITTI with Titan V).
• Fusion sampling (FS) with F-FPS and D-FPS preserves interior points and maintains enough negative points for robust classification, improving AP over D-FPS and F-FPS alone.
• Anchor-free regression with 3D center-ness yields strong localization by prioritizing candidate points near instance centers.
• KITTI val results show Ours achieving 89.71 Easy, 79.45 Moderate, 78.67 Hard APs, outperforming SECOND and VoxelNet and matching or exceeding PointPillars in moderate.
• On nuScenes, Ours achieves stronger performance than voxel-based single-stage methods and competitive results to two-stage point-based approaches, with better velocity and attribute predictions.