[Paper Review] 3D Object Detection Method Based on YOLO and K-Means for Image and Point Clouds

Xuanyu Yin, Yoko Sasaki+2 more|arXiv (Cornell University)|Apr 21, 2020

Remote Sensing and LiDAR Applications13 references4 citations

TL;DR

This paper proposes a real-time 3D object detection method combining YOLO-based 2D detection with K-means clustering on LiDAR point clouds. By projecting 2D YOLO detections onto 3D point clouds and refining them via unsupervised clustering, the method improves precision and reduces noise, achieving 0.856 precision and 0.192s inference time per frame on two 1080Ti GPUs.

ABSTRACT

Lidar based 3D object detection and classification tasks are essential for autonomous driving(AD). A lidar sensor can provide the 3D point cloud data reconstruction of the surrounding environment. However, real time detection in 3D point clouds still needs a strong algorithmic. This paper proposes a 3D object detection method based on point cloud and image which consists of there parts.(1)Lidar-camera calibration and undistorted image transformation. (2)YOLO-based detection and PointCloud extraction, (3)K-means based point cloud segmentation and detection experiment test and evaluation in depth image. In our research, camera can capture the image to make the Real-time 2D object detection by using YOLO, we transfer the bounding box to node whose function is making 3d object detection on point cloud data from Lidar. By comparing whether 2D coordinate transferred from the 3D point is in the object bounding box or not can achieve High-speed 3D object recognition function in GPU. The accuracy and precision get imporved after k-means clustering in point cloud. The speed of our detection method is a advantage faster than PointNet.

Motivation & Objective

• Address the challenge of real-time 3D object detection in LiDAR point clouds, which are irregular and computationally heavy.
• Leverage 2D image detection via YOLO to accelerate 3D detection by focusing on relevant point cloud regions.
• Improve detection accuracy and reduce noise in 3D point clouds through unsupervised K-means clustering.
• Enable high-speed, GPU-accelerated 3D object recognition by combining image-based detection with point cloud processing.
• Develop a pipeline for end-to-end 3D object detection using calibrated camera and LiDAR data, with evaluation on depth images.

Proposed method

• Perform lidar-camera calibration and undistort multi-view camera images to align 2D and 3D data.
• Apply YOLOv3 (and YOLOv1 tiny) to detect 2D objects in undistorted images, outputting bounding boxes and class labels.
• Map 2D detection bounding boxes to 3D point clouds using extrinsic and intrinsic camera parameters and time-synchronized data from rosbag.
• Extract all 3D points within each 2D bounding box and assign them class-specific colors for visualization.
• Apply K-means clustering to segment point clouds within each detected bounding box, removing noise and refining object boundaries.
• Convert final 3D detection results into 32×1024 depth images for evaluation against ground truth using pixel-level metrics.

Experimental results

Research questions

• RQ1 Can 2D object detection via YOLO be effectively transferred to 3D point clouds to enable real-time 3D detection?
• RQ2 To what extent does K-means clustering improve 3D object detection accuracy by reducing noise in point cloud regions?
• RQ3 How does the proposed method compare in speed and accuracy to existing 3D detection models like PointNet?
• RQ4 Can the fusion of image and LiDAR data through bounding box projection achieve high-precision 3D detection with minimal computational overhead?
• RQ5 What is the impact of clustering on the recall and precision of 3D object detection in complex scenes?

Key findings

• The method achieved 0.856 precision and 0.6103 recall after K-means clustering, representing a 1.7% improvement in precision and 1.5% in recall over the non-clustered baseline.
• K-means clustering reduced the number of point clouds by 31.6% on average, significantly removing noise while preserving object-relevant points.
• The average inference time was 0.192 seconds per frame with K-means clustering and 0.19 seconds without, demonstrating high real-time performance on two 1080Ti GPUs.
• The method outperformed PointNet in speed, with a 0.19s processing time per frame, while maintaining competitive accuracy (0.837 mAP) in comparison.
• The depth image-based evaluation showed that the final detection results with K-means clustering (Fig. 17) closely matched the ground truth (Fig. 15), confirming improved spatial alignment.
• In cases with no detected objects, the method reduced false positives by 91% on average, as shown by the drop from 49.15% to 0.85% of point cloud data in empty frames after clustering.