[Paper Review] HDNET: Exploiting HD Maps for 3D Object Detection

Bin Yang, Ming Liang+1 more|arXiv (Cornell University)|Dec 21, 2020

Advanced Image and Video Retrieval Techniques244 citations

TL;DR

HDNET introduces a map-aware single-stage 3D detector that fuses LiDAR with HD map priors (geometric and semantic) in BEV, and can also predict maps online from a single LiDAR sweep, achieving consistent gains and running at 20 FPS.

ABSTRACT

In this paper we show that High-Definition (HD) maps provide strong priors that can boost the performance and robustness of modern 3D object detectors. Towards this goal, we design a single stage detector that extracts geometric and semantic features from the HD maps. As maps might not be available everywhere, we also propose a map prediction module that estimates the map on the fly from raw LiDAR data. We conduct extensive experiments on KITTI as well as a large-scale 3D detection benchmark containing 1 million frames, and show that the proposed map-aware detector consistently outperforms the state-of-the-art in both mapped and un-mapped scenarios. Importantly the whole framework runs at 20 frames per second.

Motivation & Objective

• Demonstrate that HD maps provide strong priors to boost 3D object detection performance.
• Develop a single-stage BEV detector that fuses LiDAR with semantic and geometric map priors.
• Handle the absence of maps by online prediction of map priors from LiDAR data.
• Evaluate the approach on KITTI BEV and TOR4D benchmarks to establish robustness across mapped and unmapped scenarios.

Proposed method

• Represent LiDAR in BEV and augment with HD map priors (ground height relative to map, semantic road mask).
• Fuse priors by concatenating map-derived channels with discretized LiDAR BEV, and process with a fully convolutional backbone and dense detection header.
• Train with multi-task loss (focal loss for classification, smooth L1 for regression) and apply data dropout on map priors to improve robustness when maps are unavailable.
• During inference, detect from dense BEV predictions with NMS to obtain oriented 3D boxes.
• Provide an online map estimation module (ground height and road mask) using a U-Net from a single LiDAR sweep, enabling unmapped scenarios.

Experimental results

Research questions

• RQ1 Can HD maps provide useful priors to improve 3D object detection accuracy and robustness in BEV?
• RQ2 How much do semantic and geometric map priors individually contribute to detection performance?
• RQ3 Can a detector trained with map priors operate robustly when maps are unavailable by using online map prediction?
• RQ4 What are the performance and speed implications of incorporating HD maps and online map estimation in real-world benchmarks?
• RQ5 How does HDNET perform across short and long ranges compared to map-free baselines?

Key findings

• HDNET outperforms a strong map-free baseline (PIXOR++) across TOR4D and KITTI when HD maps are available.
• On TOR4D, HDNET yields AP gains of 2.42, 3.43, and 5.49 percentage points for 0–70 m, 30–50 m, and 50–70 m ranges respectively with offline maps.
• Even with online map priors (estimated from a single LiDAR sweep), HDNET achieves notable gains over the baseline (e.g., +0.91/ +1.69 / +0.92 in respective ranges).
• Using offline HD maps provides larger gains, up to +2.42 AP overall and up to +5.49 AP at long ranges, than online priors.
• On KITTI BEV, HDNET achieves an absolute AP gain of 2.87 percentage points in the moderate setting over PIXOR++, surpassing methods that use camera data or external labels.
• The framework runs at approximately 20 frames per second, suitable for real-time deployment.