[Paper Review] PolarBEVDet: Exploring Polar Representation for Multi-View 3D Object Detection in Bird's-Eye-View

Zichen Yu, Quanli Liu+3 more|arXiv (Cornell University)|Aug 29, 2024

Advanced Image and Video Retrieval Techniques4 citations

TL;DR

PolarBEVDet proposes a novel LSS-based multi-view 3D object detection framework that replaces Cartesian Bird's-Eye-View (BEV) representation with polar BEV to better match non-uniform image information distribution and preserve view symmetry. By introducing a polar view transformer, polar temporal fusion, and a polar detection head, along with 2D auxiliary supervision and spatial attention, it achieves state-of-the-art performance on nuScenes, with 52.7% NDS and 43.4% mAP.

ABSTRACT

Recently, LSS-based multi-view 3D object detection provides an economical and deployment-friendly solution for autonomous driving. However, all the existing LSS-based methods transform multi-view image features into a Cartesian Bird's-Eye-View(BEV) representation, which does not take into account the non-uniform image information distribution and hardly exploits the view symmetry. In this paper, in order to adapt the image information distribution and preserve the view symmetry by regular convolution, we propose to employ the polar BEV representation to substitute the Cartesian BEV representation. To achieve this, we elaborately tailor three modules: a polar view transformer to generate the polar BEV representation, a polar temporal fusion module for fusing historical polar BEV features and a polar detection head to predict the polar-parameterized representation of the object. In addition, we design a 2D auxiliary detection head and a spatial attention enhancement module to improve the quality of feature extraction in perspective view and BEV, respectively. Finally, we integrate the above improvements into a novel multi-view 3D object detector, PolarBEVDet. Experiments on nuScenes show that PolarBEVDet achieves the superior performance. The code is available at https://github.com/Yzichen/PolarBEVDet.git.(This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible)

Motivation & Objective

• Address the mismatch between uniform Cartesian BEV grids and non-uniform image information distribution in multi-view 3D detection.
• Preserve view symmetry across surround-view cameras, which is disrupted by translation-invariant convolutions in Cartesian BEV.
• Improve feature quality in both perspective and BEV spaces through auxiliary supervision and attention mechanisms.
• Demonstrate the generalization and superiority of polar BEV representation across different detection architectures.

Proposed method

• Introduce a polar view transformer to convert multi-view image features into a polar BEV representation, with angular and radial grid rasterization.
• Design a polar temporal fusion module to aggregate historical polar BEV features for improved temporal consistency.
• Develop a polar detection head that predicts 3D objects using polar-parameterized representations.
• Incorporate a 2D auxiliary detection head with box regression, classification, and center regression to enhance feature quality in the image view.
• Apply a spatial attention enhancement module in the BEV space to suppress background noise and emphasize foreground objects.
• Integrate all components into a unified LSS-based framework, PolarBEVDet, for end-to-end multi-view 3D detection.

Experimental results

Research questions

• RQ1 Can polar BEV representation better align with the natural distribution of image information in multi-view 3D detection?
• RQ2 Does polar BEV preserve view symmetry across surround-view cameras, reducing representation inconsistency from standard convolution?
• RQ3 How does 2D auxiliary supervision improve feature quality in the image and BEV spaces?
• RQ4 Where is the optimal insertion point for the spatial attention enhancement module in the network architecture?
• RQ5 Does the polar BEV representation generalize across different backbone architectures and detection heads?

Key findings

• PolarBEVDet achieves 52.7% NDS and 43.4% mAP on the nuScenes benchmark, outperforming its Cartesian BEV counterparts.
• The 2D auxiliary supervision improves mAP by 0.4% and NDS by 0.2% when all sub-tasks (box regression, classification, center regression) are used.
• Inserting the spatial attention enhancement module before the BEV encoder yields the best performance, increasing mAP by 0.5% and NDS by 0.4% compared to no SAE.
• The polar BEV representation reduces both translation and orientation errors, indicating improved localization accuracy.
• The polar version of BEVDet, BEVDet4D, and BEVDepth all show consistent improvements over their Cartesian counterparts, proving the universality of the polar BEV approach.
• Qualitative results show accurate detection of small objects (e.g., traffic cones, pedestrians), though distant object detection remains challenging.