[论文解读] Improving 3D Object Detection with Channel-wise Transformer

Xian-Sheng Hua, Sijia Cai+5 more|arXiv (Cornell University)|Aug 23, 2021

Advanced Neural Network Applications参考文献 38被引用 11

一句话总结

本文提出CT3D，一种新颖的两阶段3D目标检测框架，通过通道注意力Transformer架构改进提议框的精炼。通过整合提议框到点的嵌入表示与自注意力编码，以及通道注意力重加权解码器，CT3D在KITTI中等难度汽车基准上实现了81.77%的AP，达到当前最优性能。

ABSTRACT

Though 3D object detection from point clouds has achieved rapid progress in recent years, the lack of flexible and high-performance proposal refinement remains a great hurdle for existing state-of-the-art two-stage detectors. Previous works on refining 3D proposals have relied on human-designed components such as keypoints sampling, set abstraction and multi-scale feature fusion to produce powerful 3D object representations. Such methods, however, have limited ability to capture rich contextual dependencies among points. In this paper, we leverage the high-quality region proposal network and a Channel-wise Transformer architecture to constitute our two-stage 3D object detection framework (CT3D) with minimal hand-crafted design. The proposed CT3D simultaneously performs proposal-aware embedding and channel-wise context aggregation for the point features within each proposal. Specifically, CT3D uses proposal's keypoints for spatial contextual modelling and learns attention propagation in the encoding module, mapping the proposal to point embeddings. Next, a new channel-wise decoding module enriches the query-key interaction via channel-wise re-weighting to effectively merge multi-level contexts, which contributes to more accurate object predictions. Extensive experiments demonstrate that our CT3D method has superior performance and excellent scalability. Remarkably, CT3D achieves the AP of 81.77% in the moderate car category on the KITTI test 3D detection benchmark, outperforms state-of-the-art 3D detectors.

研究动机与目标

• 解决现有两阶段3D检测器在依赖手工设计组件（如关键点采样与多尺度特征融合）时，提议框精炼能力受限的问题。
• 克服传统点云/体素方法在稀疏无序点云中难以有效建模长距离依赖与上下文关系的局限。
• 设计一种灵活的端到端框架，通过注意力机制增强特征表示，且不被特定RPN架构所限制。
• 通过自注意力机制实现全局上下文建模，提升在遮挡与远距离等挑战性条件下的检测精度。
• 引入通道注意力重加权机制，增强Transformer解码器中查询与键的交互，实现多层级上下文的丰富特征融合。

提出的方法

• 提出一种提议框到点的嵌入模块，利用每个点与提议框关键点之间的相对坐标，生成几何感知的点特征。
• 在Transformer编码器中应用自注意力编码，聚合每个3D提议框内点之间的全局上下文依赖关系。
• 设计一种新颖的通道注意力解码模块，通过在通道维度重加权特征，增强查询与键的交互，提升特征表示能力。
• 将通道注意力Transformer集成到两阶段检测流程中，其中RPN生成提议框，CT3D利用基于注意力的特征学习对提议框进行精炼。
• 采用端到端训练策略，联合优化RPN与CT3D精炼头，实现对多样化点云分布的强大泛化能力。
• 利用Transformer的灵活性处理不规则点云结构，同时保持计算效率与可扩展性。

Figure 1: Overview of CT3D. The raw points are first fed into the RPN for generating 3D proposals. Then the raw points along with the corresponding proposals are processed by the channel-wise Transformer composed of the proposal-to-point encoding module and the channel-wise decoding module. Specific

实验结果

研究问题

• RQ1 可学习的注意力机制是否能在3D提议框精炼中超越手工设计组件（如关键点采样与多尺度特征融合）？
• RQ2 自注意力机制在稀疏3D点云中能多大程度上捕捉长距离依赖，以提升在遮挡或点密度低情况下的检测鲁棒性？
• RQ3 与标准解码方式相比，Transformer解码器中的通道注意力重加权机制如何提升特征表示与预测精度？
• RQ4 所提出的CT3D框架能否与多种RPN主干网络有效结合，同时保持高性能与可扩展性？
• RQ5 各组件（提议框到点嵌入、自注意力、通道注意力解码）对整体检测精度的贡献分别是什么？

主要发现

• CT3D在KITTI中等难度汽车类别上达到81.77%的平均精度（AP），超越所有已发表的SOTA方法。
• 在PointPillar RPN基础上，CT3D相比基线模型检测AP提升+4.82%，证明其在不同RPN主干网络上均具备强大泛化能力。
• 所提出的用于提议框到点嵌入的关键点减法策略，相比基线的尺寸-方向方法，性能最高提升+5.47% AP。
• 移除自注意力编码模块后性能显著下降，证实其在捕捉全局上下文与依赖关系中的关键作用。
• 扩展的通道注意力重加权机制优于标准解码与基础通道注意力重加权，所有难度级别下均取得最高AP。
• 注意力图可视化显示，CT3D在训练过程中逐步聚焦于前景点（如车辆），而减少对背景点的关注，表明其注意力学习有效。

Figure 2: Proposal-to-point encoding. The location features of raw point clouds are first modulated by the proposal information (center and corners) via subtraction operator. Then, the resulting point features are refined by the proposal-aware encoding module with multi-head self-attention mechanism