[论文解读] LION: Linear Group RNN for 3D Object Detection in Point Clouds

Zhe Liu, Jinghua Hou+5 more|arXiv (Cornell University)|Jul 25, 2024

3D Surveying and Cultural Heritage被引用 6

一句话总结

LION 引入基于窗口的 3D 主干网络，使用线性群 RNN 以实现稀疏点云中 3D 目标检测的远距离特征交互，具备 3D 空间描述符和体素生成以提升性能；LION 的变体在多个自动驾驶数据集上达到最前沿的结果（state-of-the-art 结果）。

ABSTRACT

The benefit of transformers in large-scale 3D point cloud perception tasks, such as 3D object detection, is limited by their quadratic computation cost when modeling long-range relationships. In contrast, linear RNNs have low computational complexity and are suitable for long-range modeling. Toward this goal, we propose a simple and effective window-based framework built on LInear grOup RNN (i.e., perform linear RNN for grouped features) for accurate 3D object detection, called LION. The key property is to allow sufficient feature interaction in a much larger group than transformer-based methods. However, effectively applying linear group RNN to 3D object detection in highly sparse point clouds is not trivial due to its limitation in handling spatial modeling. To tackle this problem, we simply introduce a 3D spatial feature descriptor and integrate it into the linear group RNN operators to enhance their spatial features rather than blindly increasing the number of scanning orders for voxel features. To further address the challenge in highly sparse point clouds, we propose a 3D voxel generation strategy to densify foreground features thanks to linear group RNN as a natural property of auto-regressive models. Extensive experiments verify the effectiveness of the proposed components and the generalization of our LION on different linear group RNN operators including Mamba, RWKV, and RetNet. Furthermore, it is worth mentioning that our LION-Mamba achieves state-of-the-art on Waymo, nuScenes, Argoverse V2, and ONCE dataset. Last but not least, our method supports kinds of advanced linear RNN operators (e.g., RetNet, RWKV, Mamba, xLSTM and TTT) on small but popular KITTI dataset for a quick experience with our linear RNN-based framework.

研究动机与目标

• 通过线性群 RNNs 在 3D 点云中激发并实现长距离特征交互，以降低计算开销。
• 通过整合 3D 空间特征描述符来捕捉局部几何信息，从而增强用于 3D 检测的线性 RNNs。
• 利用自回归特性的一种体素生成策略来密化稀疏前景特征。
• 演示 LION 在多种线性 RNN 算子（Mamba、RWKV、RetNet）和数据集上的泛化能力。

提出的方法

• 提出 LION，一种基于窗口的 3D 主干网络，使用线性 RNN 将数千个体素特征分组以实现长距离交互。
• 引入 3D 空间特征描述符（3D 子流形卷积 + LayerNorm + GELU），以在展平为序列时降低空间信息丢失。
• 实现体素合并和体素扩展，以在稀疏点云中实现多尺度特征学习。
• 开发一种体素生成策略，基于主干中的高响应区域自回归生成前景体素特征。
• 在 LION 模块中使用双向线性 RNN 运算符（Mamba、RWKV、RetNet），并采用 X 轴和 Y 轴的窗口分区实现丰富的特征交互。
• 提供一个实用框架，用线性 RNN 基于的方法替代或增强现有基于体素的主干。

Figure 1: (a) Comparison of different 3D backbones in terms of detection performance on Waymo [ 49 ] , nuScenes [ 4 ] , Argoverse V2 [ 59 ] and ONCE [ 34 ] datasets. Here, we adopt Mamba [ 22 ] as the default operator of our LION. Besides, we present the simplified schematic of DSVT (b) [ 57 ] and o

实验结果

研究问题

• RQ1 具备大分组窗口的线性 RNN 是否能够在稀疏点云中的 3D 目标检测中匹配或超越 Transformer 风格的主干？
• RQ2 在 3D 主干中添加 3D 空间特征描述符是否能提升线性分组 RNN 的局部空间建模？
• RQ3 通过自回归线性 RNN 的体素生成是否能使稀疏前景区域密集化并提升检测性能？
• RQ4 LION 在不同线性 RNN 运算符和多样化的 3D 检测数据集上的泛化能力如何？

主要发现

• 使用不同线性 RNN 的 LION 在 Waymo、nuScenes、Argoverse V2 和 ONCE 数据集上达到有竞争力或最先进的性能。
• 由线性 RNN 使能的更大组交互使 LION 相较于小组 Transformer 方法获得更好的长距离特征交互。
• 3D 空间特征描述符在一维序列处理过程中有效减轻空间信息丢失。
• 基于自回归特性的体素生成使前景特征更密集，在稀疏场景中提高判别能力。
• LION-Mamba 在多个数据集上实现了最先进的结果，显示 LION 框架在运算符和领域上的强泛化能力。

Figure 2: The illustration of LION, which mainly consists of several LION blocks, each paired with a voxel generation for feature enhancement and a voxel merging for down-sampling features along the height dimension. $(H,W,D)$ indicates the shape of the 3D feature map, where $H$ , $W$ , and $D$ are