[论文解读] Multi-modal Semantic SLAM for Complex Dynamic Environments

Han Wang, Jing Ying Ko+1 more|arXiv (Cornell University)|May 9, 2022

Robotics and Sensor-Based Localization被引用 5

一句话总结

该论文提出了一种多模态语义SLAM框架，通过融合几何聚类与视觉语义分割，提升了在复杂动态环境中的鲁棒性。通过引入'先看后思'机制增强特征表示，并整合多模态线索，该方法相比基线模型将定位漂移降低了59%，实现了超过10 Hz的实时性能，具备高精度的动态物体剔除与稠密静态地图构建能力。

ABSTRACT

Simultaneous Localization and Mapping (SLAM) is one of the most essential techniques in many real-world robotic applications. The assumption of static environments is common in most SLAM algorithms, which however, is not the case for most applications. Recent work on semantic SLAM aims to understand the objects in an environment and distinguish dynamic information from a scene context by performing image-based segmentation. However, the segmentation results are often imperfect or incomplete, which can subsequently reduce the quality of mapping and the accuracy of localization. In this paper, we present a robust multi-modal semantic framework to solve the SLAM problem in complex and highly dynamic environments. We propose to learn a more powerful object feature representation and deploy the mechanism of looking and thinking twice to the backbone network, which leads to a better recognition result to our baseline instance segmentation model. Moreover, both geometric-only clustering and visual semantic information are combined to reduce the effect of segmentation error due to small-scale objects, occlusion and motion blur. Thorough experiments have been conducted to evaluate the performance of the proposed method. The results show that our method can precisely identify dynamic objects under recognition imperfection and motion blur. Moreover, the proposed SLAM framework is able to efficiently build a static dense map at a processing rate of more than 10 Hz, which can be implemented in many practical applications. Both training data and the proposed method is open sourced at https://github.com/wh200720041/MMS_SLAM.

研究动机与目标

• 解决传统SLAM在动态环境中因移动物体导致漂移与制图误差的局限性。
• 提升在运动模糊、遮挡和小尺度物体等挑战性条件下实例分割的鲁棒性。
• 开发一种实时、计算高效的SLAM系统，利用多模态语义信息分离动态与静态元素。
• 实现在智能工厂等复杂工业场景中可靠、稠密的语义制图，其中存在移动的人与AGV。

提出的方法

• 通过引入'先看后思'机制增强主干网络，学习更具表现力的物体特征表示，从而提升分割精度。
• 结合仅几何的聚类与视觉语义分割，降低对不完美2D实例分割的依赖。
• 将多模态识别集成到SLAM流水线中，以过滤动态物体并保持精确的定位。
• 采用基于SOLOv2的实例分割进行视觉语义识别，通过多尺度特征学习与注意力机制进一步增强。
• 采用两阶段过滤策略：首先通过深度学习分割动态物体；其次利用几何一致性进行验证与优化。
• 通过移除检测到的动态物体，同时保留语义标签与边界框，构建稠密的3D静态地图。

实验结果

研究问题

• RQ1 多模态视觉与几何线索的融合在多大程度上能提升动态环境中语义SLAM的鲁棒性？
• RQ2 ‘先看后思’机制在多大程度上增强了物体特征表示与分割精度？
• RQ3 仅几何聚类能否有效减少由运动模糊与遮挡引起的分割误差？
• RQ4 在动态条件下，该方法与纯视觉方法及基线SLAM相比，定位漂移表现如何？
• RQ5 该系统能否在实现高精度动态物体检测与静态地图构建的同时，保持超过10 Hz的实时性能？

主要发现

• 所提出的多模态SLAM将平均平移漂移误差（ATDE）降低至0.875 cm，相比无语义识别的基线模型（4.834 cm）降低了59%。
• 最大平移漂移误差（MTDE）降低至0.502 cm，显著优于纯视觉方法（0.667 cm）与基线模型（1.877 cm）。
• 系统实现超过10 Hz的实时处理，支持在智能工厂等动态工业环境中的实际部署。
• 动态物体（如人员与AGV）被准确检测并从静态地图中移除，同时提供实时边界框与颜色编码状态可视化（红色表示人员，绿色表示AGV）。
• 消融实验与真实AGV实验验证表明，该方法能有效缓解由运动模糊、遮挡与小尺度物体引起的漂移。
• 开源实现已发布于 https://github.com/wh200720041/MMS_SLAM，支持可复现性与进一步研究。