[论文解读] On Efficient Real-Time Semantic Segmentation: A Survey

Christopher J. Holder, Muhammad Shafique|arXiv (Cornell University)|Jun 17, 2022

Advanced Neural Network Applications被引用 21

一句话总结

本综述分析适用于低内存嵌入式硬件的紧凑高效实时语义分割模型，在一致的评估设置下比较它们的潜在延迟和精度。

ABSTRACT

Semantic segmentation is the problem of assigning a class label to every pixel in an image, and is an important component of an autonomous vehicle vision stack for facilitating scene understanding and object detection. However, many of the top performing semantic segmentation models are extremely complex and cumbersome, and as such are not suited to deployment onboard autonomous vehicle platforms where computational resources are limited and low-latency operation is a vital requirement. In this survey, we take a thorough look at the works that aim to address this misalignment with more compact and efficient models capable of deployment on low-memory embedded systems while meeting the constraint of real-time inference. We discuss several of the most prominent works in the field, placing them within a taxonomy based on their major contributions, and finally we evaluate the inference speed of the discussed models under consistent hardware and software setups that represent a typical research environment with high-end GPU and a realistic deployed scenario using low-memory embedded GPU hardware. Our experimental results demonstrate that many works are capable of real-time performance on resource-constrained hardware, while illustrating the consistent trade-off between latency and accuracy.

研究动机与目标

• 评估最先进语义分割性能与低内存嵌入式系统部署约束之间的差距。
• 在一个连贯的分类体系中按主要贡献对高效实时分割工作进行分类。
• 在一致的硬件/软件环境下评估所讨论模型的推理速度，以反映真实部署场景。

提出的方法

• 回顾并按核心贡献与技术对突出的实时分割工作进行分类。
• 基于面向紧凑模型的效率导向方法提供分类法。
• 在固定、现实的硬件/软件栈下使用高端GPU和低内存嵌入式GPU进行实验性评估推理速度。

实验结果

研究问题

• RQ1 在资源受限硬件上实现实时语义分割的主要架构与算法策略是什么？
• RQ2 在一致的评估条件下，延迟与精度的权衡如何在所调研的方法中体现？
• RQ3 哪些方法在嵌入式GPU上实现了实时性能，它们的局限性是什么？
• RQ4 如何对现有的实时分割方法进行分类以指导部署选择？
• RQ5 哪些基准测试或设置最能反映现实世界的自动驾驶部署场景？

主要发现

• 许多工作在资源受限的硬件上实现了实时性能。
• 在所调研的方法中存在持续的延迟与精度权衡。