[论文解读] GFD-SSD: Gated Fusion Double SSD for Multispectral Pedestrian Detection

Yang Zheng, Izzat H. Izzat+1 more|arXiv (Cornell University)|Mar 16, 2019

Advanced Neural Network Applications参考文献 29被引用 35

一句话总结

引入门控融合双SSD（GFD-SSD），融合彩色与热像流用于多光谱行人检测，在比基于 Faster-RCNN 的融合网络更快推理的同时实现更低的漏检率。

ABSTRACT

Pedestrian detection is an essential task in autonomous driving research. In addition to typical color images, thermal images benefit the detection in dark environments. Hence, it is worthwhile to explore an integrated approach to take advantage of both color and thermal images simultaneously. In this paper, we propose a novel approach to fuse color and thermal sensors using deep neural networks (DNN). Current state-of-the-art DNN object detectors vary from two-stage to one-stage mechanisms. Two-stage detectors, like Faster-RCNN, achieve higher accuracy, while one-stage detectors such as Single Shot Detector (SSD) demonstrate faster performance. To balance the trade-off, especially in the consideration of autonomous driving applications, we investigate a fusion strategy to combine two SSDs on color and thermal inputs. Traditional fusion methods stack selected features from each channel and adjust their weights. In this paper, we propose two variations of novel Gated Fusion Units (GFU), that learn the combination of feature maps generated by the two SSD middle layers. Leveraging GFUs for the entire feature pyramid structure, we propose several mixed versions of both stack fusion and gated fusion. Experiments are conducted on the KAIST multispectral pedestrian detection dataset. Our Gated Fusion Double SSD (GFD-SSD) outperforms the stacked fusion and achieves the lowest miss rate in the benchmark, at an inference speed that is two times faster than Faster-RCNN based fusion networks.

研究动机与目标

• 通过利用彩色和热成像，在多变光照条件下实现鲁棒的行人检测。
• 开发一种融合策略，将两个 SSD 检测器与可学习的特征融合单元相结合。
• 通过探索高效的融合架构，在自动驾驶应用中在准确性和速度之间取得平衡。

提出的方法

• 提出两种新颖的门控融合单元（GFU）的变体，学习从两个 SSD 中层的特征图进行组合。
• 将 GFU 应用于整个特征金字塔，以创建混合堆叠和门控融合策略。
• 通过学习的特征级融合，而不是简单的特征堆叠，来融合彩色和热 SSD。
• 在 KAIST 多光谱行人检测数据集上评估，以与堆叠式融合基线和基于 Faster-RCNN 的融合网络进行比较。

实验结果

研究问题

• RQ1 门控融合单元能否在多光谱行人检测中学习比简单特征堆叠更有效的跨传感器特征整合？
• RQ2 在双 SSD 框架中通过 GFU 将彩色与热流融合，是否在保持实时速度的前提下提升漏检率？
• RQ3 GFD-SSD 与基于 Faster-RCNN 的融合网络在 KAIST 数据上的准确性和推理速度方面有何比较？

主要发现

• GFD-SSD 在 KAIST 多光谱行人数据集上优于堆叠式融合基线。
• 所提出的方法在评估的多种方法中实现了最低的漏检率。
• 推理速度比基于 Faster-RCNN 的融合网络快两倍。