[论文解读] Densely Nested Top-Down Flows for Salient Object Detection

Chaowei Fang, Haibin Tian+4 more|arXiv (Cornell University)|Feb 18, 2021

Visual Attention and Saliency Detection被引用 5

一句话总结

该论文提出了一种新型轻量级显著性目标检测框架——密集嵌套自顶向下流（DNTDF），通过渐进式压缩快捷路径（PCSPs）增强自顶向下的特征传播，以提升语义特征重用并缓解梯度消失问题。与EfficientNet结合后，DNTDF在DUTS-TE上的FLOPs较F3-Net减少30.48%，同时在六个基准数据集上保持了高精度，实现了SOTA性能。

ABSTRACT

With the goal of identifying pixel-wise salient object regions from each input image, salient object detection (SOD) has been receiving great attention in recent years. One kind of mainstream SOD methods is formed by a bottom-up feature encoding procedure and a top-down information decoding procedure. While numerous approaches have explored the bottom-up feature extraction for this task, the design on top-down flows still remains under-studied. To this end, this paper revisits the role of top-down modeling in salient object detection and designs a novel densely nested top-down flows (DNTDF)-based framework. In every stage of DNTDF, features from higher levels are read in via the progressive compression shortcut paths (PCSP). The notable characteristics of our proposed method are as follows. 1) The propagation of high-level features which usually have relatively strong semantic information is enhanced in the decoding procedure; 2) With the help of PCSP, the gradient vanishing issues caused by non-linear operations in top-down information flows can be alleviated; 3) Thanks to the full exploration of high-level features, the decoding process of our method is relatively memory efficient compared against those of existing methods. Integrating DNTDF with EfficientNet, we construct a highly light-weighted SOD model, with very low computational complexity. To demonstrate the effectiveness of the proposed model, comprehensive experiments are conducted on six widely-used benchmark datasets. The comparisons to the most state-of-the-art methods as well as the carefully-designed baseline models verify our insights on the top-down flow modeling for SOD. The code of this paper is available at https://github.com/new-stone-object/DNTD.

研究动机与目标

• 解决现有显著性目标检测（SOD）框架中自顶向下语义信息利用不足的问题。
• 缓解解码器阶段因非线性操作导致的反向传播中梯度消失问题。
• 设计一种内存高效的解码架构，实现高层语义特征的重用，同时不增加模型复杂度。
• 在计算成本极低的前提下实现SOTA的SOD性能，尤其当与EfficientNet等高效主干网络结合时。

提出的方法

• 提出密集嵌套自顶向下流（DNTDF），通过渐进式压缩快捷路径（PCSPs）将编码器中的高层特征传播至所有解码器阶段。
• 采用PCSPs对高层特征进行压缩与传输，避免非线性操作，从而减少反向传播过程中的梯度消失。
• 使用金字塔池化模块（PPM）从高层特征中提取全局上下文，增强语义表征能力。
• 通过重用压缩后的高层特征设计轻量级解码器，显著降低FLOPs与参数量。
• 将DNTDF模块与EfficientNet主干网络集成，构建高效SOD模型。
• 应用渐进式压缩比（最高达16），在不降低性能的前提下减少主干网络特征的冗余。

实验结果

研究问题

• RQ1 如何更有效地将高层编码器层的自顶向下语义特征传播至低层解码器层？
• RQ2 渐进式压缩快捷路径（PCSPs）是否能缓解SOD解码器中反向传播的梯度消失问题？
• RQ3 高层特征重用在多大程度上可降低SOD模型的计算复杂度，同时不损失性能？
• RQ4 在多样化基准测试中，所提出的DNTDF框架与现有SOD方法相比，在准确率与效率方面表现如何？

主要发现

• 在DUTS-TE数据集上，DNTDF结合EfficientNet-B3的S-measure达到0.892，较F3-Net高出0.007，同时仅消耗其30.48%的FLOPs。
• 使用ResNet50时，DNTDF的Fmax略高于F3-Net（0.856 vs. 0.855），且FLOPs减少2.915G。
• 在DUT-O数据集上，尽管使用更小的主干网络和更少参数，DNTDF的MAE较CSNet降低9.62%。
• 即使采用EfficientNet-B3作为主干，DNTDF的解码器FLOPs也显著低于F3-Net、ITSD、CSF和MINet。
• 消融实验表明，PCSPs与PPM均对性能提升有贡献：在包含PPM的前提下，使用4条PCSPs可使Fmax提升0.011。
• 在编码器特征上应用最高达16的压缩比，性能下降可忽略不计，实现了轻量化设计且无精度损失。