[论文解读] HorNet: Efficient High-Order Spatial Interactions with Recursive Gated Convolutions

Yongming Rao, Wenliang Zhao+4 more|arXiv (Cornell University)|Jul 28, 2022

Advanced Neural Network Applications被引用 185

一句话总结

HorNet 引入递归门控卷积（g^n Conv）以在卷积框架中实现高阶、输入自适应的时空交互，在 ImageNet、COCO 和 ADE20K 上与视觉 Transformer 和 CNNs 相媲美或更优，并通过 HorFPN 提升密集预测能力。

ABSTRACT

Recent progress in vision Transformers exhibits great success in various tasks driven by the new spatial modeling mechanism based on dot-product self-attention. In this paper, we show that the key ingredients behind the vision Transformers, namely input-adaptive, long-range and high-order spatial interactions, can also be efficiently implemented with a convolution-based framework. We present the Recursive Gated Convolution ($ extit{g}^ extit{n}$Conv) that performs high-order spatial interactions with gated convolutions and recursive designs. The new operation is highly flexible and customizable, which is compatible with various variants of convolution and extends the two-order interactions in self-attention to arbitrary orders without introducing significant extra computation. $ extit{g}^ extit{n}$Conv can serve as a plug-and-play module to improve various vision Transformers and convolution-based models. Based on the operation, we construct a new family of generic vision backbones named HorNet. Extensive experiments on ImageNet classification, COCO object detection and ADE20K semantic segmentation show HorNet outperform Swin Transformers and ConvNeXt by a significant margin with similar overall architecture and training configurations. HorNet also shows favorable scalability to more training data and larger model sizes. Apart from the effectiveness in visual encoders, we also show $ extit{g}^ extit{n}$Conv can be applied to task-specific decoders and consistently improve dense prediction performance with less computation. Our results demonstrate that $ extit{g}^ extit{n}$Conv can be a new basic module for visual modeling that effectively combines the merits of both vision Transformers and CNNs. Code is available at https://github.com/raoyongming/HorNet

研究动机与目标

• 研究vision模型中超越标准卷积和自注意力的高阶空间交互的动机。
• 提出一种卷积机制（g^n Conv）以在有界计算下实现任意阶的空间交互。
• 通过用 g^n Conv 替换自注意力来构建 HorNet 骨架，并在分类、检测和分割任务上进行验证。

提出的方法

• 引入门控卷积（g Conv）以实现输入自适应的空间混合。
• 开发递归门控卷积（g^n Conv），通过粗到细的通道分配和单一路径的逐层卷积实现 n 阶空间交互（Equation 3.3–3.5）。
• 提供有界 FLOPs 分析，显示 FLOPs(g^n Conv) < HWC(2K^2 + 11/3 C + 2)。
• 通过 7×7 核或 Global Filter (GF) 层支持长距离交互，实现大感受野。
• 通过用 g^n Conv 替换自注意力模块来构建 HorNet 骨架，并为密集预测任务建立 HorFPN 变体用于特征融合。

实验结果

研究问题

• RQ1 g^n Conv 能否在有界计算成本下实现任意阶的空间交互？
• RQ2 在可比的架构和训练设定下，HorNet 骨架是否优于 Swin Transformer 和 ConvNeXt？
• RQ3 HorFPN 变体是否在语义分割和目标检测等密集预测任务中以较低 FLOPs 提升性能？
• RQ4 在更大的数据和模型规模（ImageNet-22K 预训练）下，g^n Conv 方法是否具备可扩展性？

主要发现

• 在相似配置下，HorNet 变体在 ImageNet 分类、COCO 目标检测和 ADE20K 分割等任务上持续优于 Swin Transformer 和 ConvNeXt。
• 在 ImageNet-22K 预训练下，HorNet-L 在 ImageNet 上达到 87.7% top-1 精度，在 ADE20K val 上达到 57.9% mIoU，在 COCO val 上达到 59.2% box AP。
• HorFPN 用 g^n Conv 替代标准的 3×3 卷积进行特征融合，在语义分割和目标检测中带来改进，同时显著降低 FLOPs。
• HorNet-L 7×7 和 HorNet-L GF 在语义分割基准上超越 ConvNeXt-XL，同时 FLOPs 约降低 25%。
• GF 变体在多任务上通常获得比 7×7 变体更好的性能，突显结合全局交互的优势。