[论文解读] Glance and Focus: a Dynamic Approach to Reducing Spatial Redundancy in Image Classification

Yulin Wang, Kangchen Lv+4 more|arXiv (Cornell University)|Oct 11, 2020

Advanced Neural Network Applications被引用 87

一句话总结

GFNet通过按顺序处理小图像补丁（先看再聚焦）在强化学习引导下，达到自适应推理，计算量更低且保持准确性。

ABSTRACT

The accuracy of deep convolutional neural networks (CNNs) generally improves when fueled with high resolution images. However, this often comes at a high computational cost and high memory footprint. Inspired by the fact that not all regions in an image are task-relevant, we propose a novel framework that performs efficient image classification by processing a sequence of relatively small inputs, which are strategically selected from the original image with reinforcement learning. Such a dynamic decision process naturally facilitates adaptive inference at test time, i.e., it can be terminated once the model is sufficiently confident about its prediction and thus avoids further redundant computation. Notably, our framework is general and flexible as it is compatible with most of the state-of-the-art light-weighted CNNs (such as MobileNets, EfficientNets and RegNets), which can be conveniently deployed as the backbone feature extractor. Experiments on ImageNet show that our method consistently improves the computational efficiency of a wide variety of deep models. For example, it further reduces the average latency of the highly efficient MobileNet-V3 on an iPhone XS Max by 20% without sacrificing accuracy. Code and pre-trained models are available at https://github.com/blackfeather-wang/GFNet-Pytorch.

研究动机与目标

• 通过利用高分辨率图像分类中的时空冗余来降低计算量的动机。
• 开发一个两阶段框架（glance and focus）以自适应地处理小输入并在有信心时终止。
• 确保与轻量级骨干网络（如MobileNet、EfficientNet、RegNet）兼容，以实现高效推断。

提出的方法

• 从下采样的全图开始处理一系列小输入（glance）。
• 使用补丁提案网络定位后续聚焦步骤的判别区域。
• 使用递归分类器在每一步输出预测，并基于置信度提前终止。
• 通过策略梯度训练补丁选择，以最大化类别概率的增量增益。
• 通过可配置的置信阈值（预算/ anytime 设置）实现在线计算控制。
• 可选地对编码器进行正则化以鼓励线性可分的特征并改善定位。

实验结果

研究问题

• RQ1 在高分辨率图像上，串行的基于补丁的处理策略是否能在降低计算量的同时维持分类准确性？
• RQ2 应如何选择判别区域以及如何在图像之间自适应分配计算？
• RQ3 在预算和 anytime 推断任务中，GFNet在最先进的轻量级CNN骨干上表现如何？
• RQ4 凝视式的 glance 步骤与 focus 步骤对准确性和延迟有何影响？

主要发现

• GFNet在MobileNet-V3、RegNet-Y、EfficientNet、ResNet和DenseNet骨干上始终提升效率。
• 对于MobileNet-V3，在相似精度下GFNet将乘加运算降低最多1.4倍。
• 对于ResNets和DenseNets，GFNet在相同精度下实现约2–3倍的计算量下降。
• 在iPhone XS Max上，GFNet在相关骨干上对延迟的加速约为1.3x至2.9x。
• GFNet实现在线可调的计算以满足预算，同时对准确率的损失最小。
• 与如MSDNet等基线相比，GFNet在预算批量分类和 anytime 预测方面实现具有竞争力或更优的准确率。