[论文解读] Towards Compact ConvNets via Structure-Sparsity Regularized Filter Pruning

Shaohui Lin, Rongrong Ji+3 more|arXiv (Cornell University)|Jan 23, 2019

Advanced Neural Network Applications被引用 11

一句话总结

该论文提出了一种结构化稀疏正则化（SSR）滤波器剪枝方法，通过联合优化滤波器重要性与结构化稀疏性，采用ℓ₂,₁-和ℓ₂,₀-范数正则化，并利用交替方向乘子法（AULM）求解器实现高效、结构化的剪枝。该方法在VGG-16上实现了SOTA精度，优化速度提升至2.5倍，内存使用降低15.4倍，同时在目标检测和领域自适应等迁移学习任务中保持高性能。

ABSTRACT

The success of convolutional neural networks (CNNs) in computer vision applications has been accompanied by a significant increase of computation and memory costs, which prohibits its usage on resource-limited environments such as mobile or embedded devices. To this end, the research of CNN compression has recently become emerging. In this paper, we propose a novel filter pruning scheme, termed structured sparsity regularization (SSR), to simultaneously speedup the computation and reduce the memory overhead of CNNs, which can be well supported by various off-the-shelf deep learning libraries. Concretely, the proposed scheme incorporates two different regularizers of structured sparsity into the original objective function of filter pruning, which fully coordinates the global outputs and local pruning operations to adaptively prune filters. We further propose an Alternative Updating with Lagrange Multipliers (AULM) scheme to efficiently solve its optimization. AULM follows the principle of ADMM and alternates between promoting the structured sparsity of CNNs and optimizing the recognition loss, which leads to a very efficient solver (2.5x to the most recent work that directly solves the group sparsity-based regularization). Moreover, by imposing the structured sparsity, the online inference is extremely memory-light, since the number of filters and the output feature maps are simultaneously reduced. The proposed scheme has been deployed to a variety of state-of-the-art CNN structures including LeNet, AlexNet, VGG, ResNet and GoogLeNet over different datasets. Quantitative results demonstrate that the proposed scheme achieves superior performance over the state-of-the-art methods. We further demonstrate the proposed compression scheme for the task of transfer learning, including domain adaptation and object detection, which also show exciting performance gains over the state-of-the-arts.

研究动机与目标

• 解决非结构化稀疏CNN在资源受限环境下的效率低下与硬件不兼容问题。
• 开发一种滤波器剪枝方法，同时降低FLOPs、内存占用与参数量，同时保持模型精度。
• 通过结构化稀疏正则化实现与标准深度学习库兼容的高效、结构化剪枝。
• 利用剪枝模型提升迁移学习中领域自适应与目标检测任务的性能。
• 设计一种优化框架，通过协调全局输出与局部剪枝操作，自适应选择重要滤波器。

提出的方法

• 提出一种新颖的结构化稀疏正则化（SSR）框架，结合ℓ₂,₁-和ℓ₂,₀-范数正则化，以促进结构化滤波器剪枝。
• 采用交替方向乘子法（AULM）求解器，交替执行结构化稀疏性约束与识别损失最小化，受ADMM启发。
• 使用基于张量的卷积操作，可通过BLAS库加速，实现在标准硬件上的高效推理。
• 将AULM求解器应用于迭代剪枝过程，逐步移除不重要滤波器，删除对应输出特征图，并在后续层更新滤波器通道数。
• 实施两阶段流程：(1) 通过AULM实现自适应滤波器选择；(2) 剪枝后对网络进行微调以恢复精度。
• 支持在多种架构（LeNet、AlexNet、VGG、ResNet、GoogLeNet）和数据集（ImageNet、CUB-200、PASCAL VOC）上的部署。

实验结果

研究问题

• RQ1 结构化稀疏正则化能否在不损失精度的前提下，有效降低CNN的FLOPs与内存占用？
• RQ2 与现有基于组稀疏性的剪枝优化方法相比，AULM求解器在效率与精度方面表现如何？
• RQ3 所提出的SSR方法能否在领域自适应与目标检测等迁移学习场景中保持高性能？
• RQ4 联合优化全局输出与局部滤波器剪枝是否能带来优于独立滤波器选择的泛化性能？
• RQ5 结构化剪枝在保持与标准深度学习库兼容的前提下，能在多大程度上减少模型大小与推理时间？

主要发现

• SSR-L2,1-GAP方法在CUB-200上仅使用880万参数与44亿FLOPs，实现29.55%的Top-1错误率，内存成本降低15.4倍，理论推理速度提升3.5倍，优于VGG-16。
• 在PASCAL VOC 2007上，SSR-L2,1模型在Titan X GPU上实现2.45倍的推理速度提升，mAP仅下降0.3%，优于随机剪枝、L1与APoZ剪枝方法。
• AULM求解器相比最新直接求解组稀疏正则化的方法快2.5倍，实现高效优化。
• 剪枝后的模型在迁移学习中保持强性能，SSR-L2,1-GAP在CUB-200上微调时误差增加最低（1.95%）。
• 在多种架构与数据集上，该方法相比SOTA滤波器剪枝方法实现了更优的精度表现。
• 结构化稀疏方法通过同时减少滤波器数量与输出特征图数量，实现内存轻量级推理。