[论文解读] LeGR: Filter Pruning via Learned Global Ranking.

Ting-Wu Chin, Ruizhou Ding+2 more|arXiv (Cornell University)|Apr 28, 2019

Advanced Neural Network Applications参考文献 17被引用 22

一句话总结

LeGR 提出了一种对卷积滤波器进行学习的全局排序方法，以生成具有多样化准确率-延迟权衡的多个剪枝卷积神经网络架构，从而无需预先指定目标模型复杂度。该方法在剪枝速度上比之前的方法快2倍至3倍，同时在CIFAR-100、ResNet-56上保持或提升了性能，并在ImageNet和Bird-200上进一步得到验证。

ABSTRACT

Pruning convolutional filters has demonstrated its effectiveness in compressing ConvNets. Prior art in filter pruning requires users to specify a target model complexity (e.g., model size or FLOP count) for the resulting architecture. However, determining a target model complexity can be difficult for optimizing various embodied AI applications such as autonomous robots, drones, and user-facing applications. First, both the accuracy and the speed of ConvNets can affect the performance of the application. Second, the performance of the application can be hard to assess without evaluating ConvNets during inference. As a consequence, finding a sweet-spot between the accuracy and speed via filter pruning, which needs to be done in a trial-and-error fashion, can be time-consuming. This work takes a first step toward making this process more efficient by altering the goal of model compression to producing a set of ConvNets with various accuracy and latency trade-offs instead of producing one ConvNet targeting some pre-defined latency constraint. To this end, we propose to learn a global ranking of the filters across different layers of the ConvNet, which is used to obtain a set of ConvNet architectures that have different accuracy/latency trade-offs by pruning the bottom-ranked filters. Our proposed algorithm, LeGR, is shown to be 2x to 3x faster than prior work while having comparable or better performance when targeting seven pruned ResNet-56 with different accuracy/FLOPs profiles on the CIFAR-100 dataset. Additionally, we have evaluated LeGR on ImageNet and Bird-200 with ResNet-50 and MobileNetV2 to demonstrate its effectiveness. Code available at this https URL.

研究动机与目标

• 为解决在滤波器剪枝中手动选择目标模型复杂度所面临的挑战，该挑战耗时且依赖具体应用场景。
• 实现在无需预定义延迟约束的情况下，高效探索多种准确率-延迟权衡。
• 开发一种在所有层上学习全局滤波器排序的方法，以指导生成多样化的模型变体。
• 减少在机器人和无人机等实际AI应用中模型压缩的试错性质。
• 在保持或提升性能的同时，提高剪枝效率，适用于多种基准测试。

提出的方法

• LeGR 使用可微分排序机制，对卷积神经网络所有层的滤波器进行全局排序。
• 该方法根据滤波器的重要性进行排序，从而系统性地剪除排名最低的滤波器，以生成多个剪枝后的架构。
• 排序通过端到端训练，使用一个平衡准确率与FLOP减少的损失函数进行优化。
• 通过移除低于由目标FLOP预算决定的阈值的滤波器，按层进行剪枝。
• 该方法支持生成类似帕累托前沿的模型集合，具有不同的准确率和延迟特性。
• 该方法在ResNet-50和MobileNetV2上，于CIFAR-100、ImageNet和Bird-200上进行了评估。

实验结果

研究问题

• RQ1 是否可以学习一种全局滤波器排序机制，实现在不预先定义目标复杂度情况下的高效、多目标模型压缩？
• RQ2 与逐层或启发式剪枝相比，LeGR的全局排序在速度和性能上表现如何？
• RQ3 LeGR能否在多个数据集上生成一组多样化的剪枝模型，实现有利的准确率/FLOPs权衡？
• RQ4 所学习的排序是否能在不同架构（如ResNet-50和MobileNetV2）之间泛化？
• RQ5 LeGR在大规模基准（如ImageNet和Bird-200）上的效率和性能如何扩展？

主要发现

• LeGR 在CIFAR-100上使用ResNet-56时，剪枝速度比之前最先进方法快2至3倍。
• LeGR生成的剪枝模型在等效FLOP水平下，准确率保持或超过之前方法。
• LeGR 成功生成了一组具有不同准确率和延迟权衡的多样化剪枝模型，支持在实际应用中灵活部署。
• 该方法在更大模型和数据集上泛化良好，在ResNet-50和MobileNetV2上于ImageNet和Bird-200上表现出色。
• 所学习的全局排序有效捕捉了各层间滤波器的重要性，实现了在不同架构上的一致且高效的剪枝。
• 该方法减少了对模型复杂度进行迭代试错调优的需求，简化了具身AI系统中的模型压缩流程。