[论文解读] Do All MobileNets Quantize Poorly? Gaining Insights into the Effect of Quantization on Depthwise Separable Convolutional Networks Through the Eyes of Multi-scale Distributional Dynamics

Stone Yun, Alexander Wong|arXiv (Cornell University)|Apr 24, 2021

Advanced Neural Network Applications参考文献 27被引用 5

一句话总结

本文研究了深度可分离卷积神经网络（DWSCNNs），包括MobileNets，在后训练量化下为何会出现严重的精度下降。通过多尺度分布动力学的细粒度分析，发现通道间与层间分布之间的动态范围波动和分布不匹配，导致量化误差过度累积，从而解释了DWSCNNs相比常规CNNs量化性能较差的原因。

ABSTRACT

As the "Mobile AI" revolution continues to grow, so does the need to understand the behaviour of edge-deployed deep neural networks. In particular, MobileNets are the go-to family of deep convolutional neural networks (CNN) for mobile. However, they often have significant accuracy degradation under post-training quantization. While studies have introduced quantization-aware training and other methods to tackle this challenge, there is limited understanding into why MobileNets (and potentially depthwise-separable CNNs (DWSCNN) in general) quantize so poorly compared to other CNN architectures. Motivated to gain deeper insights into this phenomenon, we take a different strategy and study the multi-scale distributional dynamics of MobileNet-V1, a set of smaller DWSCNNs, and regular CNNs. Specifically, we investigate the impact of quantization on the weight and activation distributional dynamics as information propagates from layer to layer, as well as overall changes in distributional dynamics at the network level. This fine-grained analysis revealed significant dynamic range fluctuations and a "distributional mismatch" between channelwise and layerwise distributions in DWSCNNs that lead to increasing quantized degradation and distributional shift during information propagation. Furthermore, analysis of the activation quantization errors show that there is greater quantization error accumulation in DWSCNN compared to regular CNNs. The hope is that such insights can lead to innovative strategies for reducing such distributional dynamics changes and improve post-training quantization for mobile.

研究动机与目标

• 理解为何MobileNets及其他DWSCNNs在后训练量化下会出现严重的精度下降。
• 探究这种不良量化行为是否源于深度可分离卷积的本质特性，还是特定于MobileNet架构。
• 分析多尺度分布动力学——尤其是权重与激活分布——在各层中对量化退化的影响。
• 比较DWSCNNs与常规CNNs在量化误差累积与分布偏移方面的差异。
• 为提升边缘部署模型的后训练量化鲁棒性提供洞见。

提出的方法

• 在MobileNet-V1和一组小型DWSCNNs（DWS-ConvNets）上进行了系统性消融研究，与常规CNNs进行对比。
• 在CIFAR-10和ImageNet上以一致的训练流程训练模型，以隔离量化影响。
• 对量化前后各层的权重与激活分布进行逐层分析，重点关注动态范围波动与分布不匹配问题。
• 在推理过程中，通过QMSE、QCE和QKL-Divergence量化各层的量化误差累积。
• 分析批归一化折叠后的权重与激活，以评估量化下的分布动力学。
• 通过多次量化试验并记录2000张图像的数据，确保分布度量的统计可靠性。

Figure 1: Layerwise average precison (see Eq. 2 ) of trained weights in the DWS-Convnets. Low average precision at depthwise-conv layers indicate a mismatch between each individual channel’s dynamic range and the entire tensor’s range. The resulting distributional dynamics lead to significantly high

实验结果

研究问题

• RQ1 为何MobileNets等DWSCNNs在后训练量化下比常规CNNs遭受更严重的精度下降？
• RQ2 MobileNets的不良量化性能是否源于深度可分离卷积的本质，还是由于架构设计选择？
• RQ3 通道间与层间分布之间的动态范围波动和分布不匹配如何影响DWSCNNs中的量化误差累积？
• RQ4 在前向传播过程中，DWSCNNs与常规CNNs的量化误差度量（QMSE、QCE、QKL-Div）差异有多大？
• RQ5 多尺度分布动力学能否解释量化DWSCNNs中观察到的分布偏移与误差传播？

主要发现

• DWSCNNs在各层之间的动态范围波动显著大于常规CNNs，尤其是在深度可分离卷积层中。
• DWSCNNs中通道间与层间分布之间存在显著的分布不匹配，导致量化误差累积增加。
• MobileNet-V1在ImageNet上经后训练量化后出现灾难性的15.6%精度下降，其QMSE与QKL-Div值远高于VGG-19。
• DWSCNNs的层间QMSE、QCE与QKL-Div值始终高于常规CNNs，表明其量化误差累积更多且分布偏移更严重。
• DWSCNNs的量化行为对随机权重初始化更为敏感，表明其学习到的分布具有更高的不稳定性。
• 结果表明，通道级量化以及减少分布不匹配的方法可显著提升DWSCNNs的量化鲁棒性。

Figure 2: Comparing layerwise QMSE of Regular-ConvNets (left), DWS-ConvNets (center), and MobileNets-V1 (right). We are able to see how depthwise separable convolutional networks accumulate much more QMSE on average when traversing through a quantized DWSCNN. Note : Some outliers were removed from t