[论文解读] Conditionally Deep Hybrid Neural Networks Across Edge and Cloud

Yinghan Long, Indranil Chakraborty+1 more|arXiv (Cornell University)|May 21, 2020

Advanced Neural Network Applications参考文献 27被引用 6

一句话总结

本文提出了一种条件性深度混合神经网络，通过在边缘设备上使用量化层、在云端使用全精度层，实现边缘与云系统之间的推理分配。通过采用高置信度预测的早期退出机制，该模型在CIFAR-10上将能耗降低至原来的5.5倍，在CIFAR-100上降低至4.8倍，同时仅造成极小的准确率损失，分别实现了65%和52%的推理在边缘完成。

ABSTRACT

The pervasiveness of "Internet-of-Things" in our daily life has led to a recent surge in fog computing, encompassing a collaboration of cloud computing and edge intelligence. To that effect, deep learning has been a major driving force towards enabling such intelligent systems. However, growing model sizes in deep learning pose a significant challenge towards deployment in resource-constrained edge devices. Moreover, in a distributed intelligence environment, efficient workload distribution is necessary between edge and cloud systems. To address these challenges, we propose a conditionally deep hybrid neural network for enabling AI-based fog computing. The proposed network can be deployed in a distributed manner, consisting of quantized layers and early exits at the edge and full-precision layers on the cloud. During inference, if an early exit has high confidence in the classification results, it would allow samples to exit at the edge, and the deeper layers on the cloud are activated conditionally, which can lead to improved energy efficiency and inference latency. We perform an extensive design space exploration with the goal of minimizing energy consumption at the edge while achieving state-of-the-art classification accuracies on image classification tasks. We show that with binarized layers at the edge, the proposed conditional hybrid network can process 65% of inferences at the edge, leading to 5.5x computational energy reduction with minimal accuracy degradation on CIFAR-10 dataset. For the more complex dataset CIFAR-100, we observe that the proposed network with 4-bit quantization at the edge achieves 52% early classification at the edge with 4.8x energy reduction. The analysis gives us insights on designing efficient hybrid networks which achieve significantly higher energy efficiency than full-precision networks for edge-cloud based distributed intelligence systems.

研究动机与目标

• 为解决在能源受限的边缘设备上部署大型深度学习模型的挑战。
• 在雾计算环境中实现边缘与云之间的工作负载高效分配。
• 在保持最先进分类准确率的同时，最小化边缘端的能耗。
• 探索量化位深度与层分布的最优配置，以优化混合网络性能。

提出的方法

• 在边缘设备上部署量化（1-bit、2-bit、4-bit）层，在云端使用全精度层。
• 在量化层后引入早期退出机制，使高置信度样本可在边缘提前退出。
• 采用条件性深度学习方法，仅在必要时激活更深层的云端网络。
• 在位深度、量化层数量及早期退出位置之间进行设计空间探索。
• 通过在CIFAR-10和CIFAR-100上进行大量实验，优化能耗与准确率之间的权衡。
• 采用联合优化方式训练混合网络，以提升早期退出决策的鲁棒性。

实验结果

研究问题

• RQ1 在边缘端，量化位深度与量化层数量之间应如何平衡，才能实现能耗最小化？
• RQ2 早期退出位置的设置如何影响混合边缘-云网络中的推理准确率与能耗效率？
• RQ3 在不降低模型准确率的前提下，早期退出机制能在多大程度上减少计算能耗？
• RQ4 网络复杂度（如CIFAR-100与CIFAR-10）如何影响早期退出与量化技术的有效性？

主要发现

• 在CIFAR-10上，采用1-bit量化方案的所提网络使65%的推理样本可在边缘完成，实现5.5倍的能耗降低，且准确率损失极小。
• 在CIFAR-100上，边缘端采用4-bit量化可使52%的样本实现早期退出，能耗降低至4.8倍。
• 在ResNet-32上，使用16个二值化层与14个全精度层可实现CIFAR-10上最佳的性能-能耗权衡。
• 增加多个早期退出点（如两个）并不会显著降低云端能耗，反而可能因训练复杂度增加而降低性能。
• 早期退出层的参数量影响其鲁棒性；较大的退出层（如在24个二值化层后）可能因容量不足而降低早期退出率。
• 能耗节省主要来自云端的减少，因为全精度层的能耗远高于二值化层。