[Paper Review] ScissionLite: Accelerating Distributed Deep Neural Networks Using Transfer Layer

Hyunho Ahn, Munkyu Lee+2 more|arXiv (Cornell University)|May 5, 2021

Advanced Neural Network Applications4 citations

TL;DR

ScissionLite is a holistic framework that accelerates distributed deep neural network (DNN) inference in Industrial IoT (IIoT) by inserting a lightweight, traffic-aware Transfer Layer (TL) between the optimal slicing point of a DNN model. The TL uses a down/upsampling network to compress feature maps on the IIoT device and reconstruct them on the edge, reducing network traffic with minimal accuracy loss, achieving up to 16× latency reduction over local inference and 2.8× over state-of-the-art methods.

ABSTRACT

Industrial Internet of Things (IIoT) applications can benefit from leveraging edge computing. For example, applications underpinned by deep neural networks (DNN) models can be sliced and distributed across the IIoT device and the edge of the network for improving the overall performance of inference and for enhancing privacy of the input data, such as industrial product images. However, low network performance between IIoT devices and the edge is often a bottleneck. In this study, we develop ScissionLite, a holistic framework for accelerating distributed DNN inference using the Transfer Layer (TL). The TL is a traffic-aware layer inserted between the optimal slicing point of a DNN model slice in order to decrease the outbound network traffic without a significant accuracy drop. For the TL, we implement a new lightweight down/upsampling network for performance-limited IIoT devices. In ScissionLite, we develop ScissionTL, the Preprocessor, and the Offloader for end-to-end activities for deploying DNN slices with the TL. They decide the optimal slicing point of the DNN, prepare pre-trained DNN slices including the TL, and execute the DNN slices on an IIoT device and the edge. Employing the TL for the sliced DNN models has a negligible overhead. ScissionLite improves the inference latency by up to 16 and 2.8 times when compared to execution on the local device and an existing state-of-the-art model slicing approach respectively.

Motivation & Objective

• To address the bottleneck of limited uplink bandwidth in IIoT networks during distributed DNN inference.
• To reduce data transfer between IIoT devices and edge servers without significantly degrading inference accuracy.
• To develop a traffic-aware, lightweight solution suitable for computation-constrained IIoT devices.
• To enable end-to-end optimization of model slicing with dynamic traffic reduction via a novel Transfer Layer.
• To validate the framework on real-world, production-grade DNNs under realistic network conditions.

Proposed method

• Introduces a Transfer Layer (TL) as a traffic-aware component inserted between DNN slices to compress and decompress feature maps.
• Designs a lightweight down/upsampling neural network for the TL, optimized for low-computation IIoT devices.
• Develops ScissionTL, a Scission-based tool that re-optimizes the slicing point considering reduced traffic due to the TL.
• Employs a Preprocessor to generate and retrain the DNN slice including the TL (TLModel) using fine-tuning with SGD to recover accuracy.
• Uses an Offloader to deploy the DNN slices across the IIoT device and edge with minimal latency.
• Employs empirical benchmarking on device, edge, and cloud to determine optimal slicing points under varying network conditions.

Experimental results

Research questions

• RQ1 Can a lightweight, integrated Transfer Layer reduce data transfer in distributed DNN inference without significant accuracy degradation?
• RQ2 How does the inclusion of a TL affect the optimal slicing point selection in resource-constrained IIoT environments?
• RQ3 To what extent does the TL improve inference latency compared to local execution and existing model slicing methods?
• RQ4 Does the TL maintain acceptable accuracy when applied to large, production-grade DNNs under real-world network conditions?
• RQ5 Can the framework be generalized across different DNN architectures and network bandwidths?

Key findings

• ScissionLite reduces inference latency by up to 16 times compared to local execution on the IIoT device.
• The framework achieves a 2.8× speedup over a state-of-the-art model slicing approach under the same conditions.
• The accuracy drop due to the TL is minimal, at only 0.9% to 1.4% on top-5 ImageNet accuracy.
• The Transfer Layer introduces negligible computational overhead, making it suitable for performance-limited IIoT devices.
• The framework maintains stable performance even at low network bandwidth (e.g., 30 Mbps), demonstrating robustness to variable network conditions.
• ScissionTL identifies earlier slicing points than Scission due to reduced data traffic, indicating the TL's effectiveness in traffic-aware optimization.