[Paper Review] A Survey on Deep Neural Network Partition over Cloud, Edge and End Devices

Di Xu, X. He+2 more|arXiv (Cornell University)|Apr 20, 2023

Advanced Neural Network Applications4 citations

TL;DR

This survey presents a comprehensive analysis of deep neural network (DNN) partitioning across cloud, edge, and end devices, introducing a five-dimensional classification framework that unifies existing approaches. It proposes a standardized model for evaluating DNN partitioning strategies and identifies key challenges such as dynamic environments, privacy, and algorithmic complexity, offering a foundation for future research in distributed AI inference.

ABSTRACT

Deep neural network (DNN) partition is a research problem that involves splitting a DNN into multiple parts and offloading them to specific locations. Because of the recent advancement in multi-access edge computing and edge intelligence, DNN partition has been considered as a powerful tool for improving DNN inference performance when the computing resources of edge and end devices are limited and the remote transmission of data from these devices to clouds is costly. This paper provides a comprehensive survey on the recent advances and challenges in DNN partition approaches over the cloud, edge, and end devices based on a detailed literature collection. We review how DNN partition works in various application scenarios, and provide a unified mathematical model of the DNN partition problem. We developed a five-dimensional classification framework for DNN partition approaches, consisting of deployment locations, partition granularity, partition constraints, optimization objectives, and optimization algorithms. Each existing DNN partition approache can be perfectly defined in this framework by instantiating each dimension into specific values. In addition, we suggest a set of metrics for comparing and evaluating the DNN partition approaches. Based on this, we identify and discuss research challenges that have not yet been investigated or fully addressed. We hope that this work helps DNN partition researchers by highlighting significant future research directions in this domain.

Motivation & Objective

• To address the challenge of deploying large-scale DNNs on resource-constrained end and edge devices by enabling efficient model partitioning across heterogeneous computing environments.
• To unify the diverse DNN partitioning approaches in the literature into a coherent, extensible classification framework for systematic comparison and analysis.
• To identify and highlight open research challenges in dynamic environments, privacy protection, and algorithmic efficiency that remain under-investigated.
• To propose standardized metrics for evaluating and comparing DNN partitioning strategies across optimization objectives such as latency, energy, and resource utilization.
• To guide future research by outlining promising directions in dynamic, privacy-aware, and scalable DNN partitioning for edge and end-device inference.

Proposed method

• Proposes a five-dimensional classification framework for DNN partitioning, including deployment locations, partition granularity, constraints, optimization objectives, and optimization algorithms.
• Develops a unified mathematical model of the DNN partition problem to formalize the trade-offs between computation, communication, and resource usage.
• Introduces a set of standardized evaluation metrics, including inference latency, energy consumption, resource utilization, and privacy impact, to enable fair comparison across approaches.
• Analyzes existing DNN partitioning techniques through the lens of the proposed framework, mapping each approach to specific values in each of the five dimensions.
• Employs systematic literature review and synthesis from major digital libraries and search engines to cover recent advances in DNN partitioning from 2015 to 2023.
• Highlights limitations in current approaches, such as ignoring computation complexity of partitioning algorithms and static assumptions in offloading cost modeling.

(a) Fall detection in smart home

Experimental results

Research questions

• RQ1 How can DNN partitioning approaches be systematically classified and compared across different deployment environments (cloud, edge, end devices)?
• RQ2 What are the key performance trade-offs in DNN partitioning, and how can they be modeled and optimized under real-world constraints?
• RQ3 What are the major gaps in current DNN partitioning research, particularly regarding dynamic environments, privacy, and algorithmic efficiency?
• RQ4 How can the computational complexity of DNN partitioning algorithms be minimized to support large-scale models in real-time applications?
• RQ5 What role does cooperative inference play in enhancing privacy and performance in distributed DNN inference?

Key findings

• The five-dimensional classification framework successfully maps and categorizes existing DNN partitioning approaches, enabling a systematic understanding and comparison of diverse techniques.
• Current DNN partitioning methods often overlook the computational complexity of the partitioning algorithm itself, leading to scalability issues with large models.
• Offloading cost modeling in dynamic environments—especially with mobile edge devices—remains underdeveloped, with most studies assuming static or idealized network conditions.
• Resource utilization and fair allocation are critical but underemphasized optimization objectives, often overshadowed by latency and energy consumption.
• Privacy concerns are significant, particularly when intermediate features are transmitted to the cloud, and current approaches offer limited support for privacy-preserving inference.
• Dynamic DNN partitioning is essential for real-world IoT applications due to mobility and changing device availability, but few methods support runtime adaptation with low overhead.

(b) Smart traffic in smart city