[Paper Review] AI-oriented Medical Workload Allocation for Hierarchical Cloud/Edge/Device Computing

Tianshu Hao, Jianfeng Zhan+3 more|arXiv (Cornell University)|Feb 10, 2020

IoT and Edge/Fog Computing28 references4 citations

TL;DR

This paper proposes an AI-oriented workload allocation strategy for hierarchical cloud/edge/device systems in emergency medical settings, optimizing response time by dynamically assigning medical AI workloads (e.g., life-death prediction, breathlessness alerts) to the most suitable layer—cloud, edge, or device—based on processing and transmission latency. The method reduces total response time by up to 63% compared to fixed-layer deployment strategies.

ABSTRACT

In a hierarchically-structured cloud/edge/device computing environment, workload allocation can greatly affect the overall system performance. This paper deals with AI-oriented medical workload generated in emergency rooms (ER) or intensive care units (ICU) in metropolitan areas. The goal is to optimize AI-workload allocation to cloud clusters, edge servers, and end devices so that minimum response time can be achieved in life-saving emergency applications. In particular, we developed a new workload allocation method for the AI workload in distributed cloud/edge/device computing systems. An efficient scheduling and allocation strategy is developed in order to reduce the overall response time to satisfy multi-patient demands. We apply several ICU AI workloads from a comprehensive edge computing benchmark Edge AIBench. The healthcare AI applications involved are short-of-breath alerts, patient phenotype classification, and life-death threats. Our experimental results demonstrate the high efficiency and effectiveness in real-life health-care and emergency applications.

Motivation & Objective

• Address the challenge of minimizing response time in latency-sensitive medical AI applications within a three-tier cloud/edge/device architecture.
• Identify optimal deployment layers (cloud, edge, or device) for individual AI workloads based on computational capacity and network conditions.
• Develop a scheduling strategy for multiple concurrent medical AI jobs to minimize total and last completion response times.
• Evaluate the proposed strategy using real-world ICU datasets and edge AI benchmarks to ensure clinical relevance and practical effectiveness.

Proposed method

• Formalize response time as the sum of processing time (dependent on device computational capability) and transmission time (dependent on network bandwidth).
• Model workload allocation as a trade-off between computational load and communication overhead across the three layers.
• Propose a single-job optimization algorithm that selects the deployment layer minimizing end-to-end response time using analytical latency estimation.
• Design a heuristic-based scheduling algorithm for multi-job scenarios that prioritizes workload sequencing to minimize cumulative response time.
• Integrate workload characterization from Edge AIBench, including real medical AI workloads such as patient phenotype classification and life-death threat detection.
• Use numerical analysis and simulation to compare the proposed strategy against fixed deployment strategies (e.g., all on cloud, edge, or device).

Experimental results

Research questions

• RQ1 What is the optimal deployment layer (cloud, edge, or device) for a single AI workload in a hierarchical cloud/edge/device system to minimize response time?
• RQ2 How does dynamic workload allocation across layers affect the total response time in multi-job medical AI scenarios?
• RQ3 What scheduling strategy minimizes the overall response time when multiple latency-sensitive medical AI jobs are processed in sequence?
• RQ4 How does the proposed allocation strategy compare to fixed-layer deployment strategies in terms of response time reduction for ICU applications?

Key findings

• The proposed workload allocation strategy achieved a total response time of 150 units, which is 33% lower than deploying on the optimal layer per job (227 units).
• The strategy reduced the last completion time to 43 units, representing a 63% improvement over deployment on the cloud server (74 units).
• Compared to deploying all jobs on the edge server, the proposed method reduced response time by 63% (416 vs. 150 units).
• The strategy outperformed all baseline approaches, including deployment on end devices (366 units) and edge servers (416 units), in both total and last response time.
• The results demonstrate that dynamic layer selection based on workload characteristics and system constraints significantly improves performance over static deployment.
• The heuristic scheduling algorithm effectively reduces job queueing delays, especially in high-concurrency scenarios with prioritized medical workloads.