[Paper Review] EdgeSlice: Slicing Wireless Edge Computing Network with Decentralized Deep Reinforcement Learning

Qiang Liu, Tao Han+1 more|arXiv (Cornell University)|Mar 28, 2020

Software-Defined Networks and 5G38 references4 citations

TL;DR

EdgeSlice proposes a decentralized deep reinforcement learning (D-DRL) framework for dynamic end-to-end network slicing in wireless edge computing networks. It uses a central performance coordinator and multiple decentralized orchestration agents to optimize resource allocation across radio, transport, and computing domains, achieving significant improvements in SLA compliance, scalability, and performance over baseline methods like TARO.

ABSTRACT

5G and edge computing will serve various emerging use cases that have diverse requirements of multiple resources, e.g., radio, transportation, and computing. Network slicing is a promising technology for creating virtual networks that can be customized according to the requirements of different use cases. Provisioning network slices requires end-to-end resource orchestration which is challenging. In this paper, we design a decentralized resource orchestration system named EdgeSlice for dynamic end-to-end network slicing. EdgeSlice introduces a new decentralized deep reinforcement learning (D-DRL) method to efficiently orchestrate end-to-end resources. D-DRL is composed of a performance coordinator and multiple orchestration agents. The performance coordinator manages the resource orchestration policies in all the orchestration agents to ensure the service level agreement (SLA) of network slices. The orchestration agent learns the resource demands of network slices and orchestrates the resource allocation accordingly to optimize the performance of the slices under the constrained networking and computing resources. We design radio, transport and computing manager to enable dynamic configuration of end-to-end resources at runtime. We implement EdgeSlice on a prototype of the end-to-end wireless edge computing network with OpenAirInterface LTE network, OpenDayLight SDN switches, and CUDA GPU platform. The performance of EdgeSlice is evaluated through both experiments and trace-driven simulations. The evaluation results show that EdgeSlice achieves much improvement as compared to baseline in terms of performance, scalability, compatibility.

Motivation & Objective

• To address the challenge of dynamic, end-to-end resource orchestration in wireless edge computing networks with diverse, heterogeneous service requirements.
• To overcome the lack of closed-form models for correlating multi-domain resources (radio, transport, computing) with slice performance.
• To enable scalable, efficient, and adaptive resource allocation across geographically distributed base stations and edge servers.
• To design a model-free, decentralized learning framework that maintains service level agreement (SLA) compliance without requiring prior knowledge of performance functions.
• To implement and evaluate a prototype system that supports runtime reconfiguration of network slices across O-RAN, SDN, and GPU platforms.

Proposed method

• EdgeSlice employs a decentralized deep reinforcement learning (D-DRL) architecture with a central performance coordinator and multiple distributed orchestration agents.
• The performance coordinator enforces SLA compliance by coordinating resource orchestration policies across agents.
• Each orchestration agent learns optimal resource allocation policies using deep reinforcement learning, adapting to dynamic slice demands and resource constraints.
• Radio, transport, and computing managers enable runtime configuration of resources based on orchestration actions.
• The system is implemented on a prototype using OpenAirInterface (OAI) for radio access, OpenDayLight (ODL) for SDN-controlled transport, and CUDA-enabled GPU platforms for edge computing.
• The D-DRL framework is trained and evaluated using both real-world trace-driven simulations and a physical prototype network.

Experimental results

Research questions

• RQ1 How can end-to-end network slicing be efficiently orchestrated in wireless edge computing networks with dynamic, heterogeneous resource demands?
• RQ2 Can a decentralized deep reinforcement learning approach outperform centralized or heuristic methods in maintaining SLA compliance under unknown performance functions?
• RQ3 How does the system scale with increasing numbers of network slices and distributed infrastructure components?
• RQ4 What is the impact of spatial distribution of traffic and resource availability on slice performance and orchestration efficiency?
• RQ5 To what extent can model-free reinforcement learning effectively manage complex, multi-domain tradeoffs in network slicing?

Key findings

• EdgeSlice achieves significant performance gains over the baseline TARO method, particularly in scenarios with complex, non-linear performance functions.
• The system demonstrates superior scalability and compatibility across diverse network topologies and slice workloads.
• The decentralized D-DRL approach maintains SLA compliance while enabling dynamic, real-time resource adaptation without requiring closed-form performance models.
• Trace-driven simulations and prototype evaluation confirm that EdgeSlice reduces performance degradation and improves resource utilization under varying traffic conditions.
• The performance coordinator effectively coordinates multiple agents to ensure global SLA adherence while preserving local decision autonomy.
• The system shows robustness in handling diverse service requirements, including low-latency and high-throughput use cases, across radio, transport, and computing domains.