[Paper Review] Toward a Smart Resource Allocation Policy via Artificial Intelligence in 6G Networks: Centralized or Decentralized?

Ali Nouruzi, Atefeh Rezaei+5 more|arXiv (Cornell University)|Feb 18, 2022

Software-Defined Networks and 5G4 citations

TL;DR

This paper proposes a dynamic, AI-driven resource allocation framework for 6G networks that intelligently selects between centralized and decentralized operation modes using deep reinforcement learning (DRL) with the soft actor-critic algorithm. The framework introduces a novel Throughput Overhead Complexity (TOC) metric and demonstrates through simulation that the smart policy outperforms fixed centralized and distributed schemes in scalability, adaptability, and performance trade-offs under varying network loads.

ABSTRACT

In this paper, we design a new smart softwaredefined radio access network (RAN) architecture with important properties like flexibility and traffic awareness for sixth generation (6G) wireless networks. In particular, we consider a hierarchical resource allocation framework for the proposed smart soft-RAN model, where the software-defined network (SDN) controller is the first and foremost layer of the framework. This unit dynamically monitors the network to select a network operation type on the basis of distributed or centralized resource allocation architectures to perform decision-making intelligently. In this paper, our aim is to make the network more scalable and more flexible in terms of achievable data rate, overhead, and complexity indicators. To this end, we introduce a new metric, throughput overhead complexity (TOC), for the proposed machine learning-based algorithm, which makes a trade-off between these performance indicators. In particular, the decision making based on TOC is solved via deep reinforcement learning (DRL), which determines an appropriate resource allocation policy. Furthermore, for the selected algorithm, we employ the soft actor-critic method, which is more accurate, scalable, and robust than other learning methods. Simulation results demonstrate that the proposed smart network achieves better performance in terms of TOC compared to fixed centralized or distributed resource management schemes that lack dynamism. Moreover, our proposed algorithm outperforms conventional learning methods employed in other state-of-the-art network designs.

Motivation & Objective

• To address the limitations of fixed centralized or distributed resource allocation in 6G networks, which lack adaptability to dynamic network conditions.
• To design a flexible, software-defined RAN architecture that enables real-time, intelligent decision-making based on network state and traffic load.
• To introduce a new performance metric, Throughput Overhead Complexity (TOC), that balances data rate, overhead, and computational complexity.
• To develop a learning-based framework that autonomously selects the optimal resource allocation strategy—centralized or distributed—based on network dynamics.

Proposed method

• Design a hierarchical, software-defined RAN (soft-RAN) architecture with an SDN controller as the central decision-making layer.
• Implement a deep reinforcement learning (DRL) agent using the soft actor-critic algorithm to learn optimal switching policies between centralized and distributed resource allocation.
• Introduce a new composite metric, Throughput Overhead Complexity (TOC), to evaluate trade-offs between data rate, signaling overhead, and computational complexity.
• Train the DRL agent using environment feedback from network simulations that vary user density, channel conditions, and traffic load.
• Use continuous action spaces in the soft actor-critic method to enable fine-grained, high-accuracy policy learning compared to discrete-action methods like DQN.
• Compare the proposed dynamic framework against fixed centralized and distributed schemes under varying network conditions to validate adaptability and performance.

Experimental results

Research questions

• RQ1 How does a dynamically adaptive resource allocation policy compare to fixed centralized or distributed schemes in terms of performance trade-offs in 6G networks?
• RQ2 What is the impact of network density and traffic load on the optimal choice between centralized and decentralized resource allocation?
• RQ3 Can a deep reinforcement learning agent effectively learn to switch between centralized and distributed modes to minimize the Throughput Overhead Complexity (TOC)?
• RQ4 How does the soft actor-critic method improve performance and stability compared to other DRL methods like DDPG or DQN in this context?
• RQ5 To what extent does the proposed smart framework balance data rate, overhead, and complexity across varying network conditions?

Key findings

• The proposed smart framework achieves a TOC performance that lies between fixed centralized and distributed schemes, dynamically adapting to network conditions.
• At low user loads, the smart policy performs similarly to the centralized scheme, achieving high data rates with moderate overhead.
• At high user loads (e.g., 160 users), the smart policy transitions toward the distributed mode, outperforming the centralized scheme in TOC due to reduced overhead and complexity.
• The soft actor-critic-based algorithm achieves higher data rates and better stability than DQN and DDPG, particularly due to its ability to handle continuous action spaces.
• The TOC metric demonstrates that the smart framework reduces performance degradation in ultra-dense networks, where centralized schemes suffer from linearly increasing complexity and overhead.
• The simulation results confirm that the learning-based approach outperforms conventional DRL methods in terms of TOC, especially under high-density and dynamic traffic scenarios.