[Paper Review] The state-of-the-art review on resource allocation problem using artificial intelligence methods on various computing paradigms

Javad Hassannataj Joloudari, Sanaz Mojrian+8 more|arXiv (Cornell University)|Mar 23, 2022

Traffic Prediction and Management Techniques4 citations

TL;DR

This paper presents a comprehensive state-of-the-art review of artificial intelligence (AI)-based resource allocation in diverse computing paradigms, including cloud, fog, IoT, 5G, and vehicular networks. It focuses on deep learning and reinforcement learning techniques—such as deep Q-learning and Q-learning—to optimize key performance metrics like delay, energy consumption, and cost, demonstrating AI's effectiveness in enhancing system efficiency and scalability across distributed environments.

ABSTRACT

With the increasing growth of information through smart devices, increasing the quality level of human life requires various computational paradigms presentation including the Internet of Things, fog, and cloud. Between these three paradigms, the cloud computing paradigm as an emerging technology adds cloud layer services to the edge of the network so that resource allocation operations occur close to the end-user to reduce resource processing time and network traffic overhead. Hence, the resource allocation problem for its providers in terms of presenting a suitable platform, by using computational paradigms is considered a challenge. In general, resource allocation approaches are divided into two methods, including auction-based methods(goal, increase profits for service providers-increase user satisfaction and usability) and optimization-based methods(energy, cost, network exploitation, Runtime, reduction of time delay). In this paper, according to the latest scientific achievements, a comprehensive literature study (CLS) on artificial intelligence methods based on resource allocation optimization without considering auction-based methods in various computing environments are provided such as cloud computing, Vehicular Fog Computing, wireless, IoT, vehicular networks, 5G networks, vehicular cloud architecture,machine-to-machine communication(M2M),Train-to-Train(T2T) communication network, Peer-to-Peer(P2P) network. Since deep learning methods based on artificial intelligence are used as the most important methods in resource allocation problems; Therefore, in this paper, resource allocation approaches based on deep learning are also used in the mentioned computational environments such as deep reinforcement learning, Q-learning technique, reinforcement learning, online learning, and also Classical learning methods such as Bayesian learning, Cummins clustering, Markov decision process.

Motivation & Objective

• To analyze the latest advancements in AI-based resource allocation across heterogeneous computing environments such as cloud, fog, IoT, and 5G networks.
• To evaluate the effectiveness of deep learning and reinforcement learning methods in optimizing resource allocation under dynamic and distributed workloads.
• To identify key challenges in resource allocation, including latency, energy efficiency, and network overhead, across emerging computing paradigms.
• To provide a systematic comparison of AI techniques—especially deep reinforcement learning and Q-learning—against classical methods like Bayesian learning and Markov decision processes.
• To establish a foundation for future research by synthesizing state-of-the-art approaches and highlighting gaps in current methodologies.

Proposed method

• Conducting a comprehensive literature study (CLS) on AI-based resource allocation, excluding auction-based methods, across multiple computing paradigms.
• Focusing on deep learning-based approaches, particularly deep reinforcement learning (DRL), Q-learning, and online learning, for dynamic and adaptive resource allocation.
• Evaluating classical learning methods such as Bayesian learning, Cummins clustering, and Markov decision processes (MDPs) for resource allocation in time-varying environments.
• Analyzing the integration of AI techniques in diverse environments, including vehicular fog computing, machine-to-machine (M2M) communication, and peer-to-peer (P2P) networks.
• Comparing performance metrics such as processing delay, energy consumption, cost, and network utilization across AI and non-AI methods.
• Structuring the review around optimization goals: minimizing delay, reducing energy use, and improving cost-efficiency in distributed and edge-centric architectures.

Experimental results

Research questions

• RQ1 How do deep reinforcement learning and Q-learning techniques improve resource allocation efficiency in cloud and fog computing environments?
• RQ2 What are the relative advantages of AI-based methods like deep Q-learning over classical methods such as Markov decision processes in dynamic network environments?
• RQ3 In what ways do AI-driven resource allocation strategies reduce latency and energy consumption in 5G, IoT, and vehicular networks?
• RQ4 How do online learning and adaptive learning mechanisms enhance real-time resource allocation in distributed and mobile computing paradigms?
• RQ5 What are the key performance trade-offs and limitations of AI-based resource allocation across different computing models such as P2P, T2T, and M2M networks?

Key findings

• Deep reinforcement learning and Q-learning techniques significantly reduce processing delay and improve resource utilization in cloud and fog computing environments.
• AI-based methods outperform classical approaches like Bayesian learning and Cummins clustering in dynamic, high-variability networks such as vehicular and IoT systems.
• The integration of deep learning in resource allocation leads to a measurable reduction in energy consumption and network overhead across 5G and M2M communication networks.
• Online learning and adaptive AI models demonstrate superior scalability and responsiveness in real-time environments like peer-to-peer and train-to-train communication networks.
• Reinforcement learning-based systems achieve higher user satisfaction and system stability by dynamically adjusting to changing workloads and network conditions.
• The study confirms that AI-driven optimization methods are particularly effective in minimizing runtime and enhancing cost-efficiency in heterogeneous computing infrastructures.