[论文解读] Deep Reinforcement Learning-based Methods for Resource Scheduling in Cloud Computing: A Review and Future Directions

Guangyao Zhou, Wenhong Tian+3 more|arXiv (Cornell University)|May 10, 2021

Cloud Computing and Resource Management参考文献 160被引用 26

一句话总结

本论文综述了基于深度强化学习（DRL）的云计算资源调度，概述了模型、目标和挑战，并讨论了 DRL 在调度中的未来方向。

ABSTRACT

As the quantity and complexity of information processed by software systems increase, large-scale software systems have an increasing requirement for high-performance distributed computing systems. With the acceleration of the Internet in Web 2.0, Cloud computing as a paradigm to provide dynamic, uncertain and elastic services has shown superiorities to meet the computing needs dynamically. Without an appropriate scheduling approach, extensive Cloud computing may cause high energy consumptions and high cost, in addition that high energy consumption will cause massive carbon dioxide emissions. Moreover, inappropriate scheduling will reduce the service life of physical devices as well as increase response time to users' request. Hence, efficient scheduling of resource or optimal allocation of request, that usually a NP-hard problem, is one of the prominent issues in emerging trends of Cloud computing. Focusing on improving quality of service (QoS), reducing cost and abating contamination, researchers have conducted extensive work on resource scheduling problems of Cloud computing over years. Nevertheless, growing complexity of Cloud computing, that the super-massive distributed system, is limiting the application of scheduling approaches. Machine learning, a utility method to tackle problems in complex scenes, is used to resolve the resource scheduling of Cloud computing as an innovative idea in recent years. Deep reinforcement learning (DRL), a combination of deep learning (DL) and reinforcement learning (RL), is one branch of the machine learning and has a considerable prospect in resource scheduling of Cloud computing. This paper surveys the methods of resource scheduling with focus on DRL-based scheduling approaches in Cloud computing, also reviews the application of DRL as well as discusses challenges and future directions of DRL in scheduling of Cloud computing.

研究动机与目标

• 回顾云计算的发展，并对调度方法和优化目标进行分类。
• 总结云计算中针对不同目标的调度模型。
• 讨论 DRL 在调度中的应用并与非 DRL 方法进行比较。
• 识别挑战并提出面向云计算的 DRL 驱动调度的未来方向。

提出的方法

• 对云计算中的调度目标和优化问题进行分类（例如能源、完成时间、延迟、负载均衡、利用率、成本）。
• 分析资源调度任务与资源的数学模型与记号。
• 调查在调度问题中使用的强化学习（RL）和深度强化学习（DRL）结构与体系结构。
• 将基于 DRL 的方法与云调度文献中的非 DRL 方法进行比较。
• 讨论将 DRL 应用于现实云调度场景的挑战与未来方向。

实验结果

研究问题

• RQ1 在云计算中使用了哪些基于 DRL 的资源调度方法？
• RQ2 在常见调度目标上，DRL 方法与非 DRL 方法的比较如何？
• RQ3 云资源调度中 DRL 的主要挑战与有前景的方向是什么？

主要发现

• DRL，结合深度学习和强化学习，在复杂的云调度场景中显示出潜力。
• 本文编目了 DRL 调度方法并讨论其与非 DRL 方法的关系。
• 它确定了将 DRL 应用到云资源调度的关键挑战和未来方向。
• 该工作旨在为该领域的 DRL 应用和未来研究需求提供一个专业化的视角。