[论文解读] A Time-driven Data Placement Strategy for a Scientific Workflow Combining Edge Computing and Cloud Computing

Bing Lin, Fangning Zhu+5 more|arXiv (Cornell University)|Jan 22, 2019

IoT and Edge/Fog Computing参考文献 35被引用 11

一句话总结

本文提出一种基于自适应GA-DPSO算法的时间驱动型数据放置策略，旨在最小化混合边缘-云计算环境中科学工作流的数据传输延迟。通过将遗传算法（GA）的交叉与突变操作融入粒子群优化（PSO），该方法增强了种群多样性，减少了早熟收敛，显著降低了在边缘存储容量受限的异构数据中心间的传输时间。

ABSTRACT

Compared to traditional distributed computing environments such as grids, cloud computing provides a more cost-effective way to deploy scientific workflows. Each task of a scientific workflow requires several large datasets that are located in different datacenters from the cloud computing environment, resulting in serious data transmission delays. Edge computing reduces the data transmission delays and supports the fixed storing manner for scientific workflow private datasets, but there is a bottleneck in its storage capacity. It is a challenge to combine the advantages of both edge computing and cloud computing to rationalize the data placement of scientific workflow, and optimize the data transmission time across different datacenters. Traditional data placement strategies maintain load balancing with a given number of datacenters, which results in a large data transmission time. In this study, a self-adaptive discrete particle swarm optimization algorithm with genetic algorithm operators (GA-DPSO) was proposed to optimize the data transmission time when placing data for a scientific workflow. This approach considered the characteristics of data placement combining edge computing and cloud computing. In addition, it considered the impact factors impacting transmission delay, such as the band-width between datacenters, the number of edge datacenters, and the storage capacity of edge datacenters. The crossover operator and mutation operator of the genetic algorithm were adopted to avoid the premature convergence of the traditional particle swarm optimization algorithm, which enhanced the diversity of population evolution and effectively reduced the data transmission time. The experimental results show that the data placement strategy based on GA-DPSO can effectively reduce the data transmission time during workflow execution combining edge computing and cloud computing.

研究动机与目标

• 解决跨越边缘和云数据中心的科学工作流中高数据传输延迟的挑战。
• 通过利用边缘计算的低延迟优势和云计算的可扩展存储能力，优化数据放置。
• 克服传统数据放置策略在负载均衡与最小化传输时间方面无法有效平衡的局限性。
• 开发一种自适应优化算法，动态适应混合边缘-云环境中网络与存储约束的变化。

提出的方法

• 通过引入遗传算法（GA）的交叉与突变操作，增强自适应离散粒子群优化（DPSO）算法，以提升种群多样性并避免早熟收敛。
• 将数据放置决策建模为离散优化问题，其中每个粒子代表在边缘和云数据中心之间的一种潜在数据放置配置。
• 传输延迟被建模为数据中心间带宽、边缘数据中心数量以及边缘存储容量约束的函数。
• 适应度函数基于总数据传输时间评估数据放置配置，目标是最小化端到端工作流执行延迟。
• 算法通过结合局部最优与全局最优解信息的混合学习规则，迭代更新粒子的位置与速度。
• 该方法在真实网络与存储约束下进行评估，模拟了用于科学工作流的混合边缘-云基础设施。

实验结果

研究问题

• RQ1 如何在混合边缘-云环境中优化数据放置，以最小化科学工作流中的数据传输延迟？
• RQ2 带宽、边缘数据中心数量以及边缘存储容量对数据传输时间有何影响？
• RQ3 将遗传算法操作集成到粒子群优化中，能否改善数据放置中的收敛行为与解的质量？
• RQ4 与传统数据放置方法相比，所提出的GA-DPSO策略在传输时间与负载均衡方面表现如何？
• RQ5 该算法的自适应特性在多大程度上能够应对网络与存储条件的动态变化？

主要发现

• 基于GA-DPSO的数据放置策略相比传统负载均衡方法，显著减少了数据传输时间。
• 在DPSO中引入交叉与突变操作，提升了种群多样性并防止了早熟收敛，从而带来更优的优化性能。
• 该算法有效应对了边缘数据中心的存储容量限制，同时最小化了长距离云连接之间的数据传输。
• 通过将频繁访问的数据集智能地放置在靠近处理节点的位置，该方法实现了更低的端到端工作流执行时间。
• 实验结果表明，在不同网络与存储条件下，所提策略在传输时间减少方面优于基线方法。