[论文解读] Characterizing Commodity Serverless Computing Platforms

Jinfeng Wen, Yi Liu+3 more|arXiv (Cornell University)|Dec 2, 2020

IoT and Edge/Fog Computing参考文献 33被引用 6

一句话总结

本文通过分析官方文档的定性研究和使用开源工具TBS进行的定量基准测试，对四种主流商品化无服务器平台——AWS Lambda、Google Cloud Functions、Azure Functions和Alibaba Cloud Function Compute——进行了全面表征。研究揭示了各平台在冷启动延迟、资源效率和并发处理方面存在关键差异，为开发者和云服务提供商提供了可操作的洞察。

ABSTRACT

Serverless computing has become a new trending paradigm in cloud computing, allowing developers to focus on the development of core application logic and rapidly construct the prototype via the composition of independent functions. With the development and prosperity of serverless computing, major cloud vendors have successively rolled out their commodity serverless computing platforms. However, the characteristics of these platforms have not been systematically studied. Measuring these characteristics can help developers to select the most adequate serverless computing platform and develop their serverless-based applications in the right way. To fill this knowledge gap, we present a comprehensive study on characterizing mainstream commodity serverless computing platforms, including AWS Lambda, Google Cloud Functions, Azure Functions, and Alibaba Cloud Function Compute. Specifically, we conduct both qualitative analysis and quantitative analysis. In qualitative analysis, we compare these platforms from three aspects (i.e., development, deployment, and runtime) based on their official documentation to construct a taxonomy of characteristics. In quantitative analysis, we analyze the runtime performance of these platforms from multiple dimensions with well-designed benchmarks. First, we analyze three key factors that can influence the startup latency of serverless-based applications. Second, we compare the resource efficiency of different platforms with 16 representative benchmarks. Finally, we measure their performance difference when dealing with different concurrent requests, and explore the potential causes in a black-box fashion. Based on the results of both qualitative and quantitative analysis, we derive a series of findings and provide insightful implications for both developers and cloud vendors.

研究动机与目标

• 为解决当前对商品化无服务器平台缺乏系统性理解的问题，该问题阻碍了最优应用开发和平台选型。
• 识别并分类无服务器平台在开发、部署和运行时阶段的关键静态特性。
• 通过实证方法评估主流平台的实际运行时性能，重点关注冷启动延迟、资源效率和并发处理能力。
• 为开发者提供平台选型和配置调优的可操作建议，并为云服务厂商提供架构优化的参考。

提出的方法

• 通过对比四种平台在开发、部署和运行时三个维度的官方文档，开展定性分析，形成包含20项特性的分类体系。
• 设计并实现了一个开源基准测试工具TBS，以黑盒方式评估运行时性能。
• 测量不同编程语言、内存分配和包大小下的冷启动延迟，以隔离影响性能的关键因素。
• 使用16个代表性微基准和宏基准测试，评估资源效率，比较执行时间和资源消耗。
• 在不同请求负载下评估并发性能，并通过黑盒观察分析可扩展性和负载均衡行为。
• 结合微基准和类真实场景的宏基准测试，确保性能评估兼具精度与实际相关性。

实验结果

研究问题

• RQ1 主要商品化无服务器平台在开发、部署和运行时特性方面有何差异？开发者需考虑哪些关键约束？
• RQ2 编程语言、内存大小和包大小在多大程度上影响AWS Lambda、Google Cloud Functions、Azure Functions和Alibaba Cloud Function Compute的冷启动延迟？
• RQ3 在执行代表性工作负载时，各平台在资源效率方面表现如何？
• RQ4 在并发请求工作负载下，各平台如何扩展？哪些因素影响其调度和性能行为？

主要发现

• 冷启动延迟在不同平台间差异显著，且对包大小最为敏感，部分平台中大包可使延迟增加高达300%。
• Google Cloud Functions在所有测试编程语言下均表现出最低的平均冷启动延迟，其次为AWS Lambda，而Azure Functions和Alibaba Cloud Function Compute的基线延迟较高。
• 资源效率因工作负载类型而异：在计算密集型基准测试中，AWS Lambda和Google Cloud Functions在单位内存分配下的性能表现更优。
• 在高并发场景下，Azure Functions展现出最激进的扩展行为，但调度延迟较高；而AWS Lambda则保持一致的性能表现，波动更小。
• Alibaba Cloud Function Compute在宏基准测试中表现出每调用资源消耗最高，表明其执行环境初始化可能存在潜在效率问题。
• 编程语言的选择对冷启动延迟有可测量的影响，Node.js和Python在冷启动场景下通常优于Java和.NET。