[论文解读] EcoServe: Designing Carbon-Aware AI Inference Systems

Yueying, Zhanqiu Hu+4 more|ArXiv.org|Feb 7, 2025

Big Data and Business Intelligence被引用 3

一句话总结

EcoServe 提出一个四大支柱、面向碳的框架（Reuse、Rightsize、Reduce、Recycle）以优化 LLM 推理的运营碳与隐性碳，同时在性能微小折中下实现多达 47% 的碳减排。

ABSTRACT

The rapid increase in LLM ubiquity and scale levies unprecedented demands on computing infrastructure. These demands not only incur large compute and memory resources but also significant energy, yielding large operational and embodied carbon emissions. In this work, we present three main observations based on modeling and traces from the production deployment of two Generative AI services in a major cloud service provider. First, while GPUs dominate operational carbon, host processing systems (e.g., CPUs, memory, storage) dominate embodied carbon. Second, offline, batch inference accounts for a significant portion (up to 55\%) of serving capacity. Third, there are different levels of heterogeneity across hardware and workloads for LLM inference. Based on these observations, we design EcoServe, a carbon-aware resource provision and scheduling framework for LLM serving systems. It is based on four principles - Reduce, Reuse, Rightsize, and Recycle (4R). With a cross-stack ILP formulation and design, we demonstrate that EcoServe can lower carbon emissions by up to 47\%, compared to performance, energy, and cost-optimized design points, while maintaining performance targets and SLOs.

研究动机与目标

• 解决 AI 推理系统中运营碳与隐性碳的需求与必要性.
• 开发一个包含宿主系统与 GPU 的碳排放整体模型。
• 提出 EcoServe 的四大设计支柱，在保持 SLOs 和性能的前提下优化碳排放。
• 展示在容量规划、资源分配和调度的协同设计如何降低碳排放。
• 量化在开源与生产轨迹上的碳减排。

提出的方法

• 为 AI 系统建立一个包含宿主 CPU、内存、存储、PCB 和 GPU 组件的隐性碳模型。
• 将总碳量表征为功率、时间和碳强度的函数，纳入运营与隐性排放。
• 提出 EcoServe 的四大设计支柱（Reuse、Rightsize、Reduce、Recycle）并构建一个 ILP 以共同优化性能、效率与碳。
• 展示离线推理中 CPU 的重用以提升容量并降低隐性碳。
• 基于工作负载特征和碳效率，对在线/离线阶段的异构 GPU 进行端对端降容。
• 提出降低浪费硬件并高效回收宿主系统与加速组件的策略。

Figure 1 : (Left) Breakdown of power (TDP) and embodied carbon between host systems (CPU) and GPU. (Right) EcoServe optimizes operational and embodied carbon based on input hardware, model, and workload traces by reusing CPU for offline inference, rightsizing GPUs, reducing underutilized hardware, a

实验结果

研究问题

• RQ1 运营碳与隐性碳如何共同构成 AI 推理系统的总碳足迹？
• RQ2 跨层次的碳感知设计（Reuse、Rightsize、Reduce、Recycle）是否能够在不牺牲 SLOs 的前提下降低排放？
• RQ3 在离线推理中重用 CPU 对容量与隐性碳有何影响？
• RQ4 应如何对异构 GPU 进行供给以在不同的 LLM 阶段（提示阶段与解码阶段）优化能耗、碳与性能？
• RQ5 哪些策略在回收硬件与延长寿命之间取得平衡，以最小化总碳？

主要发现

• 相对于优化基线，EcoServe 可以在保持性能目标和 SLO 的前提下将碳排放降低多达 47%。
• 重用空闲 CPU 资源进行离线推理可通过利用未充分使用的宿主组件来提升集群容量并降低隐性碳。
• 对异构硬件进行右尺寸化并考虑离线/在线阶段，在能源、碳与性能效率方面带来显著收益。
• 通过更 lean 的硬件减少隐性碳并高效回收宿主系统，在每个支柱（Reuse、Rightsize、Reduce、Recycle）可实现 29% 到 41% 的碳 savings。
• 基于对工作负载切片（提示 vs 解码）与硬件的 ILP 联合优化可实现 1.4x–2.2x 的碳收益且性能降幅极小。

Figure 2 : Trends in bit density (left) and embodied carbon footprint (right) across varoius DRAM memory technologies for 3 different manufacturers.