[论文解读] Power Hungry Processing: Watts Driving the Cost of AI Deployment?

Alexandra Sasha Luccioni, Yacine Jernite+1 more|arXiv (Cornell University)|Nov 28, 2023

Machine Learning and Data Classification参考文献 45被引用 27

一句话总结

该论文系统性比较任务特定模型与多用途ML模型的推断能耗和碳排放，结果显示在各任务与模态下，多用途生成系统在每1000次推断上的能量消耗远高于任务特定模型。

ABSTRACT

Recent years have seen a surge in the popularity of commercial AI products based on generative, multi-purpose AI systems promising a unified approach to building machine learning (ML) models into technology. However, this ambition of ``generality'' comes at a steep cost to the environment, given the amount of energy these systems require and the amount of carbon that they emit. In this work, we propose the first systematic comparison of the ongoing inference cost of various categories of ML systems, covering both task-specific (i.e. finetuned models that carry out a single task) and `general-purpose' models, (i.e. those trained for multiple tasks). We measure deployment cost as the amount of energy and carbon required to perform 1,000 inferences on representative benchmark dataset using these models. We find that multi-purpose, generative architectures are orders of magnitude more expensive than task-specific systems for a variety of tasks, even when controlling for the number of model parameters. We conclude with a discussion around the current trend of deploying multi-purpose generative ML systems, and caution that their utility should be more intentionally weighed against increased costs in terms of energy and emissions. All the data from our study can be accessed via an interactive demo to carry out further exploration and analysis.

研究动机与目标

• 评估跨多样任务与模态的ML模型推断的能耗与碳排放。
• 在部署环境中比较任务特定模型与多用途模型。
• 量化模型规模、架构与任务类型如何影响推断能耗与排放。
• 为从业者在部署时的准确性-效率权衡与政策影响提供指导。

提出的方法

• 在5种模态的10个任务和30个数据集上选择；使用88个模型（任务特定与多用途）以代表部署使用情景。
• 在固定的AWS区域(us-west-2)上对NVIDIA A100-SXM4-80GB GPU，使用Code Carbon测量每1000次推断的能量与CO2排放。
• 每组推断集重复运行10次，未进行批量处理以反映部署变异性。
• 对比任务特定模型与多用途零-shot模型（Flan-T5和BLOOMz家族）在所选任务上的表现。
• 分析模型规模、任务类型与排放之间的关系；利用可用的训练/微调数据检查训练成本与推断成本的对等性。

Figure 1. The tasks examined in our study and the average quantity of carbon emissions they produced (in g of $CO_{2}$ ) for 1,000 queries. N.B. The y axis is in logarithmic scale.

实验结果

研究问题

• RQ1 在每1000次推断中部署不同ML模型用于常见NLP和CV任务的能耗与碳排放成本是多少？
• RQ2 在跨任务与模态的情况下，任务特定模型与多用途模型在排放方面有何差异？
• RQ3 在任务特定 vs 多用途类别中，模型规模与架构对能耗与排放有何影响？
• RQ4 对于代表性模型，推断成本何时达到与训练成本的对等？

主要发现

• 在不同任务与模态下，推断能耗与CO2排放可相差三个数量级。
• 与多用途模型相比，任务特定模型在每1000次推断的CO2排放通常要低得多，特别是在判别性任务与生成性任务之间。
• 文本到图像与图像生成任务的能量与碳排放最为高昂；文本类任务相对更高效。
• 在多用途模型中，序列到序列架构在同等规模的相似任务上往往比解码器单独模型更节能。
• 更大模型规模通常会增加排放，但架构（序列到序列 vs 解码器）与输出长度显著影响总排放。

Figure 2. The 5 modalities examined in our study, with the number of parameters of each model on the x axis and the average amount of carbon emitted for 1000 inferences on the y axis. NB: Both axes are in logarithmic scale.