[Paper Review] Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learning

Peter Henderson, Jie‐Ru Hu+4 more|arXiv (Cornell University)|Jan 31, 2020

Green IT and Sustainability306 citations

TL;DR

The paper proposes a lightweight, modular framework (experiment-impact-tracker) for real-time energy, compute, and carbon accounting in ML, demonstrates its use with case studies, and recommends practices like energy leaderboards and regionalized testing to reduce emissions.

ABSTRACT

Accurate reporting of energy and carbon usage is essential for understanding the potential climate impacts of machine learning research. We introduce a framework that makes this easier by providing a simple interface for tracking realtime energy consumption and carbon emissions, as well as generating standardized online appendices. Utilizing this framework, we create a leaderboard for energy efficient reinforcement learning algorithms to incentivize responsible research in this area as an example for other areas of machine learning. Finally, based on case studies using our framework, we propose strategies for mitigation of carbon emissions and reduction of energy consumption. By making accounting easier, we hope to further the sustainable development of machine learning experiments and spur more research into energy efficient algorithms.

Motivation & Objective

• Motivate the need for accurate, systematic energy and carbon accounting in ML research.
• Propose a lightweight, extensible framework to track energy, compute, and carbon footprints per experiment.
• Demonstrate the framework through empirical studies and derive mitigation and reporting recommendations.

Proposed method

• Introduce experiment-impact-tracker as a modular data router collecting metrics from CPUs/GPUs, Python environments, and power-tools; generate online appendices from tracked data.
• Compute total energy using etotal = PUE × Σ(pdramedram + pcpuecpu + pgpuegpu) to allocate per-process energy on shared hardware.
• Gather carbon intensity from regional grid data (e.g., electricitymap) and CAISO realtime data to compute CO2eq emissions.
• Provide per-experiment carbon impact statements (including social cost of carbon) and automated web-based appendices.
• Highlight a Reinforcement Learning Energy Leaderboard to incentivize energy-efficient RL methods.
• Analyze how FLOPs (FPOs) do not reliably predict energy use across architectures and show stronger correlations within the same architecture.

Experimental results

Research questions

• RQ1 How can ML experiments be tracked for energy and carbon impacts in a lightweight, reproducible way?
• RQ2 To what extent do common efficiency metrics (like FLOPs) correlate with real energy consumption across and within architectures?
• RQ3 What practical strategies (leaderboards, regional testing, reproducibility practices) can reduce energy use and emissions in ML research?

Key findings

• The framework enables easy automatic logging of energy, CPU/GPU power draw, per-process resource attribution, and generation of online appendices.
• Across architectures, FLOPs are not strongly correlated with energy or runtime; within a single architecture, FLOPs correlate strongly with energy and time (FLOPs example with VGG showing R2≈1.0).
• The paper reports a concrete carbon footprint example from their experiments: 8.021 kg CO2eq and 24.344 kWh consumed, with a USA-specific social cost of carbon of $0.38.
• Regional carbon intensity can drive large variations in CO2 emissions; in RL experiments, emissions can vary by region, potentially up to 30x differences based on grid mix.
• The framework advocates for systemic changes: energy-efficient leaderboards, choosing carbon-friendly regions for experiments, reducing overheads, energy-performance trade-offs, and consistent reporting of energy and carbon metrics.