[Paper Review] Carbontracker: Tracking and Predicting the Carbon Footprint of Training Deep Learning Models

Lasse F. Wolff Anthony, Benjamin Kanding+1 more|arXiv (Cornell University)|Jul 6, 2020

Advanced Neural Network ApplicationsComputer Science22 references203 citations

TL;DR

Carbontracker is an open-source Python tool that tracks and predicts energy use and carbon emissions during DL model training, and can warn users to stop training when predicted footprint is exceeded.

ABSTRACT

Deep learning (DL) can achieve impressive results across a wide variety of tasks, but this often comes at the cost of training models for extensive periods on specialized hardware accelerators. This energy-intensive workload has seen immense growth in recent years. Machine learning (ML) may become a significant contributor to climate change if this exponential trend continues. If practitioners are aware of their energy and carbon footprint, then they may actively take steps to reduce it whenever possible. In this work, we present Carbontracker, a tool for tracking and predicting the energy and carbon footprint of training DL models. We propose that energy and carbon footprint of model development and training is reported alongside performance metrics using tools like Carbontracker. We hope this will promote responsible computing in ML and encourage research into energy-efficient deep neural networks.

Motivation & Objective

• Motivate reporting of energy and carbon footprints alongside performance metrics in ML research and practice.
• Provide a practical tool to monitor, predict, and potentially intervene to reduce the environmental impact of DL model training.
• Demonstrate the tool across CNNs on medical imaging datasets and assess prediction accuracy.
• Highlight factors influencing footprint, including regional carbon intensity and hardware composition, to guide mitigation strategies.

Proposed method

• Develop a Python-based, multithreaded tool to measure power usage of GPU, CPU, and DRAM in real time.
• Predict total training energy consumption and carbon footprint using a simple linear model over a user-specified number of epochs.
• Forecast carbon intensity for the predicted duration via real-time APIs to compute CO2eq emissions.
• Allow interventions by enabling training to stop if the predicted footprint would exceed a threshold.
• Support cross-platform deployment (clusters, desktops, Colab) with plug-and-play integration into existing codebases.

Experimental results

Research questions

• RQ1 How accurately can Carbontracker predict the energy and carbon footprint of DL model training from partial monitoring?
• RQ2 What is the distribution of energy use across GPU, CPU, and DRAM components during training?
• RQ3 How do regional carbon intensities and forecast accuracy affect CO2eq estimates and recommended mitigation actions?

Key findings

• Prediction errors for energy are in the range of 4.9% to 19.1% after 1 epoch of monitoring.
• CO2eq prediction errors range from 7.3% to 19.9% for the same setup.
• Training duration predictions have 0.8% to 4.6% error across experiments.
• GPU accounts for about 50%–60% of total energy consumption, with CPU and DRAM comprising a significant share as well.
• Real-time or forecasted carbon intensity substantially improves CO2eq estimates compared with region-wide averages.
• The paper demonstrates potential emission differences across regions (e.g., Estonia vs. Sweden) and times of day, suggesting training timing and location as mitigation levers.