[Paper Review] Towards Climate Awareness in NLP Research

Daniel Hershcovich, Nicolas Webersinke+3 more|Zurich Open Repository and Archive (University of Zurich)|May 10, 2022

Topic Modeling4 citations

TL;DR

This paper proposes a practical climate performance model card to standardize and simplify reporting of environmental impact in NLP research, addressing the lack of systematic climate reporting despite growing awareness. It introduces a lightweight, accessible template requiring minimal hardware details, enabling widespread adoption to increase transparency and drive climate-conscious practices across NLP communities.

ABSTRACT

The climate impact of AI, and NLP research in particular, has become a serious issue given the enormous amount of energy that is increasingly being used for training and running computational models. Consequently, increasing focus is placed on efficient NLP. However, this important initiative lacks simple guidelines that would allow for systematic climate reporting of NLP research. We argue that this deficiency is one of the reasons why very few publications in NLP report key figures that would allow a more thorough examination of environmental impact. As a remedy, we propose a climate performance model card with the primary purpose of being practically usable with only limited information about experiments and the underlying computer hardware. We describe why this step is essential to increase awareness about the environmental impact of NLP research and, thereby, paving the way for more thorough discussions.

Motivation & Objective

• To address the lack of systematic climate impact reporting in NLP research, despite growing awareness of AI's environmental costs.
• To identify why current tools and practices fail to achieve widespread adoption in the NLP community.
• To propose a practical, lightweight model card that enables transparent climate performance reporting with minimal input data.
• To increase climate awareness in mainstream NLP by integrating environmental impact considerations into standard research reporting workflows.
• To reduce the risk of greenwashing by promoting honest, standardized, and transparent reporting of CO2 emissions from NLP experiments.

Proposed method

• Conduct a quantitative survey of 6 years of NLP literature to assess the prevalence and quality of environmental impact statements.
• Develop a taxonomy of 'efficiency' concepts in NLP to standardize terminology and reporting across studies.
• Propose a climate performance model card template that reports key environmental metrics using only basic experimental and hardware details.
• Design the model card to be compatible with existing platforms like Hugging Face, enabling integration into model sharing and publication workflows.
• Integrate principles from financial reporting to enhance transparency and reduce ambiguity in climate impact communication.
• Provide open-source templates in LaTeX and Markdown, along with a Jupyter notebook for reproducibility and tooling support.

Experimental results

Research questions

• RQ1 Why is climate impact reporting uncommon in NLP research despite growing awareness of AI’s environmental costs?
• RQ2 What are the key dimensions that influence the environmental impact of NLP experiments, and how can they be systematically reported?
• RQ3 How can a model card be designed to be both practical and transparent, requiring minimal input while still enabling meaningful climate impact assessment?
• RQ4 In what ways can standardized climate reporting reduce greenwashing and improve accountability in NLP research?
• RQ5 How can climate performance reporting be integrated into mainstream NLP workflows without imposing excessive overhead?

Key findings

• A survey of NLP papers over six years revealed that climate impact statements remain uncommon, despite increasing recognition of the issue.
• Existing automated tools for carbon footprint estimation are underutilized, largely due to complexity and lack of integration into standard reporting practices.
• The proposed climate performance model card enables transparent reporting using only basic information such as model size, training time, and hardware type.
• The model card is designed to be lightweight and practical, increasing the likelihood of adoption across diverse research settings.
• The authors emphasize that climate reporting should not be used to assess research quality, as this could disadvantage researchers with less efficient hardware.
• The model card is explicitly not intended for measuring net climate impact or higher-order effects, focusing instead on first-order emissions from model training and inference.