“Energy and Policy Considerations for Deep Learning in NLP,” by Emma Strubell, Ananya Ganesh, and Andrew McCallum, was presented at ACL 2019 in Florence. It is the seminal academic paper estimating the carbon footprint of training modern natural-language-processing models, and it put concrete numbers behind a cost the field had largely ignored.
The headline estimate was for a full neural architecture search (NAS) run to tune a transformer, including all the trial models trained along the way: about 626,155 pounds of CO2 equivalent. The paper put that in context with everyday benchmarks - a single round-trip flight from New York to San Francisco for one passenger at about 1,984 lbs, an average human’s emissions of about 11,023 lbs per year, and a car including fuel over its entire lifetime at about 126,000 lbs. A single training run of a large transformer was far smaller, but the search-and-tuning process that produces a published result was where the cost ballooned.
The paper is distinct from later work on AI data-center energy and infrastructure: its contribution was methodological, showing how to translate GPU-hours into kilowatt-hours, dollars, and CO2, and arguing for reporting these figures, prioritizing computationally efficient hardware and algorithms, and improving equitable access to compute. It helped launch the “Green AI” conversation and is one of the most cited works on the environmental cost of machine learning.