And no, I don't ridicule AI ethics researchers: the energy cost of AI is negligible to the serious issues. The real harm stems from other areas.
And no, I don't ridicule AI ethics researchers: the energy cost of AI is negligible to the serious issues. The real harm stems from other areas.
From the paper they cite
algorithm kWh-PUE
---------- -------
ELMo 275
BERT(base) 1,507
NAS 656,347
NAS was Transformer (213M parameters) for neural architecture search 2019.
They didn't have kWh numbers for GPT-2.More importantly, a human brain isn't trained from scratch at birth. It's had a few hundred million years of pretraining.
If we want to account for power supply, temperature control etc. we could use the entire calorie count of a human over 35 years, leaving us with around 30000kWh (assuming a health, not-overweight human). You could now argue that that's too high (the human is doing a lot of other things). But as you pointed out it's a bit of a pointless comparison anyways as humans don't start from scratch.
That's still too low, considering you can't just have a human eating food and educating themselves in a vacuum.
We need to account for energy going to and from school, shelter, heating, clothing. Basically all the necessities that enable someone to learn (and their professors)
That's another 5x at a minimum.
This doesn't take you anywhere.