Hacker News Reader

Comment by MAXPOOL | Hacker News Reader

Human mind (aka brain) has consumed little over 6000 kWh energy when it's 35.

From the paper they cite

    algorithm    kWh-PUE
    ----------   -------
    ELMo             275
    BERT(base)     1,507 
    NAS          656,347

NAS was Transformer (213M parameters) for neural architecture search 2019. They didn't have kWh numbers for GPT-2.

View on HN

I don't know about that 6000 KWh number.

More importantly, a human brain isn't trained from scratch at birth. It's had a few hundred million years of pretraining.

The 6000kWh number is assuming 20W over 35 years. That might be right if you only count the brain, but then you would have to revise the neural network numbers to only count CPU and memory, which would easily halve those numbers (fans take a lot of power).

If we want to account for power supply, temperature control etc. we could use the entire calorie count of a human over 35 years, leaving us with around 30000kWh (assuming a health, not-overweight human). You could now argue that that's too high (the human is doing a lot of other things). But as you pointed out it's a bit of a pointless comparison anyways as humans don't start from scratch.

> we could use the entire calorie count of a human over 35 years, leaving us with around 30000kWh (assuming a health, not-overweight human)

That's still too low, considering you can't just have a human eating food and educating themselves in a vacuum.

We need to account for energy going to and from school, shelter, heating, clothing. Basically all the necessities that enable someone to learn (and their professors)

That's another 5x at a minimum.

For the NN architectures, you'll have to factor in there production, design, the energy of the humans involved and so on and on.

This doesn't take you anywhere.

Reply on news.ycombinator.com