Given that most people have smart televisions which gather and send information for, among other things, selling to advertisers, I can't help but wonder if their calculation is just for the TV or if it includes the data center overhead - which may well be running an AI model.
You wrote: 'My hunch going in was that a lot of this is waste: dozens of companies racing to build the same thing in secret, each burning energy so that one of them can win.'
Part of that waste is all the data scrapers draw power (from botnets running on residential proxies with less efficiency than a data center) -- and the scraped data isn't shared. The scrapers cause more demand on the servers, which need more power. The server operators deploy anti-bot protections like proof-of-work, which requires more power, and the anti-anti-bot methods which use real hardware or automated full browsers which again require more power.
Any idea what that power use might be?
BTW, you did a bit of a cheat there, with "The oldest big AI workload isn’t chat at all." If recommendation systems (or Bayesian spam filtering) count as AI then we've long known that AI for everyone is easily possible, with well less than a tenth of a kettle of tea per person per day.
If you're serious about including recommendation systems, you'll need to include the advertisement infrastructure as part of the AI overhead, as all use recommendation methods to figure out how much to bid for an advertisement on your screen. Or probably LLMs these days.
How much power is going into powering online advertisement?