> The power required per user to run those GPUs seems like enough to run an air conditioner.

Not true. Maybe you're thinking of computer games? Inference requires much less energy per user session than a session of your average AAA game title. And there are more active gamers at any time than active ChatGPT users.