Hacker News Reader

Energy efficient LLM inference (for now) is as realistic as existence of perpetual motion.

I wouldn't mind if it burns 200 watts while it does the task, as long as it idles at below 30W

NVIDIA H200 idles at 75 watts. I'm not keeping my hopes high on that, either.

To be fair, if I paid $30k+ for an H200, I’d want it to be making money 24/7 rather than idling, so the idle power draw would be strictly theoretical.

It's not always about money.

Reply on news.ycombinator.com