Hacker News Reader

Comment by habitue | Hacker News Reader

I disagree, I think the fluke was the era in which we didn't have enough work to do for CPU/GPU to be at 100% 24/7. Think of this like leaving money on the table: compute should always be useful, why aren't those cores pegged to max all the time?

I think it was literally lack of imagination. We were like "well, I automated most of the paper pushing we used to do in the office, guess my job is done!" and this occupied 0.001% of a computer's time. We invented all sorts of ways for people to only pay for that tiny slice of active time (serverless, async web frameworks, etc).

Now we're in an era where we can actually use the computers we've built. I don't think we're going back

View on HN

Even once you have a core sitting in a datacenter, the value you derive from the computations must exceed the power cost (and the SWE cost to run that computation). Otherwise you're better off turning it off.

I have a stack of 1080ti and Titan V GPUs that are testament to this. :-) (which, admittedly, I should sell)

Well if you're looking to sell, get in touch!

This is where the lack of imagination comes in (not you in particular, this is everyone right now). I'm postulating something non-obvious and pretty contentious, but I think compute should always be more valuable than the cost of the power it consumes.

That's an obviously wrong statement: at some point of power consumption, you've burned the earth to ashes, and there's no returns on compute that can pay that off.

The economics of efficiency come in. You can say it is lack of imagination and all that, but the graveyard of bitcoin miners would disagree. Yes we might come up with some novel usage for computation that would somehow exceed the power costs, but like, that is not an easy problem to do even at some very low margin rate. CPU and computation is cheap and linear. LLMs are one of the few things that produce value for all that computation in recent times outside of the normal workloads we have always been progressing (e.g. serving sites, etc.).

I can't see how that edge case actually changes the practical value of the original statement, though.

If you spend 1000W on 1 gigaflops, but could have gotten 1 teraflops instead on newer hardware, you are mostly just throwing money away. Unless fab capacity is severely limited in the future or energy becomes too cheap to meter, the opportunity cost is just too great for your statement to be true.

Even if fab capacity becomes unconstrained, you'd have to buy newer GPUs to take advantage of the more efficient processes.

Computations will usually not be done if the result is less valuable than the cost. This is true of everything, not just compute.

Consider the extrema: You have a Pentium P5 running at 16W doing something like 70 MIPS (which isn't quite producing 70 MFLOPS), getting something like 4 MFLOP/Watt.

Then take a Titan V at 14.9 TFLOPs (32 bit) at about 250W, for 59,000 MFLOP/Watt.

There's almost no conceivable world in which it's worth running the P5. It literally consumes ten thousand times as much power per unit compute as a not-quite modern GPU.

For compute to "always" be more valuable than the cost of the power it consumes, the value of that compute would have to be infinite. We have no such application. And I suspect we're unlikely to. :-)

I have a stack of 1080ti and Titan V GPUs that are testament to this. :-)

https://en.wikipedia.org/wiki/SETI@home

Yeah, I was going to say just that. The statement above is like saying: "If you have an oven, it should be burning at full power all the time. All the time with your oven not burning is wasted time". No. I need an oven to burn at high power when I bake, and it's fine for it to sit idle the rest of the time.

Don't, I'm working on a startup where you running them would be welcome (and paid): https://borg.games/setup

How is that different than suggesting that all the engines we've ever made should constantly be driving load?

It may not be obvious in the same ways as engines, but computation consumes a resources like power and attention and outputs waste like heat and fried circuits. These resources and wastes interact with other systems than just computers and data centers and so efficiency and necessity needs to be considered in a bigger picture than just "let's use all the transistors all the time and see what happens!"

But 100% compute != 100% efficiency? 'Pegging' cores to their maximum for the sake of utilisation seems wasteful where (for example) a simple bash script would have sufficed.

Once the compute heavy pathways have established, I'd wager the next round of automation can utilise these established paths to lock-in on an answer rather than throwing more cycles at the problem.

Don't worry -- I can write bash scripts that eat all CPU too!

> why aren't those cores pegged to max all the time?

Power consumption and heat dissipation, mostly. Amusingly, heat even turns into a performance thing itself, since you can get better performance in bursts than sustained.

There definitely was an era of compute surplus — compute being a solution in search of a problem. I think the recent crypto hype (bitcoin, NFTs) can (at least partially) be attributed to the same issue.

For sure, once the LLM hype diminishes to more practical scale we'll figure something else to burn cycles on. And I'm not predicting demand for GPUs to die out all of a sudden, but my money isn't on the nVidias of this world.

> For sure, once the LLM hype diminishes

It haven't even started. There are so many places it can be used, expect sci-fi in the next few years. There is no way back. The only thing that may change is the AI technology under the hood. I mean LLM isn't the goal, it's only a tool. Something else may replace or extend it. Many people are actively working on this. GPUs aren't the goal either, there must be more efficient way. But still they are very good at numbers crunching and that is needed for video processing.

Effecience is NOT a goal. I want my computers ready when I want to do something. If you want effecency we should put them in universities only and make sure there is always a line of people waiting for their turn.

i want inefficient so that if I feel like a large calculation I have one ready at hand.

Why are computers different from your car or your house? Shouldn't you hire someone to drive other people around in your car 24/7? And why not fill to whatever limit fire department sets your house to 24/7?

There is so many resources we keep available and do not fully utilize. I don't see why computing devices should be different...

Reply on news.ycombinator.com