Hacker News Reader

Comment by tzs | Hacker News Reader

> Another solution is dummy calculations, which run while there are no spikes, to smooth out demand. This makes the grid see a consistent load, but it also wastes energy doing unnecessary work.

Oh god...I can see it now. Someone will try to capitalize on the hype of LLMs and the hype of cryptocurrency and try to build a combined LLM training and cryptocurrency mining facility that that runs the mining between training spikes.

View on HN

Oh man, I really, really wish that you hadn't said this and also that you were wrong.

you really think this isn't literally the first thing that happened the second hosting these models became commercially viable?

There wouldn't be any crypto or LLMs if the Fed hadn't printed $7 trillion.

Dummy calculations aren't even needed, if you allow the LLMs to pre-compute on the given context before inference:

https://arxiv.org/abs/2504.13171

It should be noted that this type of inference is less useful on time-sensitive tasks, but most tasks truthfully don't require such time sensitivity (there exists slack time between when the task is given & when questions are asked).

There are already some providers offering cheap LLM services that will give you a response within 24 hours instead of within seconds. That allows them to schedule tasks during low-request hours when they have spare capacity and use better batching. For some automated tasks this is perfectly acceptable. A bit of effort to accommodate, but easy to justify when it halves your inference costs

Any examples of such providers?

Certain tasks at OpenAI when I checked a few months ago. Embedding for one.

OpenAI [1] as well as Azure OpenAI, Anthropic [2], as well as Parasail [3] for all the "open source" models. There are others that I was thinking of, but those are the first I could find without my notes. Typically the batch API is 50% cheaper than live inference

1: https://platform.openai.com/docs/guides/batch

2: https://docs.anthropic.com/en/docs/build-with-claude/batch-p...

3: https://docs.parasail.io/parasail-docs/batch/batch-quickstar...

YCW27

From the same founders who brought you (or didn't, actually) maritime fusion

It’s ok though because YC invests in the team, better just give them another chance!!

https://infernetprotocol.com

>implying its not already happening

Reply on news.ycombinator.com