Hacker News Reader

Comment by piyh | Hacker News Reader

Sol on Cerebras is going to be expensive AF

Cerebras is an uncut whole wafer. Each wafer gets you 44GB SRAM (not a typo, SRAM, not DRAM/VRAM). A few years ago, leading process node wafer from tsmc is $20K/ea without guarantee on yield.

A single full wafer likely can run qwen3.6-27b alone. But won't be enough to run bigger models, which are pretty much all popular models.

Is it? I think waferscale might actually be cheaper per-token, it's just so many more tokens, and of course right now it's not a full buildout so the availability is limited as well. I'd imagine they'll be migrating to whichever inference method is least expensive, and I expect asics to be the ultimate answer.

I'm not familiar with economics of chips, but I presume the SRAM on the wafer is less dense than HBM so it might be eating into its cost efficiency?

Moving from either frontier intelligence or frontier latency to a single model that does both at the same time is potentially a game changer in certain industries. I can easily see e.g. hedge funds dropping tons of money on this, because it means they can now do the same thing as their competitors, but much faster. That's basically a license to print money.

What would a hedge fund want to do on this exactly? It’s too slow for hft and I’m not sure what they would be doing where ms matter but is not hft.

There's a lot of trading that isn't proper "HFT", but where speed and latency still matter. Often you'll find this employed more as slippage reduction - i.e you're going to make the trade either way, but making it faster saves you a few bps.

I'm not sure what event-based traders are doing now, but back in the day NLP sentiment analysis was all the rage, so I'm assuming they've now incorporated LLMs too.

It's not like there are only two buckets:

1. HFT doing ass-simple arbitrage where only latency matters 2. More sophisticated slower trading taking in deeper signals

Those are two points along a continuum. If you are reacting to an earnings announcement by having an LLM read the earnings release and listen to the call, getting the results a few seconds earlier lets you get your trade in a few seconds earlier. Just because "not HFT" doesn't mean "completely latency insensitive".

Exactly that. Except that ultrafast delivers this level of intelligence an order of magnitude faster. So if your competitors automatically react to news articles or financial statements with a certain level of comprehension within minutes, you can now do so in seconds.

Except they all do, and the money flows to Cerebras :)

Eventually they will. But until these models become commonly available there is a significant first mover advantage.

I am not sure this is the way to make AI more cost effective for such customers. If they are able to tweak any model for their use case it would be way more reliable and also way cheaper. In my opinion generic LLMs in the future will be just for attention economy or maybe government contracts. Everyone else will be running fine tuned free weight models or licenced closed source models (self hosted or managed).

This was a widespread opinion a few years ago, but by now it is pretty much accepted that any LLM fine tune you build today based on the best available models will be beaten by a general purpose frontier LLM within a year.

Reply on news.ycombinator.com