Hacker News Reader

> My guess is that someone will be selling tokens at less than electricity + depreciation of GCs soon, since there's a lot of competition and "smaller" data centers have overcapacity? This is speculation, correct me if I'm wrong

My guess is they are selling you the tokens, then selling your tokens (data) onto someone else.

I see these conspiratorial arguments all the time and I think people massively overestimate the value of the average users tokens.

The problems with frontier models (design taste, ability to solve novel/difficult problems, etc) cannot be solved by throwing more slop from the average user at it.

Actually, most of the main deficiencies in current models stem from the fact that their data sets aren’t curated and specialized enough.

I don't think the goal of this data is necessarily model improvement.

I think it's marketing, advertising, and product refinement.

Ex: all the things Google wants your search data for.

It's somewhat silly to think the value of that data has changed much. Advertisers want to know what's popular and getting clicks and attention. Competitors want to know what features are getting used in their markets.

In the simplest case, think of this data as improving the harness, not the model.

I wonder how much less useful it is if I use those models for open code or similar. What are you really learning about me, other than the fact that I am a technical person, which you could know by the fact that I signed up for open router to start with.

Do you run your coding harness in a completely segregated sandbox? Even folder names of what projects you have, what projects you actually work on, etc are very useful for targeted advertising.

The prompts contain sensitive personal data.

That would be valuable to advertizers for example.

Press x to doubt on the 45% number. The cheaper providers on open router are fp4 vs fp8 for official zai. There are some cheap fp8 ones (like novita) but the ui makes it seem like it's a temporary promotion, with their normal prices being almost equal to official zai (idk much about open router so not really sure what's going on with these discounts)

If you click the provider it shows the precision, 45% off at AkashML shows FP8. The drawback is the small context window, at 96k.

Then there's 43% off at StreamLake with FP8 precision and 1M context window.

There is nothing to doubt, the cheapest price on openrouter is ~45% lower than when GLM5.2 was released.

Yes it's true that it's not super clear whether these prices are permanent or short term promotions. On the other hand, there are so many providers making promotional offerings that you could probably easily switch from one to another should their prices go up?

Seminanlysis is estimating sub $1 cost per MT for ~2Trillion models. The numbers change based on throughput and quant, but it is conceivable that provider costs at scale are low enough that even $2.42 per MT on GLM 5.2 (current best price) is margin positive by a wide margin.