But you have to be willing to break out of the bubble, including getting on a plane, and I see very few people being willing to do that.
6,588 karma · joined April 20, 2009
But you have to be willing to break out of the bubble, including getting on a plane, and I see very few people being willing to do that.
Your assumption is that the benchmarks are essentially identical in difficulty, with the only difference being their age and thus whether they could have been trained on.
It is possible to trick pangram - they bias toward a low false positive and a higher false negative - but it is not true that it is essentially random.
DeepInfra sells DS v4-flash at 0.08 in, $0.18 out. Gemma4 they sell for $0.07 in, $0.34 out. OpenAI's price for luna is $0.20 in, $1.20 out.
Why would you assume OpenAI is somehow uniquely incompetent at making small, fast models? And that they're worse at serving it than DeepInfra? Any observer can see they are making money here.
I never understand why people who are convinced there is a big con just don't check market prices and see if there's money to be made.
That doesn't mean their business is great -- they're losing tons of money, but it's because they spend too much on fixed costs, and they can't stop spending money on training next generation models with no end in sight, not because the inference is margin negative, which is a flimsy idea that just clouds the actual business issue.
So we can go round and round on this, each with our made-up objections about how it's temporary or unrealistic or impossible or whatever, or we can just accept the prices as listed and use that to guide our economic decisions.
Their cache read costs are $0.50 per million, or 25% of the cost of uncached reads.
The industry standard is a 90% discount, so cache costs you 10% of uncached. So that means 5.6 Sol actually costs less per million cache reads - $0.40/million.
If you are doing a lot of agentic work where the vast bulk of your token consumption will be cached input reads, you won't get the expected cost savings from Grok.
I imagine this is the result of some problem in their serving infrastructure that I hope they will fix, because then the pricing will become actually strong. (The other possibility is that they bet on distracting people with good headline prices assuming they'd miss the bad cache pricing, but I'll give them the benefit of the doubt on that.)
"Pressing charges" is mostly a made up idea for criminal cases. However, prosecuting attorneys may not want to pick up a case if the victim is not cooperating, because it makes the case much harder to win.
The county's other option is to simply price data centers. Yes, that means they might not win the data center. If the data center can get better terms elsewhere, they do that. If the town doesn't want to concede on price, then fine.
People seem to be mad that data centers show up with less perceived value to the community than they want, so why would they care if the data center builds elsewhere because they don't agree to their price? The position strictly Pareto dominates the ban stance: at worst, they get no data center (same as ban), at best, they get a bunch of tax money.
Please educate me on how I'm "missing the point" because of how "naive" I am!
For alcohol, almost every big study has shown either no discernible effects going from zero to 1 drink per day, or actual slight benefits going from 0 to 1 that then reverse when increasing beyond that. There is not strong consensus which of these is true, but there's definitely no real evidence that 1 drink per day is harmful as measured by all-cause mortality.
See GBD 2020 in the Lancet (which showed a slight J-curve), Wood in the Lancet 2018 (optimally low risk at 100g ethanol per week, or ~1 drink per day), Zhao in Jama (no discernible negative effects at 1 drink per day, no benefits either, no J-curve), and the latest National Acadamies review showing lowest all-cause mortality at about 1 drink per day (so, a J-curve).
Alcoholism has tons of negative effects. But these seem to start at around 2 drinks per day and rise rapidly from there.
Moreover, the WHO never changed their stance when later big studies showed that the "lowest risk" level is not actually at zero drinks.
The timeline was this:
- In 2018, a study was published in the Lancet that was the Global Burden of Disease analysis. They claimed the modeled minimum risk exposure was zero drinks. But the magnitude effects were extremely small: moving from zero to one drink a day pushed the incidence rate of alcohol-associated health effects from 914 per 100,000 people to 918 per 100,000 people. This was the basis of the "no safe level."
- Four years later, an updated Global Burden of Disease analysis, also published in the Lancet, actually reversed course on this. After controlling for more factors, they found risk was actually slightly J-shaped, and for 40+ year olds the theoretical minimum risk was about 0.5 drinks per day. The WHO ignored this updated study when they wrote their opinion piece in 2023.
- Another big study with ~600k participants published in 2018 in the Lancet shows lowest total risk consumption levels at 1 drink per day, with mortality rising above that level. The WHO also ignored this.
- The JAMA study in 2023, with >4 million participants, showed no benefit to low-volume drinking, and also no increase in harms from low volume (1 drink daily) drinking. Again, the WHO ignored it.
Several other studies have shown either no discernible risk change between zero and 1 drink per day, or lowest possible risk at 1 drink per day.
But the WHO and much of the public health policy community ignores almost all of the actual credible science on this because it does not fit with their ideology. It's a huge problem with public health policy: the actual scientists and statisticians are doing great work, but if it's not what the policy people want to hear, they discard it and keep repeating their mantra.
Just make them pay!
Your electricity price shouldn't go up because of them. So just make a rate card for data centers that ensures the burden doesn't fall on residents. That's what Oregon started doing, and residents' electricity prices actually started falling, because the fixed costs of energy production were being spread across more kilowatt-hours and the data centers weren't getting a special deal.
I've also seen counties offer tax incentives. Why? Just have them pay property tax. Then you get nicer schools, more police, better roads.
Instead we have these campaigns to stop them, instead of making them a local cash cow. Loudon County gets half of all its revenue from data center property tax. Its residents were already rich, and now their local taxes are artificially low. Who wouldn't want that?
It is true that wine doesn't do anyone any good (unless you count it making people more social or something). But the effects of one glass a night is indistinguishable from zero. There are definitely statistically significant morbidity effects at 3.
I've been shocked reading public health proclamations that "anything above zero" is bad, and then reading the actual underlying paper they refer to, which clearly states exactly the opposite finding. I think it is an extension of the public policy belief that the public cannot be trusted with moderation, and so therefore must be communicated to in absolutes, and nobody will read the underlying paper anyway.
But that's not too hard to work around. You can either have a single process that owns both writing and reading that file, or you can do a data lake where you post updates as parquet files into object storage, and duckdb handles the catalog. The Quack protocol also basically fixes this (though still in beta).
With Clickhouse, you can of course still have tenant separation, but you have to do it by managing users within Clickhouse that map to users/tenants of your main app, so that you can restrict SQL access by tenant to only their data store. Not a huge deal but I just like the Unix "it's just a file" simplicity of "Tenant A gets to run arbitrary SQL against their separate read-only, no-ATTACH duckdb file".
It is almost always low information density. Sometimes I can get it to remove most of the extraordinarily cliche phrases it injects into everything, along with the tortured, gauche prose (especially Claude). But what remains is an expansion algorithm: it takes a simple idea and expands it like polyurethane foam to fill however much space it can take, the volume filled mainly by grammatical syntax and only traced with any actual semantic meaning.
And now with things like quack the sharp points around concurrency are relaxing, and it feels like the compromises of using it this way are disappearing.
Tokens are not normal software, because they have marginal cost, and I think people who are used to software economics really struggle with this. With token generation there really can be manufacturing cost efficiencies where one producer is just straight up better at serving product at a lower marginal cost.
I've previously found flash (for all the hate it gets) to be good for these kinds of things. Haiku was fine but it's ancient.
But user->AI calls are very much the rare case now, the more agentic the workload. Most of them will be tool->result->tool without the user involved. And token burn is highest with these long running agentic chains, but that's precisely where routing doesn't work because of the KV cache.
How do you deal with that?
If you think it should be different then you have to make a cogent argument why the public should get to interfere with a settlement the two sides mutually agree on.
If the idea is that there's no point in encrypting anything on a USB stick, and you just hope they don't look at it at all, sure.
But there is no threat model here that includes "check the USB stick" and does not automatically lead to finding and breaking "hidden" content.
Use the right tool for the job. It's like asking why a screwdriver isn't good at sawing wood, or calling C a terrible language because it's hard to make CRUD apps with it.
The labs are not interested in the small, fast, single purpose end of the market. Google increased their pricing on Flash so much that it stopped becoming a cheap model; instead, they released Gemma 4 open source, which is actually easier to use from a third-party inference provider than from Google.
From a total token volume perspective, these "utility" models (classifiers, simple summarizers, small OCR models) will absolutely drive enormous volumes of tokens, at low prices and margin and modest overall market size. Because the models are small and the performance requirements are modest, and because their use cases are specialized rather than general, there are poor economies of scale: they can run cost effectively on rented small GPUs, and a big player doesn't get a structural cost advantage. These models are usually 1b - 30b in size, and can run on a rented 5090. I've productized these myself: I run millions of pages through a fine tuned 1b OCR language model that runs on 5090s at a cost far lower than commercial providers.
But that's not the segment of the market where GLM 5.2, Kimi 3, etc., play. They compete with frontier capabilities, and they are not particularly cheaper than OpenAI models at a cost per task. (I do actually think they compete well with Anthropic, because Anthropic's model efficiencies are poor compared to OpenAI.) And although this part of the market may not be the bulk of the token volume, it is the bulk of the market value.
That's because a lot of human knowledge work is too generalized and fuzzy for dedicated, fine-tuned models, so they are almost entirely different markets that don't particularly compete with each other. (Though if SaaS companies successfully build around verticals that can use small models applied against well-defined jobs, there may be opportunity to push the small/big capability boundary to subsume marginally more valuable tasks that today would require mid-grade reasoning.)
- Commodity market profitability is determined by marginal cost of production. LLMs have marginal cost; traditional software does not.
- Models are not free. Downloading them is free. Running them is not. This has manufacturing economics, not software economics; the idea that they are "free" is an economic category error as it relates to their actual use
- The highest tier Chinese models are not more economical than US frontier models. Try GLM 5.2 and see how much it costs to do real work. I did, and it was more expensive than GPT 5.6.
- This is because US labs are leading on cost efficacy of inference ($/task)
- Training will decline as a percentage of costs as inference expands compute share due to agentic workloads. A big part of training now is optimizing token efficiency. It's hard to distill token efficiency; that is perhaps why Chinese LLMs are so inefficient.
- With increasing inference as % of total compute, if labs create efficient models -- which they can, because they can create highly optimized models amortized over very high inference loads -- they can be low cost producers, and be competitive at $/task rates
OpenAI really shows the way here. Their cost per task is less than half that of Anthropic because of more efficient tokenization and less verbosity. OpenAI is both cheaper and better than Chinese models for frontier work.