If you can't beat them, ban them.
109 karma · joined July 28, 2026
If you can't beat them, ban them.
Whether open-weight labs can get to the frontier on their own is irrelevant IMO. Even if all R&D were to stop today it would take a decade at least for the productivity gains from current open-weight models to trickle through the economy. For 99% of use cases of AI there simply isn't a need for further development of the frontier, where we are very clearly at diminishing returns. How many white collar workers need to write an OS or develop sophisticated software?
Think about a normal office worker - they an easily double their productivity by creating a well-thought-out partnership between the human and today's open weight AI. Humans and today's AI are very complementary. AI brings vast knowledge, pattern recognition, and infinite patience for grunt work. Human brings context and reasoning and verification.
Frontier labs are trying to replace the human altogether by trying to make AI do things that AI is weak at and humans are strong at. This is silly but they have no choice because of the massive profits they've promised their investors. It's not enough to double every knowledge worker's productivity. They want to replace the knowledge worker altogether.
This is creating a massive spike in training & inference cost that is almost certainly more expensive than the human labor its intended to replace. Fable and 5.6 sol are supposedly around 8-10T parameters. Is there enough compute on earth to make a 100T or 1Q model? We would need models that size to replace the human altogether.
> I also don’t believe we have seen anything like 1% of inference demand fulfilled globally.
I agree with this, but the massive demand for inference will force companies to be price conscious and drive them away from ultra high margin frontier models to low margin open weight models. This is why it will become a low-margin, commodity market.
Immense innovation right now is going into tiny, specialized models that run on device. See PrismML getting a mid sized QWEN model to run on an iPhone. When on-device inference becomes a thing we'll see another explosion in inference that will further leach demand away from frontier labs.
> I also don’t believe these open letters are signed by people that are all master strategists and just economically motivated.
I agree if you could read their minds they might think they're doing it for the "right reasons." But massive $$$ can create equally massive self justification/deception.
1000 employees of the anti-alien defense industry are telling us we better be worried about an impending alien invasion. Let's not spend any time asking whether this alien invasion is really coming. Let's just trust them and start regulating willy nilly.
You are correct that attacking someone's motives is un-falsifiable. So is the claim that "ai is improving therefore it will keep improving therefore it will become dangerous sometime in the future.
No one can prove whether these employees had angels or demons in their hearts. What the skeptics are asking is much simpler: Make a believable argument for (1) the danger of AI and (2) that regulation can be effective.
Your entire argument about cynicism or unfalsifiableness is simply a distraction from having that discussion. And it's a discussion reasonable, well intentioned people can have. For example, if 1000 high school science teachers (who didn't have massive stock options in frontier labs) said 80% of our students could make a bioweapon 10x more lethal than an AR15 mass shooting with nothing more than ChatGPT and an amazon account, then that would be something reasonable people could discuss. On the other hand, 1000 employees conflicted by $Bs of options saying "trust us, this is dangerous" is not a claim that can be constructively critiqued.
If your only evidence that the claim "ai dangerous - regulate ai" is true is "insiders who stand to make billions if they can regulate away the competition" told me so, that is pretty weak evidence indeed.
If a group of concerned scientists (without $$$ in stock options) demonstrated that anyone with one year of HS chemistry, an Amazon account, and a ChatGPT max plan could build a bioweapon 10x more lethal than an AR15 mass shooting or driving a truck into a crowd (because it is certainly more than 10x harder to make a bioweapon than either of these things, even with a theoretical AGI), then that would be worrisome. This is really not that high of an evidence bar but even it has not been met, but oh no we should start regulating yesterday.
By that logic Excel, the internet, calculators, and air conditioning are all force multipliers in that all of the make it easier for an expert who can make bioweapons able to do so more quickly / comfortably / easily. Obviously it wouldn't make sense to "pace" any of these technologies. Even social media or email could be considered force multiplier in that they could literally allow you to increase the number of collaborators working on supposed bioweapon. Credit cards and modern logistics force multiply your ability to purchase precursors.
Also it would be a force multiplier on the prevention/mitigation side as well. Police armed with AI to scan for suspicious precursor purchases or review security camera footage after the fact would be far more effective than before. It's not obvious that the force multiplication is greater to the bad actor or to those who oppose them.
You and others are making a claim about LLMs enabling biological dangers. Some people disagree with you, pointing out that the incremental knowledge an LLM provides over say google or a library card may not be that much.
What seems myopic to me is un-skeptically accepting "AI dangerous, must regulate" messaging from frontier labs that have trillions of dollars of vested interests in this message.
If the claim is (1) AGI coming and (2) we should so something about AGI, wouldn't it be smart to confirm (1) before worrying about (2)? Especially if certain companies have trillions of dollars of valuation that depend on society as a whole believing (1)?