HNHacker News
TopNewBestAskShowJobs

red_green_yell

109 karma · joined July 28, 2026

submissionscomments
red_green_yell··on Pacing the frontier
OAI and Anthropic specifically are under intense pressure to compete. One form of competition could be to innovate. Another form would be regulatory capture. The former seems unlikely given how quickly open labs are catching up. The latter is much easier and cheaper.

If you can't beat them, ban them.

red_green_yell··on Pacing the frontier
> we cannot currently disambiguate how easy it is to get close to the frontier in a vacuum.

Whether open-weight labs can get to the frontier on their own is irrelevant IMO. Even if all R&D were to stop today it would take a decade at least for the productivity gains from current open-weight models to trickle through the economy. For 99% of use cases of AI there simply isn't a need for further development of the frontier, where we are very clearly at diminishing returns. How many white collar workers need to write an OS or develop sophisticated software?

Think about a normal office worker - they an easily double their productivity by creating a well-thought-out partnership between the human and today's open weight AI. Humans and today's AI are very complementary. AI brings vast knowledge, pattern recognition, and infinite patience for grunt work. Human brings context and reasoning and verification.

Frontier labs are trying to replace the human altogether by trying to make AI do things that AI is weak at and humans are strong at. This is silly but they have no choice because of the massive profits they've promised their investors. It's not enough to double every knowledge worker's productivity. They want to replace the knowledge worker altogether.

This is creating a massive spike in training & inference cost that is almost certainly more expensive than the human labor its intended to replace. Fable and 5.6 sol are supposedly around 8-10T parameters. Is there enough compute on earth to make a 100T or 1Q model? We would need models that size to replace the human altogether.

> I also don’t believe we have seen anything like 1% of inference demand fulfilled globally.

I agree with this, but the massive demand for inference will force companies to be price conscious and drive them away from ultra high margin frontier models to low margin open weight models. This is why it will become a low-margin, commodity market.

Immense innovation right now is going into tiny, specialized models that run on device. See PrismML getting a mid sized QWEN model to run on an iPhone. When on-device inference becomes a thing we'll see another explosion in inference that will further leach demand away from frontier labs.

> I also don’t believe these open letters are signed by people that are all master strategists and just economically motivated.

I agree if you could read their minds they might think they're doing it for the "right reasons." But massive $$$ can create equally massive self justification/deception.

red_green_yell··on Pacing the frontier
Why is it you are so quick to skip over whether "accelerates beyond our ability to understand or control" actually happens and instead jump to "regulate now"?

1000 employees of the anti-alien defense industry are telling us we better be worried about an impending alien invasion. Let's not spend any time asking whether this alien invasion is really coming. Let's just trust them and start regulating willy nilly.

red_green_yell··on Pacing the frontier
These 1000 employees have generational wealth riding on the ability for their companies to IPO. They know that those IPOs won't happen if open weight models continue to kill their companies' profit margins. This is an obvious conflict of interest that anyone who cares to take their head out of the sand can see. Nuclear scientists didn't have this conflict of interest.

You are correct that attacking someone's motives is un-falsifiable. So is the claim that "ai is improving therefore it will keep improving therefore it will become dangerous sometime in the future.

No one can prove whether these employees had angels or demons in their hearts. What the skeptics are asking is much simpler: Make a believable argument for (1) the danger of AI and (2) that regulation can be effective.

Your entire argument about cynicism or unfalsifiableness is simply a distraction from having that discussion. And it's a discussion reasonable, well intentioned people can have. For example, if 1000 high school science teachers (who didn't have massive stock options in frontier labs) said 80% of our students could make a bioweapon 10x more lethal than an AR15 mass shooting with nothing more than ChatGPT and an amazon account, then that would be something reasonable people could discuss. On the other hand, 1000 employees conflicted by $Bs of options saying "trust us, this is dangerous" is not a claim that can be constructively critiqued.

red_green_yell··on Pacing the frontier
Employees inside frontier labs have the chance to make life changing money if they can make it to an IPO before the music stops. Have you considered that might be the reason they are so eager to claim "ai dangerous - regulate ai"? The nuclear scientists at los alamos did not have generational wealth incentivizing them to raise "concerns" to the public.

If your only evidence that the claim "ai dangerous - regulate ai" is true is "insiders who stand to make billions if they can regulate away the competition" told me so, that is pretty weak evidence indeed.

If a group of concerned scientists (without $$$ in stock options) demonstrated that anyone with one year of HS chemistry, an Amazon account, and a ChatGPT max plan could build a bioweapon 10x more lethal than an AR15 mass shooting or driving a truck into a crowd (because it is certainly more than 10x harder to make a bioweapon than either of these things, even with a theoretical AGI), then that would be worrisome. This is really not that high of an evidence bar but even it has not been met, but oh no we should start regulating yesterday.

red_green_yell··on Pacing the frontier
It sounds like your claim is "AI is a force multiplier, therefore it is dangerous and must be regulated." Not trying to strawman - I read your comment twice and I think that's what you're saying.

By that logic Excel, the internet, calculators, and air conditioning are all force multipliers in that all of the make it easier for an expert who can make bioweapons able to do so more quickly / comfortably / easily. Obviously it wouldn't make sense to "pace" any of these technologies. Even social media or email could be considered force multiplier in that they could literally allow you to increase the number of collaborators working on supposed bioweapon. Credit cards and modern logistics force multiply your ability to purchase precursors.

Also it would be a force multiplier on the prevention/mitigation side as well. Police armed with AI to scan for suspicious precursor purchases or review security camera footage after the fact would be far more effective than before. It's not obvious that the force multiplication is greater to the bad actor or to those who oppose them.

red_green_yell··on Pacing the frontier
The only thing it might accomplish is allowing OpenAI / Anthropic to squeak out an IPO before the market realizes AI will become a commodity and the massive profits these companies are promising will never materialize. They aren't in denial about this open letter's futility in terms of actually reducing the pace of AI, that's just not their goal in the first place.
red_green_yell··on Pacing the frontier
If the accuracy of your biology claim is "hard to say" then why would pushback be self-evidently myopic?

You and others are making a claim about LLMs enabling biological dangers. Some people disagree with you, pointing out that the incremental knowledge an LLM provides over say google or a library card may not be that much.

What seems myopic to me is un-skeptically accepting "AI dangerous, must regulate" messaging from frontier labs that have trillions of dollars of vested interests in this message.

red_green_yell··on Pacing the frontier
See printing press and internet!
red_green_yell··on Pacing the frontier
I think this is a spot on analogy and would two more: the hand wringing over the release of the printing press and the internet. In both cases the people in power were terrified that the masses would use their newfound power to do dangerous things. In both cases bad things did happen (market bubbles, scams, witch hunts, spread of knowledge on making poisons), but trying to put the cat in the bag by pacing the printing press or the internet would have been a fool's errand, and over the long run both inventions massively improved the lives of society overall.
red_green_yell··on Pacing the frontier
Are they lazy dismissals? Or a simple observation that LLM scaling is hitting diminishing returns way way short of AGI.

If the claim is (1) AGI coming and (2) we should so something about AGI, wouldn't it be smart to confirm (1) before worrying about (2)? Especially if certain companies have trillions of dollars of valuation that depend on society as a whole believing (1)?

← PreviousPage 2 of 2