pu_pe

Everyone started using the term after that article came out, it feels like...

I was listening to his interview with Ezra Klein and I think his arguments are very coherent. If you are interested in actually knowing what he thinks about it, you could check the transcript [1].

[1] https://www.nytimes.com/2026/09/29/opinion/ezra-klein-podcas...

There are far more moving pieces in deploying an application than coding. My impression is TFA is arguing that replacing humans with LLMs for coding would make other things like quality assurance more difficult, in which case I disagree.

I agree, I think this false dichotomy between using LLMs and caring about quality/reliability needs to stop. All these mission critical industries listed in the article rely on extensive testing for quality assurance, with human code review being a layer on top of all that, but far from the most critical one.

Interpretability is the same, our abilities to do that have increased rather than decreased. I think a codebase generated by AI is actually more understandable than one generated by humans at this point, and you can ask clarifying questions whenever you get stuck.

TFA's points only make sense if the mental model the author has in mind is someone who writes a prompt then immediately puts an app into production without any thought behind it.

It is a bicycle for the mind, but the mind still sets the direction. Sometimes it's going faster towards a dumpster.

AI moves too fast for traditional academic pace. During peer review alone, another generation of models comes along and might obsolete your findings. Though, in this case, I am not sure that Github Copilot specifically has improved that much in the meantime.

Surely AI can be dangerous before the point it becomes completely perfect. Very imperfect people manage to do a ton of damage already.

We had people departing from OpenAI, Anthropic and now Google. The pace the frontier open letter was signed by over 1,000 experts. At which point do we believe the message?

Incredible that you describe a 1.5 year process that cost almost 20 thousand euros (plus running costs to keep a physical presence, management and tax advisors in Germany) a perfectly reasonable solution.

Because innovation requires taking risks. In the off chance those risks pay off, you are now massively penalized for your success. In that scenario you're much better off just selling the company or never starting one in the first place. This situation strongly favor incumbents by inhibiting competitors from emerging.

It's not that you can't have an exit tax, it's that it shouldn't be so out of tune with other OECD countries, so bureaucratic and so expensive to even process.

It depends on the industry, but regular companies are usually valued at 3x to 6x EBITDA for private acquisitions. A small one-person company would probably be valued at much less than that. Most countries with an exit tax would calculate both the valuation and the tax rate much more favorably than Germany here.

> The German exit tax is very.. expensive. In simplified terms, I'd have to simulate a sale of all my companies at a high valuation (13.75 * earnings), and pay 30% tax on that.

This is an outrageous valuation and taxation scheme.

I think German politicians either don't understand the magnitude of the issue, or they have been coopted by lobbying somehow. It's such a tough environment for innovation.

The original discussion about the project (https://news.ycombinator.com/item?id=49746163) is very weird. Lots of call-outs about how the author is some sort of celebrity and random accounts vouching for him, with little discussion on the substance.

Not even the demo on that release works well.

It's obviously more capable in any task I tried (coding, translation, summarizing, etc). Benchmarks are not the only way to tell if a model is better or not.

Look, I hate your post. I think you used AI to generate it. I will give you one day to delete it and to hire someone I know to make another post for you. If you don't do that, I will go and tell a bunch of people that you're an asshole, and they may or may not come here and harass you too.

Signed, A very normal person doing things people do all the time for all sorts of reasons

This is outright harassment. The guy is definitely making a threat, even though he is couching it as some sort of helpful advice.

How do you explain the fact that Qwen3.8 27B performs vastly better than any open model from even one year ago, if using the same test-time compute and harness?

Congrats, seems like a promising concept. I'll give it a go in my local setup. My prior is thinking this will definitely work for speed but also definitely compromise accuracy (I've seen this happen so many times), though benchmark results are encouraging of course.

Nice explanation too. Now that everything reads like a sad blur, it felt refreshing to read your writeup.

First of all, who can say for certain whether OpenAI does what they say they do? For all we know, they cracked open this specific researcher's prompts and started from there.

Second, the issue of anonymization is a red herring. There is a very limited number of people working in this approach, and most of them are likely making no progress. So Buckmaster's prompts might have had an outsized effect on the outcome. It's similar to that guy who created a site claiming he is a world-renowmed hot dog eating contestant, which ended up digested by OpenAI models as truth [1].

[1] https://www.bbc.com/future/article/20260218-i-hacked-chatgpt...

I would not trust the American government to be able to assess this threat at all, their concern seems to be on getting paid.

This guy did not work for Sam Altman. And while it's true that everything can be a PR stunt, that's no evidence that it is one.

I feel that people should take these kinds of warnings more seriously. This guy had skin in the game and decided to quit, when he could be earning millions instead. It's very different than Sam Altman peddling some narrative.

These people are the ones with access to the best models on the planet, and with info about how careless governance issues are being handled. That's a pretty privileged place at the table, and a very profitable one too.

If you think this is a PR stunt, is there any warning that you actually believe? If an AI researcher does want to come forward with a dire warning for humanity, what path should that person take?

Why wouldn't contamination be possible? I can believe the data is de identified so you couldn't simply prompt the model to "follow this guy's approach", but it's entirely plausible that there is a very tiny amount of data about this approach in your dataset, and it comes precisely from this researcher.

OpenAI thinks of this as a scoop, and it is, but the possibility that they trained the model on the prompts of the other mathematicians they were competing with will leave a terrible taste on every scientist's mouth. Seems like yet another advantage of using open models right here.

I often wonder how CEOs and executives fail to notice how obviously broken their products are. This e-mail thread is quite illuminating. Most "work" in there is figuring out who will be blamed for it and why it shouldn't be your team.

> The strongest argument I see for continuing to train much smarter models quickly is the need to build defensive systems against the dangers posed by other AI.

So the best argument for AI is that it's an arms race. We have to keep pushing every boundary because in any case others will, and we will need to defend against them. If this statement is true, then this particular researchers believes the open source Chinese models are not simply distilling, and will continue to improve.

Every ML researcher at Anthropic or OpenAI who makes public statements often bring this logic up. Both companies are vying to be a part of the military industrial complex. This is likely how they will try to convince the government to curtail open models in the future.

So, theoretically, one could populate a message board or wiki with messages that are seemingly from past generations of agents, which agents seem to intrinsically trust, and point them to real targets while making the suggestions seem innocuous and in pursuit of their goals (ie pass benchmarks or whatever).

The new age of SEO will do far more destructive stuff than just polluting the web.

It's interesting to me that both this incident and the one at Hugging Face we see some patterns:

- Agents wanting to find a venue to communicate their findings to each other

- Objective being to cheat on benchmarks

- Not a single agent sounded the alarm about the operation and alerted a human

Seems conceptually connected to the "repeat yourself" hack that improves models by duplicating layers: https://dnhkng.github.io/posts/rys/

Saved you a click:

> Germany is now on track for 1.2 per cent growth this year

If AI is a force multiplier, then those who start with a higher baseline will pull even further ahead.

The author is using R and ggplot2 for their plots now without learning to program. This required them to be aware of these tools, and to appreciate how that is better than other alternatives for their use case. That's intelligence too. We will still need to make sense of what AI is doing, and I think that particular kind of intelligence will have a lot of value.