simianwords

> While he's never been dismissive of the technology itself

Hmm,

> Remember the relentless “AI is beating everyone in Math Olympiads” and “it’s all over for humans, no point in learning STEM? Well…

> The 2025 Math Olympiad problems are out and naturally people tested the leading LLMs on those… and… spoiler .. they all fail, badly.

>That’s it LLMs are all about storage in training data which enables recombinatorial retrieval. Don’t expect them to solve or respond to new things like a human.

> They fail with novel and unknown tasks because their ability to generalize is vastly overstated by the people invested in the technology.

> The value of LLMs is in their data. No data, no answer. You have a novel piece of work that’s never been done before? OpenAI will insist it’s worth nothing because it’s a spec in the Ocean, yet that’s not true at all.

https://www.linkedin.com/posts/georgzoeller_proof-or-bluff-e...

Today you can solve IMO problems with ~$5.

Its up to you to verify that he has dismissed the technology and that he has been wrong about its performance in mathematics.

Edit:

Even without lean I can solve IMO problems using Sol/Fable today.

You made a categorical statement about LLMs

> That’s it LLMs are all about storage in training data which enables recombinatorial retrieval. Don’t expect them to solve or respond to new things like a human

> They fail with novel and unknown tasks because their ability to generalize is vastly overstated by the people invested in the technology.

It was not contextual but categorical. And it is false today.

The author points out that the labs are still releasing models while claiming to slow the pace of frontier. This is not in contradiction. Labs have models that will be released in 6-12 months. Those are not exposed to public and these are the models referred to when we talk about “frontier”.

It’s better than both fable and Astra? I’m going to start trusting this index less and less.

We have serious proof that these models are capable of sophisticated attacks.

So what’s the trick in this honestly? OpenAI shouldn’t care about alignment and just release it? I’m genuinely asking.

This line of conspiracy theory is sooo uninteresting.

“It’s all hype”

“Government bailouts”

“Labs are fundamentally unprofitable”

And the same people who say all this will again claim OpenAI should be held responsible and arrested for when agents hack other systems.

It’s a fundamentally unthoughtout analysis. If you believe that OpenAI should be more careful and prevent future hugging face incidents, surely their decision to delay models is rational? Does it really require tinfoil analysis?

What would you say when their models cause new incidents like hugging face one?

Important to note that lower model + higher reasoning gives a different (not higher) quality of response than higher model + lower reasoning.

Some tasks are reasoning shaped by nature and you can't just throw a big model at it.

And they are not calling it a day and they have done most things possible to communicate the fact that they are building something dangerous.

But this level of isolation is what happens normally. At least in my university and another company I worked at. It wasn’t running insecure software, as far as anyone knew, it was secure

The truthers will be the first one to say I told you so when agents do another dangerous thing autonomously.

For truthers AI is simultaneously a machine that hallucinates 100% of the time but also so powerful that it can cause nuclear levels of destruction. Because something something “wrong hands”.

The model found and exploited and chained together previously unknown vulnerabilities.

How were the sandboxes poor?

> the ballyhooed OpenAI escape depended on a pedestrian failure of containment

I... I can't believe people still think this way. A whole class of security measures are no longer useful and this guy thinks it is "pedestrian".

This kind of isolation - artifactory having access to only a specific IPs - is what is used in most places, including my university and at my own job.

LLMs basically rendered this kind of isolation almost useless by bringing in the ability to chain together never before found vulnerabilities to escape the containment. This was not possible before.

How is this pedestrian??! The author might say "oh well they should have airgapped actually" in a smug way. But you can only know this retrospectively. Even then, isn't it noteworthy and a meaningfully different class of cyber attack?

> For the first time, GPT‑5.1 Instant can use adaptive reasoning to decide when to think before responding to more challenging questions, resulting in more thorough and accurate answers, while still responding quickly. This is reflected in significant improvements on math and coding evaluations like AIME 2025 and Codeforces.

https://openai.com/index/gpt-5-1/

It says literally the thing you wanted from system 2. Its almost exactly that.

This is what you said btw:

"it's that the system itself decides how to reason based on the nature of the problem it faces"

> The argument you made implies that System 2 is just a more capable System 1, which is not what the book (nor this paper, AIUI) proposes.

No, system 2 is the emergent capability to reason and increase the space of places to find the answer. Forget the paper's proposal, and look at the problem it is trying to solve. Ability to give quick answers, ability to give thought out answers, and the ability to know when to choose what. Adaptive reasoning does all three.

> This means that any System 1 will run the input once through the heuristics, using the same computational power and taking the same time whether the input is 100 tokens or 1 million tokens, for quick but perhaps wrong decision (not "answer").

No, I don't think we humans use o(1) to for understanding 1000 tokens or 2 tokens. I simply don't think that's the case. There's a new model called "Jev" and it is literally named System 1 (from the book) and even it is billed per input token.

> it's that the system itself decides how to reason based on the nature of the problem it faces, given the model's world model.

do you even know how adaptive reasoning works?

Asking earnestly, I don’t know what you mean by this reply. I know what system 1 and 2 is. But this has already been solved using same model.

This has already been solved by GPT 5 Adaptive reasoning. A single model that knows when to reason or not based on a thinking parameter we provide (like xhigh). What’s the relevancy to post it today?

edit: why is this downvoted?

His case was materially different

1. Unauthorised network access

2. Intent to distribute licensed material

The labs aren’t doing this. They are doing something similar to you and I downloading torrents. Look, I also think laws should apply somewhat equally to individuals and companies. But this is different.

Yes it’s all hype to cash in the ipo before the bubble bursts. The labs are spreading spooky things about AI so that people think these models are very capable. It is a coordinated effort amongst all labs. The reality is that these models can’t do basic mathematics.

Sure it’s not free for anyone and both companies and individuals are treated similarly. It’s not like you will be jailed for pirating movies. And neither should OpenAI. What part of this is hard to understand

Aaron was sued for distributing and not for pirating. They are different offences

So we now know without any ambiguity that the OpenAI agents hacked the us government and multiple other websites.

Maybe.. just maybe.. it is because of this that they want to pause?

Why do people keep talking about circular financing? It’s hardly a drop in the ocean and doesn’t change the underlying financials. Is it 30% of revenue? 10%? Is it priced in the market? I think it’s around 5% of revenue and already priced. It’s a nothingburger.

I would take ketamine if it helps me perform better without adverse side effects.

You’d rather paint AI as if it were a hard drug to cope with the world changing around you. I mean listen to yourself. Im not the one making conspiracy theories.

Yes it has happened to me and it’s good that it’s happening. Life is always about finding the right abstraction to work on.

Why don’t you start a company where AI is banned. Surely it will do much better. In fact, you should also believe that most companies will bad AI for the same reason.

Smh new conspiracy theories every day. “Ai is actually a drug and you only feel that it helps you but it doesn’t”

New conspiracy theory just dropped.

In what way is hype quantified and different from stock market valuations? The stock market already captures public sentiment on future earnings, I.e hype.

Speculation of what? There are models today that perform better than Haiku at 47x lower cost.

Excessive skepticism does no good.

Also the author is deliberately chooses a model that is 1 year old and was supposed to be underpowered even then. If cost was deciding factor, they could’ve chosen Luna.

  Model                       AA Index   Cost/task
  Claude 4.5 Haiku (thinking)    17       $0.21
  GPT-6 Luna (max)               37       $0.07
  GPT-6 Sol (low)                34       $0.13
  GPT-6 Sol (medium)             40       $0.25

Luna (low) scores higher and costs about 47x less per task and performs better.

And Luna (max) scores 37 while still 30% of the cost of Haiku which scored 17.

Why deliberately choose a model that is known to be poor and also costly?

Author doesn’t understand cost dynamics of the industry - that prices keep falling.

Author also needs to make a point, and can’t do so if they are using a powerful model.