For reference: I don't care about politics in the scope of standing on the shoulders of giants.
This seems like a massive effort to collect perpetual coding data to ensure longevity of coding abilities.
563 karma · joined September 8, 2014
For reference: I don't care about politics in the scope of standing on the shoulders of giants.
This seems like a massive effort to collect perpetual coding data to ensure longevity of coding abilities.
You're still likely right as things must always come down but what if it doesn't.
Regardless of being in conversation or thinking context this doesn't prevent the model from speaking the wrong answer so the paper on the illusion of thinking makes sense.
What actually seems to be happening is a form of conversational prompting. Of course with the right conversation back and forth with an LLM you can inject knowledge in a way that causes the natural distribution to shift (again - side effect of the LLM tech.) but by itself it won't naturally get the answer perfect every time.
If this extended thinking were actually working you would expect the LLM to be able to logically conclude an answer with very high accuracy 100% of the time which it does not.
Not even that, children believe anything and more so that of a computer designed to be "harmless".
More reason to decouple and think for ourselves.
1. Even if LLM architecture doesn't work out, it's wise to remember that it's the quality of training data which is a deciding factor. A pivot is easily doable since this is a constant and disconnected from the technology itself.
2. This is clearly a moonshot project but it still feels wasteful especially with context of previous iterations of models and their shortcomings-it feels like the tremendous amount of money available is being used simply because it's available.
Regardless the critical aspect is valid, AGI would be something like Cortana from Halo.
It takes a considerable amount of depth in reasoning to see and reason about the patterns / problems / solutions.
Try doing a few of them by hand to see what I mean.
Simulated worlds are complex enough to hide their own flaws just like LLMs are complex enough to lead us to believe they can reason when most of the time they are pattern matching.
The real pressure is the private hold-out set and the variations that can be added to counter this aspect.
A true AGI would be able to solve anything thrown at it which is where the authors are trying to lead AI engineering towards since LLMs have pretty much taken over.
If it starts getting too easy, they just reconsider and add harder problems.
It's like how we don't talk about the Turing Test anymore as it's no longer the best metric to determine real intelligence.
The authors are signalling to the industry that new ideas are needed and the monetary aspect is to show how serious they are about it.
It's good because as per above we have research being thrown at it which means we can iterate until we perhaps find another breakthrough.
Your experience aligns with what I've seen: 4o and o1 are better at filling gaps from each other where there should be an upward trend of intelligence instead of having to jump around as we do currently.
So the real question is: are these models forgetting their skills, do we need to make them larger? Does distillation and generalisation work?
So many questions that might end up as LLMs being unable to escape the problem of catastrophic forgetfulness.
We've seen the trend of distilling models at what seems to be the cost of more nuanced ability to iterate and achieve correct results.
I'm very convinced LLMs can go much further than we've achieved so far but I'm very welcoming of newer techniques that will improve accuracy / efficiency and adaptability.