Why don't people here on HN understand that LLMs never see ASCII or other raw characters as input?
Expecting spelling, rhyming, arithmetic or other character oriented responses will always yield disappointing results.
Expecting spelling, rhyming, arithmetic or other character oriented responses will always yield disappointing results.
I just also think it's a reason to mock people who don't try to understand those limitations and get way ahead of themselves hyping up the technology.
The entire point of this exercise is to refute the claim that LLMs are a step towards AGI, even given "agency". And we should be happy that they aren't — because supposing that AGI is possible, the way that we currently treat LLMs shows that we as a species are nowhere near ready for the consequences of creating it.
All I have to do is turn on thinking mode and the error goes away. https://chatgpt.com/share/6897e630-77f0-800c-a9bf-30d9c0e271...
You think all PhD candidates have perfect spelling? I'd wager most of them re-read their dissertation and edit it, over and over, a process that most LLMs don't have the luxury of doing.
We'd have to give up all the efficiency of tokenizing, re-train a model (a much less optimum model) for at least twice as long to get anywhere near the same results for one that just spits out ASCII.
"Intelligence" refers to a basket of different capabilities. Some of them are borderline cases that are hard to define. The stuff that GPT-5 failed to do here is not.
Things like knowing what a question means, knowing what you know and don't, counting a single digit number of items, or replying with humility if you get stuck -- these are fairly central examples of what a very, very basic intelligence should entail.
LLMs are not intelligence. There's not some groovy sense in which we and they are both intelligent, just thinking on a different wavelength. Machines do not think.
We are inundated with this anthropomorphic chatter about them, and need to constantly deprogram ourselves.
There's an immense history of humans studying animal intelligence, which has tended pretty uniformly to find that animals are more intelligent than we previously thought at any given point in time. There's a very long history of badly designed experiments which surface 'false negative' results, and are eventually overturned. A common favor in these experiments is that the design assumes that animals have the same prescriptions and/or interests as humans. (For example, trying to do operant conditioning using a color cue with animals who can't perceive the colors. Or tasks that are easy of you happen to have approachable thumbs... That kind of thing.) Experiments eventually come along which better meet the animals where they are, and find true positive results, and our estimation of the intelligence of animals creeps slightly higher.
In other words, humans, in testing intelligence, have a decided bias towards only acknowledging intelligence which is distinctly human, and failing to take into account umwelt.
LLMs have a very different umwelt than we do. If they fail a test which doesn't take that umwelt into account, it doesn't indicate non-intelligence. It is, in fact, very hard to prove non-intelligence, because intelligence is poorly defined. And we have tended consistently to make the definition loftier whenever we're threatened with not being special anymore.
"Dangerous", because they lead into thinking they do the advanced of what they don't do basically.