So yes, science has definitely gone too far.
It doesn’t invoke your human expert when it’s “stumped”. It invokes it when the question associates with training data where invoking an expert is more likely (i.e complicated question, esoteric questions, etc) — it has no knowledge and no insight into your question or its answer; it just can imitate that certain looking questions are followed by expert referrals.
We really need to stop using the wrong language for these things.
For something that has "no knowledge and no insight" it sure is damn good at taking unique social, technical, and other problems I have and offering an array of solutions, with pros/cons and probable outcomes of each.
What that might be telling you is that your problems aren’t so unique after all.
Getting a sound answer from ChatGPT is very literally like looking up some weird concern that never occurred to you before in a search engine and finding a hundred upvotes and a three page long discussion from 2012. Those upvotes and comments that happen to come up in Google are only a tiny share of the online and offline encounters with that same concern, and tell you that your completely novel-to-you problem is actually not so novel after all.
For that class of problem, ChatGPT can usefully synthesize content that can almost always look right and often be accurate. Just like Google, it has a huge database of source material and just happens to synthesize an original text similar to many samples in it rather than pointing out one specific sample.
But as you dig into more specificity, idiosyncrasy, and esoteric “problems”, the reference material gets too thin and it starts to generate noise because it lacks alternative ways of reasoning or knowing.
There might be another leap forward in the next few years, but this generation of tools only do what they do.
So while it is obviously just guessing what to say, the idea that it is just regurgitating existing material seems wrong to me. It does some pretty impressive synthesis to come up with its answers.
If you were to ask a truly nonsense question, like "Which is more powerful, a GTX 1070 or an F-16?", what would the response be?
The GTX 1070 is a powerful graphics card that is designed to render high-quality graphics for gaming, video editing, and other visual computing applications. It has impressive processing power and can handle complex graphics and visual effects with ease.
On the other hand, the F-16 is a highly advanced fighter aircraft designed for air-to-air and air-to-ground combat missions. It is a complex machine that requires a highly skilled pilot to operate and is equipped with state-of-the-art avionics, weapons systems, and engines.
In terms of raw power, the F-16 is undoubtedly more powerful than the GTX 1070, with its advanced capabilities in speed, agility, and firepower. However, it's important to note that these are two very different types of machines, and a direct comparison of their power is not really possible."
Meanwhile ostensible "expert" humans continue to regurgitate obvious falsehoods like "it's just a fancy autocomplete" or "you're being fooled by a system that just predicts the most probable next word".
Anyway, while you can call that a "world model", that world model is very brittle since its just a bunch of heuristics, if you reverse engineer the heuristic it uses it is really easy to find cases where it fails completely. Like if you ask it to compare a modern integrated GPU to an old dedicated GPU, it will say that the old dedicated GPU is faster even though it is ancient and slow, since when people talk about dedicated vs integrated GPU they almost always say that the dedicated GPU is faster.
There's a giant gap here that people keep glossing over when they treat LLMs as almost-human.
I'm not really trying to make a point here about consciousness, although I was interested by the article - it seems to imply that gpt-3 performs better at some tasks when it has a train of thought / stream of consciousness.
People keep saying that there's a giant gap, but I still don't really know what they think it is, and there seem to be a lot of points of similarity. It's very clear that these large language models are, despite their problems, much more capable than was expected by nearly everyone not very many years ago.
Multimodal models are being trained right now using text images, video, and audio. Eventually you can add data from pressure, heat, and acceleration sensors and motors (sense of touch). We can further add additional "senses" - data from RADAR/LIDAR, magnetometer, multispectrum vision, radiation sensors if desired.
AI will come to know our world very well.
"It's not appropriate to compare a graphics card (GTX 1070) to a military fighter jet (F-16) as they are completely different things designed for entirely different purposes.
A GTX 1070 is a high-end graphics card designed for gaming and other graphics-intensive tasks. It has a lot of processing power and can handle complex visual computations quickly and efficiently.
On the other hand, an F-16 is a military fighter jet designed for air-to-air and air-to-ground combat. It is equipped with advanced weapons systems, avionics, and other technologies that allow it to perform a wide range of military operations, including surveillance, reconnaissance, and combat missions.
In short, while a GTX 1070 is a powerful graphics card, it is not designed for the same purposes as an F-16, which is a highly advanced and specialized military aircraft."
Or it has a separate category in its state space for old computers and will say the same thing for all of those in these comparisons, but you can trip it up using the weakness I talked about above in most cases. Its just that the comparisons people come up with can often be answered correctly by that method since people like to compare "powerful thing to weak thing", and in those it just looks at how often each wins and decides based on that, instead of comparing them directly.
What impressed me just as much about these answers, though, was that it correctly identified that the comparison was silly. I don't believe it has seen that exact comparison before, so it seems like it has to have a granular enough categorization of tokens to know that people would say a comparison between a GPU and a computer is silly, and that those are a GPU and computer, and that one is old and the other is new.
That would mean it would identify an old GPU as faster than a modern CPU, even though it isn't, since it would slot that into the same conversation. The hard part with finding out the source for the logic is that it can map items to similar items in many ways, and there are trillions of conversations out there that it could map it to, so it can do quite a lot of heuristics that will work fairly well for naive questions even though it can't apply any direct logic.
If it could link to similar conversations that it bases its reasoning on then that would probably greatly help understand where its logic would fail. So before it can do that we probably wont be able to eliminate these failure modes, because humans aren't random enough to generate the edge cases unless they understand what logic the model used. For example, comparing an old computer to a modern GPU might seem random to you, but to ChatGPT it just sees something like "This looks like a CPU vs GPU comparison, aha so I map it to: {CPU} isn't really comparable to {GPU}, but {GPU} has much more processing power". If you could see something like that then it would be obvious where it would fail, but without that it looks like magic.
Jup, just tested with this: "what is more powerful Athlon PRO 3125GE or Geforce 6800". It said that they aren't comparable, but that the GeForce is probably faster:
"In summary, while the GeForce 6800 is likely to be more powerful than the integrated graphics in the Athlon PRO 3125GE, the two cannot be directly compared as they have different functions and are designed for different purposes."
Which is wrong, the modern integrated GPU is much faster, and they are directly comparable. So that is how it answered your question, and since I maanged to figure out what conversation I could find this failure mode where it gives the completely wrong answer.
Once you understand that ChatGPT just slots what you ask for into some conversation based on fuzzy matching then it is really easy to understand what sort of answers it can give and can't give.
That's been one of ChatGPT's most consistently impressive features, but it's clearly a function of a lot of training to encourage strings that reject comparisons when they're in different categories. Sometimes it does so based on simple heuristics like "FamousCharacter is fictional", sometimes it demonstrates a surprisingly nuanced grasp of historical periods or events so it will insist that x and y lived in different centuries or tell you that during a conflict p attempted to invade q, not the other way round. Even when it makes logical errors (no sooner has it told you that if Finland did not exist, there would be no Winter War, it suggests that the absence of Finland would not have affected anything about the outcome...) it's pretty good overall.
But the trouble with lots of training to reject silly user inputs, is when it gets it wrong it can end up insisting it's been a good Bing dealing with a bad user...
Outlook not so good.