What is mind-blowing about it? How is it actually thinking, and not a parrot? How do you separate the hype from the specific facts of the actual advances?
In other words, what are the legitimate criticisms, and if you have none, why not?
The "inside" of an LLM is a black box in that we don't know how an input arrives to an output. We know some general ideas ("it predicts the next thing"), but not the detailed route to get to the output.
Similarly, we do not know how human minds work. How does a thought arise? How does a new idea arise?
I would not argue that an LLM is identical to a human brain. But I do think there's some interesting discussion to be had on that topic.
It's as though we have two black boxes, and so many folks are willing to say, "One of these is definitely, certainly not like the other black box!"
And I don't see where that confidence comes from.
That comparison, to my silly mind, is astonishing to think about. And I've yet to see why I should (or could) dismiss the notion of similarity.
Or, simply using GPT-4; that's quite astonishing to me. I'm not a stellar coder by any means, but I've quite literally 5x-ed my productivity with GPT-4.
"Well, you must not be working on novel problems." You're right! But is "solving novel problems" a requirement for discussion now?
It sounds like you're conflating multiple different things here which makes it hard to respond to your question as-written.
When scientists say "We don't know" they mean "It would take more time than I care to spend, to investigate the particulars to answer that question." The public jumps straight to "It's magic, Jesus or aliens!"
It's a parrot. That's not in question.
Given that the output essentially is similar to what human posters online are doing in many many cases the term isn't useful to distinguish between the two.
It's certainly not working like Eliza where there are specific programmed response trees to choose from. It doesn't seem to be just copying sentences and pasting them into the output.
But also, I'm not so sure scientists "don't have the time" but rather, "literally cannot" know with regard to an LLM's "insides".
With both of them, I'm very underwhelmed with "use it like a forum" or "ask it a tech support question". Maybe because my "tech support questions" tend to be much harder than my "scripting questions", but it's not super useful. Anything it finds I actually find "better" because of context and screenshots just using a traditional search and clicking through to the blog or whatever that's talking about the issue.
Ok, now to the "It's just a parrot". I disagree strongly with that. It's no more a parrot than wikipedia authors are. Many of the Neeva AI search results read like an autogenerated Wikipedia page, complete with the source links.
Philosophically - I think a lot of people either aren't bringing philosophy and some sci fi knowledge to these arguments or don't know of them. Many of the "it doesn't understand anything" seem to be somewhat substrate chauvinist to me - I'm not sure if Data from Star Trek showed up in front of them that they'd admit he's intelligent or a person, simply because he's not organic. And you're also right - people claiming it "doesn't understand" can't or won't define what "understand" means in this context. Other terms, like "intelligence" also have changed over time. I'm pretty sure if in 1800 you asked if a machine could do calculus if it was intelligent - people would say yes. However by the 1960s computers were enough of a thing doing math that we decided math was no longer a marker of intelligence for a machine to do. Now we claim that GPT is just advanced autosuggest. Well, I guess an iPad is just an advanced calculator - but I'd argue it's transcended being a calculator in practical use. And I think GPT has transcended next word suggestions. My phone keyboard suggests 3 options for the next word, but it can't write the next paragraph for me coherently, nor does it take directions or questions and output pretty reasonable responses.
So I think I'm in the middle here - I don't discount GPT3.5 and 4, but I don't think the're Star Trek level AIs either. I don't think they are magic as it were.
I will agree that it's by no means Star Trek level, but it is about the most fascinating piece of technology I've seen in my lifetime.
And yes, that sounds silly! But therein lies my reason for writing this post.