Examples of things I'd file as silly mistakes:
- mentioning non-existing settings when asking it about a particular software.
- responding to roughly "give me a way to list all rds clusters that are running a version that will reach EOL within the next 12 months" (which I know is almost impossible to do without scraping, as the EOLs are not returned by the API, but it was worth trying) with a query that compares the version with < 12.0. It did acknowledge the error when I asked "won't that just compare the version number" and, interestingly, it then did add to the response that the API does not provide EOL for versions, but the first response was IMHO bad in a very basic way.
- providing wrong steps in an answer to "give me steps to do X", wrong in a way that would cause the process to go very bad, and then when I point out the problematic step, it responds with a seemingly random alternate step that is also wrong. I guess this part is the one that is less bad, because it loosely reminds me of people who are bad at reasoning and just give back random answers to a logic or math question.
I must say it is surprisingly good (though it will still make up facts at times) when answering questions about elisp though.
Paraphrasing Hofstadter, I'd say if I'm asked about the intelligence of chatgpt using gpt-3 I'd say it's slightly better than a thermostat or a mosquito, but not much more. Furthermore, my (admittedly not very valuable to others) gut feeling is this is not the path to get to what I'd consider an intelligent being, mostly because in my interactions, it makes lots of bad mistakes, and none of the good ones (here I'm using good and bad as qualifiers for mistakes based on my experiences as a parent, and as a mentor to junior peers, where a good mistake is something that lets you know they're on the right path even if they haven't arrived at the right answer/place yet). Try asking it about analogies, for example ("what's a good analogy to explain X to someone new to it"), the results I've gotten are underwhelming, when not just plain wrong.
Edit: s/fall/file/ Formatting.
The difference is extremely dramatic. Any experience with 3 or 3.5 is completely irrelevant for 4.
(I use 3.5 and 4, both in the chatgpt interface and via API.)
- https://lilianweng.github.io/posts/2023-03-15-prompt-enginee...
- https://help.openai.com/en/articles/6654000-best-practices-f...
Not a large percentage of humans would be able to do the tasks listed above without significant experience or training either. Average humans are also not great at reasoning.
I'm in full agreement about humans and reasoning, btw. I just don't think chatgpt (with the version I've used) is anywhere on the same league as the worst (way below-average) humans either. I do think it's quite useful as a writing aid. You do need to review what it gives you, but you'd have to do that even if you were relying on a human writing aid.
You realize Hofstadter is not delusional about LLMs at all. His view point is completely opposite of yours more logical and rational and he doesn't need to use chatGPT 4.
Hofstadter is not a normal person because he is extremely unbiased. chatGPT basically got him to do a 180 on everything he thought and talked about in every book he has written. He flipped, you subscribe to his viewpoints, and you haven't flipped.
Keep in mind he criticized early versions of gpt3 and gpt2. chatGPT changed the game.
https://www.nytimes.com/2023/07/13/opinion/ai-chatgpt-consci...
https://www.lesswrong.com/posts/kAmgdEjq2eYQkB5PP/douglas-ho...
He literally said something along the lines of his core beliefs are collapsing. Paraphrasing. I kid you not. It's abnormal. It's very rare to find a person that can change his core beliefs. Extremely rare. Let's be frank. You are not that person. What will happen is you will reinterpret reality such that it fits your core beliefs and you won't know you're doing this. It's very predictable: Using chatGPT 4 will not revise your opinion on AI because you don't want to face the truth. chatGPT 3.5 is already enough to see how we crossed a certain line here you can't admit it... so your experience will be the same with gpt4.
"Of course, it reinforces the idea that human creativity and so forth come from the brain's hardware. There is nothing else than the brain's hardware, which is neural nets. But one thing that has completely surprised me is that these LLMs and other systems like them are all feed-forward. It's like the firing of the neurons is going only in one direction. And I would never have thought that deep thinking could come out of a network that only goes in one direction, out of firing neurons in only one direction. And that doesn't make sense to me, but that just shows that I'm naive.
It also makes me feel that maybe the human mind is not so mysterious and complex and impenetrably complex as I imagined it was when I was writing Gödel, Escher, Bach and writing I Am a Strange Loop. I felt at those times, quite a number of years ago, that as I say, we were very far away from reaching anything computational that could possibly rival us. It was getting more fluid, but I didn't think it was going to happen, you know, within a very short time."
The guy literally flipped bro. He literally just admitted what he wrote in his books are wrong. The books may not be wrong but he believes it now. Degrading hundreds of pages of his own writing and his own core beliefs is a display of incredible rationality, scientific thinking and lack of bias.
I very much respect Hofstadter and the fact that he changed his opinion on GPT (I'm based on what you're saying, I have not read the links you provided yet). I'm not sure that means he changed his core beliefs though, even if he said so. At least, my understanding and interpretation of his work is that we're not the only substrate that can develop "strange loops", and that this is not limited only to organic substrates/living beings. He may have changed his view in GPT, but if he was true to his words, he must have always been open to software being able to develop a being.
I very much think AGI is possible, but:
- I don't think it will happen through LLMs, or at least not through LLMs alone, and
- I don't think it will be human-like. In fact, I expect it will feel very alien to us, and we'll feel very alien to them too. Mostly because we're not brains in jars. We eat, sleep, get laid, feel pain, etc., and all of that feeds back into the loop to make us what we are.
Read what he says. He literally says it's depressing to watch all his core beliefs get dismantled. I'm not making this up. These words came out of his mouth. Follow the links. This is not some minor realization he went through. His entire world view was changed and he's the one claiming this... Not me.
His books which I read are all about recursion and strange loops and he's shocked that LLMs are simply feed forward networks with zero feedback. He claims it debunks everything he wrote.
He has the most to lose in disbelieving what's happening with LLMs. But he's to rational to do what you or most people do. He has to accept the truth and basically admit to being utterly wrong.
Once you read the links you will not agree with him. That's the most likely behavior for most people. But think on this. He has the most to lose and he thinks the books he wrote is wrong while you think the books he wrote is right.
Given the context the conditions are ripe for him to be the most biased and you to be less so, but he believes the opposite against his own self interest. From this you know he's not biased. Most likely you are.
It has nothing to do with fear or dislike of agi. It's just common human bias no one is sure where this lunacy comes from.