The problem with ChatGPT in a box and similar is that they are not made for live interactions. If you've tried chatGPT or claude on your phone with voice conversations you will see that it takes a while to think.

Humans on the other hand start processing the moment there's a response and will [usually] respond immediately without "thinking", or if they are thinking, they will say as much, but still respond quite quickly.

It is. It will make up a story if you ask it to. Answers questions with it's own knowledge now instead of using the amazon answers site.