Yes they are. Just because the models have been overfit on the entirety of the internet doesn't mean they aren't in principle re-generating parts of their input data for the overwhelming majority of their output. Sure, stochasticity pushes the output into weird states sometimes, these are the states where the output is plain useless nonsense. Hallucinations, if you will.

Our brains are entirely too small to recognize the patterns encoded from the gigantic training data set. But they are there nonetheless.

This is easily seen if you try get *any* neural network to generalize to *any* non-trivial task (in this case, XOR for example I consider to be trivial). There are no non-flickering, stable vision models that can recognize a human/dog/cat/whatever in all the lighting settings we can without false positives. The models are only getting better because they are getting bigger and are provided with more data. This is *not* generalization.