No, reducing LLMs to just “statistical text predictor” is an absurdity; but so is leisurely stringing together observations from unstructured experimentation to form a “matter-of-fact” conclusion.
Read the ChatGPT paper and specifically hone in on “transformer” and “input processing” (paraphrased). It will give you a clearer picture of what it actually is, rather than appears to be.
I've read the relevant papers, I've even implemented some of them, and I do not think any of this gave me a better understanding of it's capabilities than playing with it as a black box for a few weeks.
No, I expect not.
But I think it’s a mistake in thinking and loose use of language to say that differential equations, or any mathematical representation, governs anything. They are models of whatever may actually be happening, and are necessarily modeling a simplification of the behavior observed.
Don’t mistake the model for reality.
Every memory you store is part-fabrication too. Only what the brain perceives as "key" is stored, scaffolded by made up events. Every memory you retrieve is even more fabrication because the brain rewrites memories every time they're recalled. This is why implanting false memories is extremely easy and why leading questions work so well. https://en.m.wikipedia.org/wiki/Lost_in_the_mall_technique
The vast majority of what you know and perceive was not derived from first principles.
Make no mistake here, you are working off a model of reality too.
https://www.scientificamerican.com/article/how-ai-knows-thin...
You can follow the multitude of things from there to the papers published that describe each of the cases in greater depth.