Oh my point was NOT that LLMs can't simulate a Turing machine reliably. I was just replying to the tweet above. The main point of this (speculative) text is that GPTs can "evolve a general learner" inside them, but are unable to give enough computing power to it, since it will dispute dispute compute and space with billions of task-specific circuits. Also, even if it could, the "computer" on which learned procedures run (inside transformers) is too limited in many senses, but not being Turing complete isn't the main issue, just one of them.
Basically I'm saying that (on my view) GPTs is kinda growing an AGI procedure inside it, but that AGI is restricted by rigid computational constraints imposed by the GPT architecture. I think something slightly more flexible might be capable of filtering out the noise more efficiently and replicate what GPTs do with an absurdly small fraction of the cost.
So, to be very clear, the point has nothing to do with Turing completeness and more to do with the ultra-limited computation and expressivity granted to internally learned functions. It is a small distinction, but a non-Turing complete model can still be extremely expressive.