"We show that transformer-based large language models are computationally universal when augmented with an external memory. Any deterministic language model that conditions on strings of bounded length is equivalent to a finite automaton, hence computationally limited. However, augmenting such models with a read-write memory creates the possibility of processing arbitrarily large inputs and, potentially, simulating any algorithm."

From "Memory Augmented Large Language Models are Computationally Universal"

https://deepai.org/publication/memory-augmented-large-langua...