It does not use transformers as a computer in any novel way. It just makes the analogy.
This actually does what this blog post pretends to
The authors decision to name their creation a "Neural Turing Machine" doesn't make it computationally equivalent.
The fact that it explicitly states:
> ...analogous to a Turing Machine or Von Neumann architecture but is differentiable end-to- end, allowing it to be efficiently trained with gradient descent.
Actually demonstrates it is not equivalent.
Note that I am not saying their approach is wrong, but that their overloading of terms suggests more than they claim.
While 'general-purpose computer' is ambiguous, a Turning machine is not as opaque.
LLMs can be TC with infinite space for input and infinite width.
For finite input, size, and depth, LLMs aren't even equivalent to primitive recursive functions IIRC.
With consent depth threshold circuits being a recent claim as the limits of LLMs.
One idea is "Transformers, analogous to general purpose computers" although that doesn't mention the meat of the article. "Transformers from scratch, a turing machine analogy" is another idea.
I'm not sure they're good titles but they don't imply that it's about using a transformer as a computer.
With the goal you state, I understand better how you came to the title.