Neural Computers provide the illusion you are interacting with an underlying operating system/application, but are actually a recurring (user input) -> (predicted pixel output) loop.
In theory, it would be impossible to tell a perfect "Neural" computer apart from a "real" one. Which is kind of a wild thought.
This isn't suprising. After all, you can use 2 neurons to model a NAND gate, and you can build up a CPU from NAND gates only.
You don't even need transformers to model a CPU using neural nets. The only problem becomes training them, which becomes very tricky from end to end, especially from shell command to output because the gradients are often going to reverse quite a bit due to binary nature, and the recurrence of memory outputs back into the input (we still don't know how to accurately train RNNS)
With LLMS, uou have to do something like have the LLM forget the KV caches after every invocation and instead rely on injected context of a state, which it then also outputs at the end of the terminal command, and then train it on that. That way given enough parameters which translates to resolution, and some command preprocessing (like splitting a complex chained shell commands in a single line), you can guarantee that for every possible input you get accurate output and state change. So instead of an efficient RNN, you basically end getting a much wider lookup table of sorts.
The issue is, of course, memory. A LLM won't remember many things. A custom built model architecture might remember much more though.