What other statistical machine learning algorithm gives you a state machine as an output?
The particular encoding -- matrix or otherwise -- is just an implementation detail.
The particular encoding -- matrix or otherwise -- is just an implementation detail.
* hidden Markov models
* autoregressive models
* learned LQR
Sure, some of those have finite memory, but RNNs are practically limited–not as easily quantified.
I see what you're saying about the single-step operation appearing unique, but I also think that RNNs can be viewed through a lens such that they look like "normal" programming concepts like generators and folding iterators.