RWKV is random chance on the PathX problem. Both of those models are > 95% on it, if I remember correctly.
I don't think Hyena can be run in RNN mode, though. But it _might_ scale better than the LRU model (second paper).
I don't think Hyena can be run in RNN mode, though. But it _might_ scale better than the LRU model (second paper).
No comments yet.