Hacker News Reader

Any autoregressive model can do what you are describing. transformers are generating one token at a time too, not all at once.

View on HN

True but memory requirements grow with sequence length. For recurrent models the memory requirement is constant. This is why I qualified with "low memory".

yes but transformers are much slower than state space models

Reply on news.ycombinator.com