Hacker News Reader

they're also just less capable models. like just adding attention on top of an RNN made them a lot better

Calculating self-attention is still quadratic though. So you get the negatives of transformers there too.