> H 7B is always faster than optimized Transformers
> (>10%, >20% and >50% end-to-end faster than FlashAttention
50% improvement on very large sequences is great. But it is not yet transformative (pun intended..)
Presumably FlashAttention already does lots of the less-than-quadratic improvement