Oh I didn't test their in-depth knowledge, just checking to know if they were aware people were writing hundreds of papers on this topic, and what impact would have a sub-quadratic attention that works as well as the regular one. Fishing to see if they remember at least one approach or two.
Many people are using it, but people that are relying on OpenAI to do all of the work for them aren't looking deep enough to realize it.
Can you point to a single major project that does? AFAIK it always leads significant regressions in real world use cases, which is why no major repo uses them.