There's a hell of a lot of matrix multiplications going on in AI/ML. Shaving off a few milliseconds for ultra large matrices could save power and money for the big players like Google, Microsoft/OpenAI and Meta.
As it stands right now, it is actually better to have a slower algorithm that uses the local memory more efficiently.