Since LLMs began being used for math I have wondered why the first task in LLM self-improvement wouldn't be improving the underlying algorithms. Even a tiny improvement in matrix multiplication or the other many algorithms that is required would yield would have a huge payoff in efficiency and power-use and a direct impact on profitability, sustainability etc.
Are there any examples of this path being pursued?