>I'm getting the impression that the field is over-saturated, with new research being published on a daily basis

The research being published is mostly about optimizations of transformer models.

There is a lot more research in other areas. For example, everyone seemed to forget about MuZero, but that was one of the most important breakthroughs in the last decade, because it was a model that could recursively search.

So if you think about tasks like driving a car, a model that can predict the evolution of a physical system (much like MuZero can predict the evolution of the game board), and make a decision, is going to perform much better than any of the vision based transformer models that are in cars today.

But there are other areas of research as well, namely in hardware with specific instructions for general compute.