As a concept infinite context will not be discovered. We can spin up huge models with large contexts that cost more but even 1M context is 1 expensive and 2 not 3trillion(size of some datasets)

Rag is not bad attention, rag is the user not knowing the context to give the LLM