"a larger model with RAG etc is still better than a small one"

This paper from DeepMind a few years ago offers a counter example to this claim.

https://arxiv.org/abs/2112.04426