I don't see anything impressive in this paper. They basically say what they want to do, no results.
They are not even original with the idea for this type of paper.
In September last year Aran Komatsuzaki posted: "Current Limitations of Language Models: What You Need is Retrieval" https://arxiv.org/abs/2009.06857
I have also a prediction market from August last year with this question:
Will a language model with at least 100B parameters trained to do external information retrieval exist before 2023?
Seems like people gave it 61% in March this year. I was at 75%.
https://www.metaculus.com/questions/4876/will-a-language-mod...