Recent paper showing how LMs "struggle with less
popular factual knowledge, and that scaling
fails to appreciably improve memorization of
factual knowledge in the tail". In other words, simply scaling LLMs will not result in replacing Google as a search engine. Approaches like theirs ie retrieval augmentation are needed.
https://akariasai.github.io/files/llm_memorization.pdf
This is essentially what I'm building for the content marketing vertical.