I also heard this. I unfortunately forget which study it was but yes, their paper spoke of likely diminishing returns at least at around 400-500B parameters for current LLM's. The recent news of GPT-4 running on 8x 220B LLM's (which doesn't equal a 8*220B size) fits that range and it's also questionable how much further we can push LLM's further by introducing multiple models like this, because this too eventually introduces problems due to granularity and picking the right model if I understood this correctly (from an earlier Hacker News discussion). (sorry for altogether no sources lol)