I don't buy that analogy. Brains do grow over time, but what's more important is the continual learning and interaction with the environment.

You need to have multiple layers in an LLM to process and refine the information. The larger the token count, etc. are the larger each layer is. The activation density of these networks is low compared to the smaller parameter models.

It may be that you need the large networks in order to learn complex tasks, but to me it seems like the big companies are throwing compute (more data, more hardware, more parameters, etc.) at the problem because that is currently easier and simpler than doing things like creating new architectures.

AFAICT, all the architecture advancements (rope, mixture of experts, reasoning, etc.) have all come from outside of the big AI companies. I think those are where we'll see the better gains, some of which are likely to make these easier to run on smaller hardware.