I wonder if we can afford the energy cost of the ever-increasing need to compile more and more information into ever larger AI baskets. Is there a point when the cost exceeds even what a Zuckerberg can pay?
I am hopeful developments like Mistral (a 7B model with the performance of ChatGPT for most of the tasks I've used it for) keeps happening. Seems like we can still compress a lot of knowledge.