I'm curious on your point 1, and I tend to disagree. The number of parameters in these large language models is increasing faster than Moore's law. Currently you need a server full of GPUs just to run inference on a PaLM model. How do you see the size shrinking so drastically? Hardware is improving on important factors like power consumption, but inference hardware needs to scale with the size of the models. Don't get me wrong, it's likely that PaLM itself can run on 2032 phone, but the real advances will be in even more scaled up models.

The future of AI will be in the data center for a long time to come. Maybe after some point the models cease to scale up and that point will be where the model would even overfit on the amount of data we can possibly give it e.g. the entire internet. The PaLM authors allude to this in their conclusion