I predict we will have distributed training systems soon on peer to peer like network for collaboratively building free models to use, if we can all pitch in some GPUs we can get 100s of thousand of individuals working on training giant LLM models together. It’s an unsolved problem currently but should have a solution soon.
Additionally today we can build sub-$10K systems to do self hosted inference on 65B weights models.
I’m seriously confused why anybody thinks anyone will even remotely care about any kind of pause is asked for.
For sure everyone with computers and a reasonable level of skills are now looking at this field. The progress will be exponential and uncontrollable in my opinion. I’m fine with that and I also think there is absolutely nothing that can stop it anyway.