if it weren't for China, half a dozen US billionaires would own the frontier, gatekeeping any model more powerful than Llama from the public. fortunately China is open-sourcing powerful models, but how long will they keep it up?
What explains the prevalence of models being trained by small organizations, or even lone individuals, on relatively affordable hardware, then?
> so training a frontier model requires billions of dollars
The term "frontier" here almost makes this a tautology -- "the kinds of models that currently cost billions of dollars to train are only being trained by people who have billions of dollars". But things are moving so fast that the frontier of the immediate moment will be well-trodden ground a few months from now.
Ground-up models are emerging at an accelerating pace, and the FOSS community is producing distillations of proprietary models rapidly.