Serious question - why doesn’t someone crowd source the funds to train a GPT scale model for open source? I assume it’s not just a matter of a ton of GPU instances?
Doesn't one need to have a bunch of "very good" data to train on? I'm under the impression that sourcing costs are large.
Yes, and the labor of the Reinforcement Learning from Human Feedback (RLHF) .. is anyone crowdsourcing this?
Anthropic and Databricks have made datasets available. OpenAssistant is crowdsourcing one.
These seem like things affordable with money and crowd sourcing of volunteer labor. In some ways it feels too valuable to leave in the hands of megacorps.