Hacker News Reader

There seems to be a big difference between efficiently training a "large-ish" model on 4-8 GPUs and a gigantic model on 1000+ GPUs. The same techniques might not work due to different warm up steps, gradient overlaps etc.

All you can see running in the wild are quantized LLaMA variants (4 or 8-bit) whereas the original model is 32-bit.

Sounds like a great Kickstarter option for a truly free model. I'd back it.

Please do it, people shouldn't put up with the apathetic siloed status quo. I'm sure people will find all sorts of beneficial uses for these models they are going to run on their own hardware!

These models are as valuable as code now.

And we know how open source eventually took over pretty much all code. I expect this to be the same, but instead of four decades, it will be four months

Whats possible now (March 2023).

The field is moving so quick.