Hacker News Reader

Comment by lagrange77 | Hacker News Reader

There should be a system in place like Folding@Home or SETI@Home for situations like this.

Agreed. This feels like the next big step in open source machine learning. I would love to be able to dedicate CPU / GPU towards chunking out open-source GPT-3 / DALL-E 2 clones so long as the trained weights are shared with the world afterwards.

That feels more long-term productive to me than crunching coins.

For what it's worth, the architecture for seti@home doesn't make much sense any more in the modern era for most types of data processing. It's simply too slow and error-prone to transfer large amounts of data between user machines. Modern radio telescope data analysis happens at datacenters with a lot of GPUs in them, just like modern AI model training.

Okay, gotcha. Just out of interest, why is it error prone? Because of some kind of merge conflicts of the results?

No, I just mean you design your system expecting the data from machine 1462 to get analyzed at some point, but that machine belongs to some random person who uninstalled your software without you knowing about it, so you never get back an analysis. Or you have bugs that come from 100 different versions of your software running because you can't simply upgrade the software on all the machines of the datacenter.

I understand.

> you can't simply upgrade the software on all the machines

You can update a website. And a website can use the local gpu.

The break the training task in smaller pieces, parallelize and distribute it part is indeed beyond my skills.

In a way, it is similar to our approach at Q Blocks. Decentralized GPU computing for ML.

Reply on news.ycombinator.com