Hacker News Reader

Comment by ww520 | Hacker News Reader

I feel that the old SETI@home project can use a comeback. Most home machines have powerful GPU these days. The ML problems are embarrassingly parallel and can be distributed to the home machines.

Just need to work out the economy for everyone involved.

View on HN

Datacenter GPUs have >1000gbps network connections between nodes, which is necessary to actually utilize GPUs with current training techniques. It's possible that a furthering of techniques used in GPT-JT[1] might make it feasible to use home computers, but even GPT-JT requires at least a 1gbps connection.

[1]: https://www.together.xyz/blog/releasing-v1-of-gpt-jt-powered...

Good point. One criterion for whether a SETI@home approach is reasonable for a class of problems is if the total processing time exceeds the network download and upload time. In SETI@home classic the work units didn't take too much time to download compared to how long it took to process a work unit.

Single ML job can’t be distributed very well. More often than not they hit network limit, even on single zone 1 GB/s network speed that we normally get. Most of the distributed workloads use something like NVLink.

That's true. Most ML algorithms have the iterate-until-converge pattern. How about some tasks like hyper parameter tuning or trying out different algorithms against the same data set? Those can be run in parallel.

ML research should aim to produce more parallel algorithms.

state of the art models don't fit on a single gpu anymore, not for a while

Is that more for training or inference use?

I realize this isn't always entirely decoupled in certain online learning approaches. I don't work in ML, am certainly not an expert, and am genuinely curious where this space is at now in terms of hardware requirements for SOTA methodologies these days, especially inference phase HW requirements for just running stuff that's out there.

https://vast.ai?

why....that sounds like Web3! (I kid I kid...)

Actually it's a good use of Web3, handling of the billing and revenue sharing portion of the problem.

Yeah, in theory it's a great idea. In practice, it always ends up as a challenge for economies of scale. Just look at Bitcoin, we can all mine it, but it's not worth it unless you scale. In the end, someone will build a data center, just to gather profits from whatever Web3 implementation you think of. Because their energy cost is lower, relatively, it will become too expensive to 'mine' at home.

It's the circle of capitalism.

Reply on news.ycombinator.com