Hacker News Reader

Comment by DeathArrow | Hacker News Reader

People were blaming cryptocurrencies miners for the prices of GPUs, when in fact it was the AI researchers who bought all the GPUs. :D

I wonder what if somebody designs an electronic currency rewarded as payment for general GPU computations instead of just computing hashes? You pay some $, to train your model and the miner gets some coins.

Every one is happy, electricity is not wasted and the GPUs gets used for a reasonable purpose.

View on HN

The current way of training is efficient when compute is located in a single place and is colocated with large quantities of training data. Distributing small parts of computation to remote computers is theoretically possible (and an active direction of research) but currently not preferable nor widely used; you really need very high bandwidth between all the nodes to constantly synchronize the hundreds-of-gigabytes sized weights they're iterating on and the resulting gradients.

This may not be true in the future. There is some work being done on distributed neural net training. I can't recall the name of the technique at the moment, but a paper came out in the last year showing results comparable with backprop that only required local communication of information (whatever this technique's alternative to gradients is).

My understanding is that proof-of-work is intentionally wasteful; the objective is to make 51% attacks (where a single entity controls at least 51% of the global hashrate) infeasible by attaching a cost to the mining process.

Making the mining process produce useful output that can be resold nullifies the purpose as it means an attacker can now mine "for free" as a byproduct of doing general-purpose computations (as opposed to tying up dedicated hardware), lowering the barrier for a 51% attack dramatically.

Yes, this is an old idea (which I really like) but it hasn't really taken off yet. GridCoin was one example, where you solved BOINC problems or RLC that's for more general computation.

The problem is that, currently, large ML models need to be trained on clusters of tightly-connected GPUs/accelerators. So it's kinda useless having a bunch of GPUs spread all over the world with huge latency and low bandwidth between them. That may change though - there are people working on it: https://github.com/learning-at-home/hivemind

It hasn't taken off because it doesn't work. PoW only works for things that are hard to calculate but easy to verify. Any meaningful result is equally hard to verify.

> Any meaningful result is equally hard to verify.

This is very much not true. A central class in complexity is NP whose problems are hard to answer but easy to verify if the answer is yes.

E.g. is there a path visiting all nodes in this graph of length less than 243000? Hard to answer but easy to check any proposed answer.

It's easy to verify ML training - inference on a test set has lower error than it did before.

Training NN ML is much slower than inference (1000x at least) because it has to calculate all of the gradients.

It'd be awesome if instead of the proof of work algorithms being based on generating useless hashes, they were instead based on computing protein folding simulations in a way that actually benefits everyone.

If everyone offer GPUs, is the same game. If I will buy more GPU I will get more money, so the average payment for a person with a single or a small bunch of GPU will be low.

And second, the principles of electronic currency are different from gold/money. That's why crypto uses GPU ;)

Reply on news.ycombinator.com