Hacker News Reader

Comment by mooneater | Hacker News Reader

This implies more centralization, as those with cheap access to vast compute gain a bigger relative edge.

Yes, unfortunately both data and compute will probably become more and more centralized. At least the algorithmic components have a chance to becone available to everyone.

Here is an off the cuff thought: What if there was (or maybe there already is?) a system which is distributed such as was SETI back in the day and its a massively distributed general AI that can be used - and people on a mass scale allow for slices of their compute to be part of the system?

So Skynet, but it lets you rent parts of itself? I'm sure someone is writing an ICO whitepaper for this right now, if they haven't already...

EOS: https://eos.io/

Raised $2.7B in its ICO, currently trading at a market cap of $10B.

FileCoin: https://filecoin.io/

Raised $257M in its ICO.

Tezos: https://tezos.com/

Raised $232M in its ICO.

Those are the 3 largest ICOs of all time, so yes, there is definitely a market for renting part of Skynet.

The actual technology may or may not be vaporware or a scam. IMHO the way you build a decentralized P2P system is to give a single really smart programmer enough to live on for a couple years and see what he comes up with, not throw a billion dollars at a Cayman Islands corporation that may or may not use it for anything productive. Sorta like what Ethereum did.

Are any of those platforms suitable for running deep learning algorithms?

I think Golem is closer https://golem.network/ And some others: https://www.investinblockchain.com/distributed-computing-blo...

But I'm skeptical of distributed computing blockchains. I think a) it's unlikely a distributed compute network can compete with highly optimized datacenters running TPUs or whatever, b) people are unlikely to trust distributed compute networks with their proprietary data (maybe acceptable for CGI rendering and some other specific use-cases)

The current machine learning paradigms are hard to distribute efficiently - they can be parallelized but require significant ongoing communication between nodes, you can't really split a problem into separate subtasks and merge them only in the end, you need to transfer all the calculated values after, say, each iteration.

I'd guess that having ten extra machines in the same rack would be more valuable than a thousand remote machines with limited network bandwidth.

> its a massively distributed general AI

You've just described a centralized system.

Centralization can happen at different layers - not all technical. The ultimate centralization is ownership, as defined legally.

One problem with the SETI model is that a lot of the increase in available compute capacity is due to specialized GPU or TPU processors, which aren't widely available outside of purpose-built data centers. Trying to offload ML workloads to general purpose CPUs would likely be quite wasteful in terms of power consumption unless you can somehow get access to graphics processors.

This is why I am working on raspberry pi based neural net things

We have learned a lot using big computing which can still inform better efficiency of AI on smaller computing units. Raspi is pretty good because it is quite limiting, but also quite capable.

Is it really worth training on a Raspberry Pi? It seems vastly underpowered compared to even modest desktop hardware.

It's worth having a self-contained option available. For something like a drone, you don't need all that much computation for on-the-fly PID tuning in response to changing weather or different piloting styles, etc.

You've hooked a NN up to a PID? How's that going? It's hard enough tuning those things by hand using squishy human brain networks.

I haven't done it myself yet, it's on the to-do list. There is a lot of academic material on PID autotuning, not always with neural networks but that seems the most straightforward way. A Raspberry Pi is probably overkill for the job, actually.

Inference on embedded hardware makes sense, but training no so much.

Reply on news.ycombinator.com