Hacker News Reader

Comment by itchyjunk | Hacker News Reader

"Federated Learning enables mobile phones to collaboratively learn a shared prediction model while keeping all the training data on device, decoupling the ability to do machine learning from the need to store the data in the cloud."

So I assume this would help with privacy in a sense that you can train model on user data without transmitting it to the server. Is this in any way similar to something Apple calls 'Differential Privacy' [0] ?

"The key idea is to use the powerful processors in modern mobile devices to compute higher quality updates than simple gradient steps."

"Careful scheduling ensures training happens only when the device is idle, plugged in, and on a free wireless connection, so there is no impact on the phone's performance."

It's crazy what the phones of near future will be doing while 'idle'.

------------------------

[0] https://www.wired.com/2016/06/apples-differential-privacy-co...

View on HN

While I think you can definitely draw some parallels, differential privacy seems more targeted at metric collection. You have to be able to mutate the data in a way that it becomes non-identifying, without corrupting the answer in aggregate. Apple would still do all their training in the cloud.

In contrast, what Google's proposing is more like distributed training. In regular SGD, you'd iterate over a bunch of tiny batches, sequentially through your whole training set. Sounds like Google's saying each device becomes it's own mini-batch, and it beams up the result, and Google will average them all out in a smart way (I didn't read the paper, but this was the gist I got from the article).

Both ideas are in the same spirit, just the implementations are very different.

Differential Privacy is much more than what Apple's PR department says, differentially private SGD is already a thing.

Well, forget Apple for a moment (that was just an example, since parent asked about them specifically): my point was what Google's describing is separate from differential privacy. There's no controlled noise or randomness being applied.

They even say at the end of the paper: "While federated learning offers many practical privacy benefits, providing stronger guarantees via differential privacy, secure multi-party computation, or their combination is an interesting direction for future work." So, the "practical privacy benefits" here is referring to the dimensionality reduction from running the raw data thru the LSTM.

This is different from differential privacy (which, btw, isn't just an apple thing). Differential privacy essentially says some responses will be lies, but that we can still get truthful aggregate information. The canonical example is the following process: Flip a coin, if it's head, tell me whether you're a communist. If it's tails, flip another coin and if that one comes up heads, tell me you're a communist, and if it's tails, tell me you're not.

From one run, you can't tell if any individual is telling the truth, but you can still estimate the number of communists from the aggregate responses.

This is doing local model training, and sending the model updates, instead of the raw data that would usually be used for training.

Chrome used differential privacy far before Apple. See the RAPPOR paper.

Here's google doing this in November 2015:

http://download.tensorflow.org/paper/whitepaper2015.pdf

Reply on news.ycombinator.com