The model was sharded into ~800MB pieces over devices with 4GB RAM. Typical device storage is 16-32GB NAND. Throughput was 0.39tps.
Compromise enough devices (think thousands of off-brand TVs), and you have plenty of redundancy.
Compromise enough devices (think thousands of off-brand TVs), and you have plenty of redundancy.
That analogy only applies to tasks with similar characteristics. Is running a computer program the same as gestating a baby?
Also I don't think this a case where they distributed the task to speed it up, but rather their goal was to compromise speed to make it runnable.